2024-09-21T03:08:27.463 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T03:08:27.488 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:08:27.600 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913456' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO83jIvsiTkOdZnxCvh0SiOhF3AFfrRdmrYxkkHTxvEkYKpAfIhEjrHLFtrrwE+T1nZjmfH/KYMDbB/g9TWOTFM= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMx7CH9GWga4Ns7u5w+5Fpe8+sjZmgh0JhcO11ILgS98Z2kVc5kpZTkkkPS1L0+XKb6jMHtCEk4acyCC0SbJyW0= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEaqEfBzomynGes8u1ka1lN9Zsc8lb2gbxIU+54qeHaPmmczuRcB4he2XtnEgCzDm89DMZb49gTwUfMdqY41Vi8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T03:08:27.600 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T03:08:27.600 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T03:08:27.601 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T03:08:27.602 INFO:teuthology.task.internal:Checking packages... 2024-09-21T03:08:27.632 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T03:08:27.633 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T03:08:27.633 INFO:teuthology.packaging:ref: None 2024-09-21T03:08:27.633 INFO:teuthology.packaging:tag: None 2024-09-21T03:08:27.633 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:08:27.633 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:08:27.633 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:08:27.775 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T03:08:27.776 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T03:08:27.786 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T03:08:27.786 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T03:08:27.811 INFO:teuthology.task.internal:Saving configuration 2024-09-21T03:08:27.831 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T03:08:27.836 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T03:08:27.854 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:03:13.928566', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO83jIvsiTkOdZnxCvh0SiOhF3AFfrRdmrYxkkHTxvEkYKpAfIhEjrHLFtrrwE+T1nZjmfH/KYMDbB/g9TWOTFM='} 2024-09-21T03:08:27.872 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:03:13.929704', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMx7CH9GWga4Ns7u5w+5Fpe8+sjZmgh0JhcO11ILgS98Z2kVc5kpZTkkkPS1L0+XKb6jMHtCEk4acyCC0SbJyW0='} 2024-09-21T03:08:27.894 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:03:13.926582', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEaqEfBzomynGes8u1ka1lN9Zsc8lb2gbxIU+54qeHaPmmczuRcB4he2XtnEgCzDm89DMZb49gTwUfMdqY41Vi8='} 2024-09-21T03:08:27.894 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T03:08:27.900 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T03:08:27.900 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T03:08:27.900 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T03:08:27.900 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T03:08:27.963 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f51ea8535b0>, signals=[15]) 2024-09-21T03:08:27.963 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T03:08:27.968 INFO:teuthology.task.internal:Opening connections... 2024-09-21T03:08:27.968 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T03:08:27.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:08:28.044 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T03:08:28.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:08:28.121 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-09-21T03:08:28.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:08:28.197 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T03:08:28.203 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-09-21T03:08:28.222 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-09-21T03:08:28.222 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-09-21T03:08:28.280 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-09-21T03:08:28.280 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-09-21T03:08:28.280 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-09-21T03:08:28.280 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:08:28.281 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-09-21T03:08:28.281 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:08:28.281 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:08:28.281 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-09-21T03:08:28.281 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:08:28.281 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:08:28.281 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:08:28.281 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:08:28.281 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:08:28.281 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:08:28.282 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-09-21T03:08:28.308 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-09-21T03:08:28.326 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-09-21T03:08:28.327 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-09-21T03:08:28.384 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Stream" 2024-09-21T03:08:28.385 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="9" 2024-09-21T03:08:28.385 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2024-09-21T03:08:28.385 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:08:28.385 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="9" 2024-09-21T03:08:28.385 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:08:28.385 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:08:28.385 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2024-09-21T03:08:28.385 INFO:teuthology.orchestra.run.smithi187.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:08:28.385 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:08:28.385 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:08:28.385 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:08:28.386 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:08:28.386 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:08:28.386 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-09-21T03:08:28.408 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-09-21T03:08:28.427 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-09-21T03:08:28.427 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-09-21T03:08:28.485 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-09-21T03:08:28.485 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-09-21T03:08:28.485 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-09-21T03:08:28.485 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:08:28.485 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-09-21T03:08:28.486 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:08:28.486 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:08:28.486 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-09-21T03:08:28.486 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:08:28.486 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:08:28.486 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:08:28.486 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:08:28.486 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:08:28.486 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:08:28.486 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-09-21T03:08:28.516 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T03:08:28.525 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T03:08:28.530 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T03:08:28.530 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:08:28.533 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:08:28.535 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:08:28.551 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T03:08:28.556 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T03:08:28.557 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:08:28.590 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:08:28.593 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:08:28.699 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T03:08:28.767 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T03:08:28.779 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T03:08:28.779 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T03:08:28.779 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T03:08:28.780 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:08:28.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:08:28.797 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-09-21T03:08:28.853 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:08:28.854 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-503.el9.x86_64 2024-09-21T03:08:28.854 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2024-09-21T03:08:37.438 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 1.2 MB/s | 8.3 MB 00:06 2024-09-21T03:08:39.089 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-21T03:08:43.215 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-21T03:08:44.632 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 19 kB 00:00 2024-09-21T03:08:45.744 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T03:08:51.175 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T03:08:53.074 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:08:53.074 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:08:53.177 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-21T03:08:53.186 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-21T03:08:53.186 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-09-21T03:08:53.186 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-21T03:08:53.186 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-21T03:08:53.186 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:08:53.186 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-21T03:08:53.186 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:08:53.186 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:08:53.186 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:08:53.187 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:08:53.187 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-21T03:08:53.187 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-21T03:08:53.187 INFO:teuthology.orchestra.run.smithi112.stdout:Install 4 Packages 2024-09-21T03:08:53.187 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:08:53.187 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 79 M 2024-09-21T03:08:53.187 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 126 M 2024-09-21T03:08:53.188 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-21T03:08:53.419 INFO:teuthology.orchestra.run.smithi112.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 341 kB/s | 36 kB 00:00 2024-09-21T03:08:53.744 INFO:teuthology.orchestra.run.smithi112.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-09-21T03:08:54.211 INFO:teuthology.orchestra.run.smithi112.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-21T03:08:54.469 INFO:teuthology.orchestra.run.smithi112.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 27 MB/s | 29 MB 00:01 2024-09-21T03:08:54.470 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:08:54.470 INFO:teuthology.orchestra.run.smithi112.stdout:Total 61 MB/s | 79 MB 00:01 2024-09-21T03:08:55.151 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-21T03:08:55.514 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-21T03:08:55.514 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-21T03:08:56.464 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-21T03:08:56.464 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-21T03:08:58.888 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-21T03:08:59.567 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:08:59.645 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:09:02.417 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:09:02.558 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:09:07.365 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:09:07.408 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:09:12.264 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:09:50.885 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:09:50.929 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:09:51.346 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:09:51.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:09:51.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:09:51.346 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:09:52.201 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:09:52.201 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:09:52.201 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-21T03:09:52.201 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:09:52.202 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:09:52.202 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:09:52.202 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:09:52.202 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:09:52.202 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-21T03:09:52.572 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:09:52.660 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:09:52.661 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:09:52.661 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:09:52.661 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:09:52.661 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:09:52.662 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T03:09:52.662 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:09:52.662 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:09:52.721 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:09:52.721 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:09:52.721 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -qi grub2-tools 2024-09-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi112.stdout:Name : grub2-tools 2024-09-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi112.stdout:Epoch : 1 2024-09-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi112.stdout:Version : 2.06 2024-09-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi112.stdout:Release : 82.el9 2024-09-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: x86_64 2024-09-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi112.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi112.stdout:Group : Unspecified 2024-09-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi112.stdout:Size : 8274670 2024-09-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi112.stdout:License : GPLv3+ 2024-09-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi112.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:Packager : builder@centos.org 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:Vendor : CentOS 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:Summary : Support tools for GRUB. 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:Description : 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:hardware devices. 2024-09-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi112.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:09:52.760 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:09:52.760 INFO:teuthology.task.kernel:Updating grub on smithi112 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:09:52.761 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:09:53.467 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2024-09-21T03:09:54.813 INFO:teuthology.orchestra.run.smithi112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:09:54.834 INFO:teuthology.orchestra.run.smithi112.stderr:done 2024-09-21T03:09:54.836 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-09-21T03:09:54.851 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.y5JFdzNnCo 2024-09-21T03:09:54.852 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.y5JFdzNnCo 2024-09-21T03:09:54.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.y5JFdzNnCo 2024-09-21T03:09:55.065 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.y5JFdzNnCo is 6KB 2024-09-21T03:09:55.085 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.y5JFdzNnCo 2024-09-21T03:09:55.100 DEBUG:teuthology.orchestra.run.smithi112:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:09:55.162 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:09:55.162 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:09:55.162 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:09:55.162 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:09:55.164 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:09:55.335 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2024-09-21T03:09:55.432 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-09-21T03:10:25.341 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T03:10:25.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T03:10:25.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:11:25.345 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T03:11:34.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:11:34.700 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-09-21T03:11:35.090 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T03:11:35.091 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:11:35.091 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-09-21T03:11:35.105 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:11:35.105 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:11:35.105 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:11:35.106 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:11:36.106 DEBUG:teuthology.parallel:result is None 2024-09-21T03:11:36.106 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T03:11:36.115 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T03:11:36.115 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:11:36.117 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:11:36.120 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:11:36.139 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T03:11:36.148 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T03:11:36.153 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T03:11:36.153 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:11:36.178 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:11:36.180 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:11:36.248 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T03:11:36.254 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T03:11:36.254 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:11:36.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:11:36.272 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:11:36.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:11:36.288 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:11:36.290 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:11:36.314 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:11:36.352 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:11:36.363 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:11:36.369 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:11:36.390 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:11:36.402 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:11:36.407 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:11:36.409 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T03:11:36.416 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T03:11:36.416 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:11:36.418 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:11:36.421 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:11:36.448 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T03:11:36.458 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T03:11:36.458 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:11:36.487 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:11:36.490 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:11:36.506 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:11:36.585 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:11:36.635 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:11:36.721 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:11:36.766 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:11:36.766 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:11:36.834 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:11:36.874 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:11:36.935 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:11:37.019 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:11:37.063 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T03:11:37.064 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:11:37.131 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:11:37.177 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:11:37.248 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:11:37.289 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:11:37.335 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T03:11:37.335 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:11:37.403 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-09-21T03:11:37.405 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2024-09-21T03:11:37.407 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-09-21T03:11:37.447 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:11:37.455 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:11:37.478 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:11:37.961 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T03:11:37.975 INFO:teuthology.task.internal:Starting timer... 2024-09-21T03:11:37.975 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T03:11:38.008 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T03:11:38.040 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T03:11:38.040 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-09-21T03:11:38.118 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-09-21T03:11:38.120 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2024-09-21T03:11:38.168 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2024-09-21T03:11:38.170 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-09-21T03:11:38.217 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-09-21T03:11:38.219 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T03:11:38.219 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-09-21T03:11:38.259 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-09-21T03:11:38.259 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2024-09-21T03:11:38.294 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2024-09-21T03:11:38.294 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-09-21T03:11:38.314 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-09-21T03:11:38.314 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-09-21T03:11:38.314 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:11:38.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:11:38.339 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:11:38.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:11:38.366 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:11:38.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:11:38.393 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T03:11:38.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:11:38.429 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:11:38.458 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:11:38.488 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T03:11:38.497 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T03:11:38.500 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T03:11:38.608 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T03:11:38.618 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T03:11:38.619 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi187.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T03:15:38.136 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-09-21T03:15:38.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T03:15:38.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:15:38.213 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-09-21T03:15:38.252 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T03:15:38.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-09-21T03:15:38.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:15:38.329 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-09-21T03:15:38.415 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-09-21T03:15:38.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-21T03:15:38.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:15:38.494 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-09-21T03:15:38.574 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-21T03:15:38.574 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T03:15:38.584 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T03:15:38.584 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:15:38.584 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:15:38.587 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:15:38.587 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:15:38.589 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:15:38.589 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:15:38.617 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:15:38.621 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:15:38.634 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:15:38.637 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:15:38.643 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:15:38.657 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:15:38.661 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-09-21T03:15:38.663 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2024-09-21T03:15:38.672 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-09-21T03:15:38.675 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2024-09-21T03:15:38.681 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-09-21T03:15:38.688 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:15:38.690 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:15:38.692 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-09-21T03:15:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:15:38.704 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:15:38.706 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:15:38.720 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:15:38.763 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:15:38.765 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:15:38.765 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-09-21T03:15:38.765 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.766 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.766 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.766 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.767 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:15:38.770 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:15:38.770 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-21T03:15:38.770 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.770 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.770 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.770 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.775 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:15:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:15:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-09-21T03:15:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:15:38.780 INFO:teuthology.run_tasks:Running task install... 2024-09-21T03:15:38.788 DEBUG:teuthology.task.install:project ceph 2024-09-21T03:15:38.788 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T03:15:38.788 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T03:15:38.788 INFO:teuthology.task.install:Using flavor: default 2024-09-21T03:15:38.794 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T03:15:38.794 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T03:15:38.794 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:15:38.795 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:15:38.797 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:15:38.797 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:15:38.798 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:15:38.798 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:15:38.945 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:15:38.945 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:15:38.950 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:15:38.950 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:15:38.954 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:15:38.954 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:15:39.054 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:15:39.054 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:15:39.054 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:15:39.060 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:15:39.060 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T03:15:39.060 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:15:39.065 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:15:39.066 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T03:15:39.066 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:15:39.081 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:15:39.081 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:15:39.086 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:15:39.086 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:15:39.092 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:15:39.092 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:15:39.147 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:15:39.154 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:15:39.157 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:15:39.222 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:15:39.229 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:15:39.233 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:15:39.286 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-09-21T03:15:39.287 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-09-21T03:15:39.293 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2024-09-21T03:15:39.294 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2024-09-21T03:15:39.297 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2024-09-21T03:15:39.298 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2024-09-21T03:15:39.509 INFO:teuthology.orchestra.run.smithi112.stdout:45 files removed 2024-09-21T03:15:39.518 INFO:teuthology.orchestra.run.smithi203.stdout:45 files removed 2024-09-21T03:15:39.524 INFO:teuthology.orchestra.run.smithi187.stdout:45 files removed 2024-09-21T03:15:39.537 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:15:39.544 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:15:39.550 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:15:40.096 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-21T03:15:40.102 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-21T03:15:40.109 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 298 kB/s | 84 kB 00:00 2024-09-21T03:15:40.388 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 93 kB/s | 24 kB 00:00 2024-09-21T03:15:40.410 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-09-21T03:15:40.664 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T03:15:40.687 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T03:15:40.694 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T03:15:41.686 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.3 MB 00:00 2024-09-21T03:15:41.777 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.3 MB 00:01 2024-09-21T03:15:42.764 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.3 MB 00:02 2024-09-21T03:15:44.112 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-21T03:15:44.221 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-21T03:15:44.565 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-21T03:15:48.629 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.5 MB 00:00 2024-09-21T03:15:49.088 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 6.5 MB 00:01 2024-09-21T03:15:49.166 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.5 MB 00:00 2024-09-21T03:15:50.155 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 19 kB 00:00 2024-09-21T03:15:50.983 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-21T03:15:51.084 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-21T03:15:51.348 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-21T03:15:52.104 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T03:15:52.453 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T03:15:56.507 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T03:15:57.485 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-21T03:15:57.933 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-21T03:15:58.642 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:15:58.642 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:15:58.653 INFO:teuthology.orchestra.run.smithi187.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:15:58.654 INFO:teuthology.orchestra.run.smithi187.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:15:58.655 INFO:teuthology.orchestra.run.smithi187.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:15:58.656 INFO:teuthology.orchestra.run.smithi187.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:15:58.658 INFO:teuthology.orchestra.run.smithi187.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:15:58.660 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:15:58.661 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:15:58.722 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-09-21T03:15:58.732 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-09-21T03:15:58.732 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2024-09-21T03:15:58.732 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-09-21T03:15:58.732 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:15:58.733 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:15:58.734 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:15:58.735 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:15:58.736 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:15:58.737 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:15:58.738 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:15:58.739 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:15:58.740 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:15:58.741 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:15:58.742 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:15:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout:Installing weak dependencies: 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-09-21T03:15:58.744 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-09-21T03:15:58.745 INFO:teuthology.orchestra.run.smithi187.stdout:Install 149 Packages 2024-09-21T03:15:58.745 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 8 Packages 2024-09-21T03:15:58.745 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:15:58.745 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 228 M 2024-09-21T03:15:58.745 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-09-21T03:15:59.458 INFO:teuthology.orchestra.run.smithi187.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 51 kB/s | 6.6 kB 00:00 2024-09-21T03:15:59.648 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:15:59.649 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:15:59.662 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:15:59.663 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:15:59.664 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:15:59.665 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:15:59.668 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:15:59.670 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:15:59.671 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:15:59.735 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-21T03:15:59.742 INFO:teuthology.orchestra.run.smithi187.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.0 MB/s | 873 kB 00:00 2024-09-21T03:15:59.744 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:15:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:15:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:15:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:15:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:15:59.749 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:15:59.750 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:15:59.751 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:15:59.752 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:15:59.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:15:59.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:15:59.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-09-21T03:15:59.756 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:15:59.757 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:15:59.757 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-21T03:15:59.757 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-21T03:15:59.757 INFO:teuthology.orchestra.run.smithi112.stdout:Install 149 Packages 2024-09-21T03:15:59.757 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 8 Packages 2024-09-21T03:15:59.757 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:15:59.757 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 228 M 2024-09-21T03:15:59.758 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-21T03:15:59.834 INFO:teuthology.orchestra.run.smithi187.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.6 MB/s | 148 kB 00:00 2024-09-21T03:15:59.917 INFO:teuthology.orchestra.run.smithi187.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 9.4 MB/s | 5.5 MB 00:00 2024-09-21T03:16:00.042 INFO:teuthology.orchestra.run.smithi187.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 12 MB/s | 1.5 MB 00:00 2024-09-21T03:16:00.185 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:16:00.186 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:16:00.197 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:16:00.198 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:16:00.199 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:16:00.199 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:16:00.201 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:16:00.203 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:16:00.204 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:16:00.266 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-09-21T03:16:00.292 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-09-21T03:16:00.293 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-09-21T03:16:00.293 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-09-21T03:16:00.293 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-09-21T03:16:00.293 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:16:00.293 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:16:00.293 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:16:00.293 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:16:00.293 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:16:00.293 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:16:00.294 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:16:00.295 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:16:00.296 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:16:00.297 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:16:00.298 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:16:00.299 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:16:00.300 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:16:00.301 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:16:00.302 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:16:00.303 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:16:00.304 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:16:00.305 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:16:00.305 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:16:00.305 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2024-09-21T03:16:00.305 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:16:00.305 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:16:00.305 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-09-21T03:16:00.305 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-09-21T03:16:00.305 INFO:teuthology.orchestra.run.smithi203.stdout:Install 149 Packages 2024-09-21T03:16:00.305 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 8 Packages 2024-09-21T03:16:00.305 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:16:00.306 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 228 M 2024-09-21T03:16:00.306 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-09-21T03:16:00.376 INFO:teuthology.orchestra.run.smithi187.stdout:(6/157): ceph-common-19.3.0-5120.g18432218.el9. 19 MB/s | 20 MB 00:01 2024-09-21T03:16:00.418 INFO:teuthology.orchestra.run.smithi187.stdout:(7/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 3.9 MB/s | 2.3 MB 00:00 2024-09-21T03:16:00.493 INFO:teuthology.orchestra.run.smithi187.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-21T03:16:00.511 INFO:teuthology.orchestra.run.smithi112.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-09-21T03:16:00.544 INFO:teuthology.orchestra.run.smithi187.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 508 kB/s | 25 kB 00:00 2024-09-21T03:16:01.035 INFO:teuthology.orchestra.run.smithi203.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-09-21T03:16:01.111 INFO:teuthology.orchestra.run.smithi112.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 873 kB 00:00 2024-09-21T03:16:01.127 INFO:teuthology.orchestra.run.smithi187.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 24 MB/s | 18 MB 00:00 2024-09-21T03:16:01.202 INFO:teuthology.orchestra.run.smithi187.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.0 MB/s | 226 kB 00:00 2024-09-21T03:16:01.203 INFO:teuthology.orchestra.run.smithi112.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.6 MB/s | 148 kB 00:00 2024-09-21T03:16:01.302 INFO:teuthology.orchestra.run.smithi187.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 271 kB/s | 27 kB 00:00 2024-09-21T03:16:01.425 INFO:teuthology.orchestra.run.smithi187.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 260 kB/s | 32 kB 00:00 2024-09-21T03:16:01.427 INFO:teuthology.orchestra.run.smithi203.stdout:(2/157): ceph-base-19.3.0-5120.g18432218.el9.x8 8.7 MB/s | 5.5 MB 00:00 2024-09-21T03:16:01.452 INFO:teuthology.orchestra.run.smithi203.stdout:(3/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.0 MB/s | 873 kB 00:00 2024-09-21T03:16:01.519 INFO:teuthology.orchestra.run.smithi203.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.6 MB/s | 148 kB 00:00 2024-09-21T03:16:01.520 INFO:teuthology.orchestra.run.smithi187.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 7.9 MB/s | 766 kB 00:00 2024-09-21T03:16:01.628 INFO:teuthology.orchestra.run.smithi187.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.4 MB/s | 158 kB 00:00 2024-09-21T03:16:01.636 INFO:teuthology.orchestra.run.smithi203.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 13 MB/s | 1.5 MB 00:00 2024-09-21T03:16:01.704 INFO:teuthology.orchestra.run.smithi187.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.7 MB/s | 127 kB 00:00 2024-09-21T03:16:01.812 INFO:teuthology.orchestra.run.smithi187.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 4.6 MB/s | 507 kB 00:00 2024-09-21T03:16:01.936 INFO:teuthology.orchestra.run.smithi203.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 17 MB/s | 5.0 MB 00:00 2024-09-21T03:16:02.079 INFO:teuthology.orchestra.run.smithi187.stdout:(18/157): ceph-radosgw-19.3.0-5120.g18432218.el 6.3 MB/s | 10 MB 00:01 2024-09-21T03:16:02.188 INFO:teuthology.orchestra.run.smithi187.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 417 kB/s | 45 kB 00:00 2024-09-21T03:16:02.195 INFO:teuthology.orchestra.run.smithi112.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.3 MB/s | 2.3 MB 00:00 2024-09-21T03:16:02.288 INFO:teuthology.orchestra.run.smithi187.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.5 MB/s | 148 kB 00:00 2024-09-21T03:16:02.411 INFO:teuthology.orchestra.run.smithi203.stdout:(7/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.4 MB/s | 2.3 MB 00:00 2024-09-21T03:16:02.422 INFO:teuthology.orchestra.run.smithi187.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 8.7 MB/s | 5.3 MB 00:00 2024-09-21T03:16:02.456 INFO:teuthology.orchestra.run.smithi187.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 965 kB/s | 161 kB 00:00 2024-09-21T03:16:02.514 INFO:teuthology.orchestra.run.smithi187.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 3.4 MB/s | 321 kB 00:00 2024-09-21T03:16:02.598 INFO:teuthology.orchestra.run.smithi187.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.2 MB/s | 99 kB 00:00 2024-09-21T03:16:02.604 INFO:teuthology.orchestra.run.smithi112.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 3.7 MB/s | 1.5 MB 00:00 2024-09-21T03:16:02.623 INFO:teuthology.orchestra.run.smithi187.stdout:(25/157): python3-rbd-19.3.0-5120.g18432218.el9 1.8 MB/s | 307 kB 00:00 2024-09-21T03:16:02.690 INFO:teuthology.orchestra.run.smithi187.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 939 kB/s | 86 kB 00:00 2024-09-21T03:16:03.074 INFO:teuthology.orchestra.run.smithi187.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 444 kB/s | 170 kB 00:00 2024-09-21T03:16:03.112 INFO:teuthology.orchestra.run.smithi203.stdout:(8/157): ceph-osd-19.3.0-5120.g18432218.el9.x86 15 MB/s | 18 MB 00:01 2024-09-21T03:16:03.124 INFO:teuthology.orchestra.run.smithi187.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 691 kB/s | 34 kB 00:00 2024-09-21T03:16:03.170 INFO:teuthology.orchestra.run.smithi203.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 434 kB/s | 25 kB 00:00 2024-09-21T03:16:03.183 INFO:teuthology.orchestra.run.smithi187.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.6 MB/s | 154 kB 00:00 2024-09-21T03:16:03.214 INFO:teuthology.orchestra.run.smithi112.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 8.2 MB/s | 5.0 MB 00:00 2024-09-21T03:16:03.395 INFO:teuthology.orchestra.run.smithi203.stdout:(10/157): ceph-common-19.3.0-5120.g18432218.el9 7.7 MB/s | 20 MB 00:02 2024-09-21T03:16:03.471 INFO:teuthology.orchestra.run.smithi203.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.9 MB/s | 226 kB 00:00 2024-09-21T03:16:03.538 INFO:teuthology.orchestra.run.smithi203.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 405 kB/s | 27 kB 00:00 2024-09-21T03:16:03.542 INFO:teuthology.orchestra.run.smithi187.stdout:(30/157): ceph-test-19.3.0-5120.g18432218.el9.x 13 MB/s | 40 MB 00:02 2024-09-21T03:16:03.600 INFO:teuthology.orchestra.run.smithi187.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 3.2 MB/s | 3.1 MB 00:00 2024-09-21T03:16:03.605 INFO:teuthology.orchestra.run.smithi203.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 476 kB/s | 32 kB 00:00 2024-09-21T03:16:03.742 INFO:teuthology.orchestra.run.smithi187.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.0 MB/s | 292 kB 00:00 2024-09-21T03:16:03.755 INFO:teuthology.orchestra.run.smithi203.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 5.0 MB/s | 766 kB 00:00 2024-09-21T03:16:03.818 INFO:teuthology.orchestra.run.smithi187.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 658 kB/s | 49 kB 00:00 2024-09-21T03:16:03.855 INFO:teuthology.orchestra.run.smithi203.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.6 MB/s | 158 kB 00:00 2024-09-21T03:16:03.901 INFO:teuthology.orchestra.run.smithi187.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 198 kB/s | 16 kB 00:00 2024-09-21T03:16:03.939 INFO:teuthology.orchestra.run.smithi203.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.5 MB/s | 127 kB 00:00 2024-09-21T03:16:04.051 INFO:teuthology.orchestra.run.smithi187.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 1.9 MB/s | 285 kB 00:00 2024-09-21T03:16:04.089 INFO:teuthology.orchestra.run.smithi203.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 3.3 MB/s | 507 kB 00:00 2024-09-21T03:16:04.204 INFO:teuthology.orchestra.run.smithi112.stdout:(7/157): ceph-osd-19.3.0-5120.g18432218.el9.x86 18 MB/s | 18 MB 00:00 2024-09-21T03:16:04.252 INFO:teuthology.orchestra.run.smithi187.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 3.8 MB/s | 784 kB 00:00 2024-09-21T03:16:04.352 INFO:teuthology.orchestra.run.smithi187.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 184 kB/s | 18 kB 00:00 2024-09-21T03:16:04.494 INFO:teuthology.orchestra.run.smithi187.stdout:(38/157): ceph-mgr-diskprediction-local-19.3.0- 7.8 MB/s | 7.4 MB 00:00 2024-09-21T03:16:04.628 INFO:teuthology.orchestra.run.smithi187.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 290 kB/s | 80 kB 00:00 2024-09-21T03:16:04.753 INFO:teuthology.orchestra.run.smithi187.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-21T03:16:04.788 INFO:teuthology.orchestra.run.smithi112.stdout:(8/157): ceph-radosgw-19.3.0-5120.g18432218.el9 18 MB/s | 10 MB 00:00 2024-09-21T03:16:04.807 INFO:teuthology.orchestra.run.smithi203.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 7.4 MB/s | 5.3 MB 00:00 2024-09-21T03:16:04.820 INFO:teuthology.orchestra.run.smithi187.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-21T03:16:04.830 INFO:teuthology.orchestra.run.smithi112.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 608 kB/s | 25 kB 00:00 2024-09-21T03:16:04.845 INFO:teuthology.orchestra.run.smithi187.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-21T03:16:04.865 INFO:teuthology.orchestra.run.smithi203.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 776 kB/s | 45 kB 00:00 2024-09-21T03:16:04.965 INFO:teuthology.orchestra.run.smithi203.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.5 MB/s | 148 kB 00:00 2024-09-21T03:16:05.037 INFO:teuthology.orchestra.run.smithi187.stdout:(43/157): ceph-mgr-dashboard-19.3.0-5120.g18432 7.6 MB/s | 14 MB 00:01 2024-09-21T03:16:05.049 INFO:teuthology.orchestra.run.smithi203.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.9 MB/s | 161 kB 00:00 2024-09-21T03:16:05.062 INFO:teuthology.orchestra.run.smithi187.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 491 kB/s | 106 kB 00:00 2024-09-21T03:16:05.080 INFO:teuthology.orchestra.run.smithi112.stdout:(10/157): ceph-base-19.3.0-5120.g18432218.el9.x 1.2 MB/s | 5.5 MB 00:04 2024-09-21T03:16:05.096 INFO:teuthology.orchestra.run.smithi187.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-09-21T03:16:05.129 INFO:teuthology.orchestra.run.smithi187.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-21T03:16:05.132 INFO:teuthology.orchestra.run.smithi203.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.8 MB/s | 321 kB 00:00 2024-09-21T03:16:05.163 INFO:teuthology.orchestra.run.smithi187.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-21T03:16:05.224 INFO:teuthology.orchestra.run.smithi203.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 3.3 MB/s | 307 kB 00:00 2024-09-21T03:16:05.281 INFO:teuthology.orchestra.run.smithi112.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.1 MB/s | 226 kB 00:00 2024-09-21T03:16:05.314 INFO:teuthology.orchestra.run.smithi187.stdout:(48/157): python3-pycparser-2.20-6.el9.noarch.r 490 kB/s | 135 kB 00:00 2024-09-21T03:16:05.325 INFO:teuthology.orchestra.run.smithi203.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 993 kB/s | 99 kB 00:00 2024-09-21T03:16:05.373 INFO:teuthology.orchestra.run.smithi112.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 295 kB/s | 27 kB 00:00 2024-09-21T03:16:05.417 INFO:teuthology.orchestra.run.smithi203.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 941 kB/s | 86 kB 00:00 2024-09-21T03:16:05.430 INFO:teuthology.orchestra.run.smithi187.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-21T03:16:05.464 INFO:teuthology.orchestra.run.smithi112.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 347 kB/s | 32 kB 00:00 2024-09-21T03:16:05.472 INFO:teuthology.orchestra.run.smithi187.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-21T03:16:05.498 INFO:teuthology.orchestra.run.smithi187.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 324 kB/s | 22 kB 00:00 2024-09-21T03:16:05.523 INFO:teuthology.orchestra.run.smithi187.stdout:(52/157): boost-program-options-1.75.0-8.el9.x8 511 kB/s | 107 kB 00:00 2024-09-21T03:16:05.548 INFO:teuthology.orchestra.run.smithi187.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-21T03:16:05.573 INFO:teuthology.orchestra.run.smithi187.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-21T03:16:05.599 INFO:teuthology.orchestra.run.smithi187.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-21T03:16:05.624 INFO:teuthology.orchestra.run.smithi187.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-21T03:16:05.658 INFO:teuthology.orchestra.run.smithi187.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.8 MB/s | 171 kB 00:00 2024-09-21T03:16:05.692 INFO:teuthology.orchestra.run.smithi187.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-21T03:16:05.716 INFO:teuthology.orchestra.run.smithi187.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-09-21T03:16:05.825 INFO:teuthology.orchestra.run.smithi187.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.9 MB/s | 662 kB 00:00 2024-09-21T03:16:05.851 INFO:teuthology.orchestra.run.smithi187.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 4.2 MB/s | 581 kB 00:00 2024-09-21T03:16:05.876 INFO:teuthology.orchestra.run.smithi187.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 486 kB/s | 24 kB 00:00 2024-09-21T03:16:05.901 INFO:teuthology.orchestra.run.smithi187.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-21T03:16:05.951 INFO:teuthology.orchestra.run.smithi187.stdout:(64/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.0 MB/s | 3.0 MB 00:00 2024-09-21T03:16:05.977 INFO:teuthology.orchestra.run.smithi187.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-21T03:16:06.000 INFO:teuthology.orchestra.run.smithi203.stdout:(26/157): rbd-mirror-19.3.0-5120.g18432218.el9. 5.3 MB/s | 3.1 MB 00:00 2024-09-21T03:16:06.002 INFO:teuthology.orchestra.run.smithi187.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-21T03:16:06.027 INFO:teuthology.orchestra.run.smithi187.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 509 kB/s | 38 kB 00:00 2024-09-21T03:16:06.051 INFO:teuthology.orchestra.run.smithi203.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T03:16:06.186 INFO:teuthology.orchestra.run.smithi187.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-21T03:16:06.290 INFO:teuthology.orchestra.run.smithi112.stdout:(14/157): ceph-test-19.3.0-5120.g18432218.el9.x 27 MB/s | 40 MB 00:01 2024-09-21T03:16:06.294 INFO:teuthology.orchestra.run.smithi187.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-21T03:16:06.323 INFO:teuthology.orchestra.run.smithi112.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 892 kB/s | 766 kB 00:00 2024-09-21T03:16:06.348 INFO:teuthology.orchestra.run.smithi112.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.7 MB/s | 158 kB 00:00 2024-09-21T03:16:06.353 INFO:teuthology.orchestra.run.smithi187.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T03:16:06.391 INFO:teuthology.orchestra.run.smithi112.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.8 MB/s | 127 kB 00:00 2024-09-21T03:16:06.410 INFO:teuthology.orchestra.run.smithi203.stdout:(28/157): ceph-test-19.3.0-5120.g18432218.el9.x 12 MB/s | 40 MB 00:03 2024-09-21T03:16:06.424 INFO:teuthology.orchestra.run.smithi112.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 6.6 MB/s | 507 kB 00:00 2024-09-21T03:16:06.435 INFO:teuthology.orchestra.run.smithi203.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 89 kB/s | 34 kB 00:00 2024-09-21T03:16:06.445 INFO:teuthology.orchestra.run.smithi187.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-21T03:16:06.460 INFO:teuthology.orchestra.run.smithi203.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:16:06.466 INFO:teuthology.orchestra.run.smithi112.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 1.1 MB/s | 45 kB 00:00 2024-09-21T03:16:06.517 INFO:teuthology.orchestra.run.smithi112.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T03:16:06.545 INFO:teuthology.orchestra.run.smithi187.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 8.7 MB/s | 4.9 MB 00:00 2024-09-21T03:16:06.567 INFO:teuthology.orchestra.run.smithi112.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T03:16:06.577 INFO:teuthology.orchestra.run.smithi203.stdout:(31/157): ceph-radosgw-19.3.0-5120.g18432218.el 2.5 MB/s | 10 MB 00:04 2024-09-21T03:16:06.579 INFO:teuthology.orchestra.run.smithi187.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-09-21T03:16:06.604 INFO:teuthology.orchestra.run.smithi187.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T03:16:06.617 INFO:teuthology.orchestra.run.smithi112.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T03:16:06.627 INFO:teuthology.orchestra.run.smithi203.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 5.8 MB/s | 292 kB 00:00 2024-09-21T03:16:06.654 INFO:teuthology.orchestra.run.smithi187.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T03:16:06.667 INFO:teuthology.orchestra.run.smithi112.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T03:16:06.677 INFO:teuthology.orchestra.run.smithi203.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 986 kB/s | 49 kB 00:00 2024-09-21T03:16:06.718 INFO:teuthology.orchestra.run.smithi112.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 2.0 MB/s | 99 kB 00:00 2024-09-21T03:16:06.728 INFO:teuthology.orchestra.run.smithi203.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 330 kB/s | 16 kB 00:00 2024-09-21T03:16:06.729 INFO:teuthology.orchestra.run.smithi187.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 3.2 MB/s | 245 kB 00:00 2024-09-21T03:16:06.768 INFO:teuthology.orchestra.run.smithi112.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T03:16:06.778 INFO:teuthology.orchestra.run.smithi203.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T03:16:06.788 INFO:teuthology.orchestra.run.smithi187.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-21T03:16:06.847 INFO:teuthology.orchestra.run.smithi187.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-09-21T03:16:06.912 INFO:teuthology.orchestra.run.smithi203.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 5.8 MB/s | 784 kB 00:00 2024-09-21T03:16:06.918 INFO:teuthology.orchestra.run.smithi112.stdout:(26/157): rbd-mirror-19.3.0-5120.g18432218.el9. 21 MB/s | 3.1 MB 00:00 2024-09-21T03:16:06.922 INFO:teuthology.orchestra.run.smithi187.stdout:(79/157): postgresql-server-13.16-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-09-21T03:16:06.947 INFO:teuthology.orchestra.run.smithi187.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-09-21T03:16:06.968 INFO:teuthology.orchestra.run.smithi112.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T03:16:06.973 INFO:teuthology.orchestra.run.smithi187.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T03:16:06.978 INFO:teuthology.orchestra.run.smithi203.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 276 kB/s | 18 kB 00:00 2024-09-21T03:16:07.011 INFO:teuthology.orchestra.run.smithi112.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 825 kB/s | 34 kB 00:00 2024-09-21T03:16:07.056 INFO:teuthology.orchestra.run.smithi187.stdout:(82/157): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-21T03:16:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:16:07.082 INFO:teuthology.orchestra.run.smithi187.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 259 kB/s | 35 kB 00:00 2024-09-21T03:16:07.115 INFO:teuthology.orchestra.run.smithi187.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-21T03:16:07.141 INFO:teuthology.orchestra.run.smithi187.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-21T03:16:07.146 INFO:teuthology.orchestra.run.smithi203.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-21T03:16:07.166 INFO:teuthology.orchestra.run.smithi187.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T03:16:07.200 INFO:teuthology.orchestra.run.smithi187.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-09-21T03:16:07.225 INFO:teuthology.orchestra.run.smithi187.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-21T03:16:07.258 INFO:teuthology.orchestra.run.smithi187.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-21T03:16:07.292 INFO:teuthology.orchestra.run.smithi187.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-09-21T03:16:07.305 INFO:teuthology.orchestra.run.smithi203.stdout:(39/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.9 MB/s | 801 kB 00:00 2024-09-21T03:16:07.317 INFO:teuthology.orchestra.run.smithi187.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-09-21T03:16:07.363 INFO:teuthology.orchestra.run.smithi203.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T03:16:07.384 INFO:teuthology.orchestra.run.smithi187.stdout:(92/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T03:16:07.409 INFO:teuthology.orchestra.run.smithi187.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 458 kB/s | 54 kB 00:00 2024-09-21T03:16:07.422 INFO:teuthology.orchestra.run.smithi203.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-21T03:16:07.435 INFO:teuthology.orchestra.run.smithi187.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-09-21T03:16:07.460 INFO:teuthology.orchestra.run.smithi187.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 877 kB/s | 44 kB 00:00 2024-09-21T03:16:07.494 INFO:teuthology.orchestra.run.smithi187.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T03:16:07.519 INFO:teuthology.orchestra.run.smithi187.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-09-21T03:16:07.552 INFO:teuthology.orchestra.run.smithi187.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-09-21T03:16:07.553 INFO:teuthology.orchestra.run.smithi112.stdout:(30/157): ceph-mgr-dashboard-19.3.0-5120.g18432 29 MB/s | 14 MB 00:00 2024-09-21T03:16:07.586 INFO:teuthology.orchestra.run.smithi187.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-21T03:16:07.611 INFO:teuthology.orchestra.run.smithi187.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 990 kB/s | 58 kB 00:00 2024-09-21T03:16:07.630 INFO:teuthology.orchestra.run.smithi203.stdout:(42/157): ceph-mgr-dashboard-19.3.0-5120.g18432 12 MB/s | 14 MB 00:01 2024-09-21T03:16:07.645 INFO:teuthology.orchestra.run.smithi187.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-21T03:16:07.664 INFO:teuthology.orchestra.run.smithi203.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-09-21T03:16:07.670 INFO:teuthology.orchestra.run.smithi187.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-21T03:16:07.704 INFO:teuthology.orchestra.run.smithi187.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-09-21T03:16:07.723 INFO:teuthology.orchestra.run.smithi203.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-21T03:16:07.781 INFO:teuthology.orchestra.run.smithi203.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-21T03:16:07.787 INFO:teuthology.orchestra.run.smithi187.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-21T03:16:07.807 INFO:teuthology.orchestra.run.smithi203.stdout:(46/157): python3-ply-3.11-14.el9.noarch.rpm 606 kB/s | 106 kB 00:00 2024-09-21T03:16:07.838 INFO:teuthology.orchestra.run.smithi187.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T03:16:07.840 INFO:teuthology.orchestra.run.smithi203.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-21T03:16:07.861 INFO:teuthology.orchestra.run.smithi112.stdout:(31/157): ceph-common-19.3.0-5120.g18432218.el9 2.7 MB/s | 20 MB 00:07 2024-09-21T03:16:07.872 INFO:teuthology.orchestra.run.smithi187.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-21T03:16:07.897 INFO:teuthology.orchestra.run.smithi187.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-21T03:16:07.961 INFO:teuthology.orchestra.run.smithi112.stdout:(32/157): ceph-mgr-diskprediction-local-19.3.0- 18 MB/s | 7.4 MB 00:00 2024-09-21T03:16:07.974 INFO:teuthology.orchestra.run.smithi203.stdout:(48/157): ceph-mgr-diskprediction-local-19.3.0- 4.9 MB/s | 7.4 MB 00:01 2024-09-21T03:16:07.986 INFO:teuthology.orchestra.run.smithi112.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.3 MB/s | 292 kB 00:00 2024-09-21T03:16:08.012 INFO:teuthology.orchestra.run.smithi112.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 985 kB/s | 49 kB 00:00 2024-09-21T03:16:08.022 INFO:teuthology.orchestra.run.smithi187.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-21T03:16:08.037 INFO:teuthology.orchestra.run.smithi112.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 328 kB/s | 16 kB 00:00 2024-09-21T03:16:08.047 INFO:teuthology.orchestra.run.smithi187.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T03:16:08.062 INFO:teuthology.orchestra.run.smithi112.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T03:16:08.073 INFO:teuthology.orchestra.run.smithi187.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T03:16:08.104 INFO:teuthology.orchestra.run.smithi112.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 445 kB/s | 18 kB 00:00 2024-09-21T03:16:08.106 INFO:teuthology.orchestra.run.smithi187.stdout:(111/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-09-21T03:16:08.140 INFO:teuthology.orchestra.run.smithi187.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-21T03:16:08.174 INFO:teuthology.orchestra.run.smithi187.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-21T03:16:08.180 INFO:teuthology.orchestra.run.smithi112.stdout:(38/157): librgw2-19.3.0-5120.g18432218.el9.x86 3.0 MB/s | 5.3 MB 00:01 2024-09-21T03:16:08.216 INFO:teuthology.orchestra.run.smithi187.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-21T03:16:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:(39/157): cephadm-19.3.0-5120.g18432218.el9.noa 4.2 MB/s | 784 kB 00:00 2024-09-21T03:16:08.249 INFO:teuthology.orchestra.run.smithi187.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-21T03:16:08.275 INFO:teuthology.orchestra.run.smithi187.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-09-21T03:16:08.300 INFO:teuthology.orchestra.run.smithi187.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-09-21T03:16:08.325 INFO:teuthology.orchestra.run.smithi187.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T03:16:08.338 INFO:teuthology.orchestra.run.smithi112.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 341 kB/s | 80 kB 00:00 2024-09-21T03:16:08.350 INFO:teuthology.orchestra.run.smithi187.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T03:16:08.376 INFO:teuthology.orchestra.run.smithi187.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-21T03:16:08.380 INFO:teuthology.orchestra.run.smithi112.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-21T03:16:08.414 INFO:teuthology.orchestra.run.smithi112.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 995 kB/s | 191 kB 00:00 2024-09-21T03:16:08.439 INFO:teuthology.orchestra.run.smithi112.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-21T03:16:08.483 INFO:teuthology.orchestra.run.smithi203.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 210 kB/s | 107 kB 00:00 2024-09-21T03:16:08.489 INFO:teuthology.orchestra.run.smithi112.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-21T03:16:08.515 INFO:teuthology.orchestra.run.smithi112.stdout:(45/157): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-09-21T03:16:08.527 INFO:teuthology.orchestra.run.smithi187.stdout:(121/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-09-21T03:16:08.540 INFO:teuthology.orchestra.run.smithi112.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-21T03:16:08.568 INFO:teuthology.orchestra.run.smithi187.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.4 MB/s | 303 kB 00:00 2024-09-21T03:16:08.574 INFO:teuthology.orchestra.run.smithi112.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-21T03:16:08.591 INFO:teuthology.orchestra.run.smithi203.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-21T03:16:08.593 INFO:teuthology.orchestra.run.smithi187.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 1.0 MB/s | 220 kB 00:00 2024-09-21T03:16:08.616 INFO:teuthology.orchestra.run.smithi112.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-09-21T03:16:08.619 INFO:teuthology.orchestra.run.smithi187.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.9 MB/s | 144 kB 00:00 2024-09-21T03:16:08.644 INFO:teuthology.orchestra.run.smithi187.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-21T03:16:08.669 INFO:teuthology.orchestra.run.smithi187.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 718 kB/s | 18 kB 00:00 2024-09-21T03:16:08.694 INFO:teuthology.orchestra.run.smithi187.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 464 kB/s | 23 kB 00:00 2024-09-21T03:16:08.720 INFO:teuthology.orchestra.run.smithi187.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-21T03:16:08.745 INFO:teuthology.orchestra.run.smithi187.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T03:16:08.779 INFO:teuthology.orchestra.run.smithi187.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-21T03:16:08.791 INFO:teuthology.orchestra.run.smithi112.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 611 kB/s | 107 kB 00:00 2024-09-21T03:16:08.800 INFO:teuthology.orchestra.run.smithi203.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 694 kB/s | 689 kB 00:00 2024-09-21T03:16:08.804 INFO:teuthology.orchestra.run.smithi187.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T03:16:08.825 INFO:teuthology.orchestra.run.smithi203.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 958 kB/s | 943 kB 00:00 2024-09-21T03:16:08.829 INFO:teuthology.orchestra.run.smithi187.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T03:16:08.858 INFO:teuthology.orchestra.run.smithi112.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-21T03:16:08.871 INFO:teuthology.orchestra.run.smithi187.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-21T03:16:08.883 INFO:teuthology.orchestra.run.smithi112.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 236 kB/s | 22 kB 00:00 2024-09-21T03:16:08.896 INFO:teuthology.orchestra.run.smithi187.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-21T03:16:08.917 INFO:teuthology.orchestra.run.smithi112.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-21T03:16:08.917 INFO:teuthology.orchestra.run.smithi203.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-09-21T03:16:08.922 INFO:teuthology.orchestra.run.smithi187.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 325 kB/s | 16 kB 00:00 2024-09-21T03:16:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 649 kB/s | 33 kB 00:00 2024-09-21T03:16:08.947 INFO:teuthology.orchestra.run.smithi187.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-21T03:16:08.959 INFO:teuthology.orchestra.run.smithi112.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-21T03:16:08.972 INFO:teuthology.orchestra.run.smithi187.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-21T03:16:08.976 INFO:teuthology.orchestra.run.smithi112.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-09-21T03:16:08.998 INFO:teuthology.orchestra.run.smithi187.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-21T03:16:09.001 INFO:teuthology.orchestra.run.smithi203.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 767 kB/s | 313 kB 00:00 2024-09-21T03:16:09.001 INFO:teuthology.orchestra.run.smithi112.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-09-21T03:16:09.023 INFO:teuthology.orchestra.run.smithi187.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:16:09.035 INFO:teuthology.orchestra.run.smithi203.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 155 kB/s | 18 kB 00:00 2024-09-21T03:16:09.057 INFO:teuthology.orchestra.run.smithi187.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-21T03:16:09.060 INFO:teuthology.orchestra.run.smithi112.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-21T03:16:09.082 INFO:teuthology.orchestra.run.smithi187.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-09-21T03:16:09.093 INFO:teuthology.orchestra.run.smithi112.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.4 MB/s | 171 kB 00:00 2024-09-21T03:16:09.110 INFO:teuthology.orchestra.run.smithi203.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-21T03:16:09.119 INFO:teuthology.orchestra.run.smithi112.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-09-21T03:16:09.132 INFO:teuthology.orchestra.run.smithi187.stdout:(142/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-09-21T03:16:09.143 INFO:teuthology.orchestra.run.smithi203.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2024-09-21T03:16:09.158 INFO:teuthology.orchestra.run.smithi187.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 741 kB/s | 74 kB 00:00 2024-09-21T03:16:09.183 INFO:teuthology.orchestra.run.smithi187.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T03:16:09.185 INFO:teuthology.orchestra.run.smithi112.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-21T03:16:09.208 INFO:teuthology.orchestra.run.smithi187.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T03:16:09.233 INFO:teuthology.orchestra.run.smithi187.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-21T03:16:09.244 INFO:teuthology.orchestra.run.smithi112.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-21T03:16:09.259 INFO:teuthology.orchestra.run.smithi187.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T03:16:09.260 INFO:teuthology.orchestra.run.smithi203.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.9 MB/s | 3.0 MB 00:00 2024-09-21T03:16:09.269 INFO:teuthology.orchestra.run.smithi112.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 293 kB/s | 24 kB 00:00 2024-09-21T03:16:09.277 INFO:teuthology.orchestra.run.smithi203.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 338 kB/s | 45 kB 00:00 2024-09-21T03:16:09.295 INFO:teuthology.orchestra.run.smithi112.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-21T03:16:09.302 INFO:teuthology.orchestra.run.smithi203.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 833 kB/s | 160 kB 00:00 2024-09-21T03:16:09.354 INFO:teuthology.orchestra.run.smithi112.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-21T03:16:09.378 INFO:teuthology.orchestra.run.smithi203.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-21T03:16:09.403 INFO:teuthology.orchestra.run.smithi203.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 244 kB/s | 24 kB 00:00 2024-09-21T03:16:09.403 INFO:teuthology.orchestra.run.smithi187.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 1.7 MB/s | 427 kB 00:00 2024-09-21T03:16:09.404 INFO:teuthology.orchestra.run.smithi112.stdout:(65/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-09-21T03:16:09.437 INFO:teuthology.orchestra.run.smithi112.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-09-21T03:16:09.463 INFO:teuthology.orchestra.run.smithi112.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-21T03:16:09.478 INFO:teuthology.orchestra.run.smithi203.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-09-21T03:16:09.529 INFO:teuthology.orchestra.run.smithi203.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-21T03:16:09.543 INFO:teuthology.orchestra.run.smithi187.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-09-21T03:16:09.563 INFO:teuthology.orchestra.run.smithi203.stdout:(65/157): libtool-2.4.6-46.el9.x86_64.rpm 2.0 MB/s | 581 kB 00:00 2024-09-21T03:16:09.595 INFO:teuthology.orchestra.run.smithi203.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T03:16:09.637 INFO:teuthology.orchestra.run.smithi203.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 359 kB/s | 38 kB 00:00 2024-09-21T03:16:09.668 INFO:teuthology.orchestra.run.smithi187.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 8.3 MB/s | 3.4 MB 00:00 2024-09-21T03:16:09.771 INFO:teuthology.orchestra.run.smithi112.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 13 MB/s | 4.9 MB 00:00 2024-09-21T03:16:09.871 INFO:teuthology.orchestra.run.smithi112.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-21T03:16:09.910 INFO:teuthology.orchestra.run.smithi187.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 6.3 MB/s | 3.2 MB 00:00 2024-09-21T03:16:09.930 INFO:teuthology.orchestra.run.smithi112.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T03:16:09.953 INFO:teuthology.orchestra.run.smithi203.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-09-21T03:16:09.968 INFO:teuthology.orchestra.run.smithi187.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 5.6 MB/s | 2.3 MB 00:00 2024-09-21T03:16:10.044 INFO:teuthology.orchestra.run.smithi187.stdout:(153/157): NetworkManager-libnm-1.51.0-1.el9.x8 4.8 MB/s | 1.8 MB 00:00 2024-09-21T03:16:10.053 INFO:teuthology.orchestra.run.smithi203.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-09-21T03:16:10.064 INFO:teuthology.orchestra.run.smithi112.stdout:(71/157): postgresql-13.16-1.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-09-21T03:16:10.077 INFO:teuthology.orchestra.run.smithi187.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.2 MB/s | 249 kB 00:00 2024-09-21T03:16:10.095 INFO:teuthology.orchestra.run.smithi203.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-21T03:16:10.111 INFO:teuthology.orchestra.run.smithi187.stdout:(155/157): NetworkManager-team-1.51.0-1.el9.x86 204 kB/s | 41 kB 00:00 2024-09-21T03:16:10.163 INFO:teuthology.orchestra.run.smithi203.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 8.2 MB/s | 4.9 MB 00:00 2024-09-21T03:16:10.186 INFO:teuthology.orchestra.run.smithi187.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2024-09-21T03:16:10.188 INFO:teuthology.orchestra.run.smithi203.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.0 MB/s | 137 kB 00:00 2024-09-21T03:16:10.288 INFO:teuthology.orchestra.run.smithi203.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-09-21T03:16:10.295 INFO:teuthology.orchestra.run.smithi187.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-09-21T03:16:10.300 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:16:10.300 INFO:teuthology.orchestra.run.smithi187.stdout:Total 20 MB/s | 228 MB 00:11 2024-09-21T03:16:10.314 INFO:teuthology.orchestra.run.smithi203.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 696 kB/s | 87 kB 00:00 2024-09-21T03:16:10.390 INFO:teuthology.orchestra.run.smithi112.stdout:(72/157): postgresql-server-13.16-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-09-21T03:16:10.414 INFO:teuthology.orchestra.run.smithi203.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 612 kB/s | 61 kB 00:00 2024-09-21T03:16:10.531 INFO:teuthology.orchestra.run.smithi112.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T03:16:10.532 INFO:teuthology.orchestra.run.smithi203.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-21T03:16:10.565 INFO:teuthology.orchestra.run.smithi112.stdout:(74/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-21T03:16:10.623 INFO:teuthology.orchestra.run.smithi112.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-21T03:16:10.648 INFO:teuthology.orchestra.run.smithi203.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-21T03:16:10.715 INFO:teuthology.orchestra.run.smithi112.stdout:(76/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.6 MB/s | 1.0 MB 00:00 2024-09-21T03:16:10.741 INFO:teuthology.orchestra.run.smithi112.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-21T03:16:10.748 INFO:teuthology.orchestra.run.smithi203.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-21T03:16:10.791 INFO:teuthology.orchestra.run.smithi112.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-21T03:16:10.849 INFO:teuthology.orchestra.run.smithi203.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-09-21T03:16:10.849 INFO:teuthology.orchestra.run.smithi112.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-09-21T03:16:10.883 INFO:teuthology.orchestra.run.smithi112.stdout:(80/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-09-21T03:16:10.908 INFO:teuthology.orchestra.run.smithi112.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-21T03:16:10.934 INFO:teuthology.orchestra.run.smithi112.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-09-21T03:16:10.965 INFO:teuthology.orchestra.run.smithi203.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-21T03:16:11.017 INFO:teuthology.orchestra.run.smithi112.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-09-21T03:16:11.074 INFO:teuthology.orchestra.run.smithi203.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-09-21T03:16:11.075 INFO:teuthology.orchestra.run.smithi112.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-21T03:16:11.134 INFO:teuthology.orchestra.run.smithi112.stdout:(85/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-09-21T03:16:11.159 INFO:teuthology.orchestra.run.smithi112.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 925 kB/s | 77 kB 00:00 2024-09-21T03:16:11.201 INFO:teuthology.orchestra.run.smithi112.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 631 kB/s | 42 kB 00:00 2024-09-21T03:16:11.227 INFO:teuthology.orchestra.run.smithi112.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-21T03:16:11.285 INFO:teuthology.orchestra.run.smithi112.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-21T03:16:11.344 INFO:teuthology.orchestra.run.smithi112.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-21T03:16:11.369 INFO:teuthology.orchestra.run.smithi112.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 51 kB 00:00 2024-09-21T03:16:11.403 INFO:teuthology.orchestra.run.smithi112.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 916 kB/s | 54 kB 00:00 2024-09-21T03:16:11.470 INFO:teuthology.orchestra.run.smithi112.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 625 kB/s | 42 kB 00:00 2024-09-21T03:16:11.528 INFO:teuthology.orchestra.run.smithi112.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-09-21T03:16:11.532 INFO:teuthology.orchestra.run.smithi203.stdout:(82/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-21T03:16:11.541 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-09-21T03:16:11.641 INFO:teuthology.orchestra.run.smithi203.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-21T03:16:11.679 INFO:teuthology.orchestra.run.smithi112.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-09-21T03:16:11.693 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-09-21T03:16:11.694 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-09-21T03:16:11.737 INFO:teuthology.orchestra.run.smithi112.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 618 kB/s | 36 kB 00:00 2024-09-21T03:16:11.795 INFO:teuthology.orchestra.run.smithi112.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-09-21T03:16:11.971 INFO:teuthology.orchestra.run.smithi112.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.7 MB/s | 304 kB 00:00 2024-09-21T03:16:12.016 INFO:teuthology.orchestra.run.smithi203.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 593 kB/s | 222 kB 00:00 2024-09-21T03:16:12.029 INFO:teuthology.orchestra.run.smithi112.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 992 kB/s | 58 kB 00:00 2024-09-21T03:16:12.130 INFO:teuthology.orchestra.run.smithi112.stdout:(100/157): python3-numpy-1.20.1-5.el9.x86_64.rp 4.2 MB/s | 5.1 MB 00:01 2024-09-21T03:16:12.155 INFO:teuthology.orchestra.run.smithi112.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 765 kB/s | 96 kB 00:00 2024-09-21T03:16:12.189 INFO:teuthology.orchestra.run.smithi112.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-21T03:16:12.208 INFO:teuthology.orchestra.run.smithi203.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 402 kB/s | 77 kB 00:00 2024-09-21T03:16:12.214 INFO:teuthology.orchestra.run.smithi112.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-09-21T03:16:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-09-21T03:16:12.325 INFO:teuthology.orchestra.run.smithi203.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 363 kB/s | 42 kB 00:00 2024-09-21T03:16:12.339 INFO:teuthology.orchestra.run.smithi112.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T03:16:12.381 INFO:teuthology.orchestra.run.smithi112.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 5.0 MB/s | 863 kB 00:00 2024-09-21T03:16:12.415 INFO:teuthology.orchestra.run.smithi112.stdout:(107/157): gperftools-libs-2.9.1-3.el9.x86_64.r 4.0 MB/s | 308 kB 00:00 2024-09-21T03:16:12.432 INFO:teuthology.orchestra.run.smithi112.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-09-21T03:16:12.583 INFO:teuthology.orchestra.run.smithi112.stdout:(109/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-09-21T03:16:12.616 INFO:teuthology.orchestra.run.smithi112.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 141 kB/s | 26 kB 00:00 2024-09-21T03:16:12.650 INFO:teuthology.orchestra.run.smithi112.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 731 kB/s | 49 kB 00:00 2024-09-21T03:16:12.675 INFO:teuthology.orchestra.run.smithi112.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.2 MB/s | 67 kB 00:00 2024-09-21T03:16:12.708 INFO:teuthology.orchestra.run.smithi112.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-21T03:16:12.775 INFO:teuthology.orchestra.run.smithi112.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-21T03:16:12.784 INFO:teuthology.orchestra.run.smithi203.stdout:(87/157): python3-protobuf-3.14.0-13.el9.noarch 587 kB/s | 269 kB 00:00 2024-09-21T03:16:12.801 INFO:teuthology.orchestra.run.smithi112.stdout:(115/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-09-21T03:16:12.826 INFO:teuthology.orchestra.run.smithi112.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 867 kB/s | 43 kB 00:00 2024-09-21T03:16:12.851 INFO:teuthology.orchestra.run.smithi112.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-21T03:16:12.877 INFO:teuthology.orchestra.run.smithi112.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-21T03:16:12.902 INFO:teuthology.orchestra.run.smithi112.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-21T03:16:12.935 INFO:teuthology.orchestra.run.smithi112.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T03:16:12.960 INFO:teuthology.orchestra.run.smithi112.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T03:16:12.986 INFO:teuthology.orchestra.run.smithi112.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T03:16:13.045 INFO:teuthology.orchestra.run.smithi112.stdout:(123/157): libarrow-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-09-21T03:16:13.070 INFO:teuthology.orchestra.run.smithi112.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-21T03:16:13.076 INFO:teuthology.orchestra.run.smithi203.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 544 kB/s | 159 kB 00:00 2024-09-21T03:16:13.095 INFO:teuthology.orchestra.run.smithi112.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-09-21T03:16:13.120 INFO:teuthology.orchestra.run.smithi112.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-21T03:16:13.146 INFO:teuthology.orchestra.run.smithi112.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-21T03:16:13.171 INFO:teuthology.orchestra.run.smithi112.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-21T03:16:13.196 INFO:teuthology.orchestra.run.smithi112.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-09-21T03:16:13.221 INFO:teuthology.orchestra.run.smithi112.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:16:13.255 INFO:teuthology.orchestra.run.smithi112.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T03:16:13.280 INFO:teuthology.orchestra.run.smithi112.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T03:16:13.322 INFO:teuthology.orchestra.run.smithi112.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-21T03:16:13.347 INFO:teuthology.orchestra.run.smithi112.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-21T03:16:13.373 INFO:teuthology.orchestra.run.smithi112.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-09-21T03:16:13.398 INFO:teuthology.orchestra.run.smithi112.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-21T03:16:13.423 INFO:teuthology.orchestra.run.smithi112.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-21T03:16:13.449 INFO:teuthology.orchestra.run.smithi112.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-21T03:16:13.474 INFO:teuthology.orchestra.run.smithi112.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:16:13.515 INFO:teuthology.orchestra.run.smithi112.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-09-21T03:16:13.516 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-09-21T03:16:13.517 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-09-21T03:16:13.535 INFO:teuthology.orchestra.run.smithi203.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 609 kB/s | 279 kB 00:00 2024-09-21T03:16:13.540 INFO:teuthology.orchestra.run.smithi112.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 544 kB/s | 36 kB 00:00 2024-09-21T03:16:13.566 INFO:teuthology.orchestra.run.smithi112.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-21T03:16:13.616 INFO:teuthology.orchestra.run.smithi112.stdout:(143/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-09-21T03:16:13.641 INFO:teuthology.orchestra.run.smithi112.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-09-21T03:16:13.651 INFO:teuthology.orchestra.run.smithi203.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 437 kB/s | 51 kB 00:00 2024-09-21T03:16:13.666 INFO:teuthology.orchestra.run.smithi112.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T03:16:13.691 INFO:teuthology.orchestra.run.smithi112.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 400 kB/s | 20 kB 00:00 2024-09-21T03:16:13.725 INFO:teuthology.orchestra.run.smithi112.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T03:16:13.751 INFO:teuthology.orchestra.run.smithi112.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T03:16:13.768 INFO:teuthology.orchestra.run.smithi203.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 460 kB/s | 54 kB 00:00 2024-09-21T03:16:13.801 INFO:teuthology.orchestra.run.smithi112.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T03:16:14.001 INFO:teuthology.orchestra.run.smithi112.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-09-21T03:16:14.093 INFO:teuthology.orchestra.run.smithi112.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-09-21T03:16:14.152 INFO:teuthology.orchestra.run.smithi112.stdout:(152/157): librbd1-19.3.0-5120.g18432218.el9.x8 7.9 MB/s | 3.2 MB 00:00 2024-09-21T03:16:14.177 INFO:teuthology.orchestra.run.smithi112.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 489 kB/s | 41 kB 00:00 2024-09-21T03:16:14.252 INFO:teuthology.orchestra.run.smithi112.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.4 MB/s | 249 kB 00:00 2024-09-21T03:16:14.294 INFO:teuthology.orchestra.run.smithi112.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-09-21T03:16:14.353 INFO:teuthology.orchestra.run.smithi112.stdout:(156/157): librados2-19.3.0-5120.g18432218.el9. 5.4 MB/s | 3.4 MB 00:00 2024-09-21T03:16:14.519 INFO:teuthology.orchestra.run.smithi112.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-09-21T03:16:14.526 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:16:14.526 INFO:teuthology.orchestra.run.smithi112.stdout:Total 15 MB/s | 228 MB 00:14 2024-09-21T03:16:14.552 INFO:teuthology.orchestra.run.smithi203.stdout:(92/157): python3-babel-2.9.1-2.el9.noarch.rpm 1.4 MB/s | 6.0 MB 00:04 2024-09-21T03:16:14.669 INFO:teuthology.orchestra.run.smithi203.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 357 kB/s | 42 kB 00:00 2024-09-21T03:16:14.786 INFO:teuthology.orchestra.run.smithi203.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 44 kB 00:00 2024-09-21T03:16:15.162 INFO:teuthology.orchestra.run.smithi203.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 589 kB/s | 221 kB 00:00 2024-09-21T03:16:15.337 INFO:teuthology.orchestra.run.smithi203.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 204 kB/s | 36 kB 00:00 2024-09-21T03:16:15.445 INFO:teuthology.orchestra.run.smithi203.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 440 kB/s | 47 kB 00:00 2024-09-21T03:16:15.739 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-21T03:16:15.891 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-21T03:16:15.891 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-21T03:16:15.904 INFO:teuthology.orchestra.run.smithi203.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 663 kB/s | 304 kB 00:00 2024-09-21T03:16:16.013 INFO:teuthology.orchestra.run.smithi203.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-09-21T03:16:16.221 INFO:teuthology.orchestra.run.smithi203.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 459 kB/s | 96 kB 00:00 2024-09-21T03:16:16.422 INFO:teuthology.orchestra.run.smithi203.stdout:(101/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 318 kB/s | 64 kB 00:00 2024-09-21T03:16:16.531 INFO:teuthology.orchestra.run.smithi203.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-09-21T03:16:16.606 INFO:teuthology.orchestra.run.smithi203.stdout:(103/157): postgresql-server-13.16-1.el9.x86_64 908 kB/s | 5.8 MB 00:06 2024-09-21T03:16:16.681 INFO:teuthology.orchestra.run.smithi203.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 148 kB/s | 22 kB 00:00 2024-09-21T03:16:16.740 INFO:teuthology.orchestra.run.smithi203.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T03:16:16.773 INFO:teuthology.orchestra.run.smithi203.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-21T03:16:16.790 INFO:teuthology.orchestra.run.smithi203.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-21T03:16:16.916 INFO:teuthology.orchestra.run.smithi203.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-21T03:16:16.933 INFO:teuthology.orchestra.run.smithi203.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-21T03:16:16.958 INFO:teuthology.orchestra.run.smithi203.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T03:16:16.991 INFO:teuthology.orchestra.run.smithi203.stdout:(111/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-21T03:16:17.017 INFO:teuthology.orchestra.run.smithi203.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-21T03:16:17.067 INFO:teuthology.orchestra.run.smithi203.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 19 MB/s | 839 kB 00:00 2024-09-21T03:16:17.093 INFO:teuthology.orchestra.run.smithi203.stdout:(114/157): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-21T03:16:17.118 INFO:teuthology.orchestra.run.smithi203.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T03:16:17.143 INFO:teuthology.orchestra.run.smithi203.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T03:16:17.169 INFO:teuthology.orchestra.run.smithi203.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-21T03:16:17.202 INFO:teuthology.orchestra.run.smithi203.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-21T03:16:17.228 INFO:teuthology.orchestra.run.smithi203.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T03:16:17.253 INFO:teuthology.orchestra.run.smithi203.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.9 MB/s | 358 kB 00:00 2024-09-21T03:16:17.278 INFO:teuthology.orchestra.run.smithi203.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T03:16:17.303 INFO:teuthology.orchestra.run.smithi203.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T03:16:17.329 INFO:teuthology.orchestra.run.smithi203.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-21T03:16:17.354 INFO:teuthology.orchestra.run.smithi203.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T03:16:17.379 INFO:teuthology.orchestra.run.smithi203.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-21T03:16:17.405 INFO:teuthology.orchestra.run.smithi203.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 918 kB/s | 23 kB 00:00 2024-09-21T03:16:17.430 INFO:teuthology.orchestra.run.smithi203.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-21T03:16:17.455 INFO:teuthology.orchestra.run.smithi203.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-21T03:16:17.514 INFO:teuthology.orchestra.run.smithi203.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.6 MB/s | 2.0 MB 00:00 2024-09-21T03:16:17.539 INFO:teuthology.orchestra.run.smithi203.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:16:17.564 INFO:teuthology.orchestra.run.smithi203.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T03:16:17.581 INFO:teuthology.orchestra.run.smithi203.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-21T03:16:17.624 INFO:teuthology.orchestra.run.smithi203.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-21T03:16:17.641 INFO:teuthology.orchestra.run.smithi203.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 997 kB/s | 16 kB 00:00 2024-09-21T03:16:17.666 INFO:teuthology.orchestra.run.smithi203.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-21T03:16:17.691 INFO:teuthology.orchestra.run.smithi203.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-21T03:16:17.716 INFO:teuthology.orchestra.run.smithi203.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-21T03:16:17.733 INFO:teuthology.orchestra.run.smithi203.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 733 kB/s | 31 kB 00:00 2024-09-21T03:16:17.758 INFO:teuthology.orchestra.run.smithi203.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:16:17.784 INFO:teuthology.orchestra.run.smithi203.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-09-21T03:16:17.809 INFO:teuthology.orchestra.run.smithi203.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-21T03:16:17.835 INFO:teuthology.orchestra.run.smithi203.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T03:16:17.860 INFO:teuthology.orchestra.run.smithi203.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T03:16:17.885 INFO:teuthology.orchestra.run.smithi203.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T03:16:17.902 INFO:teuthology.orchestra.run.smithi203.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-21T03:16:17.936 INFO:teuthology.orchestra.run.smithi203.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-21T03:16:17.961 INFO:teuthology.orchestra.run.smithi203.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T03:16:18.011 INFO:teuthology.orchestra.run.smithi203.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-21T03:16:18.098 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-21T03:16:18.098 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-21T03:16:18.270 INFO:teuthology.orchestra.run.smithi203.stdout:(149/157): librbd1-19.3.0-5120.g18432218.el9.x8 12 MB/s | 3.2 MB 00:00 2024-09-21T03:16:18.495 INFO:teuthology.orchestra.run.smithi203.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 6.3 MB/s | 3.4 MB 00:00 2024-09-21T03:16:18.704 INFO:teuthology.orchestra.run.smithi203.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 5.4 MB/s | 2.3 MB 00:00 2024-09-21T03:16:18.754 INFO:teuthology.orchestra.run.smithi203.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 819 kB/s | 41 kB 00:00 2024-09-21T03:16:18.805 INFO:teuthology.orchestra.run.smithi203.stdout:(153/157): NetworkManager-libnm-1.51.0-1.el9.x8 5.8 MB/s | 1.8 MB 00:00 2024-09-21T03:16:18.922 INFO:teuthology.orchestra.run.smithi203.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 1.5 MB/s | 249 kB 00:00 2024-09-21T03:16:19.530 INFO:teuthology.orchestra.run.smithi203.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 1.9 MB/s | 1.4 MB 00:00 2024-09-21T03:16:19.837 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-09-21T03:16:19.948 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:16:20.096 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:16:20.122 INFO:teuthology.orchestra.run.smithi203.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-09-21T03:16:20.312 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:16:20.358 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:16:20.383 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:16:20.435 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:16:20.689 INFO:teuthology.orchestra.run.smithi203.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.3 MB/s | 16 MB 00:06 2024-09-21T03:16:20.694 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:16:20.694 INFO:teuthology.orchestra.run.smithi203.stdout:Total 11 MB/s | 228 MB 00:20 2024-09-21T03:16:20.774 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:16:20.892 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:16:21.002 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:16:21.026 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:16:21.118 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:16:21.147 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:16:21.176 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:16:21.332 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:16:21.409 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:16:21.431 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:16:21.520 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:16:21.549 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:16:21.595 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:16:21.617 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:16:21.641 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:16:21.826 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:16:21.878 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-09-21T03:16:21.975 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:16:22.029 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-09-21T03:16:22.029 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-09-21T03:16:22.102 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:16:22.201 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:16:22.286 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:16:22.510 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:16:22.621 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:16:22.657 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:16:22.692 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:16:22.765 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:16:22.818 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:16:22.858 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:16:22.902 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:16:22.934 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:16:22.962 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:16:23.031 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:16:23.085 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:16:23.115 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:16:23.167 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:16:23.256 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:16:23.303 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:16:23.347 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:16:23.376 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:16:23.446 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:16:23.490 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:16:23.564 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:16:23.696 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:16:23.751 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:16:23.847 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-09-21T03:16:23.848 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-09-21T03:16:24.276 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:16:24.363 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:16:24.393 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:16:24.444 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:16:24.490 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:16:24.532 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:16:24.655 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-21T03:16:24.704 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:16:24.760 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:16:24.790 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:16:24.832 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:16:24.852 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:16:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:16:24.924 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:16:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:16:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:16:24.998 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:16:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:16:25.037 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:16:25.081 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:16:25.128 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:16:25.285 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:16:25.375 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:16:25.509 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:16:25.532 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:16:25.600 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:16:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:16:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:16:25.745 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:16:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:16:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:16:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:16:25.876 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:16:25.917 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:16:25.998 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:16:26.019 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:16:26.051 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:16:26.053 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:16:26.081 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:16:26.090 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:16:26.117 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:16:26.141 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:16:26.178 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:16:26.208 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:16:26.215 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:16:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:16:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:16:26.338 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:16:26.341 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:16:26.390 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:16:26.418 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:16:26.452 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:16:26.477 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:16:26.491 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:16:26.523 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:16:26.553 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:16:26.596 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:16:26.638 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:16:26.669 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:16:26.679 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:16:26.714 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:16:26.798 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:16:26.799 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:16:26.843 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:16:26.875 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:16:26.922 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:16:26.983 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:16:26.985 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:16:27.042 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:16:27.215 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:16:27.326 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:16:27.362 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:16:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:16:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:16:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:16:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:16:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:16:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:16:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:16:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:16:27.754 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:16:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:16:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:16:27.806 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:16:27.840 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:16:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:16:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:16:27.939 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:16:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:16:27.995 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:16:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:16:28.039 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:16:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:16:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:16:28.118 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:16:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:16:28.168 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:16:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:16:28.350 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:16:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:16:28.382 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:16:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:16:28.467 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:16:28.930 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:16:29.033 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:16:29.062 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:16:29.105 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:16:29.105 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:16:29.143 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:16:29.193 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:16:29.213 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:16:29.350 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:16:29.412 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:16:29.505 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:16:29.569 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:16:29.640 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:16:29.672 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:16:29.708 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:16:29.755 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:16:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:16:29.834 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:16:29.884 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:16:30.020 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:16:30.061 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:16:30.102 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:16:30.204 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:16:30.240 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:16:30.287 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:16:30.313 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:16:30.338 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:16:30.433 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:16:30.512 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:16:30.529 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-09-21T03:16:30.540 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:16:30.573 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:16:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:16:30.678 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:16:30.756 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:16:30.790 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:16:30.809 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:16:30.828 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:16:30.844 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:16:30.865 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:16:30.869 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:16:30.899 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:16:30.899 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:16:30.912 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:16:30.938 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:16:30.953 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:16:30.969 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:16:30.999 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:16:31.025 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:16:31.053 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:16:31.058 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:16:31.083 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:16:31.100 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:16:31.119 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:16:31.134 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:16:31.221 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:16:31.252 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:16:31.273 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:16:31.304 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:16:31.379 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:16:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:16:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:16:31.489 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:16:31.514 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:16:31.605 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:16:31.634 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:16:31.634 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:16:31.664 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:16:31.672 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:16:31.735 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:16:31.757 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:16:31.815 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:16:31.866 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:16:31.894 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:16:32.000 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:16:32.029 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:16:32.091 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:16:32.113 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:16:32.138 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:16:32.179 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:16:32.258 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:16:32.293 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:16:32.332 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:16:32.391 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:16:32.456 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:16:32.495 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:16:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:16:32.580 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:16:32.612 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:16:32.620 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:16:32.728 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:16:32.796 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:16:32.797 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:16:32.827 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:16:32.910 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:16:33.020 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:16:33.132 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:16:33.169 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:16:33.204 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:16:33.300 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:16:33.346 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:16:33.378 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:16:33.422 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:16:33.454 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:16:33.482 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:16:33.551 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:16:33.557 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:16:33.613 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:16:33.644 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:16:33.666 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:16:33.679 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:16:33.770 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:16:33.807 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:16:33.850 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:16:33.879 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:16:33.947 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:16:33.993 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:16:34.016 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:16:34.055 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:16:34.066 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:16:34.196 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:16:34.250 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:16:34.622 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:16:34.666 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:16:34.769 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:16:34.769 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:16:34.769 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:16:34.769 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:16:34.776 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:16:34.800 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:16:34.854 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:16:34.884 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:16:34.908 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:16:34.927 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:16:34.973 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:16:35.005 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:16:35.014 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:16:35.033 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:16:35.066 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:16:35.180 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:16:35.243 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:16:35.334 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:16:35.394 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:16:35.399 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:16:35.437 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:16:35.471 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:16:35.504 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:16:35.512 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:16:35.540 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:16:35.541 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:16:35.569 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:16:35.586 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:16:35.623 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:16:35.666 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:16:35.708 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:16:35.844 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:16:36.067 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:16:36.068 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:16:36.098 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:16:36.118 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:16:36.169 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:16:36.191 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:16:36.222 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:16:36.261 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:16:36.309 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:16:36.335 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:16:36.368 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:16:36.443 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:16:36.569 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:16:36.601 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:16:36.628 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:16:36.662 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:16:36.710 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:16:36.742 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:16:36.772 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:16:36.916 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:16:36.949 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:16:36.991 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:16:37.025 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:16:37.112 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:16:37.164 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:16:37.195 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:16:37.297 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:16:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:16:38.075 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:16:38.127 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:16:38.161 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:16:38.261 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:16:38.316 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:16:38.361 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:16:38.450 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:16:38.485 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:16:38.489 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:16:38.521 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:16:38.673 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:16:38.703 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:16:38.781 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:16:39.098 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:16:39.140 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:16:39.234 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:16:39.234 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:16:39.234 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:16:39.234 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:16:39.275 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:16:39.422 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:16:39.536 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:16:40.761 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:16:40.851 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:16:40.896 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:16:40.929 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:16:41.254 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:16:41.301 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:16:41.392 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:16:41.422 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:16:41.458 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:16:41.966 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:16:42.016 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:16:42.050 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:16:42.072 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:16:44.278 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:16:44.311 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:16:44.877 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:16:44.923 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:16:45.005 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:16:45.005 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:16:45.005 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:16:45.005 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:16:45.025 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:16:46.417 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:16:46.417 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2024-09-21T03:16:46.417 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2024-09-21T03:16:46.417 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2024-09-21T03:16:46.417 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2024-09-21T03:16:46.417 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2024-09-21T03:16:46.417 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /root 2024-09-21T03:16:46.417 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2024-09-21T03:16:46.418 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:16:46.603 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:16:46.619 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:16:46.619 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:46.619 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:16:46.619 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:16:48.230 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:16:48.257 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:16:48.257 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:48.257 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:16:48.257 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:16:48.257 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:16:48.257 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:16:48.489 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:16:48.513 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:16:48.513 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:48.513 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:16:48.513 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:16:48.513 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:16:48.513 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:16:48.999 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:16:49.022 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:16:49.023 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:49.023 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:16:49.023 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:16:49.023 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:16:49.023 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:16:49.137 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:16:49.194 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:16:49.249 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:16:49.436 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:16:49.482 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:16:49.561 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:16:49.677 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:16:49.824 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:16:49.858 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:16:49.889 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:16:49.925 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:16:50.344 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:16:50.374 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:16:50.406 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:16:50.472 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:16:50.497 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:16:51.167 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:16:51.167 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-09-21T03:16:51.167 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-09-21T03:16:51.168 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-09-21T03:16:51.168 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-09-21T03:16:51.168 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-09-21T03:16:51.168 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-09-21T03:16:51.168 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-09-21T03:16:51.168 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:16:51.362 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:16:51.377 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:16:51.377 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:51.378 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:16:51.378 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:16:52.981 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:16:53.008 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:16:53.008 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:53.008 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:16:53.008 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:16:53.008 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:16:53.008 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:16:53.232 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:16:53.255 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:16:53.255 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:53.255 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:16:53.255 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:16:53.255 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:16:53.255 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:16:53.714 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:16:53.724 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:16:53.747 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:16:53.748 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:53.748 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:16:53.748 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:16:53.748 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:16:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:16:53.769 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:16:53.846 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:16:53.894 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:16:53.949 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:16:54.133 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:16:54.183 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:16:54.261 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:16:54.348 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:16:54.489 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:16:54.533 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:16:54.565 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:16:54.592 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:16:54.738 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:16:54.769 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:16:54.885 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:16:55.053 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:16:55.054 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:16:55.083 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:16:55.088 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:16:55.114 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:16:55.114 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:55.114 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:16:55.114 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:16:55.114 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:16:55.114 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:16:55.115 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:16:55.164 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:16:55.176 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:16:55.182 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:16:55.205 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:16:55.275 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:16:55.329 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:16:56.297 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:16:56.333 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:16:56.357 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:16:56.357 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:56.357 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:16:56.357 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:16:56.357 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:16:56.357 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:16:56.409 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:16:56.432 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:16:56.432 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:56.432 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:16:56.432 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:16:56.484 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:16:56.507 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:16:56.507 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:56.508 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:16:56.508 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:16:56.508 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:16:56.508 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:16:56.826 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:16:56.850 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:16:56.850 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:56.851 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:16:56.851 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:16:56.851 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:16:56.851 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:16:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:16:56.993 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2024-09-21T03:16:56.993 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2024-09-21T03:16:56.993 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2024-09-21T03:16:56.993 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2024-09-21T03:16:56.993 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2024-09-21T03:16:56.994 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2024-09-21T03:16:56.994 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2024-09-21T03:16:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:16:57.156 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:16:57.172 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:16:57.172 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:57.172 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:16:57.172 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:16:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:16:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:16:58.751 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:16:58.778 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:16:58.778 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:58.778 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:16:58.778 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:16:58.778 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:16:58.778 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:16:59.011 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:16:59.034 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:16:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:16:59.385 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:16:59.420 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:16:59.530 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:16:59.533 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:16:59.553 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:16:59.553 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:59.553 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:16:59.553 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:16:59.553 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:16:59.553 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:16:59.668 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:16:59.703 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:16:59.724 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:16:59.739 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:16:59.763 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:16:59.763 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:16:59.763 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:16:59.763 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:16:59.763 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:16:59.763 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:16:59.781 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:16:59.814 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:16:59.826 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:16:59.925 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:16:59.965 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:16:59.989 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:17:00.014 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:17:00.093 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:17:00.181 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:17:00.330 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:17:00.364 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:17:00.396 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:17:00.433 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:17:00.834 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:17:00.839 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:17:00.865 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:17:00.881 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:17:00.897 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:17:00.921 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:17:00.953 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:17:00.957 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:17:00.964 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:17:00.980 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:17:00.983 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:17:00.988 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:17:01.008 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:17:01.008 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:17:01.008 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:17:01.008 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:17:01.008 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:17:01.008 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:17:01.051 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:17:01.074 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:17:01.074 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:17:01.074 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:17:01.074 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:17:01.126 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:17:01.149 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:17:01.149 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:17:01.149 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:17:01.149 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:17:01.150 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:17:01.150 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:17:01.168 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:17:01.210 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:17:01.240 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:17:01.278 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:17:01.333 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:17:01.468 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:17:01.492 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:17:01.492 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:17:01.492 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:17:01.492 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:17:01.492 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:17:01.492 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:17:01.952 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:17:01.997 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:17:02.019 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:17:02.874 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:17:02.960 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:17:03.001 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:17:03.092 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:17:03.138 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:17:03.163 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:17:03.189 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:17:03.189 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:17:03.282 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:17:03.282 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:17:03.375 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:17:03.399 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:17:03.405 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:17:03.405 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:17:03.593 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:17:03.593 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:17:03.685 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:17:03.749 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:17:04.182 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:17:04.229 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:17:05.190 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:17:05.221 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:17:05.335 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:17:05.489 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:17:05.505 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:17:05.531 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:17:05.539 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:17:05.555 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:17:05.555 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:17:05.555 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:17:05.555 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:17:05.556 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:17:05.556 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:17:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:17:05.607 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:17:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:17:05.619 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:17:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:17:05.718 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:17:05.764 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:17:05.836 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:17:05.885 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:17:05.915 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:17:05.945 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:17:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:17:06.577 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:17:06.614 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:17:06.633 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:17:06.725 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:17:06.760 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:17:06.783 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:17:06.783 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:17:06.783 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:17:06.784 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:17:06.784 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:17:06.784 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:17:06.827 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:17:06.850 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:17:06.850 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:17:06.850 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:17:06.850 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:17:06.895 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:17:06.918 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:17:06.918 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:17:06.919 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:17:06.919 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:17:06.919 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:17:06.919 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:17:07.237 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:17:07.261 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:17:07.261 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:17:07.261 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:17:07.261 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:17:07.261 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:17:07.261 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:17:07.472 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:17:07.560 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:17:07.609 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:17:07.700 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:17:07.730 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:17:07.755 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:17:07.782 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:17:07.782 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:17:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:17:07.866 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:17:07.949 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:17:07.974 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:17:07.980 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:17:07.980 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:17:08.158 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:17:08.159 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:17:08.226 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:17:08.292 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:17:10.223 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:17:10.223 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:17:10.223 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:17:10.223 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:17:10.223 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:17:10.223 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:17:10.224 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:17:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:17:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:17:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:17:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:17:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:17:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:17:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:17:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:17:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:17:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:17:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:17:10.236 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:17:10.237 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:17:10.238 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:17:10.239 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:17:10.240 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:17:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:17:10.242 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:17:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:17:10.244 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:17:10.245 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:17:10.246 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:17:10.246 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:17:10.246 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:17:10.246 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:17:10.246 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:17:10.246 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:17:11.234 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:17:11.277 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:17:11.316 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:17:11.356 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:17:11.383 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:17:11.566 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:17:11.631 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:17:11.661 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:17:11.691 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:17:11.729 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:17:12.318 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:17:12.353 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:17:12.373 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:17:12.852 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:17:12.853 INFO:teuthology.orchestra.run.smithi187.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:12.854 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.855 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.856 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:17:12.857 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:17:12.858 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:17:12.859 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:17:12.860 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:17:12.861 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:17:12.862 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:12.863 INFO:teuthology.orchestra.run.smithi187.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:17:12.864 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:17:12.864 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:17:12.864 INFO:teuthology.orchestra.run.smithi187.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:17:12.864 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:17:12.864 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:17:12.864 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:17:12.864 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:17:12.864 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-09-21T03:17:13.207 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:17:13.293 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:17:13.306 DEBUG:teuthology.parallel:result is None 2024-09-21T03:17:13.334 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:17:13.414 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:17:13.446 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:17:13.471 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:17:13.498 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:17:13.498 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:17:13.565 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:17:13.565 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:17:13.657 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:17:13.682 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:17:13.687 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:17:13.687 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:17:13.876 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:17:13.876 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:17:13.960 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:17:14.031 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:17:14.778 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:17:14.779 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:17:14.780 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:17:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:17:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:17:14.781 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:17:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:17:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:17:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:17:14.792 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:17:14.793 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:17:14.794 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:17:14.795 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:17:14.796 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:17:14.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:17:14.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:17:14.799 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:17:14.800 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:17:14.801 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:17:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:17:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:17:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:17:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:17:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:17:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:17:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:17:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:17:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:17:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:17:14.802 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:17:17.449 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:17:17.450 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:17:17.450 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-09-21T03:17:17.450 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:17.450 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:17.450 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:17.450 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:17.450 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.450 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.450 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.451 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.452 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.453 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.454 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:17:17.454 INFO:teuthology.orchestra.run.smithi112.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:17:17.454 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:17:17.454 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:17:17.454 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:17:17.454 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:17:17.454 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.454 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.454 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:17:17.454 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:17:17.455 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.456 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:17:17.457 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:17:17.458 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:17:17.459 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:17:17.460 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:17:17.461 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:17:17.462 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:17:17.462 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:17:17.462 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-21T03:17:17.897 DEBUG:teuthology.parallel:result is None 2024-09-21T03:17:20.576 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:17:20.576 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:17:20.576 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:17:20.576 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:17:20.576 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:17:20.576 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:17:20.576 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:17:20.577 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:17:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:17:20.579 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:17:20.582 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:17:20.583 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:17:20.584 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:17:20.585 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:17:20.586 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:17:20.587 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:17:20.588 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:17:20.588 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:17:20.588 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:17:20.588 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:17:20.588 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:17:20.588 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:17:20.588 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:17:20.588 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:17:20.588 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:17:20.588 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:17:20.588 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:17:23.362 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:17:23.362 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:17:23.362 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2024-09-21T03:17:23.362 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:23.362 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:23.362 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:23.362 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:17:23.362 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.363 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.364 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:17:23.365 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.366 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:17:23.367 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:17:23.368 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:17:23.369 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:17:23.370 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:17:23.371 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:17:23.372 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:17:23.373 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:17:23.374 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:17:23.374 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:17:23.374 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:17:23.374 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:17:23.374 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:17:23.374 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:17:23.374 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-09-21T03:17:23.786 DEBUG:teuthology.parallel:result is None 2024-09-21T03:17:23.787 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:17:23.929 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:17:23.956 INFO:teuthology.orchestra.run.smithi112.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:17:23.957 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:17:23.957 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:17:23.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:17:24.096 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:17:24.123 INFO:teuthology.orchestra.run.smithi187.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:17:24.123 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:17:24.123 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:17:24.125 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:17:24.266 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:17:24.290 INFO:teuthology.orchestra.run.smithi203.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:17:24.290 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:17:24.290 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:17:24.292 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T03:17:24.293 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:17:24.293 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:17:24.325 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T03:17:24.326 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:17:24.359 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T03:17:24.359 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:17:24.387 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T03:17:24.388 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:17:24.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:17:24.417 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:17:24.485 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T03:17:24.485 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:17:24.515 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:17:24.584 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T03:17:24.585 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:17:24.612 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:17:24.678 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T03:17:24.679 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:17:24.679 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:17:24.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:17:24.772 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T03:17:24.773 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:17:24.803 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:17:24.873 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T03:17:24.873 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:17:24.899 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:17:24.965 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T03:17:24.965 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:17:24.965 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:17:24.992 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:17:25.058 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T03:17:25.058 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:17:25.088 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:17:25.156 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T03:17:25.156 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:17:25.183 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:17:25.250 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T03:17:25.347 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T03:17:25.347 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:17:25.348 INFO:tasks.cephadm:Cluster fsid is 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:17:25.348 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T03:17:25.348 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.112', 'mon.b': '172.21.15.187', 'mon.c': '172.21.15.203'} 2024-09-21T03:17:25.348 INFO:tasks.cephadm:First mon is mon.a on smithi112 2024-09-21T03:17:25.348 INFO:tasks.cephadm:First mgr is x 2024-09-21T03:17:25.349 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T03:17:25.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-09-21T03:17:25.374 DEBUG:teuthology.orchestra.run.smithi187:> sudo hostname $(hostname -s) 2024-09-21T03:17:25.400 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname $(hostname -s) 2024-09-21T03:17:25.426 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T03:17:25.426 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:17:25.574 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T03:17:25.719 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:17:25.721 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:17:25.721 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:17:25.721 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:17:26.247 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:17 /home/ubuntu/cephtest/cephadm 2024-09-21T03:17:26.248 DEBUG:teuthology.orchestra.run.smithi187:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:17:26.769 INFO:teuthology.orchestra.run.smithi187.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:17 /home/ubuntu/cephtest/cephadm 2024-09-21T03:17:26.769 DEBUG:teuthology.orchestra.run.smithi203:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:17:27.290 INFO:teuthology.orchestra.run.smithi203.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:17 /home/ubuntu/cephtest/cephadm 2024-09-21T03:17:27.291 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:17:27.310 DEBUG:teuthology.orchestra.run.smithi187:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:17:27.330 DEBUG:teuthology.orchestra.run.smithi203:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:17:27.369 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T03:17:27.370 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:17:27.374 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:17:27.377 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:17:28.822 INFO:teuthology.orchestra.run.smithi187.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:17:28.835 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:17:28.912 INFO:teuthology.orchestra.run.smithi203.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:18:01.628 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-21T03:18:01.628 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:18:01.628 INFO:teuthology.orchestra.run.smithi203.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:18:01.628 INFO:teuthology.orchestra.run.smithi203.stdout: "repo_digests": [ 2024-09-21T03:18:01.628 INFO:teuthology.orchestra.run.smithi203.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:18:01.629 INFO:teuthology.orchestra.run.smithi203.stdout: ] 2024-09-21T03:18:01.629 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-21T03:18:22.838 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-09-21T03:18:22.838 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:18:22.838 INFO:teuthology.orchestra.run.smithi187.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:18:22.838 INFO:teuthology.orchestra.run.smithi187.stdout: "repo_digests": [ 2024-09-21T03:18:22.838 INFO:teuthology.orchestra.run.smithi187.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:18:22.838 INFO:teuthology.orchestra.run.smithi187.stdout: ] 2024-09-21T03:18:22.838 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-09-21T03:18:23.434 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:18:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:18:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:18:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-09-21T03:18:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:18:23.435 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:18:23.435 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:18:23.459 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-09-21T03:18:23.521 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph 2024-09-21T03:18:23.551 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph 2024-09-21T03:18:23.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-09-21T03:18:23.610 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /etc/ceph 2024-09-21T03:18:23.637 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /etc/ceph 2024-09-21T03:18:23.664 INFO:tasks.cephadm:Writing seed config... 2024-09-21T03:18:23.665 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T03:18:23.666 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-21T03:18:23.667 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-21T03:18:23.667 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T03:18:23.667 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T03:18:23.667 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-21T03:18:23.667 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T03:18:23.667 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T03:18:23.667 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T03:18:23.667 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T03:18:23.667 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T03:18:23.667 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T03:18:23.667 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-21T03:18:23.668 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:18:23.668 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T03:18:23.683 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 063b32f6-77c8-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-21T03:18:23.683 DEBUG:teuthology.orchestra.run.smithi112:mon.a> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T03:18:23.726 DEBUG:teuthology.orchestra.run.smithi112:mgr.x> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T03:18:23.768 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T03:18:23.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.112 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:18:23.970 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:18:23.970 INFO:teuthology.orchestra.run.smithi112.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.112', '--skip-admin-label'] 2024-09-21T03:18:23.971 INFO:teuthology.orchestra.run.smithi112.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T03:18:23.971 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying podman|docker is present... 2024-09-21T03:18:23.996 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:18:23.996 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying lvm2 is present... 2024-09-21T03:18:23.996 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying time synchronization is in place... 2024-09-21T03:18:24.004 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:18:24.004 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:18:24.010 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:18:24.010 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-21T03:18:24.017 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-09-21T03:18:24.023 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-09-21T03:18:24.023 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:18:24.023 INFO:teuthology.orchestra.run.smithi112.stdout:Repeating the final host check... 2024-09-21T03:18:24.045 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:18:24.045 INFO:teuthology.orchestra.run.smithi112.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T03:18:24.045 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl is present 2024-09-21T03:18:24.045 INFO:teuthology.orchestra.run.smithi112.stdout:lvcreate is present 2024-09-21T03:18:24.052 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:18:24.052 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:18:24.058 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:18:24.058 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-21T03:18:24.064 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-09-21T03:18:24.071 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-09-21T03:18:24.071 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:18:24.071 INFO:teuthology.orchestra.run.smithi112.stdout:Host looks OK 2024-09-21T03:18:24.071 INFO:teuthology.orchestra.run.smithi112.stdout:Cluster fsid: 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:18:24.071 INFO:teuthology.orchestra.run.smithi112.stdout:Acquiring lock 140331965122928 on /run/cephadm/063b32f6-77c8-11ef-baf5-efdc52797490.lock 2024-09-21T03:18:24.071 INFO:teuthology.orchestra.run.smithi112.stdout:Lock 140331965122928 acquired on /run/cephadm/063b32f6-77c8-11ef-baf5-efdc52797490.lock 2024-09-21T03:18:24.072 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 3300 ... 2024-09-21T03:18:24.072 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 6789 ... 2024-09-21T03:18:24.072 INFO:teuthology.orchestra.run.smithi112.stdout:Base mon IP(s) is [172.21.15.112:3300, 172.21.15.112:6789], mon addrv is [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-09-21T03:18:24.075 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.112 metric 100 2024-09-21T03:18:24.075 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.112 metric 100 2024-09-21T03:18:24.078 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T03:18:24.078 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-21T03:18:24.078 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-21T03:18:24.080 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T03:18:24.080 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T03:18:24.080 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:18:24.080 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-21T03:18:24.080 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T03:18:24.081 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:18:24.081 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-09-21T03:18:24.081 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-09-21T03:18:24.081 INFO:teuthology.orchestra.run.smithi112.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T03:18:24.082 INFO:teuthology.orchestra.run.smithi112.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T03:18:24.082 INFO:teuthology.orchestra.run.smithi112.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:18:24.570 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:18:24.571 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:18:24.571 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T03:18:24.571 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T03:18:24.571 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T03:18:24.571 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:18:24.571 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T03:18:25.517 INFO:teuthology.orchestra.run.smithi112.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:18:25.517 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:18:25.517 INFO:teuthology.orchestra.run.smithi112.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T03:18:26.515 INFO:teuthology.orchestra.run.smithi112.stdout:stat: stdout 167 167 2024-09-21T03:18:26.515 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial keys... 2024-09-21T03:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQACO+5m6URPNBAAsdSMAv9PxIQYBECGEdZNIg== 2024-09-21T03:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQADO+5ms2cFLRAAWKnTZG5NH7wXg1rZvCfDeg== 2024-09-21T03:18:29.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQAEO+5mvRL7IxAATBIB7SkQpIPaCOCN0meWhg== 2024-09-21T03:18:29.183 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial monmap... 2024-09-21T03:18:30.121 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:18:30.121 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T03:18:30.121 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:18:30.121 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:18:30.121 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool for a [v2:172.21.15.112:3300,v1:172.21.15.112:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:18:30.121 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = quincy 2024-09-21T03:18:30.122 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: set fsid to 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:18:30.122 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:18:30.122 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:18:30.122 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mon... 2024-09-21T03:18:39.548 INFO:teuthology.orchestra.run.smithi112.stdout:create mon.a on 2024-09-21T03:18:40.117 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T03:18:40.542 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T03:18:40.969 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-063b32f6-77c8-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-063b32f6-77c8-11ef-baf5-efdc52797490.target. 2024-09-21T03:18:40.970 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-063b32f6-77c8-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-063b32f6-77c8-11ef-baf5-efdc52797490.target. 2024-09-21T03:18:41.199 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:18:41.200 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T03:18:41.357 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-063b32f6-77c8-11ef-baf5-efdc52797490.target.wants/ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-063b32f6-77c8-11ef-baf5-efdc52797490@.service. 2024-09-21T03:18:41.879 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:41 smithi112 podman[31059]: 2024-09-21 03:18:41.822318689 +0000 UTC m=+0.327742468 container start 24cac1b20a00995285e6198213f637ce4ce83476a02ba4e97229469adce56a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T03:18:41.879 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:41 smithi112 ceph-mon[31078]: mkfs 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:18:41.925 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:18:41.925 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-09-21T03:18:41.932 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:18:41.932 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-21T03:18:41.932 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-09-21T03:18:41.932 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:18:41.933 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon to start... 2024-09-21T03:18:41.933 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon... 2024-09-21T03:18:42.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:41 smithi112 ceph-mon[31078]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:18:42.157 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:41 smithi112 bash[31059]: 24cac1b20a00995285e6198213f637ce4ce83476a02ba4e97229469adce56a84 2024-09-21T03:18:42.157 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:41 smithi112 systemd[1]: Started Ceph mon.a for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:18:43.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:18:43.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: monmap epoch 1 2024-09-21T03:18:43.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:18:43.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: last_changed 2024-09-21T03:18:29.513931+0000 2024-09-21T03:18:43.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: created 2024-09-21T03:18:29.513931+0000 2024-09-21T03:18:43.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: min_mon_release 19 (squid) 2024-09-21T03:18:43.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: election_strategy: 1 2024-09-21T03:18:43.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-09-21T03:18:43.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: fsmap 2024-09-21T03:18:43.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:18:43.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: mgrmap e1: no daemons active 2024-09-21T03:18:43.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:42 smithi112 ceph-mon[31078]: from='client.? 172.21.15.112:0/2229888903' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T03:18:43.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T03:18:43.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout id: 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:18:43.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T03:18:43.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:18:43.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout services: 2024-09-21T03:18:43.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.545156s) 2024-09-21T03:18:43.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T03:18:43.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T03:18:43.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:18:43.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout data: 2024-09-21T03:18:43.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T03:18:43.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T03:18:43.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T03:18:43.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T03:18:43.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:18:43.015 INFO:teuthology.orchestra.run.smithi112.stdout:mon is available 2024-09-21T03:18:43.015 INFO:teuthology.orchestra.run.smithi112.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T03:18:44.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:43 smithi112 ceph-mon[31078]: from='client.? 172.21.15.112:0/1460396345' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:18:44.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:43 smithi112 ceph-mon[31078]: from='client.? 172.21.15.112:0/1460396345' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T03:18:44.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:18:44.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:18:44.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:18:44.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:18:44.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:18:44.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:18:44.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:18:44.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:18:44.098 INFO:teuthology.orchestra.run.smithi112.stdout:Generating new minimal ceph.conf... 2024-09-21T03:18:45.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:44 smithi112 ceph-mon[31078]: from='client.? 172.21.15.112:0/3834668040' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:18:45.101 INFO:teuthology.orchestra.run.smithi112.stdout:Restarting the monitor... 2024-09-21T03:18:45.377 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:45 smithi112 systemd[1]: Stopping Ceph mon.a for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:18:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:45 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a[31074]: 2024-09-21T03:18:45.374+0000 7f6c189cd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:18:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:45 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a[31074]: 2024-09-21T03:18:45.374+0000 7f6c189cd640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T03:18:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:45 smithi112 podman[31318]: 2024-09-21 03:18:45.569457681 +0000 UTC m=+0.301031885 container died 24cac1b20a00995285e6198213f637ce4ce83476a02ba4e97229469adce56a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T03:18:46.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:45 smithi112 podman[31318]: 2024-09-21 03:18:45.933367835 +0000 UTC m=+0.664942026 container cleanup 24cac1b20a00995285e6198213f637ce4ce83476a02ba4e97229469adce56a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T03:18:46.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:45 smithi112 bash[31318]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:18:48.235 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:47 smithi112 podman[31330]: 2024-09-21 03:18:47.905621246 +0000 UTC m=+2.339509892 container remove 24cac1b20a00995285e6198213f637ce4ce83476a02ba4e97229469adce56a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T03:18:48.235 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:48 smithi112 systemd[1]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T03:18:48.235 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:48 smithi112 systemd[1]: Stopped Ceph mon.a for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:18:48.235 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:48 smithi112 systemd[1]: Starting Ceph mon.a for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:18:48.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:48 smithi112 podman[31423]: 2024-09-21 03:18:48.136864295 +0000 UTC m=+0.016386037 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:18:49.404 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 podman[31423]: 2024-09-21 03:18:49.135723094 +0000 UTC m=+1.015244872 container create aaae2dc90557dd4784f23ccfde80e59ce857bda592cbf99a86f9d71ac2780e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T03:18:49.596 INFO:teuthology.orchestra.run.smithi112.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T03:18:49.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 podman[31423]: 2024-09-21 03:18:49.507261288 +0000 UTC m=+1.386783018 container init aaae2dc90557dd4784f23ccfde80e59ce857bda592cbf99a86f9d71ac2780e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T03:18:49.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 podman[31423]: 2024-09-21 03:18:49.51070774 +0000 UTC m=+1.390229473 container start aaae2dc90557dd4784f23ccfde80e59ce857bda592cbf99a86f9d71ac2780e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:18:49.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: pidfile_write: ignore empty --pid-file 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: load: jerasure load: lrc 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Git sha 0 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: DB SUMMARY 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: DB Session ID: M2UVZ0LLF8BF2HKFMP2J 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T03:18:49.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T03:18:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94630 ; 2024-09-21T03:18:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:18:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:18:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:18:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:18:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:18:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:18:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.env: 0x564132c7fd60 2024-09-21T03:18:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:18:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.info_log: 0x564137577360 2024-09-21T03:18:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.statistics: (nil) 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.use_fsync: 0 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:18:49.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.db_log_dir: 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.wal_dir: 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.write_buffer_manager: 0x56413757bd60 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:18:49.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.unordered_write: 0 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.row_cache: None 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.wal_filter: None 2024-09-21T03:18:49.725 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.wal_compression: 0 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:18:49.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:18:49.727 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_open_files: -1 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Compression algorithms supported: 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: kZSTD supported: 0 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:18:49.728 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.merge_operator: 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_filter: None 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564137576fa0) 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:18:49.729 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_type: 0 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_index_type: 0 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_shortening: 1 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: checksum: 4 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: no_block_cache: 0 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache: 0x56413353b610 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_options: 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: capacity : 536870912 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_shard_bits : 4 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: strict_capacity_limit : 0 2024-09-21T03:18:49.730 INFO:journalctl@ceph.mon.a.smithi112.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_compressed: (nil) 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: persistent_cache: (nil) 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size: 4096 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size_deviation: 10 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_restart_interval: 16 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_block_restart_interval: 1 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: metadata_block_size: 4096 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: partition_filters: 0 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: use_delta_encoding: 1 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: filter_policy: bloomfilter 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: whole_key_filtering: 1 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: verify_compression: 0 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:18:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout: format_version: 5 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_index_compression: 1 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_align: 0 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout: max_auto_readahead_size: 262144 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout: prepopulate_block_cache: 0 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compression: NoCompression 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.num_levels: 7 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:18:49.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:18:49.733 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:18:49.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:18:49.735 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:18:49.736 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.ttl: 2592000 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:18:49.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 54bfa4fe-1abf-458c-83a3-2a4af0bfe162 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888729549891, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888729550998, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90856, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 283, "table_properties": {"data_size": 88897, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11567, "raw_average_key_size": 47, "raw_value_size": 82360, "raw_average_value_size": 340, "num_data_blocks": 12, "num_entries": 242, "num_filter_entries": 242, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726888729, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "54bfa4fe-1abf-458c-83a3-2a4af0bfe162", "db_session_id": "M2UVZ0LLF8BF2HKFMP2J", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888729551084, "job": 1, "event": "recovery_finished"} 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T03:18:49.738 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56413759aa80 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: DB pointer 0x564138340000 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** DB Stats ** 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-09-21T03:18:49.739 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: L0 2/0 90.59 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: Sum 2/0 90.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 127.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-09-21T03:18:49.740 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative compaction: 0.00 GB write, 25.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval compaction: 0.00 GB write, 25.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache BinnedLRUCache@0x56413353b610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: starting mon.a rank 0 at public addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] at bind addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:18:49.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: mon.a@-1(???) e1 preinit fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: mon.a@-1(???).mds e1 new map 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: mon.a@-1(???).mds e1 print_map 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout: e1 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout: btime 2024-09-21T03:18:41:875043+0000 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout: legacy client fscid: -1 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout: No filesystems configured 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:18:49.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 bash[31423]: aaae2dc90557dd4784f23ccfde80e59ce857bda592cbf99a86f9d71ac2780e2a 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 systemd[1]: Started Ceph mon.a for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: monmap epoch 1 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: last_changed 2024-09-21T03:18:29.513931+0000 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: created 2024-09-21T03:18:29.513931+0000 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: min_mon_release 19 (squid) 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: election_strategy: 1 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: fsmap 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:18:49.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:49 smithi112 ceph-mon[31442]: mgrmap e1: no daemons active 2024-09-21T03:18:50.743 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T03:18:50.773 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:18:50.773 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mgr... 2024-09-21T03:18:50.774 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T03:18:50.774 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T03:18:51.022 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:18:51.022 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T03:18:51.170 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-063b32f6-77c8-11ef-baf5-efdc52797490.target.wants/ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-063b32f6-77c8-11ef-baf5-efdc52797490@.service. 2024-09-21T03:18:51.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:51 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3478020079' entity='client.admin' 2024-09-21T03:18:51.287 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:18:51 smithi112 systemd[1]: Starting Ceph mgr.x for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:18:51.712 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:18:51 smithi112 podman[31653]: 2024-09-21 03:18:51.301244981 +0000 UTC m=+0.016115117 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:18:51.712 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:18:51 smithi112 podman[31653]: 2024-09-21 03:18:51.399794657 +0000 UTC m=+0.114664781 container create 0d4f99a98dc822d3cc18a024cf8b396cd6624acded96bbbcc533320e99e75179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:18:51.712 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:18:51 smithi112 podman[31653]: 2024-09-21 03:18:51.62734836 +0000 UTC m=+0.342218481 container init 0d4f99a98dc822d3cc18a024cf8b396cd6624acded96bbbcc533320e99e75179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T03:18:51.712 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:18:51 smithi112 podman[31653]: 2024-09-21 03:18:51.630052949 +0000 UTC m=+0.344923071 container start 0d4f99a98dc822d3cc18a024cf8b396cd6624acded96bbbcc533320e99e75179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:18:51.712 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:18:51 smithi112 bash[31653]: 0d4f99a98dc822d3cc18a024cf8b396cd6624acded96bbbcc533320e99e75179 2024-09-21T03:18:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:18:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-09-21T03:18:51.737 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:18:51.737 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-21T03:18:51.737 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-09-21T03:18:51.737 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:18:51.747 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:18:51.748 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-09-21T03:18:51.756 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:18:51.756 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-21T03:18:51.756 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-09-21T03:18:51.756 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T03:18:51.757 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr to start... 2024-09-21T03:18:51.757 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr... 2024-09-21T03:18:51.980 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:18:51 smithi112 systemd[1]: Started Ceph mgr.x for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:18:52.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:52 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3364344862' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:18:52.846 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:18:52.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:18:52.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "063b32f6-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:18:52.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:18:52.849 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:52.850 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:18:52.850 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:18:52.850 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:52.850 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:18:52.850 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:52.850 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:18:52.850 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:18:52.850 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:18:52.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:18:41:875043+0000", 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:18:52.856 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:18:41.875878+0000", 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:18:52.857 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (1/15)... 2024-09-21T03:18:55.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:55 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1343095092' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:18:55.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:18:55.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:18:55.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "063b32f6-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:18:55.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:18:55.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:18:55.966 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:18:55.967 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:18:55.968 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:18:41:875043+0000", 2024-09-21T03:18:55.969 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:18:55.970 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:55.971 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:18:41.875878+0000", 2024-09-21T03:18:55.971 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:18:55.971 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:55.971 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:18:55.971 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:18:55.971 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (2/15)... 2024-09-21T03:18:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: Activating manager daemon x 2024-09-21T03:18:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: mgrmap e2: x(active, starting, since 0.00385402s) 2024-09-21T03:18:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: from='mgr.14100 172.21.15.112:0/472170922' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:18:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: from='mgr.14100 172.21.15.112:0/472170922' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:18:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: from='mgr.14100 172.21.15.112:0/472170922' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:18:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: from='mgr.14100 172.21.15.112:0/472170922' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:18:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: from='mgr.14100 172.21.15.112:0/472170922' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:18:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: Manager daemon x is now available 2024-09-21T03:18:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: from='mgr.14100 172.21.15.112:0/472170922' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:18:57.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: from='mgr.14100 172.21.15.112:0/472170922' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:18:57.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: from='mgr.14100 172.21.15.112:0/472170922' entity='mgr.x' 2024-09-21T03:18:57.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: from='mgr.14100 172.21.15.112:0/472170922' entity='mgr.x' 2024-09-21T03:18:57.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:56 smithi112 ceph-mon[31442]: from='mgr.14100 172.21.15.112:0/472170922' entity='mgr.x' 2024-09-21T03:18:59.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:58 smithi112 ceph-mon[31442]: mgrmap e3: x(active, since 1.00733s) 2024-09-21T03:18:59.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:58 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1651513476' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:18:59.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:18:59.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:18:59.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "063b32f6-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:18:59.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:18:59.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:18:59.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:18:59.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:59.246 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:18:59.247 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:59.247 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:18:59.247 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:18:59.247 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:18:59.247 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:18:59.247 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:18:59.247 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:18:59.250 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:59.250 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:18:59.250 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:18:59.250 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:18:59.250 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:18:59.250 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:18:59.250 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:18:41:875043+0000", 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:18:59.251 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:18:41.875878+0000", 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:18:59.252 INFO:teuthology.orchestra.run.smithi112.stdout:mgr is available 2024-09-21T03:19:00.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:18:59 smithi112 ceph-mon[31442]: mgrmap e4: x(active, since 2s) 2024-09-21T03:19:00.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:19:00.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:19:00.426 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T03:19:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:19:00.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:19:00.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:19:00.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:19:00.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:19:00.428 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling cephadm module... 2024-09-21T03:19:01.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:00 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3683477610' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:19:02.019 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:01 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1130147555' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T03:19:02.020 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:01 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: ignoring --setuser ceph since I am not root 2024-09-21T03:19:02.020 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:01 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: ignoring --setgroup ceph since I am not root 2024-09-21T03:19:02.977 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:02 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1130147555' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T03:19:02.977 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:02 smithi112 ceph-mon[31442]: mgrmap e5: x(active, since 5s) 2024-09-21T03:19:03.623 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:19:03.625 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T03:19:03.625 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:19:03.626 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:19:03.626 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:19:03.626 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:19:03.626 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-09-21T03:19:03.626 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 5... 2024-09-21T03:19:03.904 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:03 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1782291408' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:19:07.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:06 smithi112 ceph-mon[31442]: Active manager daemon x restarted 2024-09-21T03:19:07.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:06 smithi112 ceph-mon[31442]: Activating manager daemon x 2024-09-21T03:19:07.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:06 smithi112 ceph-mon[31442]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T03:19:07.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:06 smithi112 ceph-mon[31442]: mgrmap e6: x(active, starting, since 0.00361018s) 2024-09-21T03:19:07.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:06 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:19:07.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:06 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:19:07.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:06 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:19:07.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:06 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:19:07.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:06 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:19:07.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:06 smithi112 ceph-mon[31442]: Manager daemon x is now available 2024-09-21T03:19:08.333 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:08.333 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:08.333 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: Found migration_current of "None". Setting to last migration. 2024-09-21T03:19:08.333 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:08.333 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:08.333 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:08.333 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:08.333 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:19:08.333 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:19:08.333 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T03:19:08.333 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: mgrmap e7: x(active, since 1.00609s) 2024-09-21T03:19:08.334 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T03:19:08.334 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:08.334 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:19:08.050+0000 7fd6e9238640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:19:08] ENGINE Error in HTTPServer.serve 2024-09-21T03:19:08.334 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: Traceback (most recent call last): 2024-09-21T03:19:08.334 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:19:08.334 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self._connections.run(self.expiration_interval) 2024-09-21T03:19:08.334 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:19:08.334 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self._run(expiration_interval) 2024-09-21T03:19:08.334 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: s = self.context.wrap_socket( 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: return self.sslsocket_class._create( 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self.do_handshake() 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self._sslobj.do_handshake() 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:19:08.335 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:19:08.336 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:19:08.336 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T03:19:08.336 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:19:08.336 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:19:08.337 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 5 is available 2024-09-21T03:19:08.337 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying orchestrator module is enabled... 2024-09-21T03:19:09.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:09 smithi112 ceph-mon[31442]: [21/Sep/2024:03:19:07] ENGINE Bus STARTING 2024-09-21T03:19:09.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:09 smithi112 ceph-mon[31442]: [21/Sep/2024:03:19:07] ENGINE Serving on http://172.21.15.112:8765 2024-09-21T03:19:09.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:09 smithi112 ceph-mon[31442]: [21/Sep/2024:03:19:08] ENGINE Serving on https://172.21.15.112:7150 2024-09-21T03:19:09.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:09 smithi112 ceph-mon[31442]: [21/Sep/2024:03:19:08] ENGINE Bus STARTED 2024-09-21T03:19:09.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:09 smithi112 ceph-mon[31442]: [21/Sep/2024:03:19:08] ENGINE Error in HTTPServer.serve 2024-09-21T03:19:09.532 INFO:journalctl@ceph.mon.a.smithi112.stdout: Traceback (most recent call last): 2024-09-21T03:19:09.532 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:19:09.532 INFO:journalctl@ceph.mon.a.smithi112.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:19:09.532 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:19:09.532 INFO:journalctl@ceph.mon.a.smithi112.stdout: self._run(expiration_interval) 2024-09-21T03:19:09.532 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:19:09.532 INFO:journalctl@ceph.mon.a.smithi112.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout: s = self.context.wrap_socket( 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout: return self.sslsocket_class._create( 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout: self.do_handshake() 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout: self._sslobj.do_handshake() 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:19:09.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:09 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2291939084' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T03:19:09.810 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T03:19:09.810 INFO:teuthology.orchestra.run.smithi112.stdout:Setting orchestrator backend to cephadm... 2024-09-21T03:19:10.457 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:10 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2291939084' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T03:19:10.457 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:10 smithi112 ceph-mon[31442]: mgrmap e8: x(active, since 2s) 2024-09-21T03:19:11.759 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:11 smithi112 ceph-mon[31442]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:11.759 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:11 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:11.759 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:11 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:12.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T03:19:12.340 INFO:teuthology.orchestra.run.smithi112.stdout:Generating ssh key... 2024-09-21T03:19:12.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:12 smithi112 ceph-mon[31442]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:13.444 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: Generating public/private rsa key pair. 2024-09-21T03:19:13.444 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: Your identification has been saved in /tmp/tmp8i2uk7f2/key 2024-09-21T03:19:13.444 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: Your public key has been saved in /tmp/tmp8i2uk7f2/key.pub 2024-09-21T03:19:13.444 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: The key fingerprint is: 2024-09-21T03:19:13.444 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: SHA256:9JBzuaMoEYlNYCbliQVWH3QI5hkx1jzdkcu3D6/xncM ceph-063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:19:13.444 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: The key's randomart image is: 2024-09-21T03:19:13.444 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: +---[RSA 3072]----+ 2024-09-21T03:19:13.444 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: |o+BXBoo...o | 2024-09-21T03:19:13.444 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: |.B+==*o. + . | 2024-09-21T03:19:13.444 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: |. +o+.. * + | 2024-09-21T03:19:13.444 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: | . . B o | 2024-09-21T03:19:13.445 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: | . S = . | 2024-09-21T03:19:13.445 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: | . . . + | 2024-09-21T03:19:13.445 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: | . . . .+ . | 2024-09-21T03:19:13.445 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: | . oo.E.| 2024-09-21T03:19:13.445 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: | ... o.| 2024-09-21T03:19:13.445 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: +----[SHA256]-----+ 2024-09-21T03:19:14.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:14 smithi112 ceph-mon[31442]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:14 smithi112 ceph-mon[31442]: Generating ssh key... 2024-09-21T03:19:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:14 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:14 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:14.954 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEjmEAN6WfGSDFHSVNJXBhmK7ztSe/TVe3GjSt0agQr2LY/6ql3nDhMbPOlujSI4KAJkSIeyRjFE6ax0fSYOBbpL8hSUKZTn/3HNoUA41efA96pSG5SzHlsK/EU7uIgV5FKV+EBOloLTvGQreWukIfiR8fG3xgHq5OHLWSaLbTIoEjiyrvOF4FhN1IMxQ8BikDU1u8M4gCLKdhVo9bki94JmNbMBUaiFLvX+pSzSJDOQBWhARCOMIbuuffDsRpwvmmnmtU0JeqprYK0JcEztb+OMWBhmzXXLOkFCZw5O0oMkqXXsWSaR8RRkI3En2rHHg7MJBb+p44bJZQITiYHkSg/graFCaGIbddfWzvrQL6dcHZ01hECDAdI8gEdB6GeIkNIbd2APPs1iXo1w23e7XCcF76AEmgD6n6HQuoIJlbL9w93tDc2+mzCU7oUBbOkpHrFv/HM7bwXSyhfutiYJipdq01S7UhSNtFNsAgf3AoL5PSgEOOt5nQ2fo7fQXUpqc= ceph-063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:19:14.954 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:19:14.954 INFO:teuthology.orchestra.run.smithi112.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T03:19:14.959 INFO:teuthology.orchestra.run.smithi112.stdout:Adding host smithi112... 2024-09-21T03:19:15.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:15 smithi112 ceph-mon[31442]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:16.317 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:16 smithi112 ceph-mon[31442]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:17.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:17 smithi112 ceph-mon[31442]: Deploying cephadm binary to smithi112 2024-09-21T03:19:19.380 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Added host 'smithi112' with addr '172.21.15.112' 2024-09-21T03:19:19.380 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mon service... 2024-09-21T03:19:19.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:19 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:19.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:19 smithi112 ceph-mon[31442]: Added host smithi112 2024-09-21T03:19:19.652 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:19 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:21.074 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T03:19:21.074 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mgr service... 2024-09-21T03:19:21.196 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:21 smithi112 ceph-mon[31442]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:21.196 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:21 smithi112 ceph-mon[31442]: Saving service mon spec with placement count:5 2024-09-21T03:19:21.196 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:21 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:22.475 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T03:19:22.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:22 smithi112 ceph-mon[31442]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:22.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:22 smithi112 ceph-mon[31442]: Saving service mgr spec with placement count:2 2024-09-21T03:19:22.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:22 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:22.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:22 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:22.724 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:22 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:24.140 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:24 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2524932189' entity='client.admin' 2024-09-21T03:19:25.408 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling the dashboard module... 2024-09-21T03:19:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:25 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:25 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2743823235' entity='client.admin' 2024-09-21T03:19:27.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:26 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3769474067' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T03:19:27.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:26 smithi112 ceph-mon[31442]: from='mgr.14118 172.21.15.112:0/33044799' entity='mgr.x' 2024-09-21T03:19:27.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:26 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: ignoring --setuser ceph since I am not root 2024-09-21T03:19:27.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:26 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: ignoring --setgroup ceph since I am not root 2024-09-21T03:19:27.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:27 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3769474067' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T03:19:27.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:27 smithi112 ceph-mon[31442]: mgrmap e9: x(active, since 19s) 2024-09-21T03:19:29.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:28 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2076526302' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:19:29.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:19:29.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T03:19:29.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:19:29.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:19:29.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:19:29.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:19:29.429 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-09-21T03:19:29.429 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 9... 2024-09-21T03:19:32.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:31 smithi112 ceph-mon[31442]: Active manager daemon x restarted 2024-09-21T03:19:32.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:31 smithi112 ceph-mon[31442]: Activating manager daemon x 2024-09-21T03:19:32.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:31 smithi112 ceph-mon[31442]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T03:19:32.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:31 smithi112 ceph-mon[31442]: mgrmap e10: x(active, starting, since 0.00393182s) 2024-09-21T03:19:32.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:31 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:19:32.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:31 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:19:32.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:31 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:19:32.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:31 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:19:32.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:31 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:19:32.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:31 smithi112 ceph-mon[31442]: Manager daemon x is now available 2024-09-21T03:19:33.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:32 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:33.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:32 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:19:33.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:32 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:19:33.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:32 smithi112 ceph-mon[31442]: mgrmap e11: x(active, since 1.00759s) 2024-09-21T03:19:33.225 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:19:33.025+0000 7fd3ebfa0640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:19:33] ENGINE Error in HTTPServer.serve 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: Traceback (most recent call last): 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self._connections.run(self.expiration_interval) 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self._run(expiration_interval) 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: s = self.context.wrap_socket( 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:19:33.226 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: return self.sslsocket_class._create( 2024-09-21T03:19:33.227 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:19:33.227 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self.do_handshake() 2024-09-21T03:19:33.227 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:19:33.227 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self._sslobj.do_handshake() 2024-09-21T03:19:33.227 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:19:33.227 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:19:33.529 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:19:33.529 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T03:19:33.529 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:19:33.529 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:19:33.529 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 9 is available 2024-09-21T03:19:33.529 INFO:teuthology.orchestra.run.smithi112.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T03:19:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-mon[31442]: [21/Sep/2024:03:19:32] ENGINE Bus STARTING 2024-09-21T03:19:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-mon[31442]: [21/Sep/2024:03:19:33] ENGINE Serving on https://172.21.15.112:7150 2024-09-21T03:19:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-mon[31442]: [21/Sep/2024:03:19:33] ENGINE Error in HTTPServer.serve 2024-09-21T03:19:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout: Traceback (most recent call last): 2024-09-21T03:19:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:19:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:19:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:19:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout: self._run(expiration_interval) 2024-09-21T03:19:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:19:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:19:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout: s = self.context.wrap_socket( 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout: return self.sslsocket_class._create( 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout: self.do_handshake() 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout: self._sslobj.do_handshake() 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-mon[31442]: [21/Sep/2024:03:19:33] ENGINE Serving on http://172.21.15.112:8765 2024-09-21T03:19:34.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:33 smithi112 ceph-mon[31442]: [21/Sep/2024:03:19:33] ENGINE Bus STARTED 2024-09-21T03:19:35.117 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:34 smithi112 ceph-mon[31442]: mgrmap e12: x(active, since 2s) 2024-09-21T03:19:35.117 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:34 smithi112 ceph-mon[31442]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:35.117 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:34 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE/DCCAuSgAwIBAgIUTAbpuxUol1oG0ZEJ9V8H8lVUR3MwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAzMTkzNFoXDTM0\nMDkyMjAzMTkzNFowGDEWMBQGA1UEAwwNMTcyLjIxLjE1LjExMjCCAiIwDQYJKoZI\nhvcNAQEBBQADggIPADCCAgoCggIBALVCi8wSzIJ5rMR/Qzb2lYeDFkcl3X5i3v+I\nTr5lH7tjEn+1L6/ZKVN1JFrP+3Mx7ldrAKF1jFT7xPpA0BkuprR1VD4OLY5BMcUX\nrSVP4Lkl9cyfCbz+Z4uiVoCsCtJfpryYWR05EcSRJ4N8tmKTorJMKXViMtUEtQGL\nGjHCHp29gXWyx1R8epgeegF6m5CjJCQ6Jj6oZKFLBp6uKIsP6CcpjygFDH/9afMo\nQzPrU+6dl2VJP5EhM1UNQwCDHTHkcnL2TnWAzsJT0zQz7wL+PAC/8+5OdMv9Jvun\nHV7IeDwAkcJMX3NPyxmTD/42i62TiH1ad4iQazlgqNVuTJdCw7JQkxfTHTnQksmj\n8D3RxybfTyd75BVkn9ZGyKN/GqvslygTAnvV3MymPGC+hPSkzGD7P2peUAxuwtrr\nhUwMWdLIuEIv0lTN3eYeLJr2yf0TwecvmcsOrlGbigJh9y+2HvEg20Q7oPZX3bHu\nYr4CFOzLx/Dnmvj0Gsj7EhY4irP2a1/cMnNO1+3T8akSL6h1WqGMKB6WxbpwUzLr\n5IhsSPKSU7QtX8LxR9KVFhLwtt2erR1ImEU7PnUiXJx96P5Gim1O5Kql6umDOQEz\nKRDmlJtpPwInLsOVv8ZuyAwoEG4j726bkfCc4nzv45ONjnFoF4ZGPGvTsb0m7pkV\nB8nY63jNAgMBAAGjPzA9MC0GA1UdEQQmMCSCCXNtaXRoaTExMoIRZGFzaGJvYXJk\nX3NlcnZlcnOHBKwVD3AwDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAgEA\njZKMKzeas6rjkK5OKBlm4D6T1Bt6/hKP0+hgaZmeZhXBk89BcSW+XEHdkl12tqIk\nmjanMzcBCiYneylQ0zwGL+YDFKQm62VdbcXT9qaBNNYpdlioxQ+Ouf1Qm1StGBWc\nzqJJc8Nd1RsHSeGkb9YVbOzOqsy1LwRvpR6GWV/a5KihjBQPbOCyyQZedyaN+RlO\nwHwv/EFLTP/8ohR16kUsPDoHqrizkoDRKFTmAhY38DL+vv+otyLzBL9V9x7mMERE\nYM0/8tLd87qDcbC6aOXYRL5DDwEMIF+KWUshUUnwqUz5ALoMLbeY4QYLo4p25nE5\nqnV8C5s2xIRo6nmrsunCijnxrj6MluOS6ueZ4/iS429TsCReRTlLFpcOHVQBtWTP\n+y4unKehi4RdVO/uClECW+Eg/L4hyR1DPfRXzn1XmMe6UaJyACuz47D+T3at7xPs\ncKt8yu95wn1AC0i1Fvzr7wypUo/9Z+ntTyyIV6+/87hRp6XV/7kqIi6gW6MmqQJ/\nz6E2BifDTMETS37VP+wpVgY7nsizcGXAT9mFYCa2fsINCzHOERjg96ON/yDD99we\nQK3leIOhR4P06dj8LsUrX9+ZEf3vU2HgvPcJt6+iFmJQ3icQpquWYSIcDxY4RaMD\n04dcIoyyBJ0dZtBBPXQtICFbVAodIV3WhXFcsGp+JOI=\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEAtUKLzBLMgnmsxH9DNvaVh4MWRyXdfmLe/4hOvmUfu2MSf7Uv\nr9kpU3UkWs/7czHuV2sAoXWMVPvE+kDQGS6mtHVUPg4tjkExxRetJU/guSX1zJ8J\nvP5ni6JWgKwK0l+mvJhZHTkRxJEng3y2YpOiskwpdWIy1QS1AYsaMcIenb2BdbLH\nVHx6mB56AXqbkKMkJDomPqhkoUsGnq4oiw/oJymPKAUMf/1p8yhDM+tT7p2XZUk/\nkSEzVQ1DAIMdMeRycvZOdYDOwlPTNDPvAv48AL/z7k50y/0m+6cdXsh4PACRwkxf\nc0/LGZMP/jaLrZOIfVp3iJBrOWCo1W5Ml0LDslCTF9MdOdCSyaPwPdHHJt9PJ3vk\nFWSf1kbIo38aq+yXKBMCe9XczKY8YL6E9KTMYPs/al5QDG7C2uuFTAxZ0si4Qi/S\nVM3d5h4smvbJ/RPB5y+Zyw6uUZuKAmH3L7Ye8SDbRDug9lfdse5ivgIU7MvH8Oea\n+PQayPsSFjiKs/ZrX9wyc07X7dPxqRIvqHVaoYwoHpbFunBTMuvkiGxI8pJTtC1f\nwvFH0pUWEvC23Z6tHUiYRTs+dSJcnH3o/kaKbU7kqqXq6YM5ATMpEOaUm2k/Aicu\nw5W/xm7IDCgQbiPvbpuR8JzifO/jk42OcWgXhkY8a9OxvSbumRUHydjreM0CAwEA\nAQKCAgBPPITyM/GUw+3jhCJ033xkXciaV5Io0r3bBNIvZvNlLh0DGMh83bi0FAxN\naofoRpaDPnJrIoCzEdB8+WI+7+bOElA+RGJjwlooZ344Rl/dcBuwVIGkTa4kG1Yn\nL44XgtrXrN2f6be7rxyP3S+5jgJcYPDqsH+6ITGjOBS+VbaejB4QvpNmenf5EbGR\n/fANXVPKRmU7lM9QtwRwz39Hsy9wFznOa86YEMaFFj+ybZgmQlvRWiDgp5vR/Vch\n3S8hBVWGreeEdEduKci1FFLSgyLlzQOsVD24grTM47PyID53NGSDhHFhbitBxckC\n/wM2YL0U1zxTxZYp4fhMPMhcg0IabRqGVJJ5Q1aGSFI5rkjg9E0lNULdv/rEoKT0\nKPNJXXMxq277SK7f22djyAGsSY2pxNYzT1EzUglVeH4S/9gDcB6YSs+S0zHu6LmR\n6owzi6xsWW6KGKiPA/TP2uUOI7aniJz/7gzyIdwisilBlr0fVf5ftthwkhknNSrZ\nleEdAjcLHOPnKQ2ozpkpUO3zr5ixWlOf4ra464RMjtZyXI/UGDKBsEH06j3GRtPm\n+j4J1XWutl75eyJPlD7WsJC7KUALvnyAC7adji0S8X7MZ9qWqDy7fgr9Qn1VJ0UX\n5idTqXh0ND8xc3vsUL8QXp0nwnonF4dF/y0gQ0C6vqjqT5YsHQKCAQEA/wWPMbUy\nVfZxPgwSwqXh6s7i64fSbJgX/mgAmJxwJ6kmCZgATAk24igPl0oFBH/9vGjxYbpy\nHDeL0FVmca0ZXshYz4fczShCzpfF3zpJmZs5cNjRJ5zaZ3XjvIDEGcKHXPwp/IFO\nRN63DOS7W5TcPrJPvLKy30RwQCHDCR+hVdmI/ra8d5VCjkug/CFkFndxMV4MF/wN\nayBNUi4tR7AFzG0xigFof3qvJ51BK6rw83mOwff1edlHERvnp1i9aP1EwTkisnRg\nOMRffECTBo9LrghxxnBZUBmvxtxBb80oMxL/VrRFwRoz0MM+90/D1n81FqR1MoWH\nwyST9AhXgrRRKwKCAQEAtfSMyy7FPXBEyRLNxjxzIiTXe1DpwX2lG2hbymUoLzlq\nZETLTUI2raFttVTLHyRDaXGT7qYlZ7UEOevR7a29jMS0IiTe1BGYPhcApM65WVtx\nq+Q2yfn+jrKNU7gf4ejheY6fVcSXPbKQxnP+4Sp8hUb5rt5zHNro99ON14LaDVgO\n0kTNvA/H/4e0Vu+Mp+gs7FNwgVHy/Cmu91K2OZFeSt4oCnILzuocw965bW53lpdJ\neFKpSmQTEUswSCIK26Vv1oVcnFxH2zV9njG4kXCdverM3a3T42LXYGf4Z+/50/rP\nZLffN0WlnASKCxoMDLHL+Ym97+5jHpmLLewFRYcx5wKCAQEA4jnzShw8N+ezk2Aw\ncD2pSJwYzumg8DaH6mYmSTtVfsldddtCQDFYX408L3A5PUGxPCc4IgK8YW2g+MIZ\na7Qrslo19nvGkrQpd41JufnB+qKpdQLRn7tUP71jtf/gdh2L6R/EuzuVzCBp7+Uk\nk1xdeLzDYViAtvDGRw3sLzbA7yBXl+VRER/2d7Z9a8zPNeLlJe8Q+WHeIW75SHKw\nNkzfB/bBoHQZWCTkJlcoze0JbGMkEcu+FkZ59aJ0gI6HEBDEokfmELqN9S22qsm5\n43LQpuTU0T0u7ZCrztPqtJwhEhXDLw4iZuewmXvqaod26dvUo25mjNfJlIHpQez4\nfHkrGwKCAQAs1e7fnk3B/ukkNvr3ryubrUJfw6dmI2TM5SFBNT36rQY3YPBLwFYi\n4NVcymEm8lOeJvuR4eTzshoxf5eCHr7Hci0FANrbnN5L5nUlJz9q8mqHNbv+dxiv\n5vWGPKe2jiEJSAihJMkzzc146VpT95kpajzaRzxMTvD7KGkaQsciZKF6VLPxdF4O\n7EyW435gXYi8Ty843o2mb0sK7Jm0g8qdXjK1qDaarPublEVS3tafB/Z1rMDEcPZ1\n23YxQBo214RAcypH6UFXlc23bn8Rb5OfGb2TnmLI9DEoM1Z1nh+MiZATPG/HeeT+\nFexCIg/4DNO7UKK5/Z71ClbOYQtHf7HfAoIBAQCoH3KQjPBmkhpQEBlJtjgXuYKy\ndGgS1TBZ+hVL5AX+chNJ+6zxMxLqqPc6L9Q1QgWSEINu/x4Dw+jITeo70pvS8J3v\no+dGBd+NoBbo3uvKqBAicSWJE39tj4l8J9FNnmLY0/tX0RSmb6bxzSVcRngMHeKS\nBRx2gA8PQ3UR2O5SHUaJRy/fxcjlm2IcjztaKWZr9MCtHgVYlC8vTsuHxb2uxCVc\nuB4WVBBjcOgPBKApNIjcnw5JFvn5o3yuXHv8bT3ywSGUOIZjJseMJRGLu4x2RwqU\nr3snJOTivuRhSPwpA/7zgFlFodkNhfdskBfgzhi8J/gcqX3TGUTtmhBx+DUm\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T03:19:36.601 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T03:19:37.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:36 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:37.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:36 smithi112 ceph-mon[31442]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:37.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:36 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:37.978 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T03:19:37.978 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial admin user... 2024-09-21T03:19:38.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:38 smithi112 ceph-mon[31442]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:38.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:38 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:39.386 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$GxZlUC7RxYw.KXHKrLt8dewWZLTMx8sQR9BkM6mZ4BPqXIKo11QkW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726888778, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T03:19:39.386 INFO:teuthology.orchestra.run.smithi112.stdout:Fetching dashboard port number... 2024-09-21T03:19:39.500 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:39 smithi112 ceph-mon[31442]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:39.500 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:39 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:40.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:40 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/101328643' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T03:19:40.803 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T03:19:40.814 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:19:40.814 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-09-21T03:19:40.823 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:19:40.823 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-09-21T03:19:40.823 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-09-21T03:19:40.823 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T03:19:40.824 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph Dashboard is now available at: 2024-09-21T03:19:40.824 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:40.825 INFO:teuthology.orchestra.run.smithi112.stdout: URL: https://smithi112.front.sepia.ceph.com:8443/ 2024-09-21T03:19:40.825 INFO:teuthology.orchestra.run.smithi112.stdout: User: admin 2024-09-21T03:19:40.825 INFO:teuthology.orchestra.run.smithi112.stdout: Password: 3gqrjgum5a 2024-09-21T03:19:40.825 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:40.825 INFO:teuthology.orchestra.run.smithi112.stdout:Saving cluster configuration to /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config directory 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: ceph telemetry on 2024-09-21T03:19:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:42.136 INFO:teuthology.orchestra.run.smithi112.stdout:For more information see: 2024-09-21T03:19:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T03:19:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:42.136 INFO:teuthology.orchestra.run.smithi112.stdout:Bootstrap complete. 2024-09-21T03:19:42.173 INFO:tasks.cephadm:Fetching config... 2024-09-21T03:19:42.174 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:19:42.174 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T03:19:42.189 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T03:19:42.189 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:19:42.189 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T03:19:42.245 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T03:19:42.245 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:19:42.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T03:19:42.310 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T03:19:42.311 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:19:42.311 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T03:19:42.365 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T03:19:42.365 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEjmEAN6WfGSDFHSVNJXBhmK7ztSe/TVe3GjSt0agQr2LY/6ql3nDhMbPOlujSI4KAJkSIeyRjFE6ax0fSYOBbpL8hSUKZTn/3HNoUA41efA96pSG5SzHlsK/EU7uIgV5FKV+EBOloLTvGQreWukIfiR8fG3xgHq5OHLWSaLbTIoEjiyrvOF4FhN1IMxQ8BikDU1u8M4gCLKdhVo9bki94JmNbMBUaiFLvX+pSzSJDOQBWhARCOMIbuuffDsRpwvmmnmtU0JeqprYK0JcEztb+OMWBhmzXXLOkFCZw5O0oMkqXXsWSaR8RRkI3En2rHHg7MJBb+p44bJZQITiYHkSg/graFCaGIbddfWzvrQL6dcHZ01hECDAdI8gEdB6GeIkNIbd2APPs1iXo1w23e7XCcF76AEmgD6n6HQuoIJlbL9w93tDc2+mzCU7oUBbOkpHrFv/HM7bwXSyhfutiYJipdq01S7UhSNtFNsAgf3AoL5PSgEOOt5nQ2fo7fQXUpqc= ceph-063b32f6-77c8-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:19:42.441 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEjmEAN6WfGSDFHSVNJXBhmK7ztSe/TVe3GjSt0agQr2LY/6ql3nDhMbPOlujSI4KAJkSIeyRjFE6ax0fSYOBbpL8hSUKZTn/3HNoUA41efA96pSG5SzHlsK/EU7uIgV5FKV+EBOloLTvGQreWukIfiR8fG3xgHq5OHLWSaLbTIoEjiyrvOF4FhN1IMxQ8BikDU1u8M4gCLKdhVo9bki94JmNbMBUaiFLvX+pSzSJDOQBWhARCOMIbuuffDsRpwvmmnmtU0JeqprYK0JcEztb+OMWBhmzXXLOkFCZw5O0oMkqXXsWSaR8RRkI3En2rHHg7MJBb+p44bJZQITiYHkSg/graFCaGIbddfWzvrQL6dcHZ01hECDAdI8gEdB6GeIkNIbd2APPs1iXo1w23e7XCcF76AEmgD6n6HQuoIJlbL9w93tDc2+mzCU7oUBbOkpHrFv/HM7bwXSyhfutiYJipdq01S7UhSNtFNsAgf3AoL5PSgEOOt5nQ2fo7fQXUpqc= ceph-063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:19:42.453 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEjmEAN6WfGSDFHSVNJXBhmK7ztSe/TVe3GjSt0agQr2LY/6ql3nDhMbPOlujSI4KAJkSIeyRjFE6ax0fSYOBbpL8hSUKZTn/3HNoUA41efA96pSG5SzHlsK/EU7uIgV5FKV+EBOloLTvGQreWukIfiR8fG3xgHq5OHLWSaLbTIoEjiyrvOF4FhN1IMxQ8BikDU1u8M4gCLKdhVo9bki94JmNbMBUaiFLvX+pSzSJDOQBWhARCOMIbuuffDsRpwvmmnmtU0JeqprYK0JcEztb+OMWBhmzXXLOkFCZw5O0oMkqXXsWSaR8RRkI3En2rHHg7MJBb+p44bJZQITiYHkSg/graFCaGIbddfWzvrQL6dcHZ01hECDAdI8gEdB6GeIkNIbd2APPs1iXo1w23e7XCcF76AEmgD6n6HQuoIJlbL9w93tDc2+mzCU7oUBbOkpHrFv/HM7bwXSyhfutiYJipdq01S7UhSNtFNsAgf3AoL5PSgEOOt5nQ2fo7fQXUpqc= ceph-063b32f6-77c8-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:19:42.493 INFO:teuthology.orchestra.run.smithi187.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEjmEAN6WfGSDFHSVNJXBhmK7ztSe/TVe3GjSt0agQr2LY/6ql3nDhMbPOlujSI4KAJkSIeyRjFE6ax0fSYOBbpL8hSUKZTn/3HNoUA41efA96pSG5SzHlsK/EU7uIgV5FKV+EBOloLTvGQreWukIfiR8fG3xgHq5OHLWSaLbTIoEjiyrvOF4FhN1IMxQ8BikDU1u8M4gCLKdhVo9bki94JmNbMBUaiFLvX+pSzSJDOQBWhARCOMIbuuffDsRpwvmmnmtU0JeqprYK0JcEztb+OMWBhmzXXLOkFCZw5O0oMkqXXsWSaR8RRkI3En2rHHg7MJBb+p44bJZQITiYHkSg/graFCaGIbddfWzvrQL6dcHZ01hECDAdI8gEdB6GeIkNIbd2APPs1iXo1w23e7XCcF76AEmgD6n6HQuoIJlbL9w93tDc2+mzCU7oUBbOkpHrFv/HM7bwXSyhfutiYJipdq01S7UhSNtFNsAgf3AoL5PSgEOOt5nQ2fo7fQXUpqc= ceph-063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:19:42.506 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEjmEAN6WfGSDFHSVNJXBhmK7ztSe/TVe3GjSt0agQr2LY/6ql3nDhMbPOlujSI4KAJkSIeyRjFE6ax0fSYOBbpL8hSUKZTn/3HNoUA41efA96pSG5SzHlsK/EU7uIgV5FKV+EBOloLTvGQreWukIfiR8fG3xgHq5OHLWSaLbTIoEjiyrvOF4FhN1IMxQ8BikDU1u8M4gCLKdhVo9bki94JmNbMBUaiFLvX+pSzSJDOQBWhARCOMIbuuffDsRpwvmmnmtU0JeqprYK0JcEztb+OMWBhmzXXLOkFCZw5O0oMkqXXsWSaR8RRkI3En2rHHg7MJBb+p44bJZQITiYHkSg/graFCaGIbddfWzvrQL6dcHZ01hECDAdI8gEdB6GeIkNIbd2APPs1iXo1w23e7XCcF76AEmgD6n6HQuoIJlbL9w93tDc2+mzCU7oUBbOkpHrFv/HM7bwXSyhfutiYJipdq01S7UhSNtFNsAgf3AoL5PSgEOOt5nQ2fo7fQXUpqc= ceph-063b32f6-77c8-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:19:42.549 INFO:teuthology.orchestra.run.smithi203.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEjmEAN6WfGSDFHSVNJXBhmK7ztSe/TVe3GjSt0agQr2LY/6ql3nDhMbPOlujSI4KAJkSIeyRjFE6ax0fSYOBbpL8hSUKZTn/3HNoUA41efA96pSG5SzHlsK/EU7uIgV5FKV+EBOloLTvGQreWukIfiR8fG3xgHq5OHLWSaLbTIoEjiyrvOF4FhN1IMxQ8BikDU1u8M4gCLKdhVo9bki94JmNbMBUaiFLvX+pSzSJDOQBWhARCOMIbuuffDsRpwvmmnmtU0JeqprYK0JcEztb+OMWBhmzXXLOkFCZw5O0oMkqXXsWSaR8RRkI3En2rHHg7MJBb+p44bJZQITiYHkSg/graFCaGIbddfWzvrQL6dcHZ01hECDAdI8gEdB6GeIkNIbd2APPs1iXo1w23e7XCcF76AEmgD6n6HQuoIJlbL9w93tDc2+mzCU7oUBbOkpHrFv/HM7bwXSyhfutiYJipdq01S7UhSNtFNsAgf3AoL5PSgEOOt5nQ2fo7fQXUpqc= ceph-063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:19:42.563 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T03:19:42.711 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:42 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2719796563' entity='client.admin' 2024-09-21T03:19:42.813 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:44.107 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T03:19:44.107 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T03:19:44.654 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:44 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1599573302' entity='client.admin' 2024-09-21T03:19:44.654 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:44 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:45.028 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:45 smithi112 ceph-mon[31442]: mgrmap e13: x(active, since 12s) 2024-09-21T03:19:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:45 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:45 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"} : dispatch 2024-09-21T03:19:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:45 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:45 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:45 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:46.462 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi187 2024-09-21T03:19:46.462 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T03:19:46.462 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:19:46.482 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T03:19:46.482 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:19:46.539 INFO:tasks.cephadm:Adding host smithi187 to orchestrator... 2024-09-21T03:19:46.540 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch host add smithi187 2024-09-21T03:19:46.887 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:46 smithi112 ceph-mon[31442]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:46 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:46 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:46 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:19:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:46 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:19:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:46 smithi112 ceph-mon[31442]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-21T03:19:47.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:46 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:47.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:46 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:47.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:46 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:48.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:47 smithi112 ceph-mon[31442]: Updating smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:19:48.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:47 smithi112 ceph-mon[31442]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:19:48.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:47 smithi112 ceph-mon[31442]: Updating smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:19:49.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:48 smithi112 ceph-mon[31442]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:50.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:49 smithi112 ceph-mon[31442]: Deploying cephadm binary to smithi187 2024-09-21T03:19:50.622 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi187' with addr '172.21.15.187' 2024-09-21T03:19:51.197 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:19:51.437 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:51.814 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:51 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:51.814 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:51 smithi112 ceph-mon[31442]: Added host smithi187 2024-09-21T03:19:51.814 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:51 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:51.814 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:51 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:51.814 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:51 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:52.098 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:52.098 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.187", "hostname": "smithi187", "labels": [], "status": ""}] 2024-09-21T03:19:52.652 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi203 2024-09-21T03:19:52.653 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T03:19:52.653 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:19:52.672 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T03:19:52.672 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:19:52.730 INFO:tasks.cephadm:Adding host smithi203 to orchestrator... 2024-09-21T03:19:52.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch host add smithi203 2024-09-21T03:19:52.969 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:53.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:52 smithi112 ceph-mon[31442]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:19:53.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:52 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:53.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:52 smithi112 ceph-mon[31442]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:19:54.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:53 smithi112 ceph-mon[31442]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:19:56.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:19:56.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: Deploying cephadm binary to smithi203 2024-09-21T03:19:56.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:56.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:56.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:56.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:56.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"} : dispatch 2024-09-21T03:19:56.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:19:56.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:19:56.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: Updating smithi187:/etc/ceph/ceph.conf 2024-09-21T03:19:56.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:56.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:56.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:55 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:56.631 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi203' with addr '172.21.15.203' 2024-09-21T03:19:56.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:56 smithi112 ceph-mon[31442]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:19:56.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:56 smithi112 ceph-mon[31442]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:19:56.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:56 smithi112 ceph-mon[31442]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:19:56.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:56 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:56.966 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:56 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:19:57.260 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:19:57.499 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:57.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:57 smithi112 ceph-mon[31442]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:19:57.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:57 smithi112 ceph-mon[31442]: Added host smithi203 2024-09-21T03:19:57.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:57 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:57.944 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:57 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:58.136 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:19:58.136 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.187", "hostname": "smithi187", "labels": [], "status": ""}, {"addr": "172.21.15.203", "hostname": "smithi203", "labels": [], "status": ""}] 2024-09-21T03:19:58.775 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T03:19:58.776 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T03:19:59.015 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:19:59.040 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:58 smithi112 ceph-mon[31442]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:19:59.040 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:58 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:19:59.040 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:58 smithi112 ceph-mon[31442]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:19:59.952 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2024-09-21T03:20:00.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:19:59 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3293825510' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:20:00.574 INFO:tasks.cephadm:Adding mon.a on smithi112 2024-09-21T03:20:00.575 INFO:tasks.cephadm:Adding mon.b on smithi187 2024-09-21T03:20:00.575 INFO:tasks.cephadm:Adding mon.c on smithi203 2024-09-21T03:20:00.575 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi112:172.21.15.112=a;smithi187:172.21.15.187=b;smithi203:172.21.15.203=c' 2024-09-21T03:20:00.824 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T03:20:00.871 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T03:20:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:00 smithi112 ceph-mon[31442]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:00 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3293825510' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:20:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:00 smithi112 ceph-mon[31442]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:20:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:00 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:00 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:00 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:00 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:00 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:00 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:00 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:00 smithi112 ceph-mon[31442]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T03:20:01.611 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mon update... 2024-09-21T03:20:02.198 DEBUG:teuthology.orchestra.run.smithi187:mon.b> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T03:20:02.201 DEBUG:teuthology.orchestra.run.smithi203:mon.c> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T03:20:02.203 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:20:02.203 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:20:02.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:02.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:02.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:02.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:02.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:02.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:02.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi187:172.21.15.187=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:20:02.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi187:172.21.15.187=b;smithi203:172.21.15.203=c;count:3 2024-09-21T03:20:02.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:02.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:02.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:02.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:02.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:02.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:02.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:02.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:02 smithi112 ceph-mon[31442]: Deploying daemon mon.c on smithi203 2024-09-21T03:20:03.075 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:03.995 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:20:03.996 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","modified":"2024-09-21T03:18:29.513931Z","created":"2024-09-21T03:18:29.513931Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T03:20:03.996 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-09-21T03:20:04.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:03 smithi112 ceph-mon[31442]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:05.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:04 smithi112 ceph-mon[31442]: from='client.? 172.21.15.203:0/1441825948' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:20:05.640 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:20:05.640 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:20:06.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:05 smithi112 ceph-mon[31442]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:06.038 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:20:06.779 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:20:06.779 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","modified":"2024-09-21T03:18:29.513931Z","created":"2024-09-21T03:18:29.513931Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T03:20:06.780 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-09-21T03:20:06.883 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.ttl: 2592000 2024-09-21T03:20:06.883 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:20:06.883 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T03:20:06.884 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a155a947-9a23-4757-8f03-8240441df08c 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888806876760, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888806877376, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726888806, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a155a947-9a23-4757-8f03-8240441df08c", "db_session_id": "T6EBMNXNJ7D6Q59U4FBX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888806877506, "job": 1, "event": "recovery_finished"} 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ac4e98ea80 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: DB pointer 0x55ac4f734000 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** DB Stats ** 2024-09-21T03:20:06.885 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:06.886 INFO:journalctl@ceph.mon.c.smithi203.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:20:06.887 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache BinnedLRUCache@0x55ac4a92f610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: using public_addr v2:172.21.15.203:0/0 -> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: starting mon.c rank -1 at public addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] at bind addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:06.888 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(???) e0 preinit fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:07.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T03:20:07.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T03:20:07.161 INFO:journalctl@ceph.mon.c.smithi203.stdout: e1 2024-09-21T03:20:07.161 INFO:journalctl@ceph.mon.c.smithi203.stdout: btime 2024-09-21T03:18:41:875043+0000 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout: legacy client fscid: -1 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout: No filesystems configured 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T03:20:07.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Deploying cephadm binary to smithi203 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:07.163 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Updating smithi187:/etc/ceph/ceph.conf 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Added host smithi203 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:07.164 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3293825510' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3293825510' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T03:20:07.165 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi187:172.21.15.187=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi187:172.21.15.187=b;smithi203:172.21.15.203=c;count:3 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:07.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:07.167 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: Deploying daemon mon.c on smithi203 2024-09-21T03:20:07.167 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:07.167 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: from='client.? 172.21.15.203:0/1441825948' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:20:07.167 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:07.167 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 ceph-mon[34723]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T03:20:07.167 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 bash[34691]: 27cd630757a959bbff34a9d186ba0f1e6218ae7ad57eab91e6c8c7fee8c4f774 2024-09-21T03:20:07.167 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:06 smithi203 systemd[1]: Started Ceph mon.c for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:20:08.511 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:20:08.511 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:20:08.776 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:20:12.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:20:12.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: mon.a calling monitor election 2024-09-21T03:20:12.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:12.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: mon.c calling monitor election 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: monmap epoch 2 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: last_changed 2024-09-21T03:20:06.907266+0000 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: created 2024-09-21T03:18:29.513931+0000 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: min_mon_release 19 (squid) 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: election_strategy: 1 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-21T03:20:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: fsmap 2024-09-21T03:20:12.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:20:12.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: mgrmap e13: x(active, since 40s) 2024-09-21T03:20:12.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: overall HEALTH_OK 2024-09-21T03:20:12.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:12.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:12.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:12.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:12.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:11 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:20:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: mon.a calling monitor election 2024-09-21T03:20:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: mon.c calling monitor election 2024-09-21T03:20:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: monmap epoch 2 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: last_changed 2024-09-21T03:20:06.907266+0000 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: created 2024-09-21T03:18:29.513931+0000 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: min_mon_release 19 (squid) 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: election_strategy: 1 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: fsmap 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: mgrmap e13: x(active, since 40s) 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: overall HEALTH_OK 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:12.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:12.413 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:12.413 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:12.413 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:11 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:12.506 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:20:12.507 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","modified":"2024-09-21T03:20:06.907266Z","created":"2024-09-21T03:18:29.513931Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:20:12.507 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-09-21T03:20:13.072 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:12 smithi203 ceph-mon[34723]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:13.072 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:12 smithi203 ceph-mon[34723]: Deploying daemon mon.b on smithi187 2024-09-21T03:20:13.072 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:12 smithi203 ceph-mon[34723]: from='client.? 172.21.15.203:0/246501619' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:20:13.072 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:12 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:12 smithi112 ceph-mon[31442]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:12 smithi112 ceph-mon[31442]: Deploying daemon mon.b on smithi187 2024-09-21T03:20:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:12 smithi112 ceph-mon[31442]: from='client.? 172.21.15.203:0/246501619' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:20:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:12 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:14.100 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:20:14.100 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:20:14.353 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:20:15.125 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:20:15.216 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","modified":"2024-09-21T03:20:06.907266Z","created":"2024-09-21T03:18:29.513931Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:20:15.216 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-09-21T03:20:15.217 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:14 smithi203 ceph-mon[34723]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:14 smithi112 ceph-mon[31442]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 podman[34583]: 2024-09-21 03:20:15.290418231 +0000 UTC m=+0.015403591 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:20:15.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 podman[34583]: 2024-09-21 03:20:15.562291782 +0000 UTC m=+0.287277129 container init 8c10b0113f0a4465f7a3b6274ba47560909db3aa943b5dcc8541945515a549e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:20:15.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 podman[34583]: 2024-09-21 03:20:15.565598133 +0000 UTC m=+0.290583483 container start 8c10b0113f0a4465f7a3b6274ba47560909db3aa943b5dcc8541945515a549e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:20:15.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:20:15.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:20:15.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: pidfile_write: ignore empty --pid-file 2024-09-21T03:20:15.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: load: jerasure load: lrc 2024-09-21T03:20:15.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:20:15.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Git sha 0 2024-09-21T03:20:15.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:20:15.932 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: DB SUMMARY 2024-09-21T03:20:15.932 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: DB Session ID: DR5C1UO4JGSSBVSCNGCK 2024-09-21T03:20:15.932 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:20:15.932 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:20:15.932 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T03:20:15.932 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-21T03:20:15.933 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-21T03:20:15.933 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:20:15.933 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:20:15.933 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:20:15.933 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:20:15.933 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:20:15.934 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:20:15.934 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.env: 0x557640bc3d60 2024-09-21T03:20:15.934 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:20:15.934 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.info_log: 0x5576455e5360 2024-09-21T03:20:15.934 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:20:15.934 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.statistics: (nil) 2024-09-21T03:20:15.935 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.use_fsync: 0 2024-09-21T03:20:15.935 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:20:15.935 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:20:15.935 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:20:15.935 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:20:15.935 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:20:15.935 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:20:15.936 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:20:15.936 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:20:15.936 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:20:15.936 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:20:15.936 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:20:15.936 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.db_log_dir: 2024-09-21T03:20:15.937 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.wal_dir: 2024-09-21T03:20:15.937 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:20:15.937 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:20:15.937 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:20:15.937 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:20:15.937 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:20:15.938 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:20:15.938 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:20:15.938 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:20:15.938 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.write_buffer_manager: 0x5576455e9d60 2024-09-21T03:20:15.938 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:20:15.938 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:20:15.939 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:20:15.939 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:20:15.939 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:20:15.939 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:20:15.939 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:20:15.939 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:20:15.939 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.unordered_write: 0 2024-09-21T03:20:15.939 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:20:15.940 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:20:15.940 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:20:15.940 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:20:15.940 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.row_cache: None 2024-09-21T03:20:15.940 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.wal_filter: None 2024-09-21T03:20:15.942 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:20:15.942 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:20:15.942 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:20:15.942 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:20:15.942 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.wal_compression: 0 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:20:15.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_open_files: -1 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:20:15.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Compression algorithms supported: 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: kZSTD supported: 0 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:20:15.945 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.merge_operator: 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_filter: None 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5576455e4fa0) 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout: index_type: 0 2024-09-21T03:20:15.946 INFO:journalctl@ceph.mon.b.smithi187.stdout: data_block_index_type: 0 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: index_shortening: 1 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: checksum: 4 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: no_block_cache: 0 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_cache: 0x5576415a9610 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_cache_options: 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: capacity : 536870912 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: num_shard_bits : 4 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: strict_capacity_limit : 0 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_cache_compressed: (nil) 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: persistent_cache: (nil) 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_size: 4096 2024-09-21T03:20:15.947 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_size_deviation: 10 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_restart_interval: 16 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: index_block_restart_interval: 1 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: metadata_block_size: 4096 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: partition_filters: 0 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: use_delta_encoding: 1 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: filter_policy: bloomfilter 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: whole_key_filtering: 1 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: verify_compression: 0 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: format_version: 5 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: enable_index_compression: 1 2024-09-21T03:20:15.948 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_align: 0 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout: max_auto_readahead_size: 262144 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout: prepopulate_block_cache: 0 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compression: NoCompression 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.num_levels: 7 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:20:15.949 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:20:15.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:20:15.951 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:20:15.952 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:20:15.953 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:20:15.954 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:20:15.954 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:20:15.954 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:20:15.954 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:20:15.954 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:20:15.954 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:20:15.954 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.ttl: 2592000 2024-09-21T03:20:15.954 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:20:15.954 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:20:15.954 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 46919164-5f73-49b1-8486-03807d1ceb4a 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888815602800, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T03:20:15.955 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888815603392, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726888815, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "46919164-5f73-49b1-8486-03807d1ceb4a", "db_session_id": "DR5C1UO4JGSSBVSCNGCK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726888815603519, "job": 1, "event": "recovery_finished"} 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557645608a80 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: DB pointer 0x5576463ae000 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout: ** DB Stats ** 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:20:15.956 INFO:journalctl@ceph.mon.b.smithi187.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: ** Compaction Stats [default] ** 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: ** Compaction Stats [default] ** 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:20:15.957 INFO:journalctl@ceph.mon.b.smithi187.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:20:15.958 INFO:journalctl@ceph.mon.b.smithi187.stdout: Block cache BinnedLRUCache@0x5576415a9610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: using public_addr v2:172.21.15.187:0/0 -> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: starting mon.b rank -1 at public addrs [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] at bind addrs [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(???) e0 preinit fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).mds e1 new map 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout: e1 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout: btime 2024-09-21T03:18:41:875043+0000 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:20:15.959 INFO:journalctl@ceph.mon.b.smithi187.stdout: legacy client fscid: -1 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout: No filesystems configured 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.960 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Deploying cephadm binary to smithi203 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Updating smithi187:/etc/ceph/ceph.conf 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:15.961 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Added host smithi203 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3293825510' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.962 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3293825510' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi187:172.21.15.187=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi187:172.21.15.187=b;smithi203:172.21.15.203=c;count:3 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Deploying daemon mon.c on smithi203 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='client.? 172.21.15.203:0/1441825948' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:20:15.964 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.a calling monitor election 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.c calling monitor election 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: monmap epoch 2 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:15.965 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: last_changed 2024-09-21T03:20:06.907266+0000 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: created 2024-09-21T03:18:29.513931+0000 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: min_mon_release 19 (squid) 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: election_strategy: 1 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: fsmap 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mgrmap e13: x(active, since 40s) 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: overall HEALTH_OK 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:15.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:15.967 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:15.967 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.967 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: Deploying daemon mon.b on smithi187 2024-09-21T03:20:15.967 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='client.? 172.21.15.203:0/246501619' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:20:15.967 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:15.967 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:15.967 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 ceph-mon[34603]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T03:20:15.967 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 bash[34583]: 8c10b0113f0a4465f7a3b6274ba47560909db3aa943b5dcc8541945515a549e0 2024-09-21T03:20:15.967 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:15 smithi187 systemd[1]: Started Ceph mon.b for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:20:16.675 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:20:16.675 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:20:16.926 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:20:21.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:20:21.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: mon.a calling monitor election 2024-09-21T03:20:21.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:21.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: mon.c calling monitor election 2024-09-21T03:20:21.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:21.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: mon.b calling monitor election 2024-09-21T03:20:21.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:21.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: monmap epoch 3 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: last_changed 2024-09-21T03:20:15.632909+0000 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: created 2024-09-21T03:18:29.513931+0000 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: min_mon_release 19 (squid) 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: election_strategy: 1 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: 2: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: fsmap 2024-09-21T03:20:21.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:20:21.040 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: mgrmap e13: x(active, since 48s) 2024-09-21T03:20:21.040 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: overall HEALTH_OK 2024-09-21T03:20:21.040 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.040 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.040 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.040 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:20 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.104 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:20:21.104 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":3,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","modified":"2024-09-21T03:20:15.632909Z","created":"2024-09-21T03:18:29.513931Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T03:20:21.104 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 3 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: mon.a calling monitor election 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: mon.c calling monitor election 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: mon.b calling monitor election 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.105 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: monmap epoch 3 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: last_changed 2024-09-21T03:20:15.632909+0000 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: created 2024-09-21T03:18:29.513931+0000 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: min_mon_release 19 (squid) 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: election_strategy: 1 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: 2: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: fsmap 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: mgrmap e13: x(active, since 48s) 2024-09-21T03:20:21.106 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: overall HEALTH_OK 2024-09-21T03:20:21.107 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.107 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.107 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.107 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:20 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: mon.a calling monitor election 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: mon.c calling monitor election 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: mon.b calling monitor election 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: monmap epoch 3 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: last_changed 2024-09-21T03:20:15.632909+0000 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: created 2024-09-21T03:18:29.513931+0000 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: min_mon_release 19 (squid) 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: election_strategy: 1 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: 2: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: fsmap 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: mgrmap e13: x(active, since 48s) 2024-09-21T03:20:21.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: overall HEALTH_OK 2024-09-21T03:20:21.182 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.182 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.182 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.182 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:20 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:21.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:21 smithi203 ceph-mon[34723]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:21.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:21 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:21.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:21 smithi203 ceph-mon[34723]: from='client.? 172.21.15.203:0/4174698870' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:20:21.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:21 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:22.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:21 smithi112 ceph-mon[31442]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:22.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:21 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:22.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:21 smithi112 ceph-mon[31442]: from='client.? 172.21.15.203:0/4174698870' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:20:22.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:21 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:22.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:21 smithi187 ceph-mon[34603]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:22.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:21 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:22.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:21 smithi187 ceph-mon[34603]: from='client.? 172.21.15.203:0/4174698870' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:20:22.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:21 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:22.533 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T03:20:22.533 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.856 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.856 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.856 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.856 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.856 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.856 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.856 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:22.856 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:20:22.856 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:22 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:22.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:22.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:20:22.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:22 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:23.177 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.178 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:23.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:23.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:20:23.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:22 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:23.264 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:24.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:23 smithi112 ceph-mon[31442]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:24.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:23 smithi112 ceph-mon[31442]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-21T03:20:24.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:23 smithi112 ceph-mon[31442]: Updating smithi187:/etc/ceph/ceph.conf 2024-09-21T03:20:24.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:23 smithi112 ceph-mon[31442]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T03:20:24.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:23 smithi112 ceph-mon[31442]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:24.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:23 smithi112 ceph-mon[31442]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:24.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:23 smithi112 ceph-mon[31442]: Updating smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:24.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:23 smithi112 ceph-mon[31442]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:20:24.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:23 smithi112 ceph-mon[31442]: Reconfiguring daemon mon.a on smithi112 2024-09-21T03:20:24.160 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:23 smithi203 ceph-mon[34723]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:24.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:23 smithi203 ceph-mon[34723]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-21T03:20:24.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:23 smithi203 ceph-mon[34723]: Updating smithi187:/etc/ceph/ceph.conf 2024-09-21T03:20:24.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:23 smithi203 ceph-mon[34723]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T03:20:24.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:23 smithi203 ceph-mon[34723]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:24.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:23 smithi203 ceph-mon[34723]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:24.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:23 smithi203 ceph-mon[34723]: Updating smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:24.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:23 smithi203 ceph-mon[34723]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:20:24.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:23 smithi203 ceph-mon[34723]: Reconfiguring daemon mon.a on smithi112 2024-09-21T03:20:24.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:23 smithi187 ceph-mon[34603]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:24.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:23 smithi187 ceph-mon[34603]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-21T03:20:24.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:23 smithi187 ceph-mon[34603]: Updating smithi187:/etc/ceph/ceph.conf 2024-09-21T03:20:24.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:23 smithi187 ceph-mon[34603]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T03:20:24.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:23 smithi187 ceph-mon[34603]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:24.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:23 smithi187 ceph-mon[34603]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:24.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:23 smithi187 ceph-mon[34603]: Updating smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:24.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:23 smithi187 ceph-mon[34603]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:20:24.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:23 smithi187 ceph-mon[34603]: Reconfiguring daemon mon.a on smithi112 2024-09-21T03:20:24.379 INFO:teuthology.orchestra.run.smithi112.stdout:# minimal ceph.conf for 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:24.380 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-09-21T03:20:24.380 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:20:24.380 INFO:teuthology.orchestra.run.smithi112.stdout: mon_host = [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-09-21T03:20:24.936 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T03:20:24.936 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:20:24.936 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:20:24.963 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:20:24.963 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:25.028 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T03:20:25.028 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:20:25.057 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T03:20:25.057 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:25.122 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T03:20:25.122 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:20:25.155 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T03:20:25.155 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:25.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:24 smithi203 ceph-mon[34723]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:25.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:24 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:25.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:24 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:25.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:24 smithi203 ceph-mon[34723]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:20:25.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:24 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:25.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:24 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:20:25.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:24 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:25.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:24 smithi203 ceph-mon[34723]: Reconfiguring daemon mon.b on smithi187 2024-09-21T03:20:25.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:24 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1063277517' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:25.225 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T03:20:25.225 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T03:20:25.247 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:24 smithi112 ceph-mon[31442]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:25.247 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:24 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:25.247 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:24 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:25.247 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:24 smithi112 ceph-mon[31442]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:20:25.247 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:24 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:25.247 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:24 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:20:25.247 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:24 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:25.247 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:24 smithi112 ceph-mon[31442]: Reconfiguring daemon mon.b on smithi187 2024-09-21T03:20:25.247 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:24 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1063277517' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:25.296 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:24 smithi187 ceph-mon[34603]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:25.296 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:24 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:25.296 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:24 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:25.296 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:24 smithi187 ceph-mon[34603]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:20:25.296 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:24 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:25.296 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:24 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:20:25.296 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:24 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:25.297 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:24 smithi187 ceph-mon[34603]: Reconfiguring daemon mon.b on smithi187 2024-09-21T03:20:25.297 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:24 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1063277517' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:25.463 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:26.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:26 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:26.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:26 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:26.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:26 smithi112 ceph-mon[31442]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:20:26.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:26 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:26.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:26 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:20:26.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:26 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:26.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:26 smithi112 ceph-mon[31442]: Reconfiguring daemon mon.c on smithi203 2024-09-21T03:20:26.577 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:26 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:26.577 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:26 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:26.577 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:26 smithi203 ceph-mon[34723]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:20:26.577 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:26 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:26.577 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:26 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:20:26.577 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:26 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:26.577 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:26 smithi203 ceph-mon[34723]: Reconfiguring daemon mon.c on smithi203 2024-09-21T03:20:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:26 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:26 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:26 smithi187 ceph-mon[34603]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:20:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:26 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:20:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:26 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:20:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:26 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:26.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:26 smithi187 ceph-mon[34603]: Reconfiguring daemon mon.c on smithi203 2024-09-21T03:20:27.537 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:27 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: ignoring --setuser ceph since I am not root 2024-09-21T03:20:27.537 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:27 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: ignoring --setgroup ceph since I am not root 2024-09-21T03:20:27.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:27 smithi112 ceph-mon[31442]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:27.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:27 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2923932216' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:20:27.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:27 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:27.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:27 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:27.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:27 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:27.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:27 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:27.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:27 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:27.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:27 smithi112 ceph-mon[31442]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:27.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:27 smithi203 ceph-mon[34723]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:27.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:27 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2923932216' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:20:27.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:27 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:27.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:27 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:27.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:27 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:27.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:27 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:27.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:27 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:27.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:27 smithi203 ceph-mon[34723]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:27 smithi187 ceph-mon[34603]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:27 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2923932216' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:20:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:27 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:27 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:27 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:27.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:27 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:27.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:27 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:27.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:27 smithi187 ceph-mon[34603]: from='mgr.14152 172.21.15.112:0/3003293533' entity='mgr.x' 2024-09-21T03:20:27.897 INFO:tasks.cephadm:Adding mgr.x on smithi112 2024-09-21T03:20:27.897 INFO:tasks.cephadm:Adding mgr.y on smithi187 2024-09-21T03:20:27.897 INFO:tasks.cephadm:Adding mgr.z on smithi203 2024-09-21T03:20:27.897 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi112=x;smithi187=y;smithi203=z' 2024-09-21T03:20:28.148 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:20:28.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:28 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2923932216' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:20:28.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:28 smithi112 ceph-mon[31442]: mgrmap e14: x(active, since 55s) 2024-09-21T03:20:28.559 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:28 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2923932216' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:20:28.559 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:28 smithi203 ceph-mon[34723]: mgrmap e14: x(active, since 55s) 2024-09-21T03:20:28.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:28 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2923932216' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:20:28.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:28 smithi187 ceph-mon[34603]: mgrmap e14: x(active, since 55s) 2024-09-21T03:20:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: Active manager daemon x restarted 2024-09-21T03:20:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: Activating manager daemon x 2024-09-21T03:20:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:20:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: mgrmap e15: x(active, starting, since 0.00721512s) 2024-09-21T03:20:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:20:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:32.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:20:32.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:20:32.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:20:32.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:20:32.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:32 smithi112 ceph-mon[31442]: Manager daemon x is now available 2024-09-21T03:20:32.650 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: Active manager daemon x restarted 2024-09-21T03:20:32.650 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: Activating manager daemon x 2024-09-21T03:20:32.650 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:20:32.650 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: mgrmap e15: x(active, starting, since 0.00721512s) 2024-09-21T03:20:32.650 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:20:32.650 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:32.650 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:32.650 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:20:32.650 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:20:32.651 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:20:32.651 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:20:32.651 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:32 smithi203 ceph-mon[34723]: Manager daemon x is now available 2024-09-21T03:20:32.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: Active manager daemon x restarted 2024-09-21T03:20:32.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: Activating manager daemon x 2024-09-21T03:20:32.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:20:32.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: mgrmap e15: x(active, starting, since 0.00721512s) 2024-09-21T03:20:32.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:20:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:20:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:20:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:20:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:20:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:20:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:20:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:32 smithi187 ceph-mon[34603]: Manager daemon x is now available 2024-09-21T03:20:33.243 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mgr update... 2024-09-21T03:20:33.534 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:33.534 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:20:33.534 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:20:33.534 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:33 smithi187 ceph-mon[34603]: mgrmap e16: x(active, since 1.00947s) 2024-09-21T03:20:33.534 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:20:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:20:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-mon[31442]: mgrmap e16: x(active, since 1.00947s) 2024-09-21T03:20:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:33.638 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:33.639 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:20:33.639 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:20:33.639 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:33 smithi203 ceph-mon[34723]: mgrmap e16: x(active, since 1.00947s) 2024-09-21T03:20:33.639 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:20:33.780+0000 7f3813979640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:20:33] ENGINE Error in HTTPServer.serve 2024-09-21T03:20:34.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: Traceback (most recent call last): 2024-09-21T03:20:34.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:20:34.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self._connections.run(self.expiration_interval) 2024-09-21T03:20:34.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:20:34.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self._run(expiration_interval) 2024-09-21T03:20:34.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:20:34.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:20:34.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:20:34.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:20:34.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:20:34.039 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: s = self.context.wrap_socket( 2024-09-21T03:20:34.039 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:20:34.039 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: return self.sslsocket_class._create( 2024-09-21T03:20:34.039 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:20:34.039 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self.do_handshake() 2024-09-21T03:20:34.039 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:20:34.039 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self._sslobj.do_handshake() 2024-09-21T03:20:34.039 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:20:34.039 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:20:33 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:20:34.045 DEBUG:teuthology.orchestra.run.smithi187:mgr.y> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T03:20:34.047 DEBUG:teuthology.orchestra.run.smithi203:mgr.z> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T03:20:34.049 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T03:20:34.050 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T03:20:34.052 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T03:20:34.052 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T03:20:34.095 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T03:20:34.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T03:20:34.138 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T03:20:34.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T03:20:34.181 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T03:20:34.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T03:20:34.224 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T03:20:34.224 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T03:20:34.268 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T03:20:34.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T03:20:34.311 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T03:20:34.311 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T03:20:34.349 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:34 smithi112 ceph-mon[31442]: Saving service mgr spec with placement smithi112=x;smithi187=y;smithi203=z;count:3 2024-09-21T03:20:34.349 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.349 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.350 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.350 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.351 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T03:20:34.351 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T03:20:34.396 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T03:20:34.396 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T03:20:34.397 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:34.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:34 smithi187 ceph-mon[34603]: Saving service mgr spec with placement smithi112=x;smithi187=y;smithi203=z;count:3 2024-09-21T03:20:34.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.444 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T03:20:34.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T03:20:34.494 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T03:20:34.494 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T03:20:34.495 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:34.550 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T03:20:34.550 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T03:20:34.597 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T03:20:34.597 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T03:20:34.641 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T03:20:34.641 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T03:20:34.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:34 smithi203 ceph-mon[34723]: Saving service mgr spec with placement smithi112=x;smithi187=y;smithi203=z;count:3 2024-09-21T03:20:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:34.696 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T03:20:34.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T03:20:34.747 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T03:20:34.747 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T03:20:34.804 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T03:20:34.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T03:20:34.853 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T03:20:34.853 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T03:20:34.897 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T03:20:34.897 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T03:20:34.941 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T03:20:34.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T03:20:34.990 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T03:20:34.990 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T03:20:35.037 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T03:20:35.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T03:20:35.097 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T03:20:35.097 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T03:20:35.141 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T03:20:35.141 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T03:20:35.185 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T03:20:35.185 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T03:20:35.232 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T03:20:35.232 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T03:20:35.292 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T03:20:35.292 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T03:20:35.339 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T03:20:35.339 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T03:20:35.396 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T03:20:35.397 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T03:20:35.466 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T03:20:35.467 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi112') 2024-09-21T03:20:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: [21/Sep/2024:03:20:33] ENGINE Bus STARTING 2024-09-21T03:20:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: [21/Sep/2024:03:20:33] ENGINE Serving on http://172.21.15.112:8765 2024-09-21T03:20:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: [21/Sep/2024:03:20:33] ENGINE Serving on https://172.21.15.112:7150 2024-09-21T03:20:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: [21/Sep/2024:03:20:33] ENGINE Bus STARTED 2024-09-21T03:20:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: [21/Sep/2024:03:20:33] ENGINE Error in HTTPServer.serve 2024-09-21T03:20:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout: Traceback (most recent call last): 2024-09-21T03:20:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:20:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:20:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:20:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: self._run(expiration_interval) 2024-09-21T03:20:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:20:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:20:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:20:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:20:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:20:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: s = self.context.wrap_socket( 2024-09-21T03:20:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:20:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: return self.sslsocket_class._create( 2024-09-21T03:20:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: self.do_handshake() 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: self._sslobj.do_handshake() 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: mgrmap e17: x(active, since 2s) 2024-09-21T03:20:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:35.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: [21/Sep/2024:03:20:33] ENGINE Bus STARTING 2024-09-21T03:20:35.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: [21/Sep/2024:03:20:33] ENGINE Serving on http://172.21.15.112:8765 2024-09-21T03:20:35.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: [21/Sep/2024:03:20:33] ENGINE Serving on https://172.21.15.112:7150 2024-09-21T03:20:35.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: [21/Sep/2024:03:20:33] ENGINE Bus STARTED 2024-09-21T03:20:35.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: [21/Sep/2024:03:20:33] ENGINE Error in HTTPServer.serve 2024-09-21T03:20:35.662 INFO:journalctl@ceph.mon.c.smithi203.stdout: Traceback (most recent call last): 2024-09-21T03:20:35.662 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:20:35.662 INFO:journalctl@ceph.mon.c.smithi203.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:20:35.662 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:20:35.662 INFO:journalctl@ceph.mon.c.smithi203.stdout: self._run(expiration_interval) 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: s = self.context.wrap_socket( 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: return self.sslsocket_class._create( 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: self.do_handshake() 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: self._sslobj.do_handshake() 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:20:35.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: mgrmap e17: x(active, since 2s) 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:35.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:35.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: [21/Sep/2024:03:20:33] ENGINE Bus STARTING 2024-09-21T03:20:35.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: [21/Sep/2024:03:20:33] ENGINE Serving on http://172.21.15.112:8765 2024-09-21T03:20:35.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: [21/Sep/2024:03:20:33] ENGINE Serving on https://172.21.15.112:7150 2024-09-21T03:20:35.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: [21/Sep/2024:03:20:33] ENGINE Bus STARTED 2024-09-21T03:20:35.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: [21/Sep/2024:03:20:33] ENGINE Error in HTTPServer.serve 2024-09-21T03:20:35.681 INFO:journalctl@ceph.mon.b.smithi187.stdout: Traceback (most recent call last): 2024-09-21T03:20:35.681 INFO:journalctl@ceph.mon.b.smithi187.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:20:35.681 INFO:journalctl@ceph.mon.b.smithi187.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:20:35.681 INFO:journalctl@ceph.mon.b.smithi187.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:20:35.682 INFO:journalctl@ceph.mon.b.smithi187.stdout: self._run(expiration_interval) 2024-09-21T03:20:35.682 INFO:journalctl@ceph.mon.b.smithi187.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:20:35.682 INFO:journalctl@ceph.mon.b.smithi187.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:20:35.682 INFO:journalctl@ceph.mon.b.smithi187.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:20:35.682 INFO:journalctl@ceph.mon.b.smithi187.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:20:35.682 INFO:journalctl@ceph.mon.b.smithi187.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:20:35.682 INFO:journalctl@ceph.mon.b.smithi187.stdout: s = self.context.wrap_socket( 2024-09-21T03:20:35.682 INFO:journalctl@ceph.mon.b.smithi187.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout: return self.sslsocket_class._create( 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout: self.do_handshake() 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout: self._sslobj.do_handshake() 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:35.684 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: mgrmap e17: x(active, since 2s) 2024-09-21T03:20:35.684 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.684 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:35.684 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"} : dispatch 2024-09-21T03:20:35.684 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:35.684 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-21T03:20:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi187:/etc/ceph/ceph.conf 2024-09-21T03:20:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T03:20:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3468014143' entity='client.admin' 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3190601349' entity='client.admin' 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Updating smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:20:36.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:20:36.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:36.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:36 smithi203 ceph-mon[34723]: Deploying daemon mgr.z on smithi203 2024-09-21T03:20:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-21T03:20:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi187:/etc/ceph/ceph.conf 2024-09-21T03:20:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T03:20:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3468014143' entity='client.admin' 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3190601349' entity='client.admin' 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Updating smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:20:36.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:20:36.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:20:36.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:36.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:36 smithi187 ceph-mon[34603]: Deploying daemon mgr.z on smithi203 2024-09-21T03:20:36.683 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi112:/etc/ceph/ceph.conf 2024-09-21T03:20:36.683 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi187:/etc/ceph/ceph.conf 2024-09-21T03:20:36.683 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-21T03:20:36.683 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:36.683 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:36.684 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:36.684 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:36.684 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:36.684 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:36.684 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.684 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.684 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.684 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3468014143' entity='client.admin' 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3190601349' entity='client.admin' 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Updating smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:20:36.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:36.686 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:36 smithi112 ceph-mon[31442]: Deploying daemon mgr.z on smithi203 2024-09-21T03:20:37.089 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi112') 2024-09-21T03:20:37.089 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi112') 2024-09-21T03:20:37.174 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:37.175 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:37.179 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:37.208 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:37.225 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:37.292 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:37.901 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:38.161 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:20:37 smithi203 systemd[1]: Starting Ceph mgr.z for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:20:38.458 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:20:38 smithi203 podman[36693]: 2024-09-21 03:20:38.179595739 +0000 UTC m=+0.111228642 container create 77c02ea8cb23adc0e3f55d7fbeb67027fc228cfead46fb620140c43ffed181df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:20:38.458 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:20:38 smithi203 podman[36693]: 2024-09-21 03:20:38.084386037 +0000 UTC m=+0.016018911 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:20:38.458 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:20:38 smithi203 podman[36693]: 2024-09-21 03:20:38.419273138 +0000 UTC m=+0.350905998 container init 77c02ea8cb23adc0e3f55d7fbeb67027fc228cfead46fb620140c43ffed181df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-z, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T03:20:38.458 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:20:38 smithi203 podman[36693]: 2024-09-21 03:20:38.421960054 +0000 UTC m=+0.353592926 container start 77c02ea8cb23adc0e3f55d7fbeb67027fc228cfead46fb620140c43ffed181df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-z, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T03:20:38.837 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:38.910 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:20:38 smithi203 ceph-mgr[36712]: -- 172.21.15.203:0/1647346122 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563dfd9c10e0 con 0x563e02764000 2024-09-21T03:20:38.911 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:20:38 smithi203 bash[36693]: 77c02ea8cb23adc0e3f55d7fbeb67027fc228cfead46fb620140c43ffed181df 2024-09-21T03:20:38.911 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:20:38 smithi203 systemd[1]: Started Ceph mgr.z for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:20:39.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:39 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1337247009' entity='client.admin' 2024-09-21T03:20:39.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:39.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:39.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:39.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:20:39.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:20:39.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:20:39.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:39.305 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:39 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1337247009' entity='client.admin' 2024-09-21T03:20:39.305 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:39.305 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:39.305 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:39.305 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:20:39.305 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:20:39.305 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:20:39.305 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:39.588 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:39 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1337247009' entity='client.admin' 2024-09-21T03:20:39.588 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:39.588 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:39.588 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:39.588 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:20:39.588 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:20:39.588 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:20:39.588 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:39.668 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:39.678 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:39.679 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:39.679 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:39.684 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:39.685 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:39.685 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:39.685 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:39.686 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:39.687 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 -- 172.21.15.112:0/2124220274 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1ce803fb70 msgr2=0x7f1ce8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 --2- 172.21.15.112:0/2124220274 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1ce803fb70 0x7f1ce8042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0005d00 tx=0x7f1cf000b410 comp rx=0 tx=0).stop 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 -- 172.21.15.112:0/2124220274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d00109e00 msgr2=0x7f1d001a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 --2- 172.21.15.112:0/2124220274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d00109e00 0x7f1d001a1ca0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800eb10 tx=0x7f1cf800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 -- 172.21.15.112:0/2124220274 shutdown_connections 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 --2- 172.21.15.112:0/2124220274 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1ce803fb70 0x7f1ce8042080 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 --2- 172.21.15.112:0/2124220274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d00109e00 0x7f1d001a1ca0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 --2- 172.21.15.112:0/2124220274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d001094b0 0x7f1d0019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 --2- 172.21.15.112:0/2124220274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d00101f70 0x7f1d0019c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 -- 172.21.15.112:0/2124220274 >> 172.21.15.112:0/2124220274 conn(0x7f1d00077c60 msgr2=0x7f1d001072c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 -- 172.21.15.112:0/2124220274 shutdown_connections 2024-09-21T03:20:40.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.111+0000 7f1d05d12640 1 -- 172.21.15.112:0/2124220274 wait complete. 2024-09-21T03:20:40.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.293+0000 7fc1f4c7f640 1 -- 172.21.15.112:0/1289380344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fc1f010ab10 con 0x7fc1f0108d20 2024-09-21T03:20:40.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.296+0000 7fc1df7fe640 1 -- 172.21.15.112:0/1289380344 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7fc1e8004000 con 0x7fc1f0108d20 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 -- 172.21.15.112:0/1289380344 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc1d803fba0 msgr2=0x7fc1d80420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 --2- 172.21.15.112:0/1289380344 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc1d803fba0 0x7fc1d80420b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000abe0 tx=0x7fc1e000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 -- 172.21.15.112:0/1289380344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1f0108d20 msgr2=0x7fc1f019e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 --2- 172.21.15.112:0/1289380344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1f0108d20 0x7fc1f019e950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009fd0 tx=0x7fc1e800def0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 -- 172.21.15.112:0/1289380344 shutdown_connections 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 --2- 172.21.15.112:0/1289380344 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc1d803fba0 0x7fc1d80420b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 --2- 172.21.15.112:0/1289380344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1f0108d20 0x7fc1f019e950 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 --2- 172.21.15.112:0/1289380344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f01083d0 0x7fc1f01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 --2- 172.21.15.112:0/1289380344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1f0102ae0 0x7fc1f0199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 -- 172.21.15.112:0/1289380344 >> 172.21.15.112:0/1289380344 conn(0x7fc1f0079760 msgr2=0x7fc1f0101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 -- 172.21.15.112:0/1289380344 shutdown_connections 2024-09-21T03:20:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.299+0000 7fc1f4c7f640 1 -- 172.21.15.112:0/1289380344 wait complete. 2024-09-21T03:20:40.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:40 smithi112 ceph-mon[31442]: Deploying daemon mgr.y on smithi187 2024-09-21T03:20:40.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:40 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3722413988' entity='client.admin' 2024-09-21T03:20:40.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:40 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2124220274' entity='client.admin' 2024-09-21T03:20:40.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:40 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1289380344' entity='client.admin' 2024-09-21T03:20:40.586 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:40 smithi187 ceph-mon[34603]: Deploying daemon mgr.y on smithi187 2024-09-21T03:20:40.586 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:40 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3722413988' entity='client.admin' 2024-09-21T03:20:40.586 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:40 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2124220274' entity='client.admin' 2024-09-21T03:20:40.586 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:40 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1289380344' entity='client.admin' 2024-09-21T03:20:40.586 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:20:40 smithi187 systemd[1]: Starting Ceph mgr.y for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:20:40.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:40 smithi203 ceph-mon[34723]: Deploying daemon mgr.y on smithi187 2024-09-21T03:20:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:40 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3722413988' entity='client.admin' 2024-09-21T03:20:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:40 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2124220274' entity='client.admin' 2024-09-21T03:20:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:40 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1289380344' entity='client.admin' 2024-09-21T03:20:40.835 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi112') 2024-09-21T03:20:40.859 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:20:40 smithi187 podman[36124]: 2024-09-21 03:20:40.584685418 +0000 UTC m=+0.105474143 container create 6187016459bf7b09211af00f60bbd8ae163353ca4487e793e8dd84d99aa4fa60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T03:20:40.859 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:20:40 smithi187 podman[36124]: 2024-09-21 03:20:40.49527941 +0000 UTC m=+0.016068144 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:20:40.859 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:20:40 smithi187 podman[36124]: 2024-09-21 03:20:40.820779432 +0000 UTC m=+0.341568169 container init 6187016459bf7b09211af00f60bbd8ae163353ca4487e793e8dd84d99aa4fa60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-y, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T03:20:40.859 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:20:40 smithi187 podman[36124]: 2024-09-21 03:20:40.823679671 +0000 UTC m=+0.344468395 container start 6187016459bf7b09211af00f60bbd8ae163353ca4487e793e8dd84d99aa4fa60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-y, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:20:40.860 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:20:40 smithi187 ceph-mgr[36143]: -- 172.21.15.187:0/3409519643 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fed2cb50e0 con 0x55fed7a58000 2024-09-21T03:20:40.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.983+0000 7fb55ec53640 1 -- 172.21.15.112:0/3761347530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5581026b0 msgr2=0x7fb558102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:40.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.983+0000 7fb55ec53640 1 --2- 172.21.15.112:0/3761347530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5581026b0 0x7fb558102ac0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009900 tx=0x7fb54c02a480 comp rx=0 tx=0).stop 2024-09-21T03:20:40.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.983+0000 7fb55ec53640 1 -- 172.21.15.112:0/3761347530 shutdown_connections 2024-09-21T03:20:40.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.983+0000 7fb55ec53640 1 --2- 172.21.15.112:0/3761347530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558110260 0x7fb558112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.983+0000 7fb55ec53640 1 --2- 172.21.15.112:0/3761347530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb558103000 0x7fb55810fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.983+0000 7fb55ec53640 1 --2- 172.21.15.112:0/3761347530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5581026b0 0x7fb558102ac0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.983+0000 7fb55ec53640 1 -- 172.21.15.112:0/3761347530 >> 172.21.15.112:0/3761347530 conn(0x7fb558079790 msgr2=0x7fb558079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:40.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.984+0000 7fb55ec53640 1 -- 172.21.15.112:0/3761347530 shutdown_connections 2024-09-21T03:20:40.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.984+0000 7fb55ec53640 1 -- 172.21.15.112:0/3761347530 wait complete. 2024-09-21T03:20:40.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.984+0000 7fb55ec53640 1 Processor -- start 2024-09-21T03:20:40.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb55ec53640 1 -- start start 2024-09-21T03:20:40.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb55ec53640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5581026b0 0x7fb55819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:40.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb55ec53640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558103000 0x7fb55819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:40.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb55ec53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb558110260 0x7fb5581a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:40.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb55ec53640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb558115a60 con 0x7fb558110260 2024-09-21T03:20:40.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb55ec53640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581158e0 con 0x7fb5581026b0 2024-09-21T03:20:40.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb55ec53640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb558115be0 con 0x7fb558103000 2024-09-21T03:20:40.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb557fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558103000 0x7fb55819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:40.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb55c9ef640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5581026b0 0x7fb55819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:40.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb557fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558103000 0x7fb55819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:49994/0 (socket says 172.21.15.112:49994) 2024-09-21T03:20:40.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb557fff640 1 -- 172.21.15.112:0/3532066610 learned_addr learned my addr 172.21.15.112:0/3532066610 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:40.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb55d1f0640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb558110260 0x7fb5581a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:40.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb557fff640 1 -- 172.21.15.112:0/3532066610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5581026b0 msgr2=0x7fb55819c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb557fff640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5581026b0 0x7fb55819c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb557fff640 1 -- 172.21.15.112:0/3532066610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb558110260 msgr2=0x7fb5581a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb557fff640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb558110260 0x7fb5581a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.985+0000 7fb557fff640 1 -- 172.21.15.112:0/3532066610 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5581a2420 con 0x7fb558103000 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.986+0000 7fb557fff640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558103000 0x7fb55819ca50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb54400d950 tx=0x7fb54400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.986+0000 7fb555ffb640 1 -- 172.21.15.112:0/3532066610 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544017030 con 0x7fb558103000 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.986+0000 7fb555ffb640 1 -- 172.21.15.112:0/3532066610 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fb544004020 con 0x7fb558103000 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.986+0000 7fb555ffb640 1 -- 172.21.15.112:0/3532066610 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544005580 con 0x7fb558103000 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.986+0000 7fb55ec53640 1 -- 172.21.15.112:0/3532066610 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5581a2710 con 0x7fb558103000 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.986+0000 7fb55ec53640 1 -- 172.21.15.112:0/3532066610 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5581a8e70 con 0x7fb558103000 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.987+0000 7fb555ffb640 1 -- 172.21.15.112:0/3532066610 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 51916+0+0 (secure 0 0 0) 0x7fb5440047a0 con 0x7fb558103000 2024-09-21T03:20:40.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.987+0000 7fb55ec53640 1 -- 172.21.15.112:0/3532066610 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb558102ac0 con 0x7fb558103000 2024-09-21T03:20:40.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.987+0000 7fb555ffb640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb54803fb70 0x7fb548042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:40.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.988+0000 7fb555ffb640 1 -- 172.21.15.112:0/3532066610 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb544054020 con 0x7fb558103000 2024-09-21T03:20:40.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.988+0000 7fb55c9ef640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb54803fb70 0x7fb548042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:40.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.988+0000 7fb55c9ef640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb54803fb70 0x7fb548042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb54c031440 tx=0x7fb54c009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:40.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:40.993+0000 7fb555ffb640 1 -- 172.21.15.112:0/3532066610 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb54401ad50 con 0x7fb558103000 2024-09-21T03:20:41.025 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:41.025 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:41.179 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:20:40 smithi187 bash[36124]: 6187016459bf7b09211af00f60bbd8ae163353ca4487e793e8dd84d99aa4fa60 2024-09-21T03:20:41.179 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:20:40 smithi187 systemd[1]: Started Ceph mgr.y for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:20:41.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.195+0000 7fb55ec53640 1 -- 172.21.15.112:0/3532066610 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7fb55810b3f0 con 0x7fb558103000 2024-09-21T03:20:41.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.200+0000 7fb555ffb640 1 -- 172.21.15.112:0/3532066610 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v16)=0 v16) ==== 116+0+0 (secure 0 0 0) 0x7fb54400fcb0 con 0x7fb558103000 2024-09-21T03:20:41.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 -- 172.21.15.112:0/3532066610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb54803fb70 msgr2=0x7fb548042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:41.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb54803fb70 0x7fb548042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb54c031440 tx=0x7fb54c009560 comp rx=0 tx=0).stop 2024-09-21T03:20:41.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 -- 172.21.15.112:0/3532066610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558103000 msgr2=0x7fb55819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:41.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558103000 0x7fb55819ca50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb54400d950 tx=0x7fb54400de10 comp rx=0 tx=0).stop 2024-09-21T03:20:41.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 -- 172.21.15.112:0/3532066610 shutdown_connections 2024-09-21T03:20:41.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb54803fb70 0x7fb548042080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb558110260 0x7fb5581a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558103000 0x7fb55819ca50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 --2- 172.21.15.112:0/3532066610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5581026b0 0x7fb55819c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 -- 172.21.15.112:0/3532066610 >> 172.21.15.112:0/3532066610 conn(0x7fb558079790 msgr2=0x7fb558111810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:41.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 -- 172.21.15.112:0/3532066610 shutdown_connections 2024-09-21T03:20:41.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.202+0000 7fb55ec53640 1 -- 172.21.15.112:0/3532066610 wait complete. 2024-09-21T03:20:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.262+0000 7fc0b61ab640 1 -- 172.21.15.112:0/2059667345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0b0109570 msgr2=0x7fc0b0109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.262+0000 7fc0b61ab640 1 --2- 172.21.15.112:0/2059667345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0b0109570 0x7fc0b0109980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009900 tx=0x7fc0a402a410 comp rx=0 tx=0).stop 2024-09-21T03:20:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.262+0000 7fc0b61ab640 1 -- 172.21.15.112:0/2059667345 shutdown_connections 2024-09-21T03:20:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.262+0000 7fc0b61ab640 1 --2- 172.21.15.112:0/2059667345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b0109ec0 0x7fc0b01106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.262+0000 7fc0b61ab640 1 --2- 172.21.15.112:0/2059667345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0b0109570 0x7fc0b0109980 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.262+0000 7fc0b61ab640 1 --2- 172.21.15.112:0/2059667345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0b0103c30 0x7fc0b0104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.262+0000 7fc0b61ab640 1 -- 172.21.15.112:0/2059667345 >> 172.21.15.112:0/2059667345 conn(0x7fc0b007b8d0 msgr2=0x7fc0b007bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:41.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.262+0000 7fc0b61ab640 1 -- 172.21.15.112:0/2059667345 shutdown_connections 2024-09-21T03:20:41.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.262+0000 7fc0b61ab640 1 -- 172.21.15.112:0/2059667345 wait complete. 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.263+0000 7fc0b61ab640 1 Processor -- start 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.263+0000 7fc0b61ab640 1 -- start start 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.263+0000 7fc0b61ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0b0103c30 0x7fc0b019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.263+0000 7fc0b61ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0b0109570 0x7fc0b019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.263+0000 7fc0b61ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b0109ec0 0x7fc0b019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.263+0000 7fc0b61ab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01138a0 con 0x7fc0b0103c30 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.263+0000 7fc0b61ab640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0113720 con 0x7fc0b0109ec0 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0b61ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0113a20 con 0x7fc0b0109570 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0b0103c30 0x7fc0b019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0affff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b0109ec0 0x7fc0b019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0b0109570 0x7fc0b019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:41.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0b0103c30 0x7fc0b019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56362/0 (socket says 172.21.15.112:56362) 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0af7fe640 1 -- 172.21.15.112:0/547262913 learned_addr learned my addr 172.21.15.112:0/547262913 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0af7fe640 1 -- 172.21.15.112:0/547262913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0b0109570 msgr2=0x7fc0b019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0af7fe640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0b0109570 0x7fc0b019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0af7fe640 1 -- 172.21.15.112:0/547262913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b0109ec0 msgr2=0x7fc0b019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0af7fe640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b0109ec0 0x7fc0b019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0af7fe640 1 -- 172.21.15.112:0/547262913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b01a02a0 con 0x7fc0b0103c30 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0affff640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b0109ec0 0x7fc0b019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0af7fe640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0b0103c30 0x7fc0b019a3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000d950 tx=0x7fc0a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0acff9640 1 -- 172.21.15.112:0/547262913 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0017070 con 0x7fc0b0103c30 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0b61ab640 1 -- 172.21.15.112:0/547262913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b01a0590 con 0x7fc0b0103c30 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0acff9640 1 -- 172.21.15.112:0/547262913 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fc0a0009c30 con 0x7fc0b0103c30 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0acff9640 1 -- 172.21.15.112:0/547262913 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a000f620 con 0x7fc0b0103c30 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.264+0000 7fc0b61ab640 1 -- 172.21.15.112:0/547262913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b00755f0 con 0x7fc0b0103c30 2024-09-21T03:20:41.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.266+0000 7fc0acff9640 1 -- 172.21.15.112:0/547262913 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 51916+0+0 (secure 0 0 0) 0x7fc0a000f7c0 con 0x7fc0b0103c30 2024-09-21T03:20:41.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.266+0000 7fc0b61ab640 1 -- 172.21.15.112:0/547262913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b0104eb0 con 0x7fc0b0103c30 2024-09-21T03:20:41.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.266+0000 7fc0acff9640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc09803fba0 0x7fc0980420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:41.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.266+0000 7fc0acff9640 1 -- 172.21.15.112:0/547262913 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc0a0054420 con 0x7fc0b0103c30 2024-09-21T03:20:41.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.266+0000 7fc0aeffd640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc09803fba0 0x7fc0980420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:41.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.266+0000 7fc0aeffd640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc09803fba0 0x7fc0980420b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009230 tx=0x7fc0a4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:41.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.271+0000 7fc0acff9640 1 -- 172.21.15.112:0/547262913 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc0a0010e20 con 0x7fc0b0103c30 2024-09-21T03:20:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.454+0000 7fc0b61ab640 1 -- 172.21.15.112:0/547262913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fc0b010bcb0 con 0x7fc0b0103c30 2024-09-21T03:20:41.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.458+0000 7fc0acff9640 1 -- 172.21.15.112:0/547262913 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v17)=0 v17) ==== 119+0+0 (secure 0 0 0) 0x7fc0a001c600 con 0x7fc0b0103c30 2024-09-21T03:20:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 -- 172.21.15.112:0/547262913 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc09803fba0 msgr2=0x7fc0980420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc09803fba0 0x7fc0980420b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009230 tx=0x7fc0a4031400 comp rx=0 tx=0).stop 2024-09-21T03:20:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 -- 172.21.15.112:0/547262913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0b0103c30 msgr2=0x7fc0b019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0b0103c30 0x7fc0b019a3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000d950 tx=0x7fc0a000de10 comp rx=0 tx=0).stop 2024-09-21T03:20:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 -- 172.21.15.112:0/547262913 shutdown_connections 2024-09-21T03:20:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc09803fba0 0x7fc0980420b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b0109ec0 0x7fc0b019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0b0109570 0x7fc0b019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 --2- 172.21.15.112:0/547262913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0b0103c30 0x7fc0b019a3f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:41.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 -- 172.21.15.112:0/547262913 >> 172.21.15.112:0/547262913 conn(0x7fc0b007b8d0 msgr2=0x7fc0b01056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:41.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 -- 172.21.15.112:0/547262913 shutdown_connections 2024-09-21T03:20:41.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:41.460+0000 7fc0b61ab640 1 -- 172.21.15.112:0/547262913 wait complete. 2024-09-21T03:20:42.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:41 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:41 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:41 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:41 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:41 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:42.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:41 smithi112 ceph-mon[31442]: from='client.? ' entity='client.admin' 2024-09-21T03:20:42.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:41 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/547262913' entity='client.admin' 2024-09-21T03:20:42.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:41 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:41 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:41 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:41 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:41 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:42.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:41 smithi203 ceph-mon[34723]: from='client.? ' entity='client.admin' 2024-09-21T03:20:42.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:41 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/547262913' entity='client.admin' 2024-09-21T03:20:42.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:41 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:41 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:41 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:41 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:42.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:41 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:42.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:41 smithi187 ceph-mon[34603]: from='client.? ' entity='client.admin' 2024-09-21T03:20:42.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:41 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/547262913' entity='client.admin' 2024-09-21T03:20:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.119+0000 7f39dcd53640 1 -- 172.21.15.112:0/1081310047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d81083a0 msgr2=0x7f39d81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.119+0000 7f39dcd53640 1 --2- 172.21.15.112:0/1081310047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d81083a0 0x7f39d81087b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009900 tx=0x7f39cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:20:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.119+0000 7f39dcd53640 1 -- 172.21.15.112:0/1081310047 shutdown_connections 2024-09-21T03:20:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.119+0000 7f39dcd53640 1 --2- 172.21.15.112:0/1081310047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39d8108cf0 0x7f39d810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.119+0000 7f39dcd53640 1 --2- 172.21.15.112:0/1081310047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d81083a0 0x7f39d81087b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.119+0000 7f39dcd53640 1 --2- 172.21.15.112:0/1081310047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39d8102a60 0x7f39d8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.119+0000 7f39dcd53640 1 -- 172.21.15.112:0/1081310047 >> 172.21.15.112:0/1081310047 conn(0x7f39d8079730 msgr2=0x7f39d8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.119+0000 7f39dcd53640 1 -- 172.21.15.112:0/1081310047 shutdown_connections 2024-09-21T03:20:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.119+0000 7f39dcd53640 1 -- 172.21.15.112:0/1081310047 wait complete. 2024-09-21T03:20:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39dcd53640 1 Processor -- start 2024-09-21T03:20:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39dcd53640 1 -- start start 2024-09-21T03:20:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39dcd53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d8102a60 0x7f39d819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39dcd53640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39d81083a0 0x7f39d819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39dcd53640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39d8108cf0 0x7f39d819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39dcd53640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8112720 con 0x7f39d8102a60 2024-09-21T03:20:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39dcd53640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81125a0 con 0x7f39d81083a0 2024-09-21T03:20:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39dcd53640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81128a0 con 0x7f39d8108cf0 2024-09-21T03:20:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39d659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d8102a60 0x7f39d819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39d659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d8102a60 0x7f39d819b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56374/0 (socket says 172.21.15.112:56374) 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39d659c640 1 -- 172.21.15.112:0/3603767282 learned_addr learned my addr 172.21.15.112:0/3603767282 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39d5d9b640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39d81083a0 0x7f39d819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39d6d9d640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39d8108cf0 0x7f39d819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39d659c640 1 -- 172.21.15.112:0/3603767282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39d8108cf0 msgr2=0x7f39d819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39d659c640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39d8108cf0 0x7f39d819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39d659c640 1 -- 172.21.15.112:0/3603767282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39d81083a0 msgr2=0x7f39d819b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39d659c640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39d81083a0 0x7f39d819b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.120+0000 7f39d659c640 1 -- 172.21.15.112:0/3603767282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d81a6380 con 0x7f39d8102a60 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.121+0000 7f39d659c640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d8102a60 0x7f39d819b0f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f39c800d950 tx=0x7f39c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.121+0000 7f39c77fe640 1 -- 172.21.15.112:0/3603767282 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8017070 con 0x7f39d8102a60 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.121+0000 7f39dcd53640 1 -- 172.21.15.112:0/3603767282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d81a6670 con 0x7f39d8102a60 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.121+0000 7f39c77fe640 1 -- 172.21.15.112:0/3603767282 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f39c8009c30 con 0x7f39d8102a60 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.121+0000 7f39c77fe640 1 -- 172.21.15.112:0/3603767282 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c800f620 con 0x7f39d8102a60 2024-09-21T03:20:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.121+0000 7f39dcd53640 1 -- 172.21.15.112:0/3603767282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d81a6b80 con 0x7f39d8102a60 2024-09-21T03:20:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.122+0000 7f39c77fe640 1 -- 172.21.15.112:0/3603767282 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 51916+0+0 (secure 0 0 0) 0x7f39c800f7c0 con 0x7f39d8102a60 2024-09-21T03:20:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.122+0000 7f39dcd53640 1 -- 172.21.15.112:0/3603767282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d8102e70 con 0x7f39d8102a60 2024-09-21T03:20:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.122+0000 7f39c77fe640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39c003fba0 0x7f39c00420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:43.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.122+0000 7f39c77fe640 1 -- 172.21.15.112:0/3603767282 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f39c8054270 con 0x7f39d8102a60 2024-09-21T03:20:43.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.122+0000 7f39d5d9b640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39c003fba0 0x7f39c00420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:43.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.123+0000 7f39d5d9b640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39c003fba0 0x7f39c00420b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f39cc000c00 tx=0x7f39cc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.127+0000 7f39c77fe640 1 -- 172.21.15.112:0/3603767282 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39c8010e20 con 0x7f39d8102a60 2024-09-21T03:20:43.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:43.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:43.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:20:43.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:20:43.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:20:43.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:43.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: Reconfiguring daemon mgr.x on smithi112 2024-09-21T03:20:43.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.295+0000 7f28bbd1b640 1 -- 172.21.15.112:0/2924132139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28b4108d00 msgr2=0x7f28b410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.295+0000 7f28bbd1b640 1 --2- 172.21.15.112:0/2924132139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28b4108d00 0x7f28b410f530 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009ca0 tx=0x7f28ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.295+0000 7f28bbd1b640 1 -- 172.21.15.112:0/2924132139 shutdown_connections 2024-09-21T03:20:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.295+0000 7f28bbd1b640 1 --2- 172.21.15.112:0/2924132139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28b4108d00 0x7f28b410f530 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.295+0000 7f28bbd1b640 1 --2- 172.21.15.112:0/2924132139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b41083b0 0x7f28b41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.295+0000 7f28bbd1b640 1 --2- 172.21.15.112:0/2924132139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b4103180 0x7f28b4103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.295+0000 7f28bbd1b640 1 -- 172.21.15.112:0/2924132139 >> 172.21.15.112:0/2924132139 conn(0x7f28b4077c60 msgr2=0x7f28b40780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.295+0000 7f28bbd1b640 1 -- 172.21.15.112:0/2924132139 shutdown_connections 2024-09-21T03:20:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.295+0000 7f28bbd1b640 1 -- 172.21.15.112:0/2924132139 wait complete. 2024-09-21T03:20:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28bbd1b640 1 Processor -- start 2024-09-21T03:20:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28bbd1b640 1 -- start start 2024-09-21T03:20:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28bbd1b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4103180 0x7f28b419b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28bbd1b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b41083b0 0x7f28b419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:43.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28bbd1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28b4108d00 0x7f28b419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28bbd1b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4112730 con 0x7f28b4108d00 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28bbd1b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28b41125b0 con 0x7f28b41083b0 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28bbd1b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f28b41128b0 con 0x7f28b4103180 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28ba2b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28b4108d00 0x7f28b419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28ba2b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28b4108d00 0x7f28b419bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56378/0 (socket says 172.21.15.112:56378) 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28ba2b8640 1 -- 172.21.15.112:0/1542512043 learned_addr learned my addr 172.21.15.112:0/1542512043 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.296+0000 7f28b92b6640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b41083b0 0x7f28b419b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28b9ab7640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4103180 0x7f28b419b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28ba2b8640 1 -- 172.21.15.112:0/1542512043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4103180 msgr2=0x7f28b419b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28ba2b8640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4103180 0x7f28b419b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28ba2b8640 1 -- 172.21.15.112:0/1542512043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b41083b0 msgr2=0x7f28b419b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28ba2b8640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b41083b0 0x7f28b419b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28ba2b8640 1 -- 172.21.15.112:0/1542512043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b41a6360 con 0x7f28b4108d00 2024-09-21T03:20:43.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28b92b6640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b41083b0 0x7f28b419b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:20:43.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28ba2b8640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28b4108d00 0x7f28b419bb20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f28ac006010 tx=0x7f28ac0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:43.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28aaffd640 1 -- 172.21.15.112:0/1542512043 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac003cf0 con 0x7f28b4108d00 2024-09-21T03:20:43.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28aaffd640 1 -- 172.21.15.112:0/1542512043 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f28ac0044b0 con 0x7f28b4108d00 2024-09-21T03:20:43.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28bbd1b640 1 -- 172.21.15.112:0/1542512043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b41a65f0 con 0x7f28b4108d00 2024-09-21T03:20:43.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28bbd1b640 1 -- 172.21.15.112:0/1542512043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b41a6b80 con 0x7f28b4108d00 2024-09-21T03:20:43.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.297+0000 7f28aaffd640 1 -- 172.21.15.112:0/1542512043 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac030a50 con 0x7f28b4108d00 2024-09-21T03:20:43.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.298+0000 7f28bbd1b640 1 -- 172.21.15.112:0/1542512043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b4103590 con 0x7f28b4108d00 2024-09-21T03:20:43.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.298+0000 7f28aaffd640 1 -- 172.21.15.112:0/1542512043 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 51916+0+0 (secure 0 0 0) 0x7f28ac030cf0 con 0x7f28b4108d00 2024-09-21T03:20:43.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.299+0000 7f28aaffd640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28a003fb70 0x7f28a0042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:43.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.299+0000 7f28aaffd640 1 -- 172.21.15.112:0/1542512043 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f28ac075b30 con 0x7f28b4108d00 2024-09-21T03:20:43.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.299+0000 7f28b9ab7640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28a003fb70 0x7f28a0042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:43.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.299+0000 7f28b9ab7640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28a003fb70 0x7f28a0042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f28b4102960 tx=0x7f28b0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:43.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.304+0000 7f28aaffd640 1 -- 172.21.15.112:0/1542512043 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f28ac03b410 con 0x7f28b4108d00 2024-09-21T03:20:43.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.315+0000 7f39dcd53640 1 -- 172.21.15.112:0/3603767282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f39d810aae0 con 0x7f39d8102a60 2024-09-21T03:20:43.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.319+0000 7f39c77fe640 1 -- 172.21.15.112:0/3603767282 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v18)=0 v18) ==== 111+0+0 (secure 0 0 0) 0x7f39c801e120 con 0x7f39d8102a60 2024-09-21T03:20:43.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.321+0000 7f39dcd53640 1 -- 172.21.15.112:0/3603767282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39c003fba0 msgr2=0x7f39c00420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:43.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.321+0000 7f39dcd53640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39c003fba0 0x7f39c00420b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f39cc000c00 tx=0x7f39cc031400 comp rx=0 tx=0).stop 2024-09-21T03:20:43.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.321+0000 7f39dcd53640 1 -- 172.21.15.112:0/3603767282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d8102a60 msgr2=0x7f39d819b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:43.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.321+0000 7f39dcd53640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d8102a60 0x7f39d819b0f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f39c800d950 tx=0x7f39c800de10 comp rx=0 tx=0).stop 2024-09-21T03:20:43.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.321+0000 7f39dcd53640 1 -- 172.21.15.112:0/3603767282 shutdown_connections 2024-09-21T03:20:43.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.321+0000 7f39dcd53640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39c003fba0 0x7f39c00420b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.321+0000 7f39dcd53640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39d8108cf0 0x7f39d819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.321+0000 7f39dcd53640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39d81083a0 0x7f39d819b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.321+0000 7f39dcd53640 1 --2- 172.21.15.112:0/3603767282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d8102a60 0x7f39d819b0f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.321+0000 7f39dcd53640 1 -- 172.21.15.112:0/3603767282 >> 172.21.15.112:0/3603767282 conn(0x7f39d8079730 msgr2=0x7f39d81044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.322+0000 7f39dcd53640 1 -- 172.21.15.112:0/3603767282 shutdown_connections 2024-09-21T03:20:43.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.322+0000 7f39dcd53640 1 -- 172.21.15.112:0/3603767282 wait complete. 2024-09-21T03:20:43.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:43.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:43.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:20:43.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:20:43.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:20:43.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:43.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: Reconfiguring daemon mgr.x on smithi112 2024-09-21T03:20:43.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: Reconfiguring daemon mgr.x on smithi112 2024-09-21T03:20:43.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:43.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.492+0000 7f28bbd1b640 1 -- 172.21.15.112:0/1542512043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f28b410aaf0 con 0x7f28b4108d00 2024-09-21T03:20:43.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.495+0000 7f28aaffd640 1 -- 172.21.15.112:0/1542512043 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v19)=0 v19) ==== 123+0+0 (secure 0 0 0) 0x7f28ac004040 con 0x7f28b4108d00 2024-09-21T03:20:43.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.496+0000 7f28aaffd640 1 -- 172.21.15.112:0/1542512043 <== mon.0 v2:172.21.15.112:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f28ac036030 con 0x7f28b4108d00 2024-09-21T03:20:43.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 -- 172.21.15.112:0/1542512043 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28a003fb70 msgr2=0x7f28a0042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:43.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28a003fb70 0x7f28a0042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f28b4102960 tx=0x7f28b0005f90 comp rx=0 tx=0).stop 2024-09-21T03:20:43.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 -- 172.21.15.112:0/1542512043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28b4108d00 msgr2=0x7f28b419bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:43.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28b4108d00 0x7f28b419bb20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f28ac006010 tx=0x7f28ac0093e0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 -- 172.21.15.112:0/1542512043 shutdown_connections 2024-09-21T03:20:43.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28a003fb70 0x7f28a0042080 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28b4108d00 0x7f28b419bb20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b41083b0 0x7f28b419b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 --2- 172.21.15.112:0/1542512043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4103180 0x7f28b419b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:43.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 -- 172.21.15.112:0/1542512043 >> 172.21.15.112:0/1542512043 conn(0x7f28b4077c60 msgr2=0x7f28b41064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:43.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 -- 172.21.15.112:0/1542512043 shutdown_connections 2024-09-21T03:20:43.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:43.498+0000 7f28bbd1b640 1 -- 172.21.15.112:0/1542512043 wait complete. 2024-09-21T03:20:44.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:44 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3603767282' entity='client.admin' 2024-09-21T03:20:44.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:44 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1542512043' entity='client.admin' 2024-09-21T03:20:44.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:44 smithi112 ceph-mon[31442]: Standby manager daemon z started 2024-09-21T03:20:44.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:44 smithi112 ceph-mon[31442]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:20:44.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:44 smithi112 ceph-mon[31442]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:20:44.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:44 smithi112 ceph-mon[31442]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:20:44.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:44 smithi112 ceph-mon[31442]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:20:44.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.622+0000 7ff0d2526640 1 -- 172.21.15.112:0/3088050622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc1083a0 msgr2=0x7ff0cc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:44.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.622+0000 7ff0d2526640 1 --2- 172.21.15.112:0/3088050622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc1083a0 0x7ff0cc1087b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009900 tx=0x7ff0c002a410 comp rx=0 tx=0).stop 2024-09-21T03:20:44.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.622+0000 7ff0d2526640 1 -- 172.21.15.112:0/3088050622 shutdown_connections 2024-09-21T03:20:44.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.622+0000 7ff0d2526640 1 --2- 172.21.15.112:0/3088050622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc108cf0 0x7ff0cc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:44.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.622+0000 7ff0d2526640 1 --2- 172.21.15.112:0/3088050622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc1083a0 0x7ff0cc1087b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:44.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.622+0000 7ff0d2526640 1 --2- 172.21.15.112:0/3088050622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0cc102a60 0x7ff0cc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:44.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.622+0000 7ff0d2526640 1 -- 172.21.15.112:0/3088050622 >> 172.21.15.112:0/3088050622 conn(0x7ff0cc079730 msgr2=0x7ff0cc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:44.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.622+0000 7ff0d2526640 1 -- 172.21.15.112:0/3088050622 shutdown_connections 2024-09-21T03:20:44.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.622+0000 7ff0d2526640 1 -- 172.21.15.112:0/3088050622 wait complete. 2024-09-21T03:20:44.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0d2526640 1 Processor -- start 2024-09-21T03:20:44.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0d2526640 1 -- start start 2024-09-21T03:20:44.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0d2526640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0cc102a60 0x7ff0cc10ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:44.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0d2526640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc1083a0 0x7ff0cc1104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:44.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0d2526640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc108cf0 0x7ff0cc110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:44.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0d2526640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc113960 con 0x7ff0cc1083a0 2024-09-21T03:20:44.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0d2526640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc1137e0 con 0x7ff0cc108cf0 2024-09-21T03:20:44.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0d2526640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc113ae0 con 0x7ff0cc102a60 2024-09-21T03:20:44.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc1083a0 0x7ff0cc1104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:44.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc1083a0 0x7ff0cc1104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56394/0 (socket says 172.21.15.112:56394) 2024-09-21T03:20:44.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0cb7fe640 1 -- 172.21.15.112:0/4083752633 learned_addr learned my addr 172.21.15.112:0/4083752633 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:44.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.623+0000 7ff0d0ac3640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc108cf0 0x7ff0cc110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:44.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0cc102a60 0x7ff0cc10ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:44.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0cb7fe640 1 -- 172.21.15.112:0/4083752633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0cc102a60 msgr2=0x7ff0cc10ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:44.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0cb7fe640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0cc102a60 0x7ff0cc10ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:44.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0cb7fe640 1 -- 172.21.15.112:0/4083752633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc108cf0 msgr2=0x7ff0cc110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:44.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0cb7fe640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc108cf0 0x7ff0cc110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:44.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0cb7fe640 1 -- 172.21.15.112:0/4083752633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc1a8670 con 0x7ff0cc1083a0 2024-09-21T03:20:44.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0cb7fe640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc1083a0 0x7ff0cc1104f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0005940 tx=0x7ff0c0005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:44.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0c97fa640 1 -- 172.21.15.112:0/4083752633 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c003f030 con 0x7ff0cc1083a0 2024-09-21T03:20:44.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0c97fa640 1 -- 172.21.15.112:0/4083752633 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff0c00043d0 con 0x7ff0cc1083a0 2024-09-21T03:20:44.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0d2526640 1 -- 172.21.15.112:0/4083752633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc1a8900 con 0x7ff0cc1083a0 2024-09-21T03:20:44.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0c97fa640 1 -- 172.21.15.112:0/4083752633 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0035070 con 0x7ff0cc1083a0 2024-09-21T03:20:44.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.624+0000 7ff0d2526640 1 -- 172.21.15.112:0/4083752633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc1a8e30 con 0x7ff0cc1083a0 2024-09-21T03:20:44.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.625+0000 7ff0d2526640 1 -- 172.21.15.112:0/4083752633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0cc102e70 con 0x7ff0cc1083a0 2024-09-21T03:20:44.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.626+0000 7ff0c97fa640 1 -- 172.21.15.112:0/4083752633 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 103195+0+0 (secure 0 0 0) 0x7ff0c0042030 con 0x7ff0cc1083a0 2024-09-21T03:20:44.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.627+0000 7ff0c97fa640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0b407b020 0x7ff0b407d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:44.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.627+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0b407b020 0x7ff0b407d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.627+0000 7ff0c97fa640 1 -- 172.21.15.112:0/4083752633 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff0c00bd160 con 0x7ff0cc1083a0 2024-09-21T03:20:44.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.627+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0b407b020 0x7ff0b407d530 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc111a00 tx=0x7ff0bc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:44.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.631+0000 7ff0c97fa640 1 -- 172.21.15.112:0/4083752633 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff0c0089b70 con 0x7ff0cc1083a0 2024-09-21T03:20:44.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:44 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3603767282' entity='client.admin' 2024-09-21T03:20:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:44 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1542512043' entity='client.admin' 2024-09-21T03:20:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:44 smithi203 ceph-mon[34723]: Standby manager daemon z started 2024-09-21T03:20:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:44 smithi203 ceph-mon[34723]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:20:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:44 smithi203 ceph-mon[34723]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:20:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:44 smithi203 ceph-mon[34723]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:20:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:44 smithi203 ceph-mon[34723]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:20:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:44 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3603767282' entity='client.admin' 2024-09-21T03:20:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:44 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1542512043' entity='client.admin' 2024-09-21T03:20:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:44 smithi187 ceph-mon[34603]: Standby manager daemon z started 2024-09-21T03:20:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:44 smithi187 ceph-mon[34603]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:20:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:44 smithi187 ceph-mon[34603]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:20:44.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:44 smithi187 ceph-mon[34603]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:20:44.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:44 smithi187 ceph-mon[34603]: from='mgr.? 172.21.15.203:0/2080675381' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:20:44.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.817+0000 7ff0d2526640 1 -- 172.21.15.112:0/4083752633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7ff0cc02e640 con 0x7ff0cc1083a0 2024-09-21T03:20:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.821+0000 7ff0c97fa640 1 -- 172.21.15.112:0/4083752633 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v20)=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7ff0c008ea20 con 0x7ff0cc1083a0 2024-09-21T03:20:44.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 -- 172.21.15.112:0/4083752633 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0b407b020 msgr2=0x7ff0b407d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:44.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0b407b020 0x7ff0b407d530 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc111a00 tx=0x7ff0bc005f90 comp rx=0 tx=0).stop 2024-09-21T03:20:44.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 -- 172.21.15.112:0/4083752633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc1083a0 msgr2=0x7ff0cc1104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:44.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc1083a0 0x7ff0cc1104f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0005940 tx=0x7ff0c0005e30 comp rx=0 tx=0).stop 2024-09-21T03:20:44.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 -- 172.21.15.112:0/4083752633 shutdown_connections 2024-09-21T03:20:44.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0b407b020 0x7ff0b407d530 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:44.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc108cf0 0x7ff0cc110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:44.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc1083a0 0x7ff0cc1104f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 --2- 172.21.15.112:0/4083752633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0cc102a60 0x7ff0cc10ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 -- 172.21.15.112:0/4083752633 >> 172.21.15.112:0/4083752633 conn(0x7ff0cc079730 msgr2=0x7ff0cc1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 -- 172.21.15.112:0/4083752633 shutdown_connections 2024-09-21T03:20:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:44.824+0000 7ff0d2526640 1 -- 172.21.15.112:0/4083752633 wait complete. 2024-09-21T03:20:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.527+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2230316168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9bc103b60 msgr2=0x7fc9bc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.527+0000 7fc9c34ea640 1 --2- 172.21.15.112:0/2230316168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9bc103b60 0x7fc9bc103f70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009ca0 tx=0x7fc9ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.527+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2230316168 shutdown_connections 2024-09-21T03:20:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.527+0000 7fc9c34ea640 1 --2- 172.21.15.112:0/2230316168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9bc109df0 0x7fc9bc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.527+0000 7fc9c34ea640 1 --2- 172.21.15.112:0/2230316168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9bc1094a0 0x7fc9bc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.527+0000 7fc9c34ea640 1 --2- 172.21.15.112:0/2230316168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9bc103b60 0x7fc9bc103f70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.527+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2230316168 >> 172.21.15.112:0/2230316168 conn(0x7fc9bc07b850 msgr2=0x7fc9bc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.527+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2230316168 shutdown_connections 2024-09-21T03:20:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.527+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2230316168 wait complete. 2024-09-21T03:20:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.528+0000 7fc9c34ea640 1 Processor -- start 2024-09-21T03:20:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.528+0000 7fc9c34ea640 1 -- start start 2024-09-21T03:20:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.528+0000 7fc9c34ea640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9bc103b60 0x7fc9bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.528+0000 7fc9c34ea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9bc1094a0 0x7fc9bc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.528+0000 7fc9c34ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9bc109df0 0x7fc9bc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.528+0000 7fc9c34ea640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc113990 con 0x7fc9bc109df0 2024-09-21T03:20:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.528+0000 7fc9c34ea640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc113810 con 0x7fc9bc1094a0 2024-09-21T03:20:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.528+0000 7fc9c34ea640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc113b10 con 0x7fc9bc103b60 2024-09-21T03:20:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.528+0000 7fc9c1a87640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9bc109df0 0x7fc9bc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c0a85640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9bc1094a0 0x7fc9bc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c1286640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9bc103b60 0x7fc9bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c1a87640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9bc109df0 0x7fc9bc076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41888/0 (socket says 172.21.15.112:41888) 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c1a87640 1 -- 172.21.15.112:0/2972873342 learned_addr learned my addr 172.21.15.112:0/2972873342 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c1a87640 1 -- 172.21.15.112:0/2972873342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9bc103b60 msgr2=0x7fc9bc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c1a87640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9bc103b60 0x7fc9bc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c1a87640 1 -- 172.21.15.112:0/2972873342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9bc1094a0 msgr2=0x7fc9bc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c1a87640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9bc1094a0 0x7fc9bc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c1a87640 1 -- 172.21.15.112:0/2972873342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9bc1a8680 con 0x7fc9bc109df0 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c0a85640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9bc1094a0 0x7fc9bc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c1a87640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9bc109df0 0x7fc9bc076190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400c970 tx=0x7fc9b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/2972873342 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4007bb0 con 0x7fc9bc109df0 2024-09-21T03:20:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/2972873342 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc9b4007d50 con 0x7fc9bc109df0 2024-09-21T03:20:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2972873342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9bc1a8970 con 0x7fc9bc109df0 2024-09-21T03:20:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/2972873342 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4004d60 con 0x7fc9bc109df0 2024-09-21T03:20:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.529+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2972873342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9bc1a8f00 con 0x7fc9bc109df0 2024-09-21T03:20:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.530+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2972873342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9bc103f70 con 0x7fc9bc109df0 2024-09-21T03:20:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.531+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/2972873342 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 103195+0+0 (secure 0 0 0) 0x7fc9b4005020 con 0x7fc9bc109df0 2024-09-21T03:20:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.531+0000 7fc9ba7fc640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc9a407b020 0x7fc9a407d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.532+0000 7fc9c1286640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc9a407b020 0x7fc9a407d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.532+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/2972873342 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc9b409c990 con 0x7fc9bc109df0 2024-09-21T03:20:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.532+0000 7fc9c1286640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc9a407b020 0x7fc9a407d530 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac006010 tx=0x7fc9ac031000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:45.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.536+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/2972873342 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9b4014030 con 0x7fc9bc109df0 2024-09-21T03:20:45.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.723+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2972873342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fc9bc10bbd0 con 0x7fc9bc109df0 2024-09-21T03:20:45.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:45 smithi112 ceph-mon[31442]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-21T03:20:45.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:45 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:20:45.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:45 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/4083752633' entity='client.admin' 2024-09-21T03:20:45.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.729+0000 7fc9ba7fc640 1 -- 172.21.15.112:0/2972873342 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v21)=0 v21) ==== 127+0+0 (secure 0 0 0) 0x7fc9b40693a0 con 0x7fc9bc109df0 2024-09-21T03:20:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2972873342 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc9a407b020 msgr2=0x7fc9a407d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc9a407b020 0x7fc9a407d530 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac006010 tx=0x7fc9ac031000 comp rx=0 tx=0).stop 2024-09-21T03:20:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2972873342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9bc109df0 msgr2=0x7fc9bc076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9bc109df0 0x7fc9bc076190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400c970 tx=0x7fc9b400ce30 comp rx=0 tx=0).stop 2024-09-21T03:20:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2972873342 shutdown_connections 2024-09-21T03:20:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc9a407b020 0x7fc9a407d530 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9bc109df0 0x7fc9bc076190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9bc1094a0 0x7fc9bc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 --2- 172.21.15.112:0/2972873342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9bc103b60 0x7fc9bc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:45.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2972873342 >> 172.21.15.112:0/2972873342 conn(0x7fc9bc07b850 msgr2=0x7fc9bc107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2972873342 shutdown_connections 2024-09-21T03:20:45.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:45.731+0000 7fc9c34ea640 1 -- 172.21.15.112:0/2972873342 wait complete. 2024-09-21T03:20:45.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:45 smithi203 ceph-mon[34723]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-21T03:20:45.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:45 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:20:45.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:45 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/4083752633' entity='client.admin' 2024-09-21T03:20:45.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:45 smithi187 ceph-mon[34603]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-21T03:20:45.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:45 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:20:45.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:45 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/4083752633' entity='client.admin' 2024-09-21T03:20:45.933 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi112') 2024-09-21T03:20:46.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.299+0000 7f4211253640 1 -- 172.21.15.112:0/2191847189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c105380 msgr2=0x7f420c10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:46.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.299+0000 7f4211253640 1 --2- 172.21.15.112:0/2191847189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c105380 0x7f420c10d8d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009900 tx=0x7f41f402a410 comp rx=0 tx=0).stop 2024-09-21T03:20:46.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.299+0000 7f4211253640 1 -- 172.21.15.112:0/2191847189 shutdown_connections 2024-09-21T03:20:46.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.299+0000 7f4211253640 1 --2- 172.21.15.112:0/2191847189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f420c10de60 0x7f420c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:46.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.299+0000 7f4211253640 1 --2- 172.21.15.112:0/2191847189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c105380 0x7f420c10d8d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:46.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.299+0000 7f4211253640 1 --2- 172.21.15.112:0/2191847189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c104a30 0x7f420c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:46.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.299+0000 7f4211253640 1 -- 172.21.15.112:0/2191847189 >> 172.21.15.112:0/2191847189 conn(0x7f420c0797b0 msgr2=0x7f420c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:46.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.299+0000 7f4211253640 1 -- 172.21.15.112:0/2191847189 shutdown_connections 2024-09-21T03:20:46.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.299+0000 7f4211253640 1 -- 172.21.15.112:0/2191847189 wait complete. 2024-09-21T03:20:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f4211253640 1 Processor -- start 2024-09-21T03:20:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f4211253640 1 -- start start 2024-09-21T03:20:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f4211253640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f420c104a30 0x7f420c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f4211253640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c105380 0x7f420c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f4211253640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c10de60 0x7f420c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:46.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f4211253640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f420c113980 con 0x7f420c105380 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f4211253640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f420c113800 con 0x7f420c104a30 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f4211253640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f420c113b00 con 0x7f420c10de60 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f420a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c105380 0x7f420c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f420a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c105380 0x7f420c19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41900/0 (socket says 172.21.15.112:41900) 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f420a59c640 1 -- 172.21.15.112:0/2389168026 learned_addr learned my addr 172.21.15.112:0/2389168026 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.300+0000 7f420b59e640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c10de60 0x7f420c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f420ad9d640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f420c104a30 0x7f420c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f420a59c640 1 -- 172.21.15.112:0/2389168026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c10de60 msgr2=0x7f420c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f420a59c640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c10de60 0x7f420c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f420a59c640 1 -- 172.21.15.112:0/2389168026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f420c104a30 msgr2=0x7f420c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f420a59c640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f420c104a30 0x7f420c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f420a59c640 1 -- 172.21.15.112:0/2389168026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c1a03a0 con 0x7f420c105380 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f420ad9d640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f420c104a30 0x7f420c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:20:46.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f420a59c640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c105380 0x7f420c19a9d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41f4005940 tx=0x7f41f40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f4203fff640 1 -- 172.21.15.112:0/2389168026 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f403f030 con 0x7f420c105380 2024-09-21T03:20:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f4203fff640 1 -- 172.21.15.112:0/2389168026 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f41f40044c0 con 0x7f420c105380 2024-09-21T03:20:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f4211253640 1 -- 172.21.15.112:0/2389168026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420c1a0630 con 0x7f420c105380 2024-09-21T03:20:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f4203fff640 1 -- 172.21.15.112:0/2389168026 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4035070 con 0x7f420c105380 2024-09-21T03:20:46.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.301+0000 7f4211253640 1 -- 172.21.15.112:0/2389168026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420c1a6df0 con 0x7f420c105380 2024-09-21T03:20:46.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.302+0000 7f4211253640 1 -- 172.21.15.112:0/2389168026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420c06bed0 con 0x7f420c105380 2024-09-21T03:20:46.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.303+0000 7f4203fff640 1 -- 172.21.15.112:0/2389168026 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 103195+0+0 (secure 0 0 0) 0x7f41f4042030 con 0x7f420c105380 2024-09-21T03:20:46.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.304+0000 7f4203fff640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41f807b020 0x7f41f807d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:46.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.304+0000 7f4203fff640 1 -- 172.21.15.112:0/2389168026 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f41f40bd290 con 0x7f420c105380 2024-09-21T03:20:46.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.304+0000 7f420ad9d640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41f807b020 0x7f41f807d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:46.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.304+0000 7f420ad9d640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41f807b020 0x7f41f807d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f41fc005fd0 tx=0x7f41fc009520 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:46.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.308+0000 7f4203fff640 1 -- 172.21.15.112:0/2389168026 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41f4089c20 con 0x7f420c105380 2024-09-21T03:20:46.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.491+0000 7f4211253640 1 -- 172.21.15.112:0/2389168026 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f420c108f40 con 0x7f420c105380 2024-09-21T03:20:46.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.495+0000 7f4203fff640 1 -- 172.21.15.112:0/2389168026 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v22)=0 v22) ==== 106+0+0 (secure 0 0 0) 0x7f41f408ead0 con 0x7f420c105380 2024-09-21T03:20:46.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.497+0000 7f4211253640 1 -- 172.21.15.112:0/2389168026 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41f807b020 msgr2=0x7f41f807d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:46.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.497+0000 7f4211253640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41f807b020 0x7f41f807d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f41fc005fd0 tx=0x7f41fc009520 comp rx=0 tx=0).stop 2024-09-21T03:20:46.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.497+0000 7f4211253640 1 -- 172.21.15.112:0/2389168026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c105380 msgr2=0x7f420c19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:46.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.497+0000 7f4211253640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c105380 0x7f420c19a9d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41f4005940 tx=0x7f41f40040c0 comp rx=0 tx=0).stop 2024-09-21T03:20:46.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.498+0000 7f4211253640 1 -- 172.21.15.112:0/2389168026 shutdown_connections 2024-09-21T03:20:46.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.498+0000 7f4211253640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41f807b020 0x7f41f807d530 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:46.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.498+0000 7f4211253640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c10de60 0x7f420c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:46.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.498+0000 7f4211253640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f420c105380 0x7f420c19a9d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:46.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.498+0000 7f4211253640 1 --2- 172.21.15.112:0/2389168026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f420c104a30 0x7f420c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:46.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.498+0000 7f4211253640 1 -- 172.21.15.112:0/2389168026 >> 172.21.15.112:0/2389168026 conn(0x7f420c0797b0 msgr2=0x7f420c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:46.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.498+0000 7f4211253640 1 -- 172.21.15.112:0/2389168026 shutdown_connections 2024-09-21T03:20:46.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:46.498+0000 7f4211253640 1 -- 172.21.15.112:0/2389168026 wait complete. 2024-09-21T03:20:46.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:46 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2972873342' entity='client.admin' 2024-09-21T03:20:46.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:46 smithi112 ceph-mon[31442]: Standby manager daemon y started 2024-09-21T03:20:46.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:46 smithi112 ceph-mon[31442]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:20:46.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:46 smithi112 ceph-mon[31442]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:20:46.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:46 smithi112 ceph-mon[31442]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:20:46.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:46 smithi112 ceph-mon[31442]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:20:46.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:46 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2389168026' entity='client.admin' 2024-09-21T03:20:47.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:46 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2972873342' entity='client.admin' 2024-09-21T03:20:47.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:46 smithi203 ceph-mon[34723]: Standby manager daemon y started 2024-09-21T03:20:47.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:46 smithi203 ceph-mon[34723]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:20:47.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:46 smithi203 ceph-mon[34723]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:20:47.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:46 smithi203 ceph-mon[34723]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:20:47.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:46 smithi203 ceph-mon[34723]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:20:47.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:46 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2389168026' entity='client.admin' 2024-09-21T03:20:47.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:46 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2972873342' entity='client.admin' 2024-09-21T03:20:47.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:46 smithi187 ceph-mon[34603]: Standby manager daemon y started 2024-09-21T03:20:47.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:46 smithi187 ceph-mon[34603]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:20:47.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:46 smithi187 ceph-mon[34603]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:20:47.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:46 smithi187 ceph-mon[34603]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:20:47.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:46 smithi187 ceph-mon[34603]: from='mgr.? 172.21.15.187:0/2389062037' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:20:47.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:46 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2389168026' entity='client.admin' 2024-09-21T03:20:47.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.432+0000 7f2e77751640 1 -- 172.21.15.112:0/1683795947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e701083a0 msgr2=0x7f2e701087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:47.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.432+0000 7f2e77751640 1 --2- 172.21.15.112:0/1683795947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e701083a0 0x7f2e701087b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009900 tx=0x7f2e6002a410 comp rx=0 tx=0).stop 2024-09-21T03:20:47.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.432+0000 7f2e77751640 1 -- 172.21.15.112:0/1683795947 shutdown_connections 2024-09-21T03:20:47.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.432+0000 7f2e77751640 1 --2- 172.21.15.112:0/1683795947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e70108cf0 0x7f2e7010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:47.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.432+0000 7f2e77751640 1 --2- 172.21.15.112:0/1683795947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e701083a0 0x7f2e701087b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:47.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.432+0000 7f2e77751640 1 --2- 172.21.15.112:0/1683795947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e70102a60 0x7f2e70102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:47.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.432+0000 7f2e77751640 1 -- 172.21.15.112:0/1683795947 >> 172.21.15.112:0/1683795947 conn(0x7f2e70079730 msgr2=0x7f2e70079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:47.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.432+0000 7f2e77751640 1 -- 172.21.15.112:0/1683795947 shutdown_connections 2024-09-21T03:20:47.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.432+0000 7f2e77751640 1 -- 172.21.15.112:0/1683795947 wait complete. 2024-09-21T03:20:47.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.432+0000 7f2e77751640 1 Processor -- start 2024-09-21T03:20:47.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e77751640 1 -- start start 2024-09-21T03:20:47.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e77751640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e70102a60 0x7f2e701993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:47.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e77751640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e701083a0 0x7f2e70199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:47.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e77751640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e70108cf0 0x7f2e7019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:47.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e77751640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70112880 con 0x7f2e701083a0 2024-09-21T03:20:47.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e77751640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70112700 con 0x7f2e70102a60 2024-09-21T03:20:47.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e77751640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70112a00 con 0x7f2e70108cf0 2024-09-21T03:20:47.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e74cec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e701083a0 0x7f2e70199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e74cec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e701083a0 0x7f2e70199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41904/0 (socket says 172.21.15.112:41904) 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e74cec640 1 -- 172.21.15.112:0/618519907 learned_addr learned my addr 172.21.15.112:0/618519907 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e754ed640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e70102a60 0x7f2e701993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e75cee640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e70108cf0 0x7f2e7019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e74cec640 1 -- 172.21.15.112:0/618519907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e70108cf0 msgr2=0x7f2e7019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e74cec640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e70108cf0 0x7f2e7019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e74cec640 1 -- 172.21.15.112:0/618519907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e70102a60 msgr2=0x7f2e701993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e74cec640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e70102a60 0x7f2e701993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e74cec640 1 -- 172.21.15.112:0/618519907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7019f320 con 0x7f2e701083a0 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.433+0000 7f2e74cec640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e701083a0 0x7f2e70199950 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e60005940 tx=0x7f2e600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.434+0000 7f2e667fc640 1 -- 172.21.15.112:0/618519907 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6003f030 con 0x7f2e701083a0 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.434+0000 7f2e667fc640 1 -- 172.21.15.112:0/618519907 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2e60031800 con 0x7f2e701083a0 2024-09-21T03:20:47.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.434+0000 7f2e667fc640 1 -- 172.21.15.112:0/618519907 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60035070 con 0x7f2e701083a0 2024-09-21T03:20:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.434+0000 7f2e77751640 1 -- 172.21.15.112:0/618519907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7019f5b0 con 0x7f2e701083a0 2024-09-21T03:20:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.434+0000 7f2e77751640 1 -- 172.21.15.112:0/618519907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7006ae60 con 0x7f2e701083a0 2024-09-21T03:20:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.435+0000 7f2e77751640 1 -- 172.21.15.112:0/618519907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e70103ce0 con 0x7f2e701083a0 2024-09-21T03:20:47.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.436+0000 7f2e667fc640 1 -- 172.21.15.112:0/618519907 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f2e60042030 con 0x7f2e701083a0 2024-09-21T03:20:47.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.437+0000 7f2e667fc640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2e5c0b6550 0x7f2e5c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:47.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.437+0000 7f2e754ed640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2e5c0b6550 0x7f2e5c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:47.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.437+0000 7f2e667fc640 1 -- 172.21.15.112:0/618519907 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f2e60106040 con 0x7f2e701083a0 2024-09-21T03:20:47.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.438+0000 7f2e754ed640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2e5c0b6550 0x7f2e5c0b8a60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c005fd0 tx=0x7f2e6c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:47.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.440+0000 7f2e667fc640 1 -- 172.21.15.112:0/618519907 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2e600ce4d0 con 0x7f2e701083a0 2024-09-21T03:20:47.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.626+0000 7f2e77751640 1 -- 172.21.15.112:0/618519907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f2e70102e70 con 0x7f2e701083a0 2024-09-21T03:20:47.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.631+0000 7f2e667fc640 1 -- 172.21.15.112:0/618519907 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) ==== 138+0+0 (secure 0 0 0) 0x7f2e70102e70 con 0x7f2e701083a0 2024-09-21T03:20:47.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 -- 172.21.15.112:0/618519907 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2e5c0b6550 msgr2=0x7f2e5c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:47.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2e5c0b6550 0x7f2e5c0b8a60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c005fd0 tx=0x7f2e6c009500 comp rx=0 tx=0).stop 2024-09-21T03:20:47.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 -- 172.21.15.112:0/618519907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e701083a0 msgr2=0x7f2e70199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:47.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e701083a0 0x7f2e70199950 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e60005940 tx=0x7f2e600042f0 comp rx=0 tx=0).stop 2024-09-21T03:20:47.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 -- 172.21.15.112:0/618519907 shutdown_connections 2024-09-21T03:20:47.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2e5c0b6550 0x7f2e5c0b8a60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:47.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e70108cf0 0x7f2e7019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:47.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e701083a0 0x7f2e70199950 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:47.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 --2- 172.21.15.112:0/618519907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e70102a60 0x7f2e701993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:47.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 -- 172.21.15.112:0/618519907 >> 172.21.15.112:0/618519907 conn(0x7f2e70079730 msgr2=0x7f2e70104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:47.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 -- 172.21.15.112:0/618519907 shutdown_connections 2024-09-21T03:20:47.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:47.633+0000 7f2e77751640 1 -- 172.21.15.112:0/618519907 wait complete. 2024-09-21T03:20:48.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:47 smithi112 ceph-mon[31442]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-21T03:20:48.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:20:48.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:47 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/618519907' entity='client.admin' 2024-09-21T03:20:48.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:47 smithi203 ceph-mon[34723]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-21T03:20:48.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:20:48.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:47 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/618519907' entity='client.admin' 2024-09-21T03:20:48.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:47 smithi187 ceph-mon[34603]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-21T03:20:48.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:20:48.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:47 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/618519907' entity='client.admin' 2024-09-21T03:20:49.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.028+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/672231396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8410de10 msgr2=0x7f4f84110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:49.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.028+0000 7f4f8a4fd640 1 --2- 172.21.15.112:0/672231396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8410de10 0x7f4f84110300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009ca0 tx=0x7f4f7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.028+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/672231396 shutdown_connections 2024-09-21T03:20:49.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.028+0000 7f4f8a4fd640 1 --2- 172.21.15.112:0/672231396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f8410de10 0x7f4f84110300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.028+0000 7f4f8a4fd640 1 --2- 172.21.15.112:0/672231396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84105300 0x7f4f8410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.028+0000 7f4f8a4fd640 1 --2- 172.21.15.112:0/672231396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f841049b0 0x7f4f84104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.028+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/672231396 >> 172.21.15.112:0/672231396 conn(0x7f4f84079730 msgr2=0x7f4f84079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:49.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.028+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/672231396 shutdown_connections 2024-09-21T03:20:49.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.029+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/672231396 wait complete. 2024-09-21T03:20:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.029+0000 7f4f8a4fd640 1 Processor -- start 2024-09-21T03:20:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.029+0000 7f4f8a4fd640 1 -- start start 2024-09-21T03:20:49.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f8a4fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841049b0 0x7f4f8419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:49.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f8a4fd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84105300 0x7f4f8419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:49.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f8a4fd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f8410de10 0x7f4f8419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:49.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f8a4fd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84113950 con 0x7f4f841049b0 2024-09-21T03:20:49.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f8a4fd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841137d0 con 0x7f4f8410de10 2024-09-21T03:20:49.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f8a4fd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84113ad0 con 0x7f4f84105300 2024-09-21T03:20:49.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f83fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841049b0 0x7f4f8419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f83fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841049b0 0x7f4f8419a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41928/0 (socket says 172.21.15.112:41928) 2024-09-21T03:20:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f83fff640 1 -- 172.21.15.112:0/65637924 learned_addr learned my addr 172.21.15.112:0/65637924 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f837fe640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84105300 0x7f4f8419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f88a9a640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f8410de10 0x7f4f8419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f83fff640 1 -- 172.21.15.112:0/65637924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84105300 msgr2=0x7f4f8419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f83fff640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84105300 0x7f4f8419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f83fff640 1 -- 172.21.15.112:0/65637924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f8410de10 msgr2=0x7f4f8419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f83fff640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f8410de10 0x7f4f8419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f83fff640 1 -- 172.21.15.112:0/65637924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f841a02f0 con 0x7f4f841049b0 2024-09-21T03:20:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f88a9a640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f8410de10 0x7f4f8419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:20:49.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f83fff640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841049b0 0x7f4f8419a380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400e9c0 tx=0x7f4f7400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.030+0000 7f4f817fa640 1 -- 172.21.15.112:0/65637924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7400cd80 con 0x7f4f841049b0 2024-09-21T03:20:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.031+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/65637924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f841a05e0 con 0x7f4f841049b0 2024-09-21T03:20:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.031+0000 7f4f817fa640 1 -- 172.21.15.112:0/65637924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4f740044b0 con 0x7f4f841049b0 2024-09-21T03:20:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.031+0000 7f4f817fa640 1 -- 172.21.15.112:0/65637924 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74010430 con 0x7f4f841049b0 2024-09-21T03:20:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.031+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/65637924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f841a6d40 con 0x7f4f841049b0 2024-09-21T03:20:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.032+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/65637924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f84104dc0 con 0x7f4f841049b0 2024-09-21T03:20:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.033+0000 7f4f817fa640 1 -- 172.21.15.112:0/65637924 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f4f74003f10 con 0x7f4f841049b0 2024-09-21T03:20:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.034+0000 7f4f817fa640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f6c0b64d0 0x7f4f6c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.034+0000 7f4f837fe640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f6c0b64d0 0x7f4f6c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.034+0000 7f4f817fa640 1 -- 172.21.15.112:0/65637924 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4f740e5b80 con 0x7f4f841049b0 2024-09-21T03:20:49.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.034+0000 7f4f837fe640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f6c0b64d0 0x7f4f6c0b89e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f84102450 tx=0x7f4f7800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:49.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.038+0000 7f4f817fa640 1 -- 172.21.15.112:0/65637924 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4f740ae010 con 0x7f4f841049b0 2024-09-21T03:20:49.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.222+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/65637924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f4f8419b4b0 con 0x7f4f841049b0 2024-09-21T03:20:49.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.236+0000 7f4f817fa640 1 -- 172.21.15.112:0/65637924 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) ==== 133+0+0 (secure 0 0 0) 0x7f4f740b23f0 con 0x7f4f841049b0 2024-09-21T03:20:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/65637924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f6c0b64d0 msgr2=0x7f4f6c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f6c0b64d0 0x7f4f6c0b89e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f84102450 tx=0x7f4f7800b390 comp rx=0 tx=0).stop 2024-09-21T03:20:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/65637924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841049b0 msgr2=0x7f4f8419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841049b0 0x7f4f8419a380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400e9c0 tx=0x7f4f7400ee80 comp rx=0 tx=0).stop 2024-09-21T03:20:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/65637924 shutdown_connections 2024-09-21T03:20:49.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f6c0b64d0 0x7f4f6c0b89e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f8410de10 0x7f4f8419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f84105300 0x7f4f8419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 --2- 172.21.15.112:0/65637924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f841049b0 0x7f4f8419a380 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/65637924 >> 172.21.15.112:0/65637924 conn(0x7f4f84079730 msgr2=0x7f4f84100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/65637924 shutdown_connections 2024-09-21T03:20:49.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.239+0000 7f4f8a4fd640 1 -- 172.21.15.112:0/65637924 wait complete. 2024-09-21T03:20:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.502+0000 7f8a67328640 1 -- 172.21.15.112:0/2109925782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a60103b80 msgr2=0x7f8a60103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.502+0000 7f8a67328640 1 --2- 172.21.15.112:0/2109925782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a60103b80 0x7f8a60103f90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009900 tx=0x7f8a5402a410 comp rx=0 tx=0).stop 2024-09-21T03:20:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.502+0000 7f8a67328640 1 -- 172.21.15.112:0/2109925782 shutdown_connections 2024-09-21T03:20:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.502+0000 7f8a67328640 1 --2- 172.21.15.112:0/2109925782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a60109e10 0x7f8a60110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.502+0000 7f8a67328640 1 --2- 172.21.15.112:0/2109925782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a601094c0 0x7f8a601098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.502+0000 7f8a67328640 1 --2- 172.21.15.112:0/2109925782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a60103b80 0x7f8a60103f90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.502+0000 7f8a67328640 1 -- 172.21.15.112:0/2109925782 >> 172.21.15.112:0/2109925782 conn(0x7f8a600ff5b0 msgr2=0x7f8a60101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.502+0000 7f8a67328640 1 -- 172.21.15.112:0/2109925782 shutdown_connections 2024-09-21T03:20:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.502+0000 7f8a67328640 1 -- 172.21.15.112:0/2109925782 wait complete. 2024-09-21T03:20:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.503+0000 7f8a67328640 1 Processor -- start 2024-09-21T03:20:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.503+0000 7f8a67328640 1 -- start start 2024-09-21T03:20:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.503+0000 7f8a67328640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a60103b80 0x7f8a6019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:49.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.503+0000 7f8a67328640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a601094c0 0x7f8a6019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:49.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.503+0000 7f8a67328640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a60109e10 0x7f8a6019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:49.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.503+0000 7f8a67328640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a60113a00 con 0x7f8a60103b80 2024-09-21T03:20:49.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.503+0000 7f8a67328640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a60113880 con 0x7f8a60109e10 2024-09-21T03:20:49.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.503+0000 7f8a67328640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8a60113b80 con 0x7f8a601094c0 2024-09-21T03:20:49.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a650c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a60103b80 0x7f8a6019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a650c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a60103b80 0x7f8a6019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41974/0 (socket says 172.21.15.112:41974) 2024-09-21T03:20:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a650c4640 1 -- 172.21.15.112:0/1679515836 learned_addr learned my addr 172.21.15.112:0/1679515836 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a648c3640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a601094c0 0x7f8a6019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a658c5640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a60109e10 0x7f8a6019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a650c4640 1 -- 172.21.15.112:0/1679515836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a601094c0 msgr2=0x7f8a6019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a650c4640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a601094c0 0x7f8a6019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a650c4640 1 -- 172.21.15.112:0/1679515836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a60109e10 msgr2=0x7f8a6019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:49.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a650c4640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a60109e10 0x7f8a6019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a650c4640 1 -- 172.21.15.112:0/1679515836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a601a02a0 con 0x7f8a60103b80 2024-09-21T03:20:49.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a650c4640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a60103b80 0x7f8a6019a3f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8a54004770 tx=0x7f8a54003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:49.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a527fc640 1 -- 172.21.15.112:0/1679515836 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5403f070 con 0x7f8a60103b80 2024-09-21T03:20:49.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a527fc640 1 -- 172.21.15.112:0/1679515836 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8a5403bd00 con 0x7f8a60103b80 2024-09-21T03:20:49.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a67328640 1 -- 172.21.15.112:0/1679515836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a601a0530 con 0x7f8a60103b80 2024-09-21T03:20:49.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a527fc640 1 -- 172.21.15.112:0/1679515836 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54035030 con 0x7f8a60103b80 2024-09-21T03:20:49.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.504+0000 7f8a67328640 1 -- 172.21.15.112:0/1679515836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a601a0a60 con 0x7f8a60103b80 2024-09-21T03:20:49.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.505+0000 7f8a67328640 1 -- 172.21.15.112:0/1679515836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a60104e00 con 0x7f8a60103b80 2024-09-21T03:20:49.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.507+0000 7f8a527fc640 1 -- 172.21.15.112:0/1679515836 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f8a54042070 con 0x7f8a60103b80 2024-09-21T03:20:49.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.508+0000 7f8a527fc640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8a4c0b6580 0x7f8a4c0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:49.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.508+0000 7f8a648c3640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8a4c0b6580 0x7f8a4c0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:49.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.508+0000 7f8a527fc640 1 -- 172.21.15.112:0/1679515836 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8a54106390 con 0x7f8a60103b80 2024-09-21T03:20:49.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.509+0000 7f8a648c3640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8a4c0b6580 0x7f8a4c0b8a90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a60109050 tx=0x7f8a5c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:49.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.511+0000 7f8a527fc640 1 -- 172.21.15.112:0/1679515836 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8a601a0a60 con 0x7f8a60103b80 2024-09-21T03:20:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.709+0000 7f8a67328640 1 -- 172.21.15.112:0/1679515836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f8a6019b520 con 0x7f8a60103b80 2024-09-21T03:20:49.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.715+0000 7f8a527fc640 1 -- 172.21.15.112:0/1679515836 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v26)=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f8a540d2be0 con 0x7f8a60103b80 2024-09-21T03:20:49.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 -- 172.21.15.112:0/1679515836 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8a4c0b6580 msgr2=0x7f8a4c0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:49.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8a4c0b6580 0x7f8a4c0b8a90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a60109050 tx=0x7f8a5c00b390 comp rx=0 tx=0).stop 2024-09-21T03:20:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 -- 172.21.15.112:0/1679515836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a60103b80 msgr2=0x7f8a6019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a60103b80 0x7f8a6019a3f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8a54004770 tx=0x7f8a54003ca0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 -- 172.21.15.112:0/1679515836 shutdown_connections 2024-09-21T03:20:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8a4c0b6580 0x7f8a4c0b8a90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a60109e10 0x7f8a6019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a601094c0 0x7f8a6019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 --2- 172.21.15.112:0/1679515836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a60103b80 0x7f8a6019a3f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 -- 172.21.15.112:0/1679515836 >> 172.21.15.112:0/1679515836 conn(0x7f8a600ff5b0 msgr2=0x7f8a601019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 -- 172.21.15.112:0/1679515836 shutdown_connections 2024-09-21T03:20:49.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.718+0000 7f8a67328640 1 -- 172.21.15.112:0/1679515836 wait complete. 2024-09-21T03:20:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.125+0000 7f08b5776640 1 -- 172.21.15.112:0/2096134123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08b0102a60 msgr2=0x7f08b0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.125+0000 7f08b5776640 1 --2- 172.21.15.112:0/2096134123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08b0102a60 0x7f08b0102e70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009900 tx=0x7f08a002a410 comp rx=0 tx=0).stop 2024-09-21T03:20:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.125+0000 7f08b5776640 1 -- 172.21.15.112:0/2096134123 shutdown_connections 2024-09-21T03:20:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.125+0000 7f08b5776640 1 --2- 172.21.15.112:0/2096134123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08b0108cf0 0x7f08b010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.125+0000 7f08b5776640 1 --2- 172.21.15.112:0/2096134123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b01083a0 0x7f08b01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.125+0000 7f08b5776640 1 --2- 172.21.15.112:0/2096134123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08b0102a60 0x7f08b0102e70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.125+0000 7f08b5776640 1 -- 172.21.15.112:0/2096134123 >> 172.21.15.112:0/2096134123 conn(0x7f08b0079730 msgr2=0x7f08b0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:50.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.125+0000 7f08b5776640 1 -- 172.21.15.112:0/2096134123 shutdown_connections 2024-09-21T03:20:50.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.125+0000 7f08b5776640 1 -- 172.21.15.112:0/2096134123 wait complete. 2024-09-21T03:20:50.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08b5776640 1 Processor -- start 2024-09-21T03:20:50.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08b5776640 1 -- start start 2024-09-21T03:20:50.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08b5776640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08b0102a60 0x7f08b0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08b5776640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b01083a0 0x7f08b0199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08b5776640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08b0108cf0 0x7f08b019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08b5776640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0112840 con 0x7f08b0108cf0 2024-09-21T03:20:50.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08b5776640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01126c0 con 0x7f08b0102a60 2024-09-21T03:20:50.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08b5776640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01129c0 con 0x7f08b01083a0 2024-09-21T03:20:50.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08b0108cf0 0x7f08b019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08b0108cf0 0x7f08b019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42004/0 (socket says 172.21.15.112:42004) 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08af7fe640 1 -- 172.21.15.112:0/2376410651 learned_addr learned my addr 172.21.15.112:0/2376410651 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08aeffd640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08b0102a60 0x7f08b0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.126+0000 7f08ae7fc640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b01083a0 0x7f08b0199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f08af7fe640 1 -- 172.21.15.112:0/2376410651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b01083a0 msgr2=0x7f08b0199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f08af7fe640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b01083a0 0x7f08b0199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f08af7fe640 1 -- 172.21.15.112:0/2376410651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08b0102a60 msgr2=0x7f08b0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f08af7fe640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08b0102a60 0x7f08b0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f08af7fe640 1 -- 172.21.15.112:0/2376410651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b019f1e0 con 0x7f08b0108cf0 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f08ae7fc640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b01083a0 0x7f08b0199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f08af7fe640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08b0108cf0 0x7f08b019eab0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f08a800eaa0 tx=0x7f08a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f0897fff640 1 -- 172.21.15.112:0/2376410651 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a800cdb0 con 0x7f08b0108cf0 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f08b5776640 1 -- 172.21.15.112:0/2376410651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b019f4d0 con 0x7f08b0108cf0 2024-09-21T03:20:50.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f0897fff640 1 -- 172.21.15.112:0/2376410651 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f08a8005020 con 0x7f08b0108cf0 2024-09-21T03:20:50.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f0897fff640 1 -- 172.21.15.112:0/2376410651 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a80116d0 con 0x7f08b0108cf0 2024-09-21T03:20:50.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.127+0000 7f08b5776640 1 -- 172.21.15.112:0/2376410651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b01a5c30 con 0x7f08b0108cf0 2024-09-21T03:20:50.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.128+0000 7f08b5776640 1 -- 172.21.15.112:0/2376410651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b0102e70 con 0x7f08b0108cf0 2024-09-21T03:20:50.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.129+0000 7f0897fff640 1 -- 172.21.15.112:0/2376410651 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f08a8004a40 con 0x7f08b0108cf0 2024-09-21T03:20:50.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.130+0000 7f0897fff640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08980b64d0 0x7f08980b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.131+0000 7f08aeffd640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08980b64d0 0x7f08980b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.131+0000 7f0897fff640 1 -- 172.21.15.112:0/2376410651 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f08a80e5d70 con 0x7f08b0108cf0 2024-09-21T03:20:50.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.131+0000 7f08aeffd640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08980b64d0 0x7f08980b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009230 tx=0x7f08a00058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:50.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.134+0000 7f0897fff640 1 -- 172.21.15.112:0/2376410651 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f08a8016030 con 0x7f08b0108cf0 2024-09-21T03:20:50.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:50 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/65637924' entity='client.admin' 2024-09-21T03:20:50.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:50 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3212011703' entity='client.admin' 2024-09-21T03:20:50.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:50 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1679515836' entity='client.admin' 2024-09-21T03:20:50.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.323+0000 7f08b5776640 1 -- 172.21.15.112:0/2376410651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f08b010aaf0 con 0x7f08b0108cf0 2024-09-21T03:20:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.329+0000 7f0897fff640 1 -- 172.21.15.112:0/2376410651 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v27)=0 v27) ==== 120+0+0 (secure 0 0 0) 0x7f08a80ae160 con 0x7f08b0108cf0 2024-09-21T03:20:50.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7f08b5776640 1 -- 172.21.15.112:0/2376410651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08980b64d0 msgr2=0x7f08980b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7f08b5776640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08980b64d0 0x7f08980b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009230 tx=0x7f08a00058d0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7f08b5776640 1 -- 172.21.15.112:0/2376410651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08b0108cf0 msgr2=0x7f08b019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7f08b5776640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08b0108cf0 0x7f08b019eab0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f08a800eaa0 tx=0x7f08a800ef60 comp rx=0 tx=0).stop 2024-09-21T03:20:50.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7f08b5776640 1 -- 172.21.15.112:0/2376410651 shutdown_connections 2024-09-21T03:20:50.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7f08b5776640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08980b64d0 0x7f08980b89e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7f08b5776640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08b0108cf0 0x7f08b019eab0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7f08b5776640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b01083a0 0x7f08b0199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7f08b5776640 1 --2- 172.21.15.112:0/2376410651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08b0102a60 0x7f08b0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7f08b5776640 1 -- 172.21.15.112:0/2376410651 >> 172.21.15.112:0/2376410651 conn(0x7f08b0079730 msgr2=0x7f08b0101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.333+0000 7f08b5776640 1 -- 172.21.15.112:0/2376410651 shutdown_connections 2024-09-21T03:20:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.333+0000 7f08b5776640 1 -- 172.21.15.112:0/2376410651 wait complete. 2024-09-21T03:20:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7efd4b87b640 1 -- 172.21.15.112:0/858500334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd44109e00 msgr2=0x7efd44110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7efd4b87b640 1 --2- 172.21.15.112:0/858500334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd44109e00 0x7efd44110630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009ca0 tx=0x7efd3c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7efd4b87b640 1 -- 172.21.15.112:0/858500334 shutdown_connections 2024-09-21T03:20:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7efd4b87b640 1 --2- 172.21.15.112:0/858500334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd44109e00 0x7efd44110630 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7efd4b87b640 1 --2- 172.21.15.112:0/858500334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd441094b0 0x7efd441098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7efd4b87b640 1 --2- 172.21.15.112:0/858500334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd44101f70 0x7efd44102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7efd4b87b640 1 -- 172.21.15.112:0/858500334 >> 172.21.15.112:0/858500334 conn(0x7efd44077c60 msgr2=0x7efd440780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7efd4b87b640 1 -- 172.21.15.112:0/858500334 shutdown_connections 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.332+0000 7efd4b87b640 1 -- 172.21.15.112:0/858500334 wait complete. 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.333+0000 7efd4b87b640 1 Processor -- start 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.333+0000 7efd4b87b640 1 -- start start 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.333+0000 7efd4b87b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd44101f70 0x7efd4419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.333+0000 7efd4b87b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd441094b0 0x7efd4419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.333+0000 7efd4b87b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd44109e00 0x7efd4419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.333+0000 7efd4b87b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd441139a0 con 0x7efd44109e00 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.333+0000 7efd4b87b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd44113820 con 0x7efd441094b0 2024-09-21T03:20:50.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.333+0000 7efd4b87b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efd44113b20 con 0x7efd44101f70 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd49e18640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd44109e00 0x7efd4419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd49e18640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd44109e00 0x7efd4419fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42024/0 (socket says 172.21.15.112:42024) 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd49e18640 1 -- 172.21.15.112:0/2453572676 learned_addr learned my addr 172.21.15.112:0/2453572676 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd48e16640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd441094b0 0x7efd4419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd49617640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd44101f70 0x7efd4419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd49e18640 1 -- 172.21.15.112:0/2453572676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd44101f70 msgr2=0x7efd4419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd49e18640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd44101f70 0x7efd4419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd49e18640 1 -- 172.21.15.112:0/2453572676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd441094b0 msgr2=0x7efd4419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd49e18640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd441094b0 0x7efd4419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd49e18640 1 -- 172.21.15.112:0/2453572676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd441a0280 con 0x7efd44109e00 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd49e18640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd44109e00 0x7efd4419fb50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd3c005e30 tx=0x7efd3c009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd3a7fc640 1 -- 172.21.15.112:0/2453572676 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c003c30 con 0x7efd44109e00 2024-09-21T03:20:50.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd3a7fc640 1 -- 172.21.15.112:0/2453572676 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efd3c003dd0 con 0x7efd44109e00 2024-09-21T03:20:50.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd3a7fc640 1 -- 172.21.15.112:0/2453572676 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c004550 con 0x7efd44109e00 2024-09-21T03:20:50.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd4b87b640 1 -- 172.21.15.112:0/2453572676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd441a0510 con 0x7efd44109e00 2024-09-21T03:20:50.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.334+0000 7efd4b87b640 1 -- 172.21.15.112:0/2453572676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd441a6cd0 con 0x7efd44109e00 2024-09-21T03:20:50.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.335+0000 7efd4b87b640 1 -- 172.21.15.112:0/2453572676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd44102380 con 0x7efd44109e00 2024-09-21T03:20:50.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.337+0000 7efd3a7fc640 1 -- 172.21.15.112:0/2453572676 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7efd3c004710 con 0x7efd44109e00 2024-09-21T03:20:50.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.337+0000 7efd3a7fc640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd300b64d0 0x7efd300b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:50.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.338+0000 7efd49617640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd300b64d0 0x7efd300b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:50.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.338+0000 7efd3a7fc640 1 -- 172.21.15.112:0/2453572676 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7efd3c106760 con 0x7efd44109e00 2024-09-21T03:20:50.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.338+0000 7efd49617640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd300b64d0 0x7efd300b89e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efd40009bd0 tx=0x7efd40005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:50.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.341+0000 7efd3a7fc640 1 -- 172.21.15.112:0/2453572676 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efd3c0cebf0 con 0x7efd44109e00 2024-09-21T03:20:50.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.526+0000 7efd4b87b640 1 -- 172.21.15.112:0/2453572676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7efd4419b440 con 0x7efd44109e00 2024-09-21T03:20:50.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.532+0000 7efd3a7fc640 1 -- 172.21.15.112:0/2453572676 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v28)=0 v28) ==== 125+0+0 (secure 0 0 0) 0x7efd4419b440 con 0x7efd44109e00 2024-09-21T03:20:50.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 -- 172.21.15.112:0/2453572676 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd300b64d0 msgr2=0x7efd300b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd300b64d0 0x7efd300b89e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efd40009bd0 tx=0x7efd40005f90 comp rx=0 tx=0).stop 2024-09-21T03:20:50.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 -- 172.21.15.112:0/2453572676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd44109e00 msgr2=0x7efd4419fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:50.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd44109e00 0x7efd4419fb50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd3c005e30 tx=0x7efd3c009870 comp rx=0 tx=0).stop 2024-09-21T03:20:50.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 -- 172.21.15.112:0/2453572676 shutdown_connections 2024-09-21T03:20:50.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd300b64d0 0x7efd300b89e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd44109e00 0x7efd4419fb50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd441094b0 0x7efd4419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 --2- 172.21.15.112:0/2453572676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd44101f70 0x7efd4419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:50.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 -- 172.21.15.112:0/2453572676 >> 172.21.15.112:0/2453572676 conn(0x7efd44077c60 msgr2=0x7efd44107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:50.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 -- 172.21.15.112:0/2453572676 shutdown_connections 2024-09-21T03:20:50.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:50.535+0000 7efd4b87b640 1 -- 172.21.15.112:0/2453572676 wait complete. 2024-09-21T03:20:50.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:50 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/65637924' entity='client.admin' 2024-09-21T03:20:50.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:50 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3212011703' entity='client.admin' 2024-09-21T03:20:50.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:50 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1679515836' entity='client.admin' 2024-09-21T03:20:50.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:50 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/65637924' entity='client.admin' 2024-09-21T03:20:50.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:50 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3212011703' entity='client.admin' 2024-09-21T03:20:50.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:50 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1679515836' entity='client.admin' 2024-09-21T03:20:51.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.405+0000 7f920f920640 1 -- 172.21.15.112:0/3930149196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9208102a90 msgr2=0x7f9208102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:51.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.405+0000 7f920f920640 1 --2- 172.21.15.112:0/3930149196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9208102a90 0x7f9208102ea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009ca0 tx=0x7f91f802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.405+0000 7f920f920640 1 -- 172.21.15.112:0/3930149196 shutdown_connections 2024-09-21T03:20:51.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.405+0000 7f920f920640 1 --2- 172.21.15.112:0/3930149196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9208108d20 0x7f920810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.405+0000 7f920f920640 1 --2- 172.21.15.112:0/3930149196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92081083d0 0x7f92081087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.405+0000 7f920f920640 1 --2- 172.21.15.112:0/3930149196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9208102a90 0x7f9208102ea0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.405+0000 7f920f920640 1 -- 172.21.15.112:0/3930149196 >> 172.21.15.112:0/3930149196 conn(0x7f9208079730 msgr2=0x7f9208079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.405+0000 7f920f920640 1 -- 172.21.15.112:0/3930149196 shutdown_connections 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.406+0000 7f920f920640 1 -- 172.21.15.112:0/3930149196 wait complete. 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.406+0000 7f920f920640 1 Processor -- start 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.406+0000 7f920f920640 1 -- start start 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.406+0000 7f920f920640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9208102a90 0x7f9208199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.406+0000 7f920f920640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92081083d0 0x7f92081998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920f920640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9208108d20 0x7f920819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920f920640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92081128c0 con 0x7f9208102a90 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920f920640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9208112740 con 0x7f92081083d0 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920f920640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9208112a40 con 0x7f9208108d20 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920d6bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9208102a90 0x7f9208199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:51.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920d6bc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9208102a90 0x7f9208199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41948/0 (socket says 172.21.15.112:41948) 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920d6bc640 1 -- 172.21.15.112:0/3212011703 learned_addr learned my addr 172.21.15.112:0/3212011703 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920debd640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9208108d20 0x7f920819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920cebb640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92081083d0 0x7f92081998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920d6bc640 1 -- 172.21.15.112:0/3212011703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9208108d20 msgr2=0x7f920819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920d6bc640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9208108d20 0x7f920819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920d6bc640 1 -- 172.21.15.112:0/3212011703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92081083d0 msgr2=0x7f92081998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920d6bc640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92081083d0 0x7f92081998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920d6bc640 1 -- 172.21.15.112:0/3212011703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920819f2c0 con 0x7f9208102a90 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f920d6bc640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9208102a90 0x7f9208199350 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f91f8005e30 tx=0x7f91f80307f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.407+0000 7f92067fc640 1 -- 172.21.15.112:0/3212011703 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8031830 con 0x7f9208102a90 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.408+0000 7f920f920640 1 -- 172.21.15.112:0/3212011703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920819f550 con 0x7f9208102a90 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.408+0000 7f92067fc640 1 -- 172.21.15.112:0/3212011703 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f91f803b890 con 0x7f9208102a90 2024-09-21T03:20:51.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.408+0000 7f92067fc640 1 -- 172.21.15.112:0/3212011703 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f803a770 con 0x7f9208102a90 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.408+0000 7f920f920640 1 -- 172.21.15.112:0/3212011703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920806bb80 con 0x7f9208102a90 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.409+0000 7f920f920640 1 -- 172.21.15.112:0/3212011703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9208103d10 con 0x7f9208102a90 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.410+0000 7f92067fc640 1 -- 172.21.15.112:0/3212011703 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f91f803a910 con 0x7f9208102a90 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.411+0000 7f92067fc640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91f00b64d0 0x7f91f00b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.411+0000 7f920cebb640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91f00b64d0 0x7f91f00b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.411+0000 7f92067fc640 1 -- 172.21.15.112:0/3212011703 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f91f8106a30 con 0x7f9208102a90 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.411+0000 7f920cebb640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91f00b64d0 0x7f91f00b89e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9208079f40 tx=0x7f91fc00f7a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.419+0000 7f92067fc640 1 -- 172.21.15.112:0/3212011703 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91f80ceec0 con 0x7f9208102a90 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.612+0000 7f920f920640 1 -- 172.21.15.112:0/3212011703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f920806c5a0 con 0x7f9208102a90 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.618+0000 7f92067fc640 1 -- 172.21.15.112:0/3212011703 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v25)=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f91f80098b0 con 0x7f9208102a90 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.620+0000 7f920f920640 1 -- 172.21.15.112:0/3212011703 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91f00b64d0 msgr2=0x7f91f00b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:51.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.620+0000 7f920f920640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91f00b64d0 0x7f91f00b89e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9208079f40 tx=0x7f91fc00f7a0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.620+0000 7f920f920640 1 -- 172.21.15.112:0/3212011703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9208102a90 msgr2=0x7f9208199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:51.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.620+0000 7f920f920640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9208102a90 0x7f9208199350 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f91f8005e30 tx=0x7f91f80307f0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.620+0000 7f920f920640 1 -- 172.21.15.112:0/3212011703 shutdown_connections 2024-09-21T03:20:51.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.620+0000 7f920f920640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91f00b64d0 0x7f91f00b89e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.620+0000 7f920f920640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9208108d20 0x7f920819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.620+0000 7f920f920640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92081083d0 0x7f92081998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.620+0000 7f920f920640 1 --2- 172.21.15.112:0/3212011703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9208102a90 0x7f9208199350 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:51.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.620+0000 7f920f920640 1 -- 172.21.15.112:0/3212011703 >> 172.21.15.112:0/3212011703 conn(0x7f9208079730 msgr2=0x7f9208106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:51.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.621+0000 7f920f920640 1 -- 172.21.15.112:0/3212011703 shutdown_connections 2024-09-21T03:20:51.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:49.621+0000 7f920f920640 1 -- 172.21.15.112:0/3212011703 wait complete. 2024-09-21T03:20:51.383 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:51.383 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:51.383 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:51.469 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:51 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2376410651' entity='client.admin' 2024-09-21T03:20:51.469 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:51 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2453572676' entity='client.admin' 2024-09-21T03:20:51.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:51 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2376410651' entity='client.admin' 2024-09-21T03:20:51.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:51 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2453572676' entity='client.admin' 2024-09-21T03:20:51.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:51 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2376410651' entity='client.admin' 2024-09-21T03:20:51.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:51 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2453572676' entity='client.admin' 2024-09-21T03:20:53.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:53 smithi187 ceph-mon[34603]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:53.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:53.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:53.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:53.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:53.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:53.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:53.476 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:53.476 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:53 smithi203 ceph-mon[34723]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:53.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:53 smithi112 ceph-mon[31442]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:53.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:53.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:53.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:20:53.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:20:53.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:20:53.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:20:55.520 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:55.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.523+0000 7f9f67d0c640 1 -- 172.21.15.112:0/2359158001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f60108cf0 msgr2=0x7f9f6010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.523+0000 7f9f67d0c640 1 --2- 172.21.15.112:0/2359158001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f60108cf0 0x7f9f6010f520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009ca0 tx=0x7f9f5802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.523+0000 7f9f67d0c640 1 -- 172.21.15.112:0/2359158001 shutdown_connections 2024-09-21T03:20:55.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.523+0000 7f9f67d0c640 1 --2- 172.21.15.112:0/2359158001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f60108cf0 0x7f9f6010f520 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.523+0000 7f9f67d0c640 1 --2- 172.21.15.112:0/2359158001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f601083a0 0x7f9f601087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.523+0000 7f9f67d0c640 1 --2- 172.21.15.112:0/2359158001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f60102a60 0x7f9f60102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.523+0000 7f9f67d0c640 1 -- 172.21.15.112:0/2359158001 >> 172.21.15.112:0/2359158001 conn(0x7f9f60079730 msgr2=0x7f9f60079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:55.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.523+0000 7f9f67d0c640 1 -- 172.21.15.112:0/2359158001 shutdown_connections 2024-09-21T03:20:55.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.523+0000 7f9f67d0c640 1 -- 172.21.15.112:0/2359158001 wait complete. 2024-09-21T03:20:55.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.524+0000 7f9f67d0c640 1 Processor -- start 2024-09-21T03:20:55.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.524+0000 7f9f67d0c640 1 -- start start 2024-09-21T03:20:55.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.524+0000 7f9f67d0c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f60102a60 0x7f9f6010ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.524+0000 7f9f67d0c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f601083a0 0x7f9f601104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.524+0000 7f9f67d0c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f60108cf0 0x7f9f60110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.524+0000 7f9f67d0c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f601139a0 con 0x7f9f60108cf0 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.524+0000 7f9f67d0c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f60113820 con 0x7f9f60102a60 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.524+0000 7f9f67d0c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f60113b20 con 0x7f9f601083a0 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f662a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f60108cf0 0x7f9f60110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f662a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f60108cf0 0x7f9f60110a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58062/0 (socket says 172.21.15.112:58062) 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f662a9640 1 -- 172.21.15.112:0/1881455556 learned_addr learned my addr 172.21.15.112:0/1881455556 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f65aa8640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f60102a60 0x7f9f6010ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f652a7640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f601083a0 0x7f9f601104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f662a9640 1 -- 172.21.15.112:0/1881455556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f601083a0 msgr2=0x7f9f601104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f662a9640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f601083a0 0x7f9f601104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f662a9640 1 -- 172.21.15.112:0/1881455556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f60102a60 msgr2=0x7f9f6010ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f662a9640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f60102a60 0x7f9f6010ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f662a9640 1 -- 172.21.15.112:0/1881455556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f601a86b0 con 0x7f9f60108cf0 2024-09-21T03:20:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f662a9640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f60108cf0 0x7f9f60110a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f58006010 tx=0x7f9f580093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f56ffd640 1 -- 172.21.15.112:0/1881455556 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58003b70 con 0x7f9f60108cf0 2024-09-21T03:20:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f56ffd640 1 -- 172.21.15.112:0/1881455556 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9f58003d10 con 0x7f9f60108cf0 2024-09-21T03:20:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f67d0c640 1 -- 172.21.15.112:0/1881455556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f601a8940 con 0x7f9f60108cf0 2024-09-21T03:20:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.525+0000 7f9f56ffd640 1 -- 172.21.15.112:0/1881455556 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58004600 con 0x7f9f60108cf0 2024-09-21T03:20:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.526+0000 7f9f67d0c640 1 -- 172.21.15.112:0/1881455556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f601a8e80 con 0x7f9f60108cf0 2024-09-21T03:20:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.527+0000 7f9f67d0c640 1 -- 172.21.15.112:0/1881455556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f60102e70 con 0x7f9f60108cf0 2024-09-21T03:20:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.528+0000 7f9f56ffd640 1 -- 172.21.15.112:0/1881455556 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f9f58030740 con 0x7f9f60108cf0 2024-09-21T03:20:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.529+0000 7f9f56ffd640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9f4c0b64d0 0x7f9f4c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.529+0000 7f9f65aa8640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9f4c0b64d0 0x7f9f4c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.529+0000 7f9f56ffd640 1 -- 172.21.15.112:0/1881455556 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9f581066f0 con 0x7f9f60108cf0 2024-09-21T03:20:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.530+0000 7f9f65aa8640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9f4c0b64d0 0x7f9f4c0b89e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9f60111a00 tx=0x7f9f5c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:55.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.537+0000 7f9f56ffd640 1 -- 172.21.15.112:0/1881455556 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9f580ceb80 con 0x7f9f60108cf0 2024-09-21T03:20:55.643 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:55 smithi112 ceph-mon[31442]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:55.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:55 smithi203 ceph-mon[34723]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:55.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:55 smithi187 ceph-mon[34603]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:55.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.724+0000 7f9f67d0c640 1 -- 172.21.15.112:0/1881455556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f9f6002e640 con 0x7f9f60108cf0 2024-09-21T03:20:55.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.724+0000 7f9f56ffd640 1 -- 172.21.15.112:0/1881455556 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v28)=0 v28) ==== 108+0+0 (secure 0 0 0) 0x7f9f580d2f60 con 0x7f9f60108cf0 2024-09-21T03:20:55.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 -- 172.21.15.112:0/1881455556 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9f4c0b64d0 msgr2=0x7f9f4c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9f4c0b64d0 0x7f9f4c0b89e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9f60111a00 tx=0x7f9f5c005f90 comp rx=0 tx=0).stop 2024-09-21T03:20:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 -- 172.21.15.112:0/1881455556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f60108cf0 msgr2=0x7f9f60110a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f60108cf0 0x7f9f60110a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f58006010 tx=0x7f9f580093d0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 -- 172.21.15.112:0/1881455556 shutdown_connections 2024-09-21T03:20:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9f4c0b64d0 0x7f9f4c0b89e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f60108cf0 0x7f9f60110a30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f601083a0 0x7f9f601104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 --2- 172.21.15.112:0/1881455556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f60102a60 0x7f9f6010ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 -- 172.21.15.112:0/1881455556 >> 172.21.15.112:0/1881455556 conn(0x7f9f60079730 msgr2=0x7f9f601052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 -- 172.21.15.112:0/1881455556 shutdown_connections 2024-09-21T03:20:55.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:55.727+0000 7f9f67d0c640 1 -- 172.21.15.112:0/1881455556 wait complete. 2024-09-21T03:20:56.409 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:20:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.420+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/2987890900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd64108cf0 msgr2=0x7fbd6410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:56.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.420+0000 7fbd6b6ae640 1 --2- 172.21.15.112:0/2987890900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd64108cf0 0x7fbd6410f520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009ca0 tx=0x7fbd5c02e990 comp rx=0 tx=0).stop 2024-09-21T03:20:56.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.420+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/2987890900 shutdown_connections 2024-09-21T03:20:56.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.421+0000 7fbd6b6ae640 1 --2- 172.21.15.112:0/2987890900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd64108cf0 0x7fbd6410f520 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:56.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.421+0000 7fbd6b6ae640 1 --2- 172.21.15.112:0/2987890900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd641083a0 0x7fbd641087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:56.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.421+0000 7fbd6b6ae640 1 --2- 172.21.15.112:0/2987890900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd64102a60 0x7fbd64102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:56.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.421+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/2987890900 >> 172.21.15.112:0/2987890900 conn(0x7fbd64079730 msgr2=0x7fbd64079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:56.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.421+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/2987890900 shutdown_connections 2024-09-21T03:20:56.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.421+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/2987890900 wait complete. 2024-09-21T03:20:56.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd6b6ae640 1 Processor -- start 2024-09-21T03:20:56.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd6b6ae640 1 -- start start 2024-09-21T03:20:56.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd6b6ae640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd64102a60 0x7fbd64198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd6b6ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd641083a0 0x7fbd64198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd6b6ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd64108cf0 0x7fbd6419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd6b6ae640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd64111740 con 0x7fbd641083a0 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd6b6ae640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbd641115c0 con 0x7fbd64102a60 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd6b6ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbd641118c0 con 0x7fbd64108cf0 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd68c49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd641083a0 0x7fbd64198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd68c49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd641083a0 0x7fbd64198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58076/0 (socket says 172.21.15.112:58076) 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd68c49640 1 -- 172.21.15.112:0/3579634408 learned_addr learned my addr 172.21.15.112:0/3579634408 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.422+0000 7fbd6944a640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd64102a60 0x7fbd64198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.423+0000 7fbd69c4b640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd64108cf0 0x7fbd6419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.423+0000 7fbd68c49640 1 -- 172.21.15.112:0/3579634408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd64108cf0 msgr2=0x7fbd6419d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:56.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.423+0000 7fbd68c49640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd64108cf0 0x7fbd6419d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:56.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.423+0000 7fbd68c49640 1 -- 172.21.15.112:0/3579634408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd64102a60 msgr2=0x7fbd64198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:56.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.423+0000 7fbd68c49640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd64102a60 0x7fbd64198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:56.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.423+0000 7fbd68c49640 1 -- 172.21.15.112:0/3579634408 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6419e0e0 con 0x7fbd641083a0 2024-09-21T03:20:56.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.423+0000 7fbd69c4b640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd64108cf0 0x7fbd6419d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:20:56.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.423+0000 7fbd68c49640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd641083a0 0x7fbd64198710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400e9c0 tx=0x7fbd5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:56.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.423+0000 7fbd5a7fc640 1 -- 172.21.15.112:0/3579634408 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5400cd80 con 0x7fbd641083a0 2024-09-21T03:20:56.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.424+0000 7fbd5a7fc640 1 -- 172.21.15.112:0/3579634408 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd540054b0 con 0x7fbd641083a0 2024-09-21T03:20:56.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.424+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/3579634408 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6419e3d0 con 0x7fbd641083a0 2024-09-21T03:20:56.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.424+0000 7fbd5a7fc640 1 -- 172.21.15.112:0/3579634408 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54010430 con 0x7fbd641083a0 2024-09-21T03:20:56.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.424+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/3579634408 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd641a4b30 con 0x7fbd641083a0 2024-09-21T03:20:56.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.425+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/3579634408 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd64102e70 con 0x7fbd641083a0 2024-09-21T03:20:56.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.427+0000 7fbd5a7fc640 1 -- 172.21.15.112:0/3579634408 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7fbd54004f70 con 0x7fbd641083a0 2024-09-21T03:20:56.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.427+0000 7fbd5a7fc640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbd500b6650 0x7fbd500b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:56.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.427+0000 7fbd6944a640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbd500b6650 0x7fbd500b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:56.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.427+0000 7fbd5a7fc640 1 -- 172.21.15.112:0/3579634408 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fbd540e5aa0 con 0x7fbd641083a0 2024-09-21T03:20:56.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.428+0000 7fbd6944a640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbd500b6650 0x7fbd500b8b60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000bb10 tx=0x7fbd6000d040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:56.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.432+0000 7fbd5a7fc640 1 -- 172.21.15.112:0/3579634408 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd540adf30 con 0x7fbd641083a0 2024-09-21T03:20:56.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.612+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/3579634408 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fbd6402e640 con 0x7fbd641083a0 2024-09-21T03:20:56.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.618+0000 7fbd5a7fc640 1 -- 172.21.15.112:0/3579634408 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v29)=0 v29) ==== 120+0+0 (secure 0 0 0) 0x7fbd540b2310 con 0x7fbd641083a0 2024-09-21T03:20:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/3579634408 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbd500b6650 msgr2=0x7fbd500b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbd500b6650 0x7fbd500b8b60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000bb10 tx=0x7fbd6000d040 comp rx=0 tx=0).stop 2024-09-21T03:20:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/3579634408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd641083a0 msgr2=0x7fbd64198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd641083a0 0x7fbd64198710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400e9c0 tx=0x7fbd5400ee80 comp rx=0 tx=0).stop 2024-09-21T03:20:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/3579634408 shutdown_connections 2024-09-21T03:20:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbd500b6650 0x7fbd500b8b60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd64108cf0 0x7fbd6419d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd641083a0 0x7fbd64198710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:56.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 --2- 172.21.15.112:0/3579634408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd64102a60 0x7fbd64198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/3579634408 >> 172.21.15.112:0/3579634408 conn(0x7fbd64079730 msgr2=0x7fbd64104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/3579634408 shutdown_connections 2024-09-21T03:20:56.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:56.621+0000 7fbd6b6ae640 1 -- 172.21.15.112:0/3579634408 wait complete. 2024-09-21T03:20:57.389 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:57 smithi112 ceph-mon[31442]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:57.390 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:57 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3579634408' entity='client.admin' 2024-09-21T03:20:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:57 smithi203 ceph-mon[34723]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:57 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3579634408' entity='client.admin' 2024-09-21T03:20:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:57 smithi187 ceph-mon[34603]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:57 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3579634408' entity='client.admin' 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.778+0000 7f3fb35e1640 1 -- 172.21.15.112:0/3708885336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac102ae0 msgr2=0x7f3fac102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.778+0000 7f3fb35e1640 1 --2- 172.21.15.112:0/3708885336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac102ae0 0x7f3fac102ef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009ca0 tx=0x7f3fa802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.778+0000 7f3fb35e1640 1 -- 172.21.15.112:0/3708885336 shutdown_connections 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.778+0000 7f3fb35e1640 1 --2- 172.21.15.112:0/3708885336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac108d70 0x7f3fac10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.778+0000 7f3fb35e1640 1 --2- 172.21.15.112:0/3708885336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac108420 0x7f3fac108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.778+0000 7f3fb35e1640 1 --2- 172.21.15.112:0/3708885336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac102ae0 0x7f3fac102ef0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.778+0000 7f3fb35e1640 1 -- 172.21.15.112:0/3708885336 >> 172.21.15.112:0/3708885336 conn(0x7f3fac079770 msgr2=0x7f3fac079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.778+0000 7f3fb35e1640 1 -- 172.21.15.112:0/3708885336 shutdown_connections 2024-09-21T03:20:57.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.778+0000 7f3fb35e1640 1 -- 172.21.15.112:0/3708885336 wait complete. 2024-09-21T03:20:57.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb35e1640 1 Processor -- start 2024-09-21T03:20:57.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb35e1640 1 -- start start 2024-09-21T03:20:57.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb35e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac102ae0 0x7f3fac19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb35e1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac108420 0x7f3fac19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb35e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108d70 0x7f3fac19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb35e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac112910 con 0x7f3fac108d70 2024-09-21T03:20:57.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb35e1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac112790 con 0x7f3fac108420 2024-09-21T03:20:57.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb35e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac112a90 con 0x7f3fac102ae0 2024-09-21T03:20:57.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb1b7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108d70 0x7f3fac19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb1b7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108d70 0x7f3fac19bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58096/0 (socket says 172.21.15.112:58096) 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb1b7e640 1 -- 172.21.15.112:0/580449335 learned_addr learned my addr 172.21.15.112:0/580449335 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb137d640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac102ae0 0x7f3fac19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb0b7c640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac108420 0x7f3fac19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb1b7e640 1 -- 172.21.15.112:0/580449335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac102ae0 msgr2=0x7f3fac19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb1b7e640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac102ae0 0x7f3fac19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb1b7e640 1 -- 172.21.15.112:0/580449335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac108420 msgr2=0x7f3fac19b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb1b7e640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac108420 0x7f3fac19b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.779+0000 7f3fb1b7e640 1 -- 172.21.15.112:0/580449335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fac1a65a0 con 0x7f3fac108d70 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.780+0000 7f3fb1b7e640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108d70 0x7f3fac19bd60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400d750 tx=0x7f3fa400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.780+0000 7f3fa27fc640 1 -- 172.21.15.112:0/580449335 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4017070 con 0x7f3fac108d70 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.780+0000 7f3fa27fc640 1 -- 172.21.15.112:0/580449335 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3fa40054b0 con 0x7f3fac108d70 2024-09-21T03:20:57.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.780+0000 7f3fa27fc640 1 -- 172.21.15.112:0/580449335 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4004ba0 con 0x7f3fac108d70 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.780+0000 7f3fb35e1640 1 -- 172.21.15.112:0/580449335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fac1a6890 con 0x7f3fac108d70 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.780+0000 7f3fb35e1640 1 -- 172.21.15.112:0/580449335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fac1a6d80 con 0x7f3fac108d70 2024-09-21T03:20:57.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.781+0000 7f3fb35e1640 1 -- 172.21.15.112:0/580449335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fac103d60 con 0x7f3fac108d70 2024-09-21T03:20:57.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.782+0000 7f3fa27fc640 1 -- 172.21.15.112:0/580449335 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f3fa4004d40 con 0x7f3fac108d70 2024-09-21T03:20:57.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.783+0000 7f3fa27fc640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3f980b6550 0x7f3f980b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.783+0000 7f3fb137d640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3f980b6550 0x7f3f980b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.783+0000 7f3fa27fc640 1 -- 172.21.15.112:0/580449335 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3fa40e5030 con 0x7f3fac108d70 2024-09-21T03:20:57.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.784+0000 7f3fb137d640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3f980b6550 0x7f3f980b8a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8006010 tx=0x7f3fa8031000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:57.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.787+0000 7f3fa27fc640 1 -- 172.21.15.112:0/580449335 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3fa40b18a0 con 0x7f3fac108d70 2024-09-21T03:20:57.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.972+0000 7f3fb35e1640 1 -- 172.21.15.112:0/580449335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f3fac02e640 con 0x7f3fac108d70 2024-09-21T03:20:57.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.966+0000 7fae4c03b640 1 -- 172.21.15.112:0/860042404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae44102ae0 msgr2=0x7fae44102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.966+0000 7fae4c03b640 1 --2- 172.21.15.112:0/860042404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae44102ae0 0x7fae44102ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae40009900 tx=0x7fae4002a410 comp rx=0 tx=0).stop 2024-09-21T03:20:57.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.966+0000 7fae4c03b640 1 -- 172.21.15.112:0/860042404 shutdown_connections 2024-09-21T03:20:57.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.966+0000 7fae4c03b640 1 --2- 172.21.15.112:0/860042404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44108d20 0x7fae4410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.966+0000 7fae4c03b640 1 --2- 172.21.15.112:0/860042404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae441083d0 0x7fae441087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.966+0000 7fae4c03b640 1 --2- 172.21.15.112:0/860042404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae44102ae0 0x7fae44102ef0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.966+0000 7fae4c03b640 1 -- 172.21.15.112:0/860042404 >> 172.21.15.112:0/860042404 conn(0x7fae44079760 msgr2=0x7fae44079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.966+0000 7fae4c03b640 1 -- 172.21.15.112:0/860042404 shutdown_connections 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.966+0000 7fae4c03b640 1 -- 172.21.15.112:0/860042404 wait complete. 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae4c03b640 1 Processor -- start 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae4c03b640 1 -- start start 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae4c03b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44102ae0 0x7fae44104280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae4c03b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae441083d0 0x7fae441047c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae4c03b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae44108d20 0x7fae44104d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae4c03b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae44112870 con 0x7fae44108d20 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae4c03b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fae441126f0 con 0x7fae44102ae0 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae4c03b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fae441129f0 con 0x7fae441083d0 2024-09-21T03:20:57.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae4a5d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae44108d20 0x7fae44104d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae4a5d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae44108d20 0x7fae44104d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58102/0 (socket says 172.21.15.112:58102) 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae4a5d8640 1 -- 172.21.15.112:0/3144243323 learned_addr learned my addr 172.21.15.112:0/3144243323 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae49dd7640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44102ae0 0x7fae44104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.967+0000 7fae495d6640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae441083d0 0x7fae441047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.968+0000 7fae4a5d8640 1 -- 172.21.15.112:0/3144243323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae441083d0 msgr2=0x7fae441047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.968+0000 7fae4a5d8640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae441083d0 0x7fae441047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.968+0000 7fae4a5d8640 1 -- 172.21.15.112:0/3144243323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44102ae0 msgr2=0x7fae44104280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.968+0000 7fae4a5d8640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44102ae0 0x7fae44104280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.968+0000 7fae4a5d8640 1 -- 172.21.15.112:0/3144243323 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae441a85e0 con 0x7fae44108d20 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.968+0000 7fae4a5d8640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae44108d20 0x7fae44104d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00cce0 tx=0x7fae3c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.968+0000 7fae3affd640 1 -- 172.21.15.112:0/3144243323 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00f450 con 0x7fae44108d20 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.968+0000 7fae3affd640 1 -- 172.21.15.112:0/3144243323 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fae3c00fa80 con 0x7fae44108d20 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.968+0000 7fae4c03b640 1 -- 172.21.15.112:0/3144243323 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae441a88d0 con 0x7fae44108d20 2024-09-21T03:20:57.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.968+0000 7fae3affd640 1 -- 172.21.15.112:0/3144243323 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c018ae0 con 0x7fae44108d20 2024-09-21T03:20:57.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.968+0000 7fae4c03b640 1 -- 172.21.15.112:0/3144243323 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae441a8e10 con 0x7fae44108d20 2024-09-21T03:20:57.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.969+0000 7fae4c03b640 1 -- 172.21.15.112:0/3144243323 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae44102ef0 con 0x7fae44108d20 2024-09-21T03:20:57.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.970+0000 7fae3affd640 1 -- 172.21.15.112:0/3144243323 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7fae3c00f5f0 con 0x7fae44108d20 2024-09-21T03:20:57.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.971+0000 7fae3affd640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fae300b6550 0x7fae300b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:57.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.971+0000 7fae49dd7640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fae300b6550 0x7fae300b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:57.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.971+0000 7fae3affd640 1 -- 172.21.15.112:0/3144243323 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fae3c0e5e50 con 0x7fae44108d20 2024-09-21T03:20:57.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.972+0000 7fae49dd7640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fae300b6550 0x7fae300b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae44105cd0 tx=0x7fae400058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:57.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.977+0000 7fae3affd640 1 -- 172.21.15.112:0/3144243323 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fae3c014030 con 0x7fae44108d20 2024-09-21T03:20:57.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.977+0000 7f3fa27fc640 1 -- 172.21.15.112:0/580449335 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v29)=0 v29) ==== 106+0+0 (secure 0 0 0) 0x7f3fa40b67e0 con 0x7f3fac108d70 2024-09-21T03:20:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 -- 172.21.15.112:0/580449335 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3f980b6550 msgr2=0x7f3f980b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3f980b6550 0x7f3f980b8a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8006010 tx=0x7f3fa8031000 comp rx=0 tx=0).stop 2024-09-21T03:20:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 -- 172.21.15.112:0/580449335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108d70 msgr2=0x7f3fac19bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108d70 0x7f3fac19bd60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400d750 tx=0x7f3fa400dc10 comp rx=0 tx=0).stop 2024-09-21T03:20:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 -- 172.21.15.112:0/580449335 shutdown_connections 2024-09-21T03:20:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3f980b6550 0x7f3f980b8a60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3fac108d70 0x7f3fac19bd60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac108420 0x7f3fac19b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 --2- 172.21.15.112:0/580449335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac102ae0 0x7f3fac19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 -- 172.21.15.112:0/580449335 >> 172.21.15.112:0/580449335 conn(0x7f3fac079770 msgr2=0x7f3fac1065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:57.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 -- 172.21.15.112:0/580449335 shutdown_connections 2024-09-21T03:20:57.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:57.980+0000 7f3fb35e1640 1 -- 172.21.15.112:0/580449335 wait complete. 2024-09-21T03:20:58.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.162+0000 7fae4c03b640 1 -- 172.21.15.112:0/3144243323 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fae441a9240 con 0x7fae44108d20 2024-09-21T03:20:58.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.166+0000 7fae3affd640 1 -- 172.21.15.112:0/3144243323 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v30)=0 v30) ==== 116+0+0 (secure 0 0 0) 0x7fae3c0ae2e0 con 0x7fae44108d20 2024-09-21T03:20:58.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 -- 172.21.15.112:0/3144243323 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fae300b6550 msgr2=0x7fae300b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fae300b6550 0x7fae300b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae44105cd0 tx=0x7fae400058d0 comp rx=0 tx=0).stop 2024-09-21T03:20:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 -- 172.21.15.112:0/3144243323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae44108d20 msgr2=0x7fae44104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae44108d20 0x7fae44104d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00cce0 tx=0x7fae3c00a620 comp rx=0 tx=0).stop 2024-09-21T03:20:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 -- 172.21.15.112:0/3144243323 shutdown_connections 2024-09-21T03:20:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fae300b6550 0x7fae300b8a60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae44108d20 0x7fae44104d00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae441083d0 0x7fae441047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 --2- 172.21.15.112:0/3144243323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44102ae0 0x7fae44104280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 -- 172.21.15.112:0/3144243323 >> 172.21.15.112:0/3144243323 conn(0x7fae44079760 msgr2=0x7fae44101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 -- 172.21.15.112:0/3144243323 shutdown_connections 2024-09-21T03:20:58.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:58.169+0000 7fae4c03b640 1 -- 172.21.15.112:0/3144243323 wait complete. 2024-09-21T03:20:59.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.401+0000 7f4277d83640 1 -- 172.21.15.112:0/4134428260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42701083b0 msgr2=0x7f42701087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.401+0000 7f4277d83640 1 --2- 172.21.15.112:0/4134428260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42701083b0 0x7f42701087c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4264009900 tx=0x7f426402a410 comp rx=0 tx=0).stop 2024-09-21T03:20:59.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.401+0000 7f4277d83640 1 -- 172.21.15.112:0/4134428260 shutdown_connections 2024-09-21T03:20:59.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.401+0000 7f4277d83640 1 --2- 172.21.15.112:0/4134428260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4270108d00 0x7f427010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.401+0000 7f4277d83640 1 --2- 172.21.15.112:0/4134428260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42701083b0 0x7f42701087c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.401+0000 7f4277d83640 1 --2- 172.21.15.112:0/4134428260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4270103140 0x7f4270103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.401+0000 7f4277d83640 1 -- 172.21.15.112:0/4134428260 >> 172.21.15.112:0/4134428260 conn(0x7f4270077c60 msgr2=0x7f42700780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:59.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.401+0000 7f4277d83640 1 -- 172.21.15.112:0/4134428260 shutdown_connections 2024-09-21T03:20:59.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.401+0000 7f4277d83640 1 -- 172.21.15.112:0/4134428260 wait complete. 2024-09-21T03:20:59.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4277d83640 1 Processor -- start 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4277d83640 1 -- start start 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4277d83640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4270103140 0x7f42701992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4277d83640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42701083b0 0x7f4270199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4277d83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4270108d00 0x7f427019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4277d83640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42701128a0 con 0x7f4270108d00 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4277d83640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4270112720 con 0x7f4270103140 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4277d83640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4270112a20 con 0x7f42701083b0 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4276320640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4270108d00 0x7f427019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4276320640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4270108d00 0x7f427019eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58118/0 (socket says 172.21.15.112:58118) 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4276320640 1 -- 172.21.15.112:0/2936669154 learned_addr learned my addr 172.21.15.112:0/2936669154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4275b1f640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4270103140 0x7f42701992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f427531e640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42701083b0 0x7f4270199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4276320640 1 -- 172.21.15.112:0/2936669154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42701083b0 msgr2=0x7f4270199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4276320640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42701083b0 0x7f4270199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4276320640 1 -- 172.21.15.112:0/2936669154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4270103140 msgr2=0x7f42701992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4276320640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4270103140 0x7f42701992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.402+0000 7f4276320640 1 -- 172.21.15.112:0/2936669154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427019f240 con 0x7f4270108d00 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.403+0000 7f4275b1f640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4270103140 0x7f42701992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.403+0000 7f4276320640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4270108d00 0x7f427019eb10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f426000eaa0 tx=0x7f426000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.403+0000 7f426effd640 1 -- 172.21.15.112:0/2936669154 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426000cdb0 con 0x7f4270108d00 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.403+0000 7f426effd640 1 -- 172.21.15.112:0/2936669154 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4260005020 con 0x7f4270108d00 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.403+0000 7f4277d83640 1 -- 172.21.15.112:0/2936669154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427019f530 con 0x7f4270108d00 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.403+0000 7f426effd640 1 -- 172.21.15.112:0/2936669154 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42600116d0 con 0x7f4270108d00 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.403+0000 7f4277d83640 1 -- 172.21.15.112:0/2936669154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42701a5c90 con 0x7f4270108d00 2024-09-21T03:20:59.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.404+0000 7f4277d83640 1 -- 172.21.15.112:0/2936669154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4270103550 con 0x7f4270108d00 2024-09-21T03:20:59.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.405+0000 7f426effd640 1 -- 172.21.15.112:0/2936669154 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f4260004a40 con 0x7f4270108d00 2024-09-21T03:20:59.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.406+0000 7f426effd640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f42580b6550 0x7f42580b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:20:59.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.406+0000 7f4275b1f640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f42580b6550 0x7f42580b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:20:59.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.406+0000 7f426effd640 1 -- 172.21.15.112:0/2936669154 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f42600e5c90 con 0x7f4270108d00 2024-09-21T03:20:59.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.407+0000 7f4275b1f640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f42580b6550 0x7f42580b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4268009fc0 tx=0x7f4268005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:20:59.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.409+0000 7f426effd640 1 -- 172.21.15.112:0/2936669154 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4260016030 con 0x7f4270108d00 2024-09-21T03:20:59.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:59 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3144243323' entity='client.admin' 2024-09-21T03:20:59.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:20:59 smithi187 ceph-mon[34603]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:59.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:59 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3144243323' entity='client.admin' 2024-09-21T03:20:59.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:20:59 smithi112 ceph-mon[31442]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:20:59.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.591+0000 7f4277d83640 1 -- 172.21.15.112:0/2936669154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f427019a400 con 0x7f4270108d00 2024-09-21T03:20:59.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.598+0000 7f426effd640 1 -- 172.21.15.112:0/2936669154 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v31)=0 v31) ==== 129+0+0 (secure 0 0 0) 0x7f427019a400 con 0x7f4270108d00 2024-09-21T03:20:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 -- 172.21.15.112:0/2936669154 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f42580b6550 msgr2=0x7f42580b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f42580b6550 0x7f42580b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4268009fc0 tx=0x7f4268005f90 comp rx=0 tx=0).stop 2024-09-21T03:20:59.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 -- 172.21.15.112:0/2936669154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4270108d00 msgr2=0x7f427019eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:20:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4270108d00 0x7f427019eb10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f426000eaa0 tx=0x7f426000ef60 comp rx=0 tx=0).stop 2024-09-21T03:20:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 -- 172.21.15.112:0/2936669154 shutdown_connections 2024-09-21T03:20:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f42580b6550 0x7f42580b8a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4270108d00 0x7f427019eb10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42701083b0 0x7f4270199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 --2- 172.21.15.112:0/2936669154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4270103140 0x7f42701992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:20:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 -- 172.21.15.112:0/2936669154 >> 172.21.15.112:0/2936669154 conn(0x7f4270077c60 msgr2=0x7f4270104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:20:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 -- 172.21.15.112:0/2936669154 shutdown_connections 2024-09-21T03:20:59.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:20:59.601+0000 7f4277d83640 1 -- 172.21.15.112:0/2936669154 wait complete. 2024-09-21T03:20:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:59 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3144243323' entity='client.admin' 2024-09-21T03:20:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:20:59 smithi203 ceph-mon[34723]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:00.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.408+0000 7f6fafdc9640 1 -- 172.21.15.112:0/996920186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa81083c0 msgr2=0x7f6fa81087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:00.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.408+0000 7f6fafdc9640 1 --2- 172.21.15.112:0/996920186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa81083c0 0x7f6fa81087d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009900 tx=0x7f6fa402a410 comp rx=0 tx=0).stop 2024-09-21T03:21:00.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.408+0000 7f6fafdc9640 1 -- 172.21.15.112:0/996920186 shutdown_connections 2024-09-21T03:21:00.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.408+0000 7f6fafdc9640 1 --2- 172.21.15.112:0/996920186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa8108d10 0x7f6fa810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:00.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.408+0000 7f6fafdc9640 1 --2- 172.21.15.112:0/996920186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa81083c0 0x7f6fa81087d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:00.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.408+0000 7f6fafdc9640 1 --2- 172.21.15.112:0/996920186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa8102a80 0x7f6fa8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:00.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.408+0000 7f6fafdc9640 1 -- 172.21.15.112:0/996920186 >> 172.21.15.112:0/996920186 conn(0x7f6fa8079740 msgr2=0x7f6fa8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:00.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.408+0000 7f6fafdc9640 1 -- 172.21.15.112:0/996920186 shutdown_connections 2024-09-21T03:21:00.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.408+0000 7f6fafdc9640 1 -- 172.21.15.112:0/996920186 wait complete. 2024-09-21T03:21:00.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fafdc9640 1 Processor -- start 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fafdc9640 1 -- start start 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fafdc9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa8102a80 0x7f6fa819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fafdc9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa81083c0 0x7f6fa819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fafdc9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa8108d10 0x7f6fa819bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fafdc9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa8112860 con 0x7f6fa8108d10 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fafdc9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa81126e0 con 0x7f6fa8102a80 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fafdc9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa81129e0 con 0x7f6fa81083c0 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fae366640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa8108d10 0x7f6fa819bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fae366640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa8108d10 0x7f6fa819bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58136/0 (socket says 172.21.15.112:58136) 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fae366640 1 -- 172.21.15.112:0/188921706 learned_addr learned my addr 172.21.15.112:0/188921706 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fadb65640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa8102a80 0x7f6fa819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:00.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.409+0000 7f6fad364640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa81083c0 0x7f6fa819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6fae366640 1 -- 172.21.15.112:0/188921706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa81083c0 msgr2=0x7f6fa819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6fae366640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa81083c0 0x7f6fa819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6fae366640 1 -- 172.21.15.112:0/188921706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa8102a80 msgr2=0x7f6fa819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6fae366640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa8102a80 0x7f6fa819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6fae366640 1 -- 172.21.15.112:0/188921706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa81a6540 con 0x7f6fa8108d10 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6fad364640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa81083c0 0x7f6fa819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6fae366640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa8108d10 0x7f6fa819bd90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000cce0 tx=0x7f6fa000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6f9affd640 1 -- 172.21.15.112:0/188921706 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000f450 con 0x7f6fa8108d10 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6f9affd640 1 -- 172.21.15.112:0/188921706 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6fa000fa80 con 0x7f6fa8108d10 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6fafdc9640 1 -- 172.21.15.112:0/188921706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa81a6830 con 0x7f6fa8108d10 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6f9affd640 1 -- 172.21.15.112:0/188921706 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0018ae0 con 0x7f6fa8108d10 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.410+0000 7f6fafdc9640 1 -- 172.21.15.112:0/188921706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa81a6d40 con 0x7f6fa8108d10 2024-09-21T03:21:00.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.411+0000 7f6fafdc9640 1 -- 172.21.15.112:0/188921706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa8103d00 con 0x7f6fa8108d10 2024-09-21T03:21:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.412+0000 7f6f9affd640 1 -- 172.21.15.112:0/188921706 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f6fa000f5f0 con 0x7f6fa8108d10 2024-09-21T03:21:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.413+0000 7f6f9affd640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6f940b6550 0x7f6f940b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:00.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.413+0000 7f6fadb65640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6f940b6550 0x7f6f940b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.413+0000 7f6f9affd640 1 -- 172.21.15.112:0/188921706 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6fa00e5e80 con 0x7f6fa8108d10 2024-09-21T03:21:00.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.414+0000 7f6fadb65640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6f940b6550 0x7f6f940b8a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0058c0 tx=0x7f6f9c00a250 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:00.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.418+0000 7f6f9affd640 1 -- 172.21.15.112:0/188921706 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6fa0014030 con 0x7f6fa8108d10 2024-09-21T03:21:00.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.599+0000 7f6fafdc9640 1 -- 172.21.15.112:0/188921706 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f6fa802e640 con 0x7f6fa8108d10 2024-09-21T03:21:00.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.608+0000 7f6f9affd640 1 -- 172.21.15.112:0/188921706 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v32)=0 v32) ==== 121+0+0 (secure 0 0 0) 0x7f6fa00ae310 con 0x7f6fa8108d10 2024-09-21T03:21:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 -- 172.21.15.112:0/188921706 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6f940b6550 msgr2=0x7f6f940b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6f940b6550 0x7f6f940b8a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0058c0 tx=0x7f6f9c00a250 comp rx=0 tx=0).stop 2024-09-21T03:21:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 -- 172.21.15.112:0/188921706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa8108d10 msgr2=0x7f6fa819bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa8108d10 0x7f6fa819bd90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000cce0 tx=0x7f6fa000a620 comp rx=0 tx=0).stop 2024-09-21T03:21:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 -- 172.21.15.112:0/188921706 shutdown_connections 2024-09-21T03:21:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6f940b6550 0x7f6f940b8a60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fa8108d10 0x7f6fa819bd90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa81083c0 0x7f6fa819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 --2- 172.21.15.112:0/188921706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fa8102a80 0x7f6fa819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 -- 172.21.15.112:0/188921706 >> 172.21.15.112:0/188921706 conn(0x7f6fa8079740 msgr2=0x7f6fa81045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:00.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 -- 172.21.15.112:0/188921706 shutdown_connections 2024-09-21T03:21:00.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:00.612+0000 7f6fafdc9640 1 -- 172.21.15.112:0/188921706 wait complete. 2024-09-21T03:21:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:00 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2936669154' entity='client.admin' 2024-09-21T03:21:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:00 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:00 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:00 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:00 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:00 smithi112 ceph-mon[31442]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:00.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:00 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2936669154' entity='client.admin' 2024-09-21T03:21:00.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:00 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:00.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:00 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:00.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:00 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:00.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:00 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:00.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:00 smithi203 ceph-mon[34723]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:00.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:00 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2936669154' entity='client.admin' 2024-09-21T03:21:00.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:00 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:00.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:00 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:00.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:00 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:00.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:00 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:00.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:00 smithi187 ceph-mon[34603]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.047+0000 7f0a8ea17640 1 -- 172.21.15.112:0/2016201978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a88102510 msgr2=0x7f0a8810f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.047+0000 7f0a8ea17640 1 --2- 172.21.15.112:0/2016201978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a88102510 0x7f0a8810f910 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009900 tx=0x7f0a7002a410 comp rx=0 tx=0).stop 2024-09-21T03:21:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.047+0000 7f0a8ea17640 1 -- 172.21.15.112:0/2016201978 shutdown_connections 2024-09-21T03:21:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.047+0000 7f0a8ea17640 1 --2- 172.21.15.112:0/2016201978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a88110000 0x7f0a88112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.047+0000 7f0a8ea17640 1 --2- 172.21.15.112:0/2016201978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a88102510 0x7f0a8810f910 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.047+0000 7f0a8ea17640 1 --2- 172.21.15.112:0/2016201978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a88101ae0 0x7f0a88101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.047+0000 7f0a8ea17640 1 -- 172.21.15.112:0/2016201978 >> 172.21.15.112:0/2016201978 conn(0x7f0a88077800 msgr2=0x7f0a88077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:01.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.047+0000 7f0a8ea17640 1 -- 172.21.15.112:0/2016201978 shutdown_connections 2024-09-21T03:21:01.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.047+0000 7f0a8ea17640 1 -- 172.21.15.112:0/2016201978 wait complete. 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a8ea17640 1 Processor -- start 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a8ea17640 1 -- start start 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a8ea17640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a88101ae0 0x7f0a8819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a8ea17640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a88102510 0x7f0a8819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a8ea17640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a88110000 0x7f0a881a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a8ea17640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88115c50 con 0x7f0a88101ae0 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a8ea17640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88115ad0 con 0x7f0a88102510 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a8ea17640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88115dd0 con 0x7f0a88110000 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a88101ae0 0x7f0a8819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a88101ae0 0x7f0a8819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58156/0 (socket says 172.21.15.112:58156) 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a87fff640 1 -- 172.21.15.112:0/1901156014 learned_addr learned my addr 172.21.15.112:0/1901156014 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:21:01.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.048+0000 7f0a7ffff640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a88102510 0x7f0a8819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a8cfb4640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a88110000 0x7f0a881a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a87fff640 1 -- 172.21.15.112:0/1901156014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a88110000 msgr2=0x7f0a881a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a87fff640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a88110000 0x7f0a881a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a87fff640 1 -- 172.21.15.112:0/1901156014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a88102510 msgr2=0x7f0a8819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a87fff640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a88102510 0x7f0a8819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a87fff640 1 -- 172.21.15.112:0/1901156014 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a881a2690 con 0x7f0a88101ae0 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a8cfb4640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a88110000 0x7f0a881a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a87fff640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a88101ae0 0x7f0a8819c720 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800b4a0 tx=0x7f0a7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a85ffb640 1 -- 172.21.15.112:0/1901156014 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78014070 con 0x7f0a88101ae0 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a85ffb640 1 -- 172.21.15.112:0/1901156014 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0a78010aa0 con 0x7f0a88101ae0 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a8ea17640 1 -- 172.21.15.112:0/1901156014 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a881a2980 con 0x7f0a88101ae0 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a85ffb640 1 -- 172.21.15.112:0/1901156014 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78017b80 con 0x7f0a88101ae0 2024-09-21T03:21:01.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.049+0000 7f0a8ea17640 1 -- 172.21.15.112:0/1901156014 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a881a90e0 con 0x7f0a88101ae0 2024-09-21T03:21:01.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.050+0000 7f0a8ea17640 1 -- 172.21.15.112:0/1901156014 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a88101f70 con 0x7f0a88101ae0 2024-09-21T03:21:01.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.052+0000 7f0a85ffb640 1 -- 172.21.15.112:0/1901156014 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f0a78017070 con 0x7f0a88101ae0 2024-09-21T03:21:01.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.052+0000 7f0a85ffb640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0a740b6550 0x7f0a740b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:01.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.052+0000 7f0a7ffff640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0a740b6550 0x7f0a740b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:01.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.052+0000 7f0a85ffb640 1 -- 172.21.15.112:0/1901156014 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0a780e4980 con 0x7f0a88101ae0 2024-09-21T03:21:01.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.053+0000 7f0a7ffff640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0a740b6550 0x7f0a740b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0a7002aa00 tx=0x7f0a70000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:01.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.057+0000 7f0a85ffb640 1 -- 172.21.15.112:0/1901156014 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a780ace10 con 0x7f0a88101ae0 2024-09-21T03:21:01.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.247+0000 7f0a8ea17640 1 -- 172.21.15.112:0/1901156014 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f0a8819d850 con 0x7f0a88101ae0 2024-09-21T03:21:01.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.252+0000 7f0a85ffb640 1 -- 172.21.15.112:0/1901156014 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v33)=0 v33) ==== 123+0+0 (secure 0 0 0) 0x7f0a780b11f0 con 0x7f0a88101ae0 2024-09-21T03:21:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.253+0000 7f0a85ffb640 1 -- 172.21.15.112:0/1901156014 <== mon.0 v2:172.21.15.112:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a780ea080 con 0x7f0a88101ae0 2024-09-21T03:21:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 -- 172.21.15.112:0/1901156014 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0a740b6550 msgr2=0x7f0a740b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0a740b6550 0x7f0a740b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0a7002aa00 tx=0x7f0a70000bc0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 -- 172.21.15.112:0/1901156014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a88101ae0 msgr2=0x7f0a8819c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a88101ae0 0x7f0a8819c720 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800b4a0 tx=0x7f0a7800b960 comp rx=0 tx=0).stop 2024-09-21T03:21:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 -- 172.21.15.112:0/1901156014 shutdown_connections 2024-09-21T03:21:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0a740b6550 0x7f0a740b8a60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a88110000 0x7f0a881a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a88102510 0x7f0a8819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 --2- 172.21.15.112:0/1901156014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a88101ae0 0x7f0a8819c720 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 -- 172.21.15.112:0/1901156014 >> 172.21.15.112:0/1901156014 conn(0x7f0a88077800 msgr2=0x7f0a88110740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 -- 172.21.15.112:0/1901156014 shutdown_connections 2024-09-21T03:21:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.255+0000 7f0a8ea17640 1 -- 172.21.15.112:0/1901156014 wait complete. 2024-09-21T03:21:01.760 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:01 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/188921706' entity='client.admin' 2024-09-21T03:21:01.760 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:01 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1901156014' entity='client.admin' 2024-09-21T03:21:01.761 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:01 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/677432974' entity='client.admin' 2024-09-21T03:21:01.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.775+0000 7fd9e4fef640 1 -- 172.21.15.112:0/359726508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0108420 msgr2=0x7fd9e0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:01.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.775+0000 7fd9e4fef640 1 --2- 172.21.15.112:0/359726508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0108420 0x7fd9e0108830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009900 tx=0x7fd9d402a410 comp rx=0 tx=0).stop 2024-09-21T03:21:01.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.775+0000 7fd9e4fef640 1 -- 172.21.15.112:0/359726508 shutdown_connections 2024-09-21T03:21:01.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.775+0000 7fd9e4fef640 1 --2- 172.21.15.112:0/359726508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9e0108d70 0x7fd9e010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.775+0000 7fd9e4fef640 1 --2- 172.21.15.112:0/359726508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0108420 0x7fd9e0108830 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.775+0000 7fd9e4fef640 1 --2- 172.21.15.112:0/359726508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e0102ae0 0x7fd9e0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.775+0000 7fd9e4fef640 1 -- 172.21.15.112:0/359726508 >> 172.21.15.112:0/359726508 conn(0x7fd9e0079770 msgr2=0x7fd9e0079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:01.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.776+0000 7fd9e4fef640 1 -- 172.21.15.112:0/359726508 shutdown_connections 2024-09-21T03:21:01.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.776+0000 7fd9e4fef640 1 -- 172.21.15.112:0/359726508 wait complete. 2024-09-21T03:21:01.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.776+0000 7fd9e4fef640 1 Processor -- start 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9e4fef640 1 -- start start 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9e4fef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102ae0 0x7fd9e019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9e4fef640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9e0108420 0x7fd9e019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9e4fef640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e0108d70 0x7fd9e019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9e4fef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0112a30 con 0x7fd9e0102ae0 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9e4fef640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01128b0 con 0x7fd9e0108d70 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9e4fef640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0112bb0 con 0x7fd9e0108420 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9de59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102ae0 0x7fd9e019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9de59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102ae0 0x7fd9e019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58192/0 (socket says 172.21.15.112:58192) 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9de59c640 1 -- 172.21.15.112:0/3075938052 learned_addr learned my addr 172.21.15.112:0/3075938052 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9ddd9b640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9e0108420 0x7fd9e019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9ded9d640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e0108d70 0x7fd9e019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:01.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9de59c640 1 -- 172.21.15.112:0/3075938052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9e0108420 msgr2=0x7fd9e019b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9de59c640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9e0108420 0x7fd9e019b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9de59c640 1 -- 172.21.15.112:0/3075938052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e0108d70 msgr2=0x7fd9e019be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9de59c640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e0108d70 0x7fd9e019be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.777+0000 7fd9de59c640 1 -- 172.21.15.112:0/3075938052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e01a6660 con 0x7fd9e0102ae0 2024-09-21T03:21:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.778+0000 7fd9de59c640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102ae0 0x7fd9e019b3a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c890 tx=0x7fd9d000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.778+0000 7fd9cf7fe640 1 -- 172.21.15.112:0/3075938052 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0004220 con 0x7fd9e0102ae0 2024-09-21T03:21:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.778+0000 7fd9e4fef640 1 -- 172.21.15.112:0/3075938052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e01a6950 con 0x7fd9e0102ae0 2024-09-21T03:21:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.778+0000 7fd9cf7fe640 1 -- 172.21.15.112:0/3075938052 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9d00043c0 con 0x7fd9e0102ae0 2024-09-21T03:21:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.778+0000 7fd9cf7fe640 1 -- 172.21.15.112:0/3075938052 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0004a00 con 0x7fd9e0102ae0 2024-09-21T03:21:01.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.778+0000 7fd9e4fef640 1 -- 172.21.15.112:0/3075938052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e01a6e60 con 0x7fd9e0102ae0 2024-09-21T03:21:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.779+0000 7fd9e4fef640 1 -- 172.21.15.112:0/3075938052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e0103d60 con 0x7fd9e0102ae0 2024-09-21T03:21:01.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.780+0000 7fd9cf7fe640 1 -- 172.21.15.112:0/3075938052 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7fd9d00079b0 con 0x7fd9e0102ae0 2024-09-21T03:21:01.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.781+0000 7fd9cf7fe640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd9c80b64d0 0x7fd9c80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:01.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.781+0000 7fd9ddd9b640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd9c80b64d0 0x7fd9c80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:01.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.781+0000 7fd9cf7fe640 1 -- 172.21.15.112:0/3075938052 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd9d00e5910 con 0x7fd9e0102ae0 2024-09-21T03:21:01.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.782+0000 7fd9ddd9b640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd9c80b64d0 0x7fd9c80b89e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009230 tx=0x7fd9d4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:01.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.784+0000 7fd9cf7fe640 1 -- 172.21.15.112:0/3075938052 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9d0014030 con 0x7fd9e0102ae0 2024-09-21T03:21:01.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:01 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/188921706' entity='client.admin' 2024-09-21T03:21:01.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:01 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1901156014' entity='client.admin' 2024-09-21T03:21:01.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:01 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/677432974' entity='client.admin' 2024-09-21T03:21:01.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:01 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/188921706' entity='client.admin' 2024-09-21T03:21:01.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:01 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1901156014' entity='client.admin' 2024-09-21T03:21:01.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:01 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/677432974' entity='client.admin' 2024-09-21T03:21:01.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.967+0000 7fd9e4fef640 1 -- 172.21.15.112:0/3075938052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fd9e006bf90 con 0x7fd9e0102ae0 2024-09-21T03:21:01.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.973+0000 7fd9cf7fe640 1 -- 172.21.15.112:0/3075938052 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v35)=0 v35) ==== 121+0+0 (secure 0 0 0) 0x7fd9d00adda0 con 0x7fd9e0102ae0 2024-09-21T03:21:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 -- 172.21.15.112:0/3075938052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd9c80b64d0 msgr2=0x7fd9c80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd9c80b64d0 0x7fd9c80b89e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009230 tx=0x7fd9d4031400 comp rx=0 tx=0).stop 2024-09-21T03:21:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 -- 172.21.15.112:0/3075938052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102ae0 msgr2=0x7fd9e019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102ae0 0x7fd9e019b3a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c890 tx=0x7fd9d000cd50 comp rx=0 tx=0).stop 2024-09-21T03:21:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 -- 172.21.15.112:0/3075938052 shutdown_connections 2024-09-21T03:21:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd9c80b64d0 0x7fd9c80b89e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e0108d70 0x7fd9e019be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9e0108420 0x7fd9e019b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 --2- 172.21.15.112:0/3075938052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102ae0 0x7fd9e019b3a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 -- 172.21.15.112:0/3075938052 >> 172.21.15.112:0/3075938052 conn(0x7fd9e0079770 msgr2=0x7fd9e01065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 -- 172.21.15.112:0/3075938052 shutdown_connections 2024-09-21T03:21:01.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.975+0000 7fd9e4fef640 1 -- 172.21.15.112:0/3075938052 wait complete. 2024-09-21T03:21:02.633 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi112') 2024-09-21T03:21:02.634 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi112') 2024-09-21T03:21:02.634 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi112') 2024-09-21T03:21:02.634 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi112') 2024-09-21T03:21:02.634 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi112') 2024-09-21T03:21:02.634 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi112') 2024-09-21T03:21:02.634 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi112') 2024-09-21T03:21:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.300+0000 7f4c38a08640 1 -- 172.21.15.112:0/4063162849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c341083c0 msgr2=0x7f4c341087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.300+0000 7f4c38a08640 1 --2- 172.21.15.112:0/4063162849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c341083c0 0x7f4c341087d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009900 tx=0x7f4c1c02a410 comp rx=0 tx=0).stop 2024-09-21T03:21:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.300+0000 7f4c38a08640 1 -- 172.21.15.112:0/4063162849 shutdown_connections 2024-09-21T03:21:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.300+0000 7f4c38a08640 1 --2- 172.21.15.112:0/4063162849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c34108d10 0x7f4c3410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.300+0000 7f4c38a08640 1 --2- 172.21.15.112:0/4063162849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c341083c0 0x7f4c341087d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.300+0000 7f4c38a08640 1 --2- 172.21.15.112:0/4063162849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c34102a80 0x7f4c34102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.300+0000 7f4c38a08640 1 -- 172.21.15.112:0/4063162849 >> 172.21.15.112:0/4063162849 conn(0x7f4c34079740 msgr2=0x7f4c34079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.300+0000 7f4c38a08640 1 -- 172.21.15.112:0/4063162849 shutdown_connections 2024-09-21T03:21:02.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.300+0000 7f4c38a08640 1 -- 172.21.15.112:0/4063162849 wait complete. 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.301+0000 7f4c38a08640 1 Processor -- start 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.301+0000 7f4c38a08640 1 -- start start 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.301+0000 7f4c38a08640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c34102a80 0x7f4c3419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c38a08640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c341083c0 0x7f4c3419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c38a08640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c34108d10 0x7f4c3419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c38a08640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34112980 con 0x7f4c34102a80 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c38a08640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34112800 con 0x7f4c341083c0 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c38a08640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34112b00 con 0x7f4c34108d10 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c3259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c34102a80 0x7f4c3419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c3259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c34102a80 0x7f4c3419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58174/0 (socket says 172.21.15.112:58174) 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c3259c640 1 -- 172.21.15.112:0/677432974 learned_addr learned my addr 172.21.15.112:0/677432974 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c32d9d640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c34108d10 0x7f4c3419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c29d9b640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c341083c0 0x7f4c3419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c3259c640 1 -- 172.21.15.112:0/677432974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c34108d10 msgr2=0x7f4c3419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c3259c640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c34108d10 0x7f4c3419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c3259c640 1 -- 172.21.15.112:0/677432974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c341083c0 msgr2=0x7f4c3419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c3259c640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c341083c0 0x7f4c3419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c3259c640 1 -- 172.21.15.112:0/677432974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c341a64e0 con 0x7f4c34102a80 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c32d9d640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c34108d10 0x7f4c3419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c29d9b640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c341083c0 0x7f4c3419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c3259c640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c34102a80 0x7f4c3419b2b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c920 tx=0x7f4c2400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c2bfff640 1 -- 172.21.15.112:0/677432974 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24007c70 con 0x7f4c34102a80 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.302+0000 7f4c2bfff640 1 -- 172.21.15.112:0/677432974 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c24007e10 con 0x7f4c34102a80 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.303+0000 7f4c38a08640 1 -- 172.21.15.112:0/677432974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c341a67d0 con 0x7f4c34102a80 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.303+0000 7f4c2bfff640 1 -- 172.21.15.112:0/677432974 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24018820 con 0x7f4c34102a80 2024-09-21T03:21:02.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.303+0000 7f4c38a08640 1 -- 172.21.15.112:0/677432974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c341a6d30 con 0x7f4c34102a80 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.304+0000 7f4c38a08640 1 -- 172.21.15.112:0/677432974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c34103d00 con 0x7f4c34102a80 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.306+0000 7f4c2bfff640 1 -- 172.21.15.112:0/677432974 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f4c2400f450 con 0x7f4c34102a80 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.307+0000 7f4c2bfff640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4c200b6550 0x7f4c200b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.307+0000 7f4c29d9b640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4c200b6550 0x7f4c200b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.307+0000 7f4c2bfff640 1 -- 172.21.15.112:0/677432974 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4c240e5b50 con 0x7f4c34102a80 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.307+0000 7f4c29d9b640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4c200b6550 0x7f4c200b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c000c00 tx=0x7f4c1c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.310+0000 7f4c2bfff640 1 -- 172.21.15.112:0/677432974 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c24014030 con 0x7f4c34102a80 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.495+0000 7f4c38a08640 1 -- 172.21.15.112:0/677432974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f4c3406bf70 con 0x7f4c34102a80 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.501+0000 7f4c2bfff640 1 -- 172.21.15.112:0/677432974 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v34)=0 v34) ==== 121+0+0 (secure 0 0 0) 0x7f4c3406bf70 con 0x7f4c34102a80 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.503+0000 7f4c38a08640 1 -- 172.21.15.112:0/677432974 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4c200b6550 msgr2=0x7f4c200b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.503+0000 7f4c38a08640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4c200b6550 0x7f4c200b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c000c00 tx=0x7f4c1c031400 comp rx=0 tx=0).stop 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.503+0000 7f4c38a08640 1 -- 172.21.15.112:0/677432974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c34102a80 msgr2=0x7f4c3419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.503+0000 7f4c38a08640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c34102a80 0x7f4c3419b2b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c920 tx=0x7f4c2400cde0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.503+0000 7f4c38a08640 1 -- 172.21.15.112:0/677432974 shutdown_connections 2024-09-21T03:21:02.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.503+0000 7f4c38a08640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4c200b6550 0x7f4c200b8a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.503+0000 7f4c38a08640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c34108d10 0x7f4c3419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.503+0000 7f4c38a08640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c341083c0 0x7f4c3419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.503+0000 7f4c38a08640 1 --2- 172.21.15.112:0/677432974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c34102a80 0x7f4c3419b2b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:02.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.503+0000 7f4c38a08640 1 -- 172.21.15.112:0/677432974 >> 172.21.15.112:0/677432974 conn(0x7f4c34079740 msgr2=0x7f4c34104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:02.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.504+0000 7f4c38a08640 1 -- 172.21.15.112:0/677432974 shutdown_connections 2024-09-21T03:21:02.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:01.504+0000 7f4c38a08640 1 -- 172.21.15.112:0/677432974 wait complete. 2024-09-21T03:21:03.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:02 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3075938052' entity='client.admin' 2024-09-21T03:21:03.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:02 smithi112 ceph-mon[31442]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:03.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:02 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3075938052' entity='client.admin' 2024-09-21T03:21:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:02 smithi203 ceph-mon[34723]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:03.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:02 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3075938052' entity='client.admin' 2024-09-21T03:21:03.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:02 smithi187 ceph-mon[34603]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.283+0000 7fa432b61640 1 -- 172.21.15.112:0/3312420768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa42c103c30 msgr2=0x7fa42c104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.283+0000 7fa432b61640 1 --2- 172.21.15.112:0/3312420768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa42c103c30 0x7fa42c104040 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa424009900 tx=0x7fa42402a410 comp rx=0 tx=0).stop 2024-09-21T03:21:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.283+0000 7fa432b61640 1 -- 172.21.15.112:0/3312420768 shutdown_connections 2024-09-21T03:21:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.283+0000 7fa432b61640 1 --2- 172.21.15.112:0/3312420768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa42c109ec0 0x7fa42c1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.283+0000 7fa432b61640 1 --2- 172.21.15.112:0/3312420768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa42c109570 0x7fa42c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.283+0000 7fa432b61640 1 --2- 172.21.15.112:0/3312420768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa42c103c30 0x7fa42c104040 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.283+0000 7fa432b61640 1 -- 172.21.15.112:0/3312420768 >> 172.21.15.112:0/3312420768 conn(0x7fa42c07b8d0 msgr2=0x7fa42c07bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.283+0000 7fa432b61640 1 -- 172.21.15.112:0/3312420768 shutdown_connections 2024-09-21T03:21:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.283+0000 7fa432b61640 1 -- 172.21.15.112:0/3312420768 wait complete. 2024-09-21T03:21:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.283+0000 7fa432b61640 1 Processor -- start 2024-09-21T03:21:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.283+0000 7fa432b61640 1 -- start start 2024-09-21T03:21:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa432b61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa42c103c30 0x7fa42c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa432b61640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa42c109570 0x7fa42c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa432b61640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa42c109ec0 0x7fa42c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa432b61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c113ae0 con 0x7fa42c103c30 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa432b61640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c113960 con 0x7fa42c109570 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa432b61640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c113c60 con 0x7fa42c109ec0 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa4308fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa42c103c30 0x7fa42c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa4308fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa42c103c30 0x7fa42c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58222/0 (socket says 172.21.15.112:58222) 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa4308fd640 1 -- 172.21.15.112:0/3351092950 learned_addr learned my addr 172.21.15.112:0/3351092950 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa42bfff640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa42c109570 0x7fa42c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa4310fe640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa42c109ec0 0x7fa42c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa4308fd640 1 -- 172.21.15.112:0/3351092950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa42c109ec0 msgr2=0x7fa42c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa4308fd640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa42c109ec0 0x7fa42c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa4308fd640 1 -- 172.21.15.112:0/3351092950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa42c109570 msgr2=0x7fa42c19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa4308fd640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa42c109570 0x7fa42c19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa4308fd640 1 -- 172.21.15.112:0/3351092950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c1a0360 con 0x7fa42c103c30 2024-09-21T03:21:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa4310fe640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa42c109ec0 0x7fa42c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:21:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.284+0000 7fa4308fd640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa42c103c30 0x7fa42c19a4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa424005940 tx=0x7fa4240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.285+0000 7fa429ffb640 1 -- 172.21.15.112:0/3351092950 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42403f030 con 0x7fa42c103c30 2024-09-21T03:21:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.285+0000 7fa429ffb640 1 -- 172.21.15.112:0/3351092950 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa424003cb0 con 0x7fa42c103c30 2024-09-21T03:21:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.285+0000 7fa429ffb640 1 -- 172.21.15.112:0/3351092950 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424035070 con 0x7fa42c103c30 2024-09-21T03:21:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.285+0000 7fa432b61640 1 -- 172.21.15.112:0/3351092950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42c1a05f0 con 0x7fa42c103c30 2024-09-21T03:21:04.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.285+0000 7fa432b61640 1 -- 172.21.15.112:0/3351092950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42c1a0a80 con 0x7fa42c103c30 2024-09-21T03:21:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.286+0000 7fa432b61640 1 -- 172.21.15.112:0/3351092950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42c104040 con 0x7fa42c103c30 2024-09-21T03:21:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.287+0000 7fa429ffb640 1 -- 172.21.15.112:0/3351092950 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7fa424042030 con 0x7fa42c103c30 2024-09-21T03:21:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.288+0000 7fa429ffb640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa4180b6500 0x7fa4180b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.288+0000 7fa42bfff640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa4180b6500 0x7fa4180b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.289+0000 7fa429ffb640 1 -- 172.21.15.112:0/3351092950 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa424105d00 con 0x7fa42c103c30 2024-09-21T03:21:04.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.289+0000 7fa42bfff640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa4180b6500 0x7fa4180b8a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa41c007350 tx=0x7fa41c0072e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:04.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.293+0000 7fa429ffb640 1 -- 172.21.15.112:0/3351092950 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa4240ce190 con 0x7fa42c103c30 2024-09-21T03:21:04.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.474+0000 7fa432b61640 1 -- 172.21.15.112:0/3351092950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fa42c02e640 con 0x7fa42c103c30 2024-09-21T03:21:04.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.478+0000 7fa429ffb640 1 -- 172.21.15.112:0/3351092950 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v36)=0 v36) ==== 118+0+0 (secure 0 0 0) 0x7fa4240d2570 con 0x7fa42c103c30 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 -- 172.21.15.112:0/3351092950 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa4180b6500 msgr2=0x7fa4180b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa4180b6500 0x7fa4180b8a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa41c007350 tx=0x7fa41c0072e0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 -- 172.21.15.112:0/3351092950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa42c103c30 msgr2=0x7fa42c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa42c103c30 0x7fa42c19a4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa424005940 tx=0x7fa4240042f0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 -- 172.21.15.112:0/3351092950 shutdown_connections 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa4180b6500 0x7fa4180b8a10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa42c109ec0 0x7fa42c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa42c109570 0x7fa42c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 --2- 172.21.15.112:0/3351092950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa42c103c30 0x7fa42c19a4b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 -- 172.21.15.112:0/3351092950 >> 172.21.15.112:0/3351092950 conn(0x7fa42c07b8d0 msgr2=0x7fa42c107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 -- 172.21.15.112:0/3351092950 shutdown_connections 2024-09-21T03:21:04.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:04.481+0000 7fa432b61640 1 -- 172.21.15.112:0/3351092950 wait complete. 2024-09-21T03:21:05.349 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:05 smithi112 ceph-mon[31442]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:05.350 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:05 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3351092950' entity='client.admin' 2024-09-21T03:21:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:05 smithi203 ceph-mon[34723]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:05 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3351092950' entity='client.admin' 2024-09-21T03:21:05.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi112') 2024-09-21T03:21:05.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi112') 2024-09-21T03:21:05.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi112') 2024-09-21T03:21:05.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi112') 2024-09-21T03:21:05.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi112') 2024-09-21T03:21:05.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi112') 2024-09-21T03:21:05.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi112') 2024-09-21T03:21:05.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi112') 2024-09-21T03:21:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:05 smithi187 ceph-mon[34603]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:05 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3351092950' entity='client.admin' 2024-09-21T03:21:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.100+0000 7f3563b1c640 1 -- 172.21.15.112:0/2474403253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c1052d0 msgr2=0x7f355c10d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.100+0000 7f3563b1c640 1 --2- 172.21.15.112:0/2474403253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c1052d0 0x7f355c10d910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3550009900 tx=0x7f355002a410 comp rx=0 tx=0).stop 2024-09-21T03:21:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.101+0000 7f3563b1c640 1 -- 172.21.15.112:0/2474403253 shutdown_connections 2024-09-21T03:21:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.101+0000 7f3563b1c640 1 --2- 172.21.15.112:0/2474403253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c10de50 0x7f355c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.101+0000 7f3563b1c640 1 --2- 172.21.15.112:0/2474403253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c1052d0 0x7f355c10d910 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.101+0000 7f3563b1c640 1 --2- 172.21.15.112:0/2474403253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f355c1048a0 0x7f355c104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.101+0000 7f3563b1c640 1 -- 172.21.15.112:0/2474403253 >> 172.21.15.112:0/2474403253 conn(0x7f355c079740 msgr2=0x7f355c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:06.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.101+0000 7f3563b1c640 1 -- 172.21.15.112:0/2474403253 shutdown_connections 2024-09-21T03:21:06.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.101+0000 7f3563b1c640 1 -- 172.21.15.112:0/2474403253 wait complete. 2024-09-21T03:21:06.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.101+0000 7f3563b1c640 1 Processor -- start 2024-09-21T03:21:06.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.101+0000 7f3563b1c640 1 -- start start 2024-09-21T03:21:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f3563b1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c1048a0 0x7f355c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f3563b1c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c1052d0 0x7f355c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f3563b1c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f355c10de50 0x7f355c1a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f3563b1c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f355c113a90 con 0x7f355c1048a0 2024-09-21T03:21:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f3563b1c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f355c113910 con 0x7f355c10de50 2024-09-21T03:21:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f3563b1c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f355c113c10 con 0x7f355c1052d0 2024-09-21T03:21:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35618b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c1048a0 0x7f355c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:06.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35618b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c1048a0 0x7f355c19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59314/0 (socket says 172.21.15.112:59314) 2024-09-21T03:21:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35618b8640 1 -- 172.21.15.112:0/2006321093 learned_addr learned my addr 172.21.15.112:0/2006321093 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:21:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35620b9640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f355c10de50 0x7f355c1a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35610b7640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c1052d0 0x7f355c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35618b8640 1 -- 172.21.15.112:0/2006321093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c1052d0 msgr2=0x7f355c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35618b8640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c1052d0 0x7f355c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35618b8640 1 -- 172.21.15.112:0/2006321093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f355c10de50 msgr2=0x7f355c1a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35618b8640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f355c10de50 0x7f355c1a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35618b8640 1 -- 172.21.15.112:0/2006321093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c1a25b0 con 0x7f355c1048a0 2024-09-21T03:21:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35610b7640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c1052d0 0x7f355c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35620b9640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f355c10de50 0x7f355c1a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:06.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.102+0000 7f35618b8640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c1048a0 0x7f355c19c700 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f354c00d950 tx=0x7f354c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:06.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.103+0000 7f355affd640 1 -- 172.21.15.112:0/2006321093 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c017020 con 0x7f355c1048a0 2024-09-21T03:21:06.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.103+0000 7f3563b1c640 1 -- 172.21.15.112:0/2006321093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c1a28a0 con 0x7f355c1048a0 2024-09-21T03:21:06.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.103+0000 7f355affd640 1 -- 172.21.15.112:0/2006321093 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f354c009c30 con 0x7f355c1048a0 2024-09-21T03:21:06.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.103+0000 7f355affd640 1 -- 172.21.15.112:0/2006321093 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c00f600 con 0x7f355c1048a0 2024-09-21T03:21:06.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.103+0000 7f3563b1c640 1 -- 172.21.15.112:0/2006321093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c1a90c0 con 0x7f355c1048a0 2024-09-21T03:21:06.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.104+0000 7f3563b1c640 1 -- 172.21.15.112:0/2006321093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355c06c1a0 con 0x7f355c1048a0 2024-09-21T03:21:06.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.105+0000 7f355affd640 1 -- 172.21.15.112:0/2006321093 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f354c01a020 con 0x7f355c1048a0 2024-09-21T03:21:06.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.106+0000 7f355affd640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f35440b6500 0x7f35440b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:06.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.106+0000 7f35610b7640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f35440b6500 0x7f35440b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:06.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.106+0000 7f355affd640 1 -- 172.21.15.112:0/2006321093 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f354c0e56a0 con 0x7f355c1048a0 2024-09-21T03:21:06.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.106+0000 7f35610b7640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f35440b6500 0x7f35440b8a10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3550000c00 tx=0x7f3550032da0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:06.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.110+0000 7f355affd640 1 -- 172.21.15.112:0/2006321093 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f354c021020 con 0x7f355c1048a0 2024-09-21T03:21:06.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.290+0000 7f3563b1c640 1 -- 172.21.15.112:0/2006321093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f355c1a9380 con 0x7f355c1048a0 2024-09-21T03:21:06.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.294+0000 7f355affd640 1 -- 172.21.15.112:0/2006321093 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v37)=0 v37) ==== 119+0+0 (secure 0 0 0) 0x7f354c0adb30 con 0x7f355c1048a0 2024-09-21T03:21:06.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.296+0000 7f3563b1c640 1 -- 172.21.15.112:0/2006321093 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f35440b6500 msgr2=0x7f35440b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:06.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.296+0000 7f3563b1c640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f35440b6500 0x7f35440b8a10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3550000c00 tx=0x7f3550032da0 comp rx=0 tx=0).stop 2024-09-21T03:21:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.296+0000 7f3563b1c640 1 -- 172.21.15.112:0/2006321093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c1048a0 msgr2=0x7f355c19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.296+0000 7f3563b1c640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c1048a0 0x7f355c19c700 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f354c00d950 tx=0x7f354c00de10 comp rx=0 tx=0).stop 2024-09-21T03:21:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.297+0000 7f3563b1c640 1 -- 172.21.15.112:0/2006321093 shutdown_connections 2024-09-21T03:21:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.297+0000 7f3563b1c640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f35440b6500 0x7f35440b8a10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.297+0000 7f3563b1c640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f355c10de50 0x7f355c1a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.297+0000 7f3563b1c640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c1052d0 0x7f355c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.297+0000 7f3563b1c640 1 --2- 172.21.15.112:0/2006321093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f355c1048a0 0x7f355c19c700 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.297+0000 7f3563b1c640 1 -- 172.21.15.112:0/2006321093 >> 172.21.15.112:0/2006321093 conn(0x7f355c079740 msgr2=0x7f355c10cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.297+0000 7f3563b1c640 1 -- 172.21.15.112:0/2006321093 shutdown_connections 2024-09-21T03:21:06.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:06.297+0000 7f3563b1c640 1 -- 172.21.15.112:0/2006321093 wait complete. 2024-09-21T03:21:06.926 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi112') 2024-09-21T03:21:06.926 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi112') 2024-09-21T03:21:06.926 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi112') 2024-09-21T03:21:06.927 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi112') 2024-09-21T03:21:06.927 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi112') 2024-09-21T03:21:06.927 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi112') 2024-09-21T03:21:06.927 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi112') 2024-09-21T03:21:06.927 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi112') 2024-09-21T03:21:06.927 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi112') 2024-09-21T03:21:06.927 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '063b32f6-77c8-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi112') 2024-09-21T03:21:06.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T03:21:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:07.559 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:07 smithi112 ceph-mon[31442]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:07.559 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:07 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2006321093' entity='client.admin' 2024-09-21T03:21:07.560 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:07 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:07.560 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:07 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:07.560 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:07 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:07.560 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:07 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:07.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.645+0000 7f69a1545640 1 -- 172.21.15.112:0/3889890136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f699c108cf0 msgr2=0x7f699c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:07.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.645+0000 7f69a1545640 1 --2- 172.21.15.112:0/3889890136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f699c108cf0 0x7f699c10f520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6994009900 tx=0x7f699402a410 comp rx=0 tx=0).stop 2024-09-21T03:21:07.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.645+0000 7f69a1545640 1 -- 172.21.15.112:0/3889890136 shutdown_connections 2024-09-21T03:21:07.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.645+0000 7f69a1545640 1 --2- 172.21.15.112:0/3889890136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f699c108cf0 0x7f699c10f520 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:07.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.645+0000 7f69a1545640 1 --2- 172.21.15.112:0/3889890136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f699c1083a0 0x7f699c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:07.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.645+0000 7f69a1545640 1 --2- 172.21.15.112:0/3889890136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f699c102a60 0x7f699c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:07.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.645+0000 7f69a1545640 1 -- 172.21.15.112:0/3889890136 >> 172.21.15.112:0/3889890136 conn(0x7f699c079730 msgr2=0x7f699c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:07.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.645+0000 7f69a1545640 1 -- 172.21.15.112:0/3889890136 shutdown_connections 2024-09-21T03:21:07.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.645+0000 7f69a1545640 1 -- 172.21.15.112:0/3889890136 wait complete. 2024-09-21T03:21:07.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.646+0000 7f69a1545640 1 Processor -- start 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.646+0000 7f69a1545640 1 -- start start 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.646+0000 7f69a1545640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f699c102a60 0x7f699c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.646+0000 7f69a1545640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f699c1083a0 0x7f699c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.646+0000 7f69a1545640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f699c108cf0 0x7f699c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.646+0000 7f69a1545640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f699c113a80 con 0x7f699c102a60 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.646+0000 7f69a1545640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f699c113900 con 0x7f699c1083a0 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.646+0000 7f69a1545640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f699c113c00 con 0x7f699c108cf0 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.646+0000 7f699affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f699c102a60 0x7f699c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.646+0000 7f699affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f699c102a60 0x7f699c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59330/0 (socket says 172.21.15.112:59330) 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.646+0000 7f699affd640 1 -- 172.21.15.112:0/2296285840 learned_addr learned my addr 172.21.15.112:0/2296285840 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f699a7fc640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f699c1083a0 0x7f699c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f699b7fe640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f699c108cf0 0x7f699c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:07.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f699affd640 1 -- 172.21.15.112:0/2296285840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f699c108cf0 msgr2=0x7f699c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f699affd640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f699c108cf0 0x7f699c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f699affd640 1 -- 172.21.15.112:0/2296285840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f699c1083a0 msgr2=0x7f699c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f699affd640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f699c1083a0 0x7f699c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f699affd640 1 -- 172.21.15.112:0/2296285840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c1a0400 con 0x7f699c102a60 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f699a7fc640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f699c1083a0 0x7f699c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f699affd640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f699c102a60 0x7f699c19a490 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f698c00ea80 tx=0x7f698c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f6993fff640 1 -- 172.21.15.112:0/2296285840 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c00ce40 con 0x7f699c102a60 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f69a1545640 1 -- 172.21.15.112:0/2296285840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c1a06f0 con 0x7f699c102a60 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f6993fff640 1 -- 172.21.15.112:0/2296285840 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698c005020 con 0x7f699c102a60 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f6993fff640 1 -- 172.21.15.112:0/2296285840 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0116d0 con 0x7f699c102a60 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.647+0000 7f69a1545640 1 -- 172.21.15.112:0/2296285840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c1a6e50 con 0x7f699c102a60 2024-09-21T03:21:07.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.648+0000 7f69a1545640 1 -- 172.21.15.112:0/2296285840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699c102e70 con 0x7f699c102a60 2024-09-21T03:21:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.651+0000 7f6993fff640 1 -- 172.21.15.112:0/2296285840 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f698c0054d0 con 0x7f699c102a60 2024-09-21T03:21:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.651+0000 7f6993fff640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69880b6550 0x7f69880b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.652+0000 7f699a7fc640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69880b6550 0x7f69880b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.652+0000 7f6993fff640 1 -- 172.21.15.112:0/2296285840 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f698c0e5f10 con 0x7f699c102a60 2024-09-21T03:21:07.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.652+0000 7f699a7fc640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69880b6550 0x7f69880b8a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f699c105ed0 tx=0x7f6984005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:07.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.655+0000 7f6993fff640 1 -- 172.21.15.112:0/2296285840 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f698c016030 con 0x7f699c102a60 2024-09-21T03:21:07.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:07 smithi203 ceph-mon[34723]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:07.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:07 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2006321093' entity='client.admin' 2024-09-21T03:21:07.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:07 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:07.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:07 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:07.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:07 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:07.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:07 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:07 smithi187 ceph-mon[34603]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:07 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2006321093' entity='client.admin' 2024-09-21T03:21:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:07 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:07 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:07 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:07 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:07.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.832+0000 7f69a1545640 1 -- 172.21.15.112:0/2296285840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f699c19b5c0 con 0x7f699c102a60 2024-09-21T03:21:07.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.833+0000 7f6993fff640 1 -- 172.21.15.112:0/2296285840 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+22127 (secure 0 0 0) 0x7f699c19b5c0 con 0x7f699c102a60 2024-09-21T03:21:07.835 INFO:teuthology.orchestra.run.smithi112.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global dev auth_debug true 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global basic log_to_file true 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global basic log_to_journald false 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global basic log_to_stderr false 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global advanced mon_allow_pool_delete true 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T03:21:07.836 INFO:teuthology.orchestra.run.smithi112.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:global dev ms_die_on_bug true 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:global dev ms_die_on_old_message true 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:global dev ms_die_on_skipped_message false 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced debug_mon 20/20 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced debug_ms 1/1 2024-09-21T03:21:07.837 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced debug_paxos 20/20 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:21:07.838 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_client 20/20 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:21:07.839 INFO:teuthology.orchestra.run.smithi112.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd dev bdev_debug_aio true 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced bdev_enable_discard true 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced bluestore_allocator bitmap * 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced debug_bluefs 1/20 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced debug_bluestore 1/20 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced debug_ms 1/1 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced debug_osd 20/20 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced debug_rocksdb 4/10 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd dev osd_debug_op_order true 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd dev osd_debug_shutdown true 2024-09-21T03:21:07.840 INFO:teuthology.orchestra.run.smithi112.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_max_omap_entries_per_request 10 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T03:21:07.841 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds 20/20 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced debug_ms 2/2 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_bal_split_size 100 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_frag true 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds dev mds_verify_scatter true 2024-09-21T03:21:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:21:07.843 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:21:07.843 INFO:teuthology.orchestra.run.smithi112.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:21:07.843 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced client_mount_timeout 600 2024-09-21T03:21:07.843 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_client 20/20 2024-09-21T03:21:07.843 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced debug_ms 1/1 2024-09-21T03:21:07.843 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:21:07.843 INFO:teuthology.orchestra.run.smithi112.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:21:07.843 INFO:teuthology.orchestra.run.smithi112.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T03:21:07.843 INFO:teuthology.orchestra.run.smithi112.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T03:21:07.843 INFO:teuthology.orchestra.run.smithi112.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T03:21:07.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 -- 172.21.15.112:0/2296285840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69880b6550 msgr2=0x7f69880b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:07.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69880b6550 0x7f69880b8a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f699c105ed0 tx=0x7f6984005f90 comp rx=0 tx=0).stop 2024-09-21T03:21:07.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 -- 172.21.15.112:0/2296285840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f699c102a60 msgr2=0x7f699c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:07.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f699c102a60 0x7f699c19a490 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f698c00ea80 tx=0x7f698c00ef40 comp rx=0 tx=0).stop 2024-09-21T03:21:07.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 -- 172.21.15.112:0/2296285840 shutdown_connections 2024-09-21T03:21:07.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69880b6550 0x7f69880b8a60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:07.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f699c108cf0 0x7f699c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:07.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f699c1083a0 0x7f699c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:07.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 --2- 172.21.15.112:0/2296285840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f699c102a60 0x7f699c19a490 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:07.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 -- 172.21.15.112:0/2296285840 >> 172.21.15.112:0/2296285840 conn(0x7f699c079730 msgr2=0x7f699c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:07.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 -- 172.21.15.112:0/2296285840 shutdown_connections 2024-09-21T03:21:07.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:07.836+0000 7f69a1545640 1 -- 172.21.15.112:0/2296285840 wait complete. 2024-09-21T03:21:08.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:08 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2296285840' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:21:08.498 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T03:21:08.498 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:21:08.498 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:21:08.514 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:21:08.515 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-09-21T03:21:08.569 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:21:08.569 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:08.569 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T03:21:08.569 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:08.570 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:08.570 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 03:19:45.400639702 +0000 2024-09-21T03:21:08.570 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 03:14:57.819430821 +0000 2024-09-21T03:21:08.570 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 03:14:57.819430821 +0000 2024-09-21T03:21:08.570 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 03:14:57.819430821 +0000 2024-09-21T03:21:08.570 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:21:08.633 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T03:21:08.633 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T03:21:08.633 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000219227 s, 2.3 MB/s 2024-09-21T03:21:08.634 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:21:08.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:08 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2296285840' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:21:08.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:08 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2296285840' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:21:08.690 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-09-21T03:21:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:21:08.746 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:08.746 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-21T03:21:08.746 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:08.746 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:08.746 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 03:19:45.433638792 +0000 2024-09-21T03:21:08.746 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 03:14:58.142421876 +0000 2024-09-21T03:21:08.747 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 03:14:58.142421876 +0000 2024-09-21T03:21:08.747 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 03:14:58.142421876 +0000 2024-09-21T03:21:08.747 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:21:08.809 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T03:21:08.809 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T03:21:08.809 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.00019064 s, 2.7 MB/s 2024-09-21T03:21:08.810 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:21:08.866 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-09-21T03:21:08.921 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:21:08.921 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:08.922 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T03:21:08.922 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:08.922 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:08.922 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 03:19:45.466637880 +0000 2024-09-21T03:21:08.922 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 03:14:58.456413180 +0000 2024-09-21T03:21:08.922 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 03:14:58.456413180 +0000 2024-09-21T03:21:08.922 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 03:14:58.456413180 +0000 2024-09-21T03:21:08.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:21:08.985 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T03:21:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T03:21:08.986 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000254601 s, 2.0 MB/s 2024-09-21T03:21:08.987 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:21:09.043 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-09-21T03:21:09.098 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:21:09.098 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:09.098 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-21T03:21:09.098 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:09.098 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:09.098 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 03:19:45.498636997 +0000 2024-09-21T03:21:09.099 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 03:14:58.777404290 +0000 2024-09-21T03:21:09.099 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 03:14:58.777404290 +0000 2024-09-21T03:21:09.099 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 03:14:58.777404290 +0000 2024-09-21T03:21:09.099 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:21:09.161 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T03:21:09.161 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T03:21:09.161 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000173969 s, 2.9 MB/s 2024-09-21T03:21:09.162 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:21:09.218 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-09-21T03:21:09.219 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:21:09.238 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:21:09.238 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2024-09-21T03:21:09.297 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:21:09.297 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:09.297 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T03:21:09.297 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:09.297 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:09.297 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-09-21 03:20:40.326895065 +0000 2024-09-21T03:21:09.298 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-09-21 03:15:11.594007326 +0000 2024-09-21T03:21:09.298 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-09-21 03:15:11.594007326 +0000 2024-09-21T03:21:09.298 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-09-21 03:15:11.594007326 +0000 2024-09-21T03:21:09.298 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:21:09.368 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-09-21T03:21:09.368 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-09-21T03:21:09.368 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000287967 s, 1.8 MB/s 2024-09-21T03:21:09.370 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:21:09.428 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2024-09-21T03:21:09.485 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:21:09.485 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:09.486 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T03:21:09.486 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:09.486 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:09.486 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-09-21 03:20:40.326895065 +0000 2024-09-21T03:21:09.486 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-09-21 03:15:11.919001854 +0000 2024-09-21T03:21:09.486 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-09-21 03:15:11.919001854 +0000 2024-09-21T03:21:09.486 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-09-21 03:15:11.919001854 +0000 2024-09-21T03:21:09.486 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:21:09.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:09 smithi112 ceph-mon[31442]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:09.551 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-09-21T03:21:09.551 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-09-21T03:21:09.551 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000209413 s, 2.4 MB/s 2024-09-21T03:21:09.552 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:09 smithi187 ceph-mon[34603]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:09.552 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:21:09.610 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2024-09-21T03:21:09.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:09 smithi203 ceph-mon[34723]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:09.666 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:21:09.666 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:09.666 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-21T03:21:09.667 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:09.667 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:09.667 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-09-21 03:20:40.327895049 +0000 2024-09-21T03:21:09.667 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-09-21 03:15:12.244996366 +0000 2024-09-21T03:21:09.667 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-09-21 03:15:12.244996366 +0000 2024-09-21T03:21:09.667 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-09-21 03:15:12.244996366 +0000 2024-09-21T03:21:09.667 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:21:09.731 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-09-21T03:21:09.731 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-09-21T03:21:09.731 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000218086 s, 2.3 MB/s 2024-09-21T03:21:09.732 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:21:09.789 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2024-09-21T03:21:09.845 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:21:09.846 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:09.846 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T03:21:09.846 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:09.846 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:09.846 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-09-21 03:20:40.327895049 +0000 2024-09-21T03:21:09.846 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-09-21 03:15:12.555991130 +0000 2024-09-21T03:21:09.846 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-09-21 03:15:12.555991130 +0000 2024-09-21T03:21:09.846 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-09-21 03:15:12.555991130 +0000 2024-09-21T03:21:09.846 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:21:09.909 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-09-21T03:21:09.910 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-09-21T03:21:09.910 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000176865 s, 2.9 MB/s 2024-09-21T03:21:09.911 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:21:09.967 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-21T03:21:09.967 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:21:09.987 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:21:09.987 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-09-21T03:21:10.046 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:21:10.046 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:10.046 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T03:21:10.046 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:10.046 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:10.047 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-21 03:20:37.896441714 +0000 2024-09-21T03:21:10.047 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-21 03:15:19.255858517 +0000 2024-09-21T03:21:10.047 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-21 03:15:19.255858517 +0000 2024-09-21T03:21:10.047 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-21 03:15:19.255858517 +0000 2024-09-21T03:21:10.047 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:21:10.116 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-21T03:21:10.116 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-21T03:21:10.116 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000285142 s, 1.8 MB/s 2024-09-21T03:21:10.117 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:21:10.175 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-09-21T03:21:10.231 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:21:10.232 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:10.232 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-21T03:21:10.232 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:10.232 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:10.232 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-21 03:20:37.897441688 +0000 2024-09-21T03:21:10.232 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-21 03:15:19.606849320 +0000 2024-09-21T03:21:10.232 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-21 03:15:19.606849320 +0000 2024-09-21T03:21:10.232 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-21 03:15:19.606849320 +0000 2024-09-21T03:21:10.232 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:21:10.297 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-21T03:21:10.297 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-21T03:21:10.297 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000203622 s, 2.5 MB/s 2024-09-21T03:21:10.298 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:21:10.355 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-09-21T03:21:10.411 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:21:10.411 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:10.412 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-21T03:21:10.412 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:10.412 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:10.412 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-21 03:20:37.897441688 +0000 2024-09-21T03:21:10.412 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-21 03:15:19.926840936 +0000 2024-09-21T03:21:10.412 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-21 03:15:19.926840936 +0000 2024-09-21T03:21:10.412 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-21 03:15:19.926840936 +0000 2024-09-21T03:21:10.412 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:21:10.475 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-21T03:21:10.475 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-21T03:21:10.475 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000188641 s, 2.7 MB/s 2024-09-21T03:21:10.476 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:21:10.532 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-09-21T03:21:10.587 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:21:10.588 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:21:10.588 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T03:21:10.588 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:21:10.588 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:21:10.588 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-21 03:20:37.897441688 +0000 2024-09-21T03:21:10.588 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-21 03:15:20.264832081 +0000 2024-09-21T03:21:10.588 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-21 03:15:20.264832081 +0000 2024-09-21T03:21:10.588 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-21 03:15:20.264832081 +0000 2024-09-21T03:21:10.588 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:21:10.650 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-21T03:21:10.650 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-21T03:21:10.651 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000190348 s, 2.7 MB/s 2024-09-21T03:21:10.652 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:21:10.708 INFO:tasks.cephadm:Deploying osd.0 on smithi112 with /dev/vg_nvme/lv_4... 2024-09-21T03:21:10.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:21:10.958 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:11.349 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:11 smithi112 ceph-mon[31442]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:11.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:11 smithi203 ceph-mon[34723]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:11 smithi187 ceph-mon[34603]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:13.262 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:21:13.287 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-09-21T03:21:13.532 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:21:13.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:13 smithi112 ceph-mon[31442]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:13.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:13 smithi203 ceph-mon[34723]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:13 smithi187 ceph-mon[34603]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:13.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.978+0000 7ff9d64d6640 1 -- 172.21.15.112:0/2202607433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d01094a0 msgr2=0x7ff9d01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:13.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.978+0000 7ff9d64d6640 1 --2- 172.21.15.112:0/2202607433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d01094a0 0x7ff9d01098b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009900 tx=0x7ff9b802a410 comp rx=0 tx=0).stop 2024-09-21T03:21:13.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.978+0000 7ff9d64d6640 1 -- 172.21.15.112:0/2202607433 shutdown_connections 2024-09-21T03:21:13.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.978+0000 7ff9d64d6640 1 --2- 172.21.15.112:0/2202607433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d0109df0 0x7ff9d0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:13.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.978+0000 7ff9d64d6640 1 --2- 172.21.15.112:0/2202607433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d01094a0 0x7ff9d01098b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:13.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.978+0000 7ff9d64d6640 1 --2- 172.21.15.112:0/2202607433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d0103b60 0x7ff9d0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:13.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.978+0000 7ff9d64d6640 1 -- 172.21.15.112:0/2202607433 >> 172.21.15.112:0/2202607433 conn(0x7ff9d007b850 msgr2=0x7ff9d007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:13.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.978+0000 7ff9d64d6640 1 -- 172.21.15.112:0/2202607433 shutdown_connections 2024-09-21T03:21:13.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.978+0000 7ff9d64d6640 1 -- 172.21.15.112:0/2202607433 wait complete. 2024-09-21T03:21:13.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d64d6640 1 Processor -- start 2024-09-21T03:21:13.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d64d6640 1 -- start start 2024-09-21T03:21:13.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d64d6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d0103b60 0x7ff9d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:13.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d64d6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d01094a0 0x7ff9d0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:13.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d64d6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d0109df0 0x7ff9d0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:13.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d64d6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d0113ab0 con 0x7ff9d0109df0 2024-09-21T03:21:13.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d64d6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d0113930 con 0x7ff9d01094a0 2024-09-21T03:21:13.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d64d6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d0113c30 con 0x7ff9d0103b60 2024-09-21T03:21:13.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d4a73640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d0109df0 0x7ff9d0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:13.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d4a73640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d0109df0 0x7ff9d0076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59340/0 (socket says 172.21.15.112:59340) 2024-09-21T03:21:13.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d4a73640 1 -- 172.21.15.112:0/1700598338 learned_addr learned my addr 172.21.15.112:0/1700598338 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:21:13.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9cffff640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d0103b60 0x7ff9d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:13.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9c7fff640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d01094a0 0x7ff9d0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:13.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d4a73640 1 -- 172.21.15.112:0/1700598338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d0103b60 msgr2=0x7ff9d0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:13.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d4a73640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d0103b60 0x7ff9d0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:13.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d4a73640 1 -- 172.21.15.112:0/1700598338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d01094a0 msgr2=0x7ff9d0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:13.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d4a73640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d01094a0 0x7ff9d0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:13.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.979+0000 7ff9d4a73640 1 -- 172.21.15.112:0/1700598338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d01a8820 con 0x7ff9d0109df0 2024-09-21T03:21:13.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.980+0000 7ff9d4a73640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d0109df0 0x7ff9d0076190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800eaa0 tx=0x7ff9c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:13.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.980+0000 7ff9cdffb640 1 -- 172.21.15.112:0/1700598338 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c800cdb0 con 0x7ff9d0109df0 2024-09-21T03:21:13.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.980+0000 7ff9d64d6640 1 -- 172.21.15.112:0/1700598338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d01a8b10 con 0x7ff9d0109df0 2024-09-21T03:21:13.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.980+0000 7ff9cdffb640 1 -- 172.21.15.112:0/1700598338 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c8005020 con 0x7ff9d0109df0 2024-09-21T03:21:13.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.980+0000 7ff9cdffb640 1 -- 172.21.15.112:0/1700598338 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80116d0 con 0x7ff9d0109df0 2024-09-21T03:21:13.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.980+0000 7ff9d64d6640 1 -- 172.21.15.112:0/1700598338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d01a9070 con 0x7ff9d0109df0 2024-09-21T03:21:13.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.981+0000 7ff9d64d6640 1 -- 172.21.15.112:0/1700598338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d0103f70 con 0x7ff9d0109df0 2024-09-21T03:21:13.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.983+0000 7ff9cdffb640 1 -- 172.21.15.112:0/1700598338 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7ff9c80054a0 con 0x7ff9d0109df0 2024-09-21T03:21:13.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.983+0000 7ff9cdffb640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9bc0b64d0 0x7ff9bc0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:13.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.983+0000 7ff9cffff640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9bc0b64d0 0x7ff9bc0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:13.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.983+0000 7ff9cdffb640 1 -- 172.21.15.112:0/1700598338 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff9c80e5c70 con 0x7ff9d0109df0 2024-09-21T03:21:13.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.984+0000 7ff9cffff640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9bc0b64d0 0x7ff9bc0b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0077160 tx=0x7ff9c000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:13.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:13.987+0000 7ff9cdffb640 1 -- 172.21.15.112:0/1700598338 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff9c8016030 con 0x7ff9d0109df0 2024-09-21T03:21:14.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:14.174+0000 7ff9d64d6640 1 -- 172.21.15.112:0/1700598338 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff9d0063800 con 0x7ff9bc0b64d0 2024-09-21T03:21:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:14 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:21:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:14 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:21:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:14 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:14.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:14 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:21:14.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:14 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:21:14.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:14 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:14.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:14 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:21:14.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:14 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:21:14.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:14 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:15.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:15 smithi112 ceph-mon[31442]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:15.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:15 smithi112 ceph-mon[31442]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:15.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:15 smithi203 ceph-mon[34723]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:15.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:15 smithi203 ceph-mon[34723]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:15 smithi187 ceph-mon[34603]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:15 smithi187 ceph-mon[34603]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:17.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:17 smithi203 ceph-mon[34723]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:17.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:17 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3610085574' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ec165c44-c973-4f2b-b1ca-a88a9bf48968"} : dispatch 2024-09-21T03:21:17.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:17 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3610085574' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec165c44-c973-4f2b-b1ca-a88a9bf48968"}]': finished 2024-09-21T03:21:17.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:17 smithi203 ceph-mon[34723]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:21:17.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:17 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:17.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:17 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3404717739' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:21:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:17 smithi187 ceph-mon[34603]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:17 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3610085574' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ec165c44-c973-4f2b-b1ca-a88a9bf48968"} : dispatch 2024-09-21T03:21:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:17 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3610085574' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec165c44-c973-4f2b-b1ca-a88a9bf48968"}]': finished 2024-09-21T03:21:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:17 smithi187 ceph-mon[34603]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:21:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:17 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:17 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3404717739' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:21:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:17 smithi112 ceph-mon[31442]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:17 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3610085574' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ec165c44-c973-4f2b-b1ca-a88a9bf48968"} : dispatch 2024-09-21T03:21:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:17 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3610085574' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec165c44-c973-4f2b-b1ca-a88a9bf48968"}]': finished 2024-09-21T03:21:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:17 smithi112 ceph-mon[31442]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:21:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:17 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:17 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3404717739' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:21:19.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:19 smithi203 ceph-mon[34723]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:19.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:19 smithi187 ceph-mon[34603]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:19 smithi112 ceph-mon[31442]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:21.650 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:21 smithi112 ceph-mon[31442]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:21.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:21 smithi203 ceph-mon[34723]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:21 smithi187 ceph-mon[34603]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:23.525 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:23 smithi112 ceph-mon[31442]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:23.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:23 smithi203 ceph-mon[34723]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:23.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:23 smithi187 ceph-mon[34603]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:25.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:25 smithi203 ceph-mon[34723]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:25.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:25 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:21:25.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:25 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:25 smithi187 ceph-mon[34603]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:25 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:21:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:25 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:25 smithi112 ceph-mon[31442]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:25 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:21:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:25 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:26.428 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:26 smithi112 ceph-mon[31442]: Deploying daemon osd.0 on smithi112 2024-09-21T03:21:26.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:26 smithi203 ceph-mon[34723]: Deploying daemon osd.0 on smithi112 2024-09-21T03:21:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:26 smithi187 ceph-mon[34603]: Deploying daemon osd.0 on smithi112 2024-09-21T03:21:27.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:27 smithi203 ceph-mon[34723]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:27 smithi187 ceph-mon[34603]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:27.756 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:27 smithi112 ceph-mon[31442]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:29 smithi203 ceph-mon[34723]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:29 smithi187 ceph-mon[34603]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:29.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:29 smithi112 ceph-mon[31442]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:31.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:31 smithi112 ceph-mon[31442]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:31.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:31 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:31.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:31 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:31.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:31 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:31 smithi203 ceph-mon[34723]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:31 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:31 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:31 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:31 smithi187 ceph-mon[34603]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:31 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:31 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:31 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:33.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:33 smithi112 ceph-mon[31442]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:33.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:33 smithi203 ceph-mon[34723]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:33.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:33 smithi187 ceph-mon[34603]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:34.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.359+0000 7ff9cdffb640 1 -- 172.21.15.112:0/1700598338 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff9d0063800 con 0x7ff9bc0b64d0 2024-09-21T03:21:34.361 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 0 on host 'smithi112' 2024-09-21T03:21:34.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.361+0000 7ff9d64d6640 1 -- 172.21.15.112:0/1700598338 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9bc0b64d0 msgr2=0x7ff9bc0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:34.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.361+0000 7ff9d64d6640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9bc0b64d0 0x7ff9bc0b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0077160 tx=0x7ff9c000b390 comp rx=0 tx=0).stop 2024-09-21T03:21:34.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.361+0000 7ff9d64d6640 1 -- 172.21.15.112:0/1700598338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d0109df0 msgr2=0x7ff9d0076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:34.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.361+0000 7ff9d64d6640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d0109df0 0x7ff9d0076190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800eaa0 tx=0x7ff9c800ef60 comp rx=0 tx=0).stop 2024-09-21T03:21:34.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.361+0000 7ff9d64d6640 1 -- 172.21.15.112:0/1700598338 shutdown_connections 2024-09-21T03:21:34.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.361+0000 7ff9d64d6640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9bc0b64d0 0x7ff9bc0b89e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:34.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.361+0000 7ff9d64d6640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9d0109df0 0x7ff9d0076190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:34.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.361+0000 7ff9d64d6640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d01094a0 0x7ff9d0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:34.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.361+0000 7ff9d64d6640 1 --2- 172.21.15.112:0/1700598338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9d0103b60 0x7ff9d0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:34.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.361+0000 7ff9d64d6640 1 -- 172.21.15.112:0/1700598338 >> 172.21.15.112:0/1700598338 conn(0x7ff9d007b850 msgr2=0x7ff9d0105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:34.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.362+0000 7ff9d64d6640 1 -- 172.21.15.112:0/1700598338 shutdown_connections 2024-09-21T03:21:34.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:21:34.362+0000 7ff9d64d6640 1 -- 172.21.15.112:0/1700598338 wait complete. 2024-09-21T03:21:34.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:34.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:34.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:34 smithi112 ceph-mon[31442]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:34.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:34.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:34 smithi203 ceph-mon[34723]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:34 smithi187 ceph-mon[34603]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:34.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:34.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:35.024 DEBUG:teuthology.orchestra.run.smithi112:osd.0> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T03:21:35.026 INFO:tasks.cephadm:Deploying osd.1 on smithi187 with /dev/vg_nvme/lv_4... 2024-09-21T03:21:35.026 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:21:35.265 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:21:37.239 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:21:36 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0[48575]: 2024-09-21T03:21:36.979+0000 7f16e0fcb640 -1 osd.0 0 log_to_monitors true 2024-09-21T03:21:37.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:37 smithi112 ceph-mon[31442]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:37.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:37 smithi112 ceph-mon[31442]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:21:37.632 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:37 smithi187 ceph-mon[34603]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:37.632 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:37 smithi187 ceph-mon[34603]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:21:37.635 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:21:37.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:37 smithi203 ceph-mon[34723]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:37.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:37 smithi203 ceph-mon[34723]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:21:37.661 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi187:vg_nvme/lv_4 2024-09-21T03:21:37.911 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:21:38.386 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.362+0000 7f12e1d5a640 1 -- 172.21.15.187:0/2130582080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12dc102a60 msgr2=0x7f12dc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:38.386 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.362+0000 7f12e1d5a640 1 --2- 172.21.15.187:0/2130582080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12dc102a60 0x7f12dc102e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009900 tx=0x7f12cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:21:38.386 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.362+0000 7f12e1d5a640 1 -- 172.21.15.187:0/2130582080 shutdown_connections 2024-09-21T03:21:38.386 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.362+0000 7f12e1d5a640 1 --2- 172.21.15.187:0/2130582080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12dc108cf0 0x7f12dc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:38.386 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.362+0000 7f12e1d5a640 1 --2- 172.21.15.187:0/2130582080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12dc1083a0 0x7f12dc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.362+0000 7f12e1d5a640 1 --2- 172.21.15.187:0/2130582080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12dc102a60 0x7f12dc102e70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.362+0000 7f12e1d5a640 1 -- 172.21.15.187:0/2130582080 >> 172.21.15.187:0/2130582080 conn(0x7f12dc079730 msgr2=0x7f12dc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.362+0000 7f12e1d5a640 1 -- 172.21.15.187:0/2130582080 shutdown_connections 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.362+0000 7f12e1d5a640 1 -- 172.21.15.187:0/2130582080 wait complete. 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.362+0000 7f12e1d5a640 1 Processor -- start 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.362+0000 7f12e1d5a640 1 -- start start 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12e1d5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12dc102a60 0x7f12dc19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12e1d5a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12dc1083a0 0x7f12dc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12e1d5a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12dc108cf0 0x7f12dc19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12e1d5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc1129b0 con 0x7f12dc102a60 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12e1d5a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc112830 con 0x7f12dc108cf0 2024-09-21T03:21:38.387 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12e1d5a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc112b30 con 0x7f12dc1083a0 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12dbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12dc108cf0 0x7f12dc19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12dbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12dc108cf0 0x7f12dc19fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:53580/0 (socket says 172.21.15.187:53580) 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12dbfff640 1 -- 172.21.15.187:0/1384847559 learned_addr learned my addr 172.21.15.187:0/1384847559 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12d3fff640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12dc1083a0 0x7f12dc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12db7fe640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12dc102a60 0x7f12dc19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12dbfff640 1 -- 172.21.15.187:0/1384847559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12dc1083a0 msgr2=0x7f12dc19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12dbfff640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12dc1083a0 0x7f12dc19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12dbfff640 1 -- 172.21.15.187:0/1384847559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12dc102a60 msgr2=0x7f12dc19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12dbfff640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12dc102a60 0x7f12dc19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12dbfff640 1 -- 172.21.15.187:0/1384847559 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc1a0480 con 0x7f12dc108cf0 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12db7fe640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12dc102a60 0x7f12dc19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:21:38.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12dbfff640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12dc108cf0 0x7f12dc19fd50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f12d400eaa0 tx=0x7f12d400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.363+0000 7f12d97fa640 1 -- 172.21.15.187:0/1384847559 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d400cdb0 con 0x7f12dc108cf0 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.364+0000 7f12d97fa640 1 -- 172.21.15.187:0/1384847559 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d4005020 con 0x7f12dc108cf0 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.364+0000 7f12e1d5a640 1 -- 172.21.15.187:0/1384847559 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12dc1a0770 con 0x7f12dc108cf0 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.364+0000 7f12d97fa640 1 -- 172.21.15.187:0/1384847559 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d40116d0 con 0x7f12dc108cf0 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.364+0000 7f12e1d5a640 1 -- 172.21.15.187:0/1384847559 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12dc1a6ed0 con 0x7f12dc108cf0 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.365+0000 7f12e1d5a640 1 -- 172.21.15.187:0/1384847559 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12dc102e70 con 0x7f12dc108cf0 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.366+0000 7f12d97fa640 1 -- 172.21.15.187:0/1384847559 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f12d40051d0 con 0x7f12dc108cf0 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.367+0000 7f12d97fa640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f12c80b6550 0x7f12c80b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.367+0000 7f12db7fe640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f12c80b6550 0x7f12c80b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.367+0000 7f12d97fa640 1 -- 172.21.15.187:0/1384847559 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1702+0+0 (secure 0 0 0) 0x7f12d40e5d90 con 0x7f12dc108cf0 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.368+0000 7f12db7fe640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f12c80b6550 0x7f12c80b8a60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f12cc000c00 tx=0x7f12cc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:21:38.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.371+0000 7f12d97fa640 1 -- 172.21.15.187:0/1384847559 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12d4016030 con 0x7f12dc108cf0 2024-09-21T03:21:38.559 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:38.557+0000 7f12e1d5a640 1 -- 172.21.15.187:0/1384847559 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f12dc101670 con 0x7f12c80b6550 2024-09-21T03:21:38.559 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:21:38.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:21:38.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:21:38.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:38.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:21:38.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:38.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:38.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:38 smithi187 ceph-mon[34603]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:21:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:21:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:21:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:21:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:21:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:38 smithi203 ceph-mon[34723]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:21:38.787 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0[48575]: 2024-09-21T03:21:38.327+0000 7f16dcf4d640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T03:21:38.787 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0[48575]: 2024-09-21T03:21:38.331+0000 7f16d8d4b640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:21:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:21:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:21:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:21:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:21:38.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:38.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:38.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:38.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:38 smithi112 ceph-mon[31442]: from='osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:21:39.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:39 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi112 2024-09-21T03:21:39.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:39 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi112 to 17088M 2024-09-21T03:21:39.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:39 smithi187 ceph-mon[34603]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:39.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:39 smithi187 ceph-mon[34603]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:21:39.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:39.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:39.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:21:39.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:21:39.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:39 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi112 2024-09-21T03:21:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:39 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi112 to 17088M 2024-09-21T03:21:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:39 smithi203 ceph-mon[34723]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:39 smithi203 ceph-mon[34723]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:21:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:21:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:21:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:39 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi112 2024-09-21T03:21:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:39 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi112 to 17088M 2024-09-21T03:21:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:39 smithi112 ceph-mon[31442]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:39 smithi112 ceph-mon[31442]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:21:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:21:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:21:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:40.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:40 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:21:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:40 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:21:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:40 smithi203 ceph-mon[34723]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:40 smithi203 ceph-mon[34723]: osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796] boot 2024-09-21T03:21:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:40 smithi203 ceph-mon[34723]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:21:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:40 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:40 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:21:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:40 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:21:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:40 smithi187 ceph-mon[34603]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:40 smithi187 ceph-mon[34603]: osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796] boot 2024-09-21T03:21:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:40 smithi187 ceph-mon[34603]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:21:40.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:40 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:40 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:21:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:40 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:21:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:40 smithi112 ceph-mon[31442]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:21:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:40 smithi112 ceph-mon[31442]: osd.0 [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796] boot 2024-09-21T03:21:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:40 smithi112 ceph-mon[31442]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:21:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:40 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:21:41.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:41 smithi203 ceph-mon[34723]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:41.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:41 smithi203 ceph-mon[34723]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:21:41.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:41 smithi203 ceph-mon[34723]: from='client.? 172.21.15.187:0/2163161116' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "200471f7-a0d9-4b5e-8be4-0033c6a82688"} : dispatch 2024-09-21T03:21:41.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:41 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "200471f7-a0d9-4b5e-8be4-0033c6a82688"} : dispatch 2024-09-21T03:21:41.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:41 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "200471f7-a0d9-4b5e-8be4-0033c6a82688"}]': finished 2024-09-21T03:21:41.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:41 smithi203 ceph-mon[34723]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:21:41.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:41 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:21:41.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:41 smithi187 ceph-mon[34603]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:41.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:41 smithi187 ceph-mon[34603]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:21:41.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:41 smithi187 ceph-mon[34603]: from='client.? 172.21.15.187:0/2163161116' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "200471f7-a0d9-4b5e-8be4-0033c6a82688"} : dispatch 2024-09-21T03:21:41.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:41 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "200471f7-a0d9-4b5e-8be4-0033c6a82688"} : dispatch 2024-09-21T03:21:41.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:41 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "200471f7-a0d9-4b5e-8be4-0033c6a82688"}]': finished 2024-09-21T03:21:41.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:41 smithi187 ceph-mon[34603]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:21:41.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:41 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:21:41.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:41 smithi112 ceph-mon[31442]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:21:41.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:41 smithi112 ceph-mon[31442]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:21:41.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:41 smithi112 ceph-mon[31442]: from='client.? 172.21.15.187:0/2163161116' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "200471f7-a0d9-4b5e-8be4-0033c6a82688"} : dispatch 2024-09-21T03:21:41.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:41 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "200471f7-a0d9-4b5e-8be4-0033c6a82688"} : dispatch 2024-09-21T03:21:41.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:41 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "200471f7-a0d9-4b5e-8be4-0033c6a82688"}]': finished 2024-09-21T03:21:41.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:41 smithi112 ceph-mon[31442]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:21:41.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:41 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:21:42.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:42 smithi203 ceph-mon[34723]: from='client.? 172.21.15.187:0/3695867022' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:21:42.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:42 smithi187 ceph-mon[34603]: from='client.? 172.21.15.187:0/3695867022' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:21:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:42 smithi112 ceph-mon[31442]: from='client.? 172.21.15.187:0/3695867022' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:21:43.598 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:43 smithi187 ceph-mon[34603]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:43.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:43 smithi203 ceph-mon[34723]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:43 smithi112 ceph-mon[31442]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:45.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:45 smithi203 ceph-mon[34723]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:45.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:45 smithi187 ceph-mon[34603]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:45 smithi112 ceph-mon[31442]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:47.511 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:47 smithi187 ceph-mon[34603]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:47.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:47 smithi203 ceph-mon[34723]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:47.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:47 smithi112 ceph-mon[31442]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:49.613 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:49 smithi187 ceph-mon[34603]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:49.614 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:49 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:21:49.614 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:49 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:49 smithi203 ceph-mon[34723]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:49 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:21:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:49 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:49 smithi112 ceph-mon[31442]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:49 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:21:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:49 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:50.634 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:50 smithi187 ceph-mon[34603]: Deploying daemon osd.1 on smithi187 2024-09-21T03:21:50.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:50 smithi203 ceph-mon[34723]: Deploying daemon osd.1 on smithi187 2024-09-21T03:21:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:50 smithi112 ceph-mon[31442]: Deploying daemon osd.1 on smithi187 2024-09-21T03:21:51.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:51 smithi187 ceph-mon[34603]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:51.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:51 smithi203 ceph-mon[34723]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:51.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:51 smithi112 ceph-mon[31442]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:52.673 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:52 smithi187 ceph-mon[34603]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:52.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:52 smithi112 ceph-mon[31442]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:52.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:52 smithi203 ceph-mon[34723]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:55.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:55 smithi203 ceph-mon[34723]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:55.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:55 smithi187 ceph-mon[34603]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:55.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:55 smithi112 ceph-mon[31442]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:56.620 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:56 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:56.620 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:56 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:56.620 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:56 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:56.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:56 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:56.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:56 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:56.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:56 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:56 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:21:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:56 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:56.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:56 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:57.531 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:57 smithi187 ceph-mon[34603]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:57.531 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:57.531 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:57.531 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:57.531 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:57.531 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:57 smithi203 ceph-mon[34723]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:57.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:57 smithi112 ceph-mon[31442]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:57.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:57.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:57.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:21:57.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:21:57.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:21:59.421 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:21:59 smithi187 ceph-mon[34603]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:59.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:21:59 smithi203 ceph-mon[34723]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:21:59.739 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.736+0000 7f12d97fa640 1 -- 172.21.15.187:0/1384847559 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f12dc101670 con 0x7f12c80b6550 2024-09-21T03:21:59.739 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 1 on host 'smithi187' 2024-09-21T03:21:59.741 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 -- 172.21.15.187:0/1384847559 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f12c80b6550 msgr2=0x7f12c80b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:59.741 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f12c80b6550 0x7f12c80b8a60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f12cc000c00 tx=0x7f12cc031400 comp rx=0 tx=0).stop 2024-09-21T03:21:59.741 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 -- 172.21.15.187:0/1384847559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12dc108cf0 msgr2=0x7f12dc19fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:21:59.741 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12dc108cf0 0x7f12dc19fd50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f12d400eaa0 tx=0x7f12d400ef60 comp rx=0 tx=0).stop 2024-09-21T03:21:59.741 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 -- 172.21.15.187:0/1384847559 shutdown_connections 2024-09-21T03:21:59.741 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f12c80b6550 0x7f12c80b8a60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:59.742 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12dc108cf0 0x7f12dc19fd50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:59.742 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12dc1083a0 0x7f12dc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:59.742 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 --2- 172.21.15.187:0/1384847559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12dc102a60 0x7f12dc19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:21:59.742 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 -- 172.21.15.187:0/1384847559 >> 172.21.15.187:0/1384847559 conn(0x7f12dc079730 msgr2=0x7f12dc1066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:21:59.742 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 -- 172.21.15.187:0/1384847559 shutdown_connections 2024-09-21T03:21:59.742 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:21:59.739+0000 7f12e1d5a640 1 -- 172.21.15.187:0/1384847559 wait complete. 2024-09-21T03:21:59.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:21:59 smithi112 ceph-mon[31442]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:00.398 DEBUG:teuthology.orchestra.run.smithi187:osd.1> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T03:22:00.400 INFO:tasks.cephadm:Deploying osd.2 on smithi203 with /dev/vg_nvme/lv_4... 2024-09-21T03:22:00.400 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:22:00.625 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:00 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:00.625 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:00 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:00.625 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:00 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:00.652 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:22:00.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:00 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:00.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:00 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:00.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:00 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:00 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:00 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:00 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:01.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:01 smithi203 ceph-mon[34723]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:01.584 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:01 smithi187 ceph-mon[34603]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:01.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:01 smithi112 ceph-mon[31442]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:01.900 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:22:01 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1[39251]: 2024-09-21T03:22:01.829+0000 7f830457b640 -1 osd.1 0 log_to_monitors true 2024-09-21T03:22:02.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:02 smithi203 ceph-mon[34723]: from='osd.1 [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:22:02.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:02 smithi203 ceph-mon[34723]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:22:02.662 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:02 smithi187 ceph-mon[34603]: from='osd.1 [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:22:02.662 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:02 smithi187 ceph-mon[34603]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:22:02.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:02 smithi112 ceph-mon[31442]: from='osd.1 [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:22:02.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:02 smithi112 ceph-mon[31442]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:22:03.119 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:22:03.145 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi203:vg_nvme/lv_4 2024-09-21T03:22:03.391 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:22:03.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:03.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:22:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: from='osd.1 [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:22:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:22:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:22:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:03.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:03 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:22:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: from='osd.1 [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:22:03.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:22:03.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:03.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:22:03.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:03.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:03.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:03.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:03 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:03.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:22:03.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: from='osd.1 [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:22:03.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:22:03.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:03.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:22:03.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:03.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:03.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:03.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:03 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:03.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.835+0000 7fc4073e2640 1 -- 172.21.15.203:0/767344523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4001027b0 msgr2=0x7fc400102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:03.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.835+0000 7fc4073e2640 1 --2- 172.21.15.203:0/767344523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4001027b0 0x7fc400102bc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009900 tx=0x7fc3fc02a410 comp rx=0 tx=0).stop 2024-09-21T03:22:03.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.835+0000 7fc4073e2640 1 -- 172.21.15.203:0/767344523 shutdown_connections 2024-09-21T03:22:03.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.835+0000 7fc4073e2640 1 --2- 172.21.15.203:0/767344523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc400107bf0 0x7fc4001080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:03.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.835+0000 7fc4073e2640 1 --2- 172.21.15.203:0/767344523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc400103100 0x7fc400107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:03.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.835+0000 7fc4073e2640 1 --2- 172.21.15.203:0/767344523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4001027b0 0x7fc400102bc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:03.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.835+0000 7fc4073e2640 1 -- 172.21.15.203:0/767344523 >> 172.21.15.203:0/767344523 conn(0x7fc400079710 msgr2=0x7fc400079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:03.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.835+0000 7fc4073e2640 1 -- 172.21.15.203:0/767344523 shutdown_connections 2024-09-21T03:22:03.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.835+0000 7fc4073e2640 1 -- 172.21.15.203:0/767344523 wait complete. 2024-09-21T03:22:03.839 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.835+0000 7fc4073e2640 1 Processor -- start 2024-09-21T03:22:03.839 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.835+0000 7fc4073e2640 1 -- start start 2024-09-21T03:22:03.839 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc4073e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4001027b0 0x7fc4001a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:03.839 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc4073e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc400103100 0x7fc4001a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:03.839 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc4073e2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc400107bf0 0x7fc4001a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:03.839 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc4073e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc40011a0a0 con 0x7fc400103100 2024-09-21T03:22:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc4073e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc400119f20 con 0x7fc4001027b0 2024-09-21T03:22:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc4073e2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc40011a220 con 0x7fc400107bf0 2024-09-21T03:22:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc40597f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc400107bf0 0x7fc4001a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc40597f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc400107bf0 0x7fc4001a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:39672/0 (socket says 172.21.15.203:39672) 2024-09-21T03:22:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc40517e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4001027b0 0x7fc4001a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc40597f640 1 -- 172.21.15.203:0/4251922840 learned_addr learned my addr 172.21.15.203:0/4251922840 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-21T03:22:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc40497d640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc400103100 0x7fc4001a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:03.840 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc40597f640 1 -- 172.21.15.203:0/4251922840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4001027b0 msgr2=0x7fc4001a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc40597f640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4001027b0 0x7fc4001a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc40597f640 1 -- 172.21.15.203:0/4251922840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc400103100 msgr2=0x7fc4001a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc40597f640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc400103100 0x7fc4001a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc40597f640 1 -- 172.21.15.203:0/4251922840 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4001a69a0 con 0x7fc400107bf0 2024-09-21T03:22:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc40597f640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc400107bf0 0x7fc4001a6330 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800cce0 tx=0x7fc3f800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.836+0000 7fc3f67fc640 1 -- 172.21.15.203:0/4251922840 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f800f450 con 0x7fc400107bf0 2024-09-21T03:22:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.837+0000 7fc3f67fc640 1 -- 172.21.15.203:0/4251922840 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3f800faa0 con 0x7fc400107bf0 2024-09-21T03:22:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.837+0000 7fc4073e2640 1 -- 172.21.15.203:0/4251922840 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4001a6c90 con 0x7fc400107bf0 2024-09-21T03:22:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.837+0000 7fc3f67fc640 1 -- 172.21.15.203:0/4251922840 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8018b70 con 0x7fc400107bf0 2024-09-21T03:22:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.837+0000 7fc4073e2640 1 -- 172.21.15.203:0/4251922840 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4001ad4b0 con 0x7fc400107bf0 2024-09-21T03:22:03.841 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.838+0000 7fc4073e2640 1 -- 172.21.15.203:0/4251922840 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40006c280 con 0x7fc400107bf0 2024-09-21T03:22:03.842 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.839+0000 7fc3f67fc640 1 -- 172.21.15.203:0/4251922840 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7fc3f8018d10 con 0x7fc400107bf0 2024-09-21T03:22:03.842 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.840+0000 7fc3f67fc640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3ec0b6500 0x7fc3ec0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:03.842 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.840+0000 7fc3f67fc640 1 -- 172.21.15.203:0/4251922840 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7fc3f80e5f30 con 0x7fc400107bf0 2024-09-21T03:22:03.842 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.840+0000 7fc40517e640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3ec0b6500 0x7fc3ec0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:03.842 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.841+0000 7fc40517e640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3ec0b6500 0x7fc3ec0b8a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009230 tx=0x7fc3fc0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:03.844 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:03.843+0000 7fc3f67fc640 1 -- 172.21.15.203:0/4251922840 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3f8014030 con 0x7fc400107bf0 2024-09-21T03:22:04.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:04.032+0000 7fc4073e2640 1 -- 172.21.15.203:0/4251922840 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc400106d20 con 0x7fc3ec0b6500 2024-09-21T03:22:04.622 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:04 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi187 2024-09-21T03:22:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:04 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi187 to 17088M 2024-09-21T03:22:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:04 smithi203 ceph-mon[34723]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:22:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:04 smithi203 ceph-mon[34723]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:22:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:04 smithi203 ceph-mon[34723]: from='osd.1 ' entity='osd.1' 2024-09-21T03:22:04.679 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1[39251]: 2024-09-21T03:22:04.221+0000 7f83004fd640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T03:22:04.679 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1[39251]: 2024-09-21T03:22:04.356+0000 7f82f82f3640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:22:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi187 2024-09-21T03:22:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi187 to 17088M 2024-09-21T03:22:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-mon[34603]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:22:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-mon[34603]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:22:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:04 smithi187 ceph-mon[34603]: from='osd.1 ' entity='osd.1' 2024-09-21T03:22:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:04 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi187 2024-09-21T03:22:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:04 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi187 to 17088M 2024-09-21T03:22:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:04 smithi112 ceph-mon[31442]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:22:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:04 smithi112 ceph-mon[31442]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:22:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:04.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:04.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:04 smithi112 ceph-mon[31442]: from='osd.1 ' entity='osd.1' 2024-09-21T03:22:05.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:05 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:22:05.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:05 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:22:05.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:05 smithi203 ceph-mon[34723]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:05.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:05 smithi203 ceph-mon[34723]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:05.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:05 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:05.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:05 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:05 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:22:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:05 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:22:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:05 smithi187 ceph-mon[34603]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:05 smithi187 ceph-mon[34603]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:05 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:05.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:05 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:05 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:22:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:05 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:22:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:05 smithi112 ceph-mon[31442]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:05 smithi112 ceph-mon[31442]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:05 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:05 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:06 smithi203 ceph-mon[34723]: osd.1 [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548] boot 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:06 smithi203 ceph-mon[34723]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:06 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:06 smithi203 ceph-mon[34723]: from='client.? 172.21.15.203:0/3953037072' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0afbab16-dfb7-45d0-895c-846a437a19b4"} : dispatch 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:06 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0afbab16-dfb7-45d0-895c-846a437a19b4"} : dispatch 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:06 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0afbab16-dfb7-45d0-895c-846a437a19b4"}]': finished 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:06 smithi203 ceph-mon[34723]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:22:06.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:06 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:06.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:06 smithi187 ceph-mon[34603]: osd.1 [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548] boot 2024-09-21T03:22:06.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:06 smithi187 ceph-mon[34603]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:22:06.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:06 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:06.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:06 smithi187 ceph-mon[34603]: from='client.? 172.21.15.203:0/3953037072' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0afbab16-dfb7-45d0-895c-846a437a19b4"} : dispatch 2024-09-21T03:22:06.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:06 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0afbab16-dfb7-45d0-895c-846a437a19b4"} : dispatch 2024-09-21T03:22:06.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:06 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0afbab16-dfb7-45d0-895c-846a437a19b4"}]': finished 2024-09-21T03:22:06.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:06 smithi187 ceph-mon[34603]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:22:06.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:06 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:06 smithi112 ceph-mon[31442]: osd.1 [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548] boot 2024-09-21T03:22:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:06 smithi112 ceph-mon[31442]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:22:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:06 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:22:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:06 smithi112 ceph-mon[31442]: from='client.? 172.21.15.203:0/3953037072' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0afbab16-dfb7-45d0-895c-846a437a19b4"} : dispatch 2024-09-21T03:22:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:06 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0afbab16-dfb7-45d0-895c-846a437a19b4"} : dispatch 2024-09-21T03:22:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:06 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0afbab16-dfb7-45d0-895c-846a437a19b4"}]': finished 2024-09-21T03:22:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:06 smithi112 ceph-mon[31442]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T03:22:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:06 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:07.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:07 smithi203 ceph-mon[34723]: pgmap v50: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:07.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:07 smithi203 ceph-mon[34723]: from='client.? 172.21.15.203:0/4106221137' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:07 smithi187 ceph-mon[34603]: pgmap v50: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:07 smithi187 ceph-mon[34603]: from='client.? 172.21.15.203:0/4106221137' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:07.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:07 smithi112 ceph-mon[31442]: pgmap v50: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:22:07.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:07 smithi112 ceph-mon[31442]: from='client.? 172.21.15.203:0/4106221137' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:08.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:08 smithi203 ceph-mon[34723]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:08.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:08 smithi187 ceph-mon[34603]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:08.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:08 smithi112 ceph-mon[31442]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:11 smithi203 ceph-mon[34723]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:11 smithi187 ceph-mon[34603]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:11 smithi112 ceph-mon[31442]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:13.570 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:13 smithi203 ceph-mon[34723]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:13 smithi187 ceph-mon[34603]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:13.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:13 smithi112 ceph-mon[31442]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:15.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:15 smithi203 ceph-mon[34723]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:15.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:15 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:22:15.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:15 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:15 smithi187 ceph-mon[34603]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:15 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:22:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:15 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:15.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:15 smithi112 ceph-mon[31442]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:15.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:15 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:22:15.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:15 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:16.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:16 smithi203 ceph-mon[34723]: Deploying daemon osd.2 on smithi203 2024-09-21T03:22:16.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:16 smithi187 ceph-mon[34603]: Deploying daemon osd.2 on smithi203 2024-09-21T03:22:16.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:16 smithi112 ceph-mon[31442]: Deploying daemon osd.2 on smithi203 2024-09-21T03:22:17.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:17 smithi203 ceph-mon[34723]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:17 smithi187 ceph-mon[34603]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:17 smithi112 ceph-mon[31442]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:19.551 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:19 smithi203 ceph-mon[34723]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:19.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:19 smithi187 ceph-mon[34603]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:19 smithi112 ceph-mon[31442]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:21 smithi203 ceph-mon[34723]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:21 smithi187 ceph-mon[34603]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:21 smithi112 ceph-mon[31442]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:23.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:23 smithi203 ceph-mon[34723]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:23.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:23.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:23.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:23.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:23.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:23.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:23 smithi187 ceph-mon[34603]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:23.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:23.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:23.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:23.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:23.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:23.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:23 smithi112 ceph-mon[31442]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:23.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:23.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:23.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:23.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:23.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:24.897 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.895+0000 7fc3f67fc640 1 -- 172.21.15.203:0/4251922840 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc400106d20 con 0x7fc3ec0b6500 2024-09-21T03:22:24.897 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 2 on host 'smithi203' 2024-09-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 -- 172.21.15.203:0/4251922840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3ec0b6500 msgr2=0x7fc3ec0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3ec0b6500 0x7fc3ec0b8a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009230 tx=0x7fc3fc0058d0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 -- 172.21.15.203:0/4251922840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc400107bf0 msgr2=0x7fc4001a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc400107bf0 0x7fc4001a6330 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800cce0 tx=0x7fc3f800a620 comp rx=0 tx=0).stop 2024-09-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 -- 172.21.15.203:0/4251922840 shutdown_connections 2024-09-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3ec0b6500 0x7fc3ec0b8a10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc400107bf0 0x7fc4001a6330 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc400103100 0x7fc4001a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 --2- 172.21.15.203:0/4251922840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4001027b0 0x7fc4001a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:24.901 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 -- 172.21.15.203:0/4251922840 >> 172.21.15.203:0/4251922840 conn(0x7fc400079710 msgr2=0x7fc400105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:24.901 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 -- 172.21.15.203:0/4251922840 shutdown_connections 2024-09-21T03:22:24.901 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:22:24.898+0000 7fc4073e2640 1 -- 172.21.15.203:0/4251922840 wait complete. 2024-09-21T03:22:25.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:25 smithi203 ceph-mon[34723]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:25.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:25 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:25.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:25 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:25.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:25 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:25.489 DEBUG:teuthology.orchestra.run.smithi203:osd.2> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T03:22:25.491 INFO:tasks.cephadm:Deploying osd.3 on smithi112 with /dev/vg_nvme/lv_3... 2024-09-21T03:22:25.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:22:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:25 smithi187 ceph-mon[34603]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:25 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:25.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:25 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:25.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:25 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:25.749 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:25 smithi112 ceph-mon[31442]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:25 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:25 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:25 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:27.161 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:22:26 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2[39869]: 2024-09-21T03:22:26.980+0000 7f7220ece640 -1 osd.2 0 log_to_monitors true 2024-09-21T03:22:27.645 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:27 smithi203 ceph-mon[34723]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:27.645 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:27 smithi203 ceph-mon[34723]: from='osd.2 [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:22:27.645 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:27 smithi203 ceph-mon[34723]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:22:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:27 smithi187 ceph-mon[34603]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:27 smithi187 ceph-mon[34603]: from='osd.2 [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:22:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:27 smithi187 ceph-mon[34603]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:22:27.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:27 smithi112 ceph-mon[31442]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:27.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:27 smithi112 ceph-mon[31442]: from='osd.2 [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:22:27.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:27 smithi112 ceph-mon[31442]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:22:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:22:28.054 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-09-21T03:22:28.304 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: from='osd.2 [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:28 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: from='osd.2 [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:28.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:28 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:22:28.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:22:28.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:28.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: from='osd.2 [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:22:28.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:22:28.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:28.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:28.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:28.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:28 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.762+0000 7feb99602640 1 -- 172.21.15.112:0/3015655488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb94105380 msgr2=0x7feb9410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.762+0000 7feb99602640 1 --2- 172.21.15.112:0/3015655488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb94105380 0x7feb9410d8d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb88009900 tx=0x7feb8802a410 comp rx=0 tx=0).stop 2024-09-21T03:22:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.762+0000 7feb99602640 1 -- 172.21.15.112:0/3015655488 shutdown_connections 2024-09-21T03:22:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.762+0000 7feb99602640 1 --2- 172.21.15.112:0/3015655488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb9410de90 0x7feb94110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.762+0000 7feb99602640 1 --2- 172.21.15.112:0/3015655488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb94105380 0x7feb9410d8d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.762+0000 7feb99602640 1 --2- 172.21.15.112:0/3015655488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94104a30 0x7feb94104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.762+0000 7feb99602640 1 -- 172.21.15.112:0/3015655488 >> 172.21.15.112:0/3015655488 conn(0x7feb94079770 msgr2=0x7feb94079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.762+0000 7feb99602640 1 -- 172.21.15.112:0/3015655488 shutdown_connections 2024-09-21T03:22:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.762+0000 7feb99602640 1 -- 172.21.15.112:0/3015655488 wait complete. 2024-09-21T03:22:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb99602640 1 Processor -- start 2024-09-21T03:22:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb99602640 1 -- start start 2024-09-21T03:22:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb99602640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94104a30 0x7feb9419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb99602640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb94105380 0x7feb9419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb99602640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb9410de90 0x7feb9419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb99602640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb94113af0 con 0x7feb94105380 2024-09-21T03:22:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb99602640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb94113970 con 0x7feb9410de90 2024-09-21T03:22:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb99602640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb94113c70 con 0x7feb94104a30 2024-09-21T03:22:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb927fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb94105380 0x7feb9419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb927fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb94105380 0x7feb9419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40578/0 (socket says 172.21.15.112:40578) 2024-09-21T03:22:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb927fc640 1 -- 172.21.15.112:0/2008756590 learned_addr learned my addr 172.21.15.112:0/2008756590 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:22:28.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb92ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94104a30 0x7feb9419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.763+0000 7feb937fe640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb9410de90 0x7feb9419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb927fc640 1 -- 172.21.15.112:0/2008756590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94104a30 msgr2=0x7feb9419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb927fc640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94104a30 0x7feb9419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb927fc640 1 -- 172.21.15.112:0/2008756590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb9410de90 msgr2=0x7feb9419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb927fc640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb9410de90 0x7feb9419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb927fc640 1 -- 172.21.15.112:0/2008756590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb941a0470 con 0x7feb94105380 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb92ffd640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94104a30 0x7feb9419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb927fc640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb94105380 0x7feb9419aaa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feb88006010 tx=0x7feb88004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb7bfff640 1 -- 172.21.15.112:0/2008756590 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8803f030 con 0x7feb94105380 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb99602640 1 -- 172.21.15.112:0/2008756590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb941a0700 con 0x7feb94105380 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb7bfff640 1 -- 172.21.15.112:0/2008756590 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb88003aa0 con 0x7feb94105380 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb99602640 1 -- 172.21.15.112:0/2008756590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb941a6ec0 con 0x7feb94105380 2024-09-21T03:22:28.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.764+0000 7feb7bfff640 1 -- 172.21.15.112:0/2008756590 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88035070 con 0x7feb94105380 2024-09-21T03:22:28.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.765+0000 7feb99602640 1 -- 172.21.15.112:0/2008756590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9406c260 con 0x7feb94105380 2024-09-21T03:22:28.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.767+0000 7feb7bfff640 1 -- 172.21.15.112:0/2008756590 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7feb88042030 con 0x7feb94105380 2024-09-21T03:22:28.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.767+0000 7feb7bfff640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feb7c0b64d0 0x7feb7c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:28.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.767+0000 7feb92ffd640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feb7c0b64d0 0x7feb7c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:28.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.767+0000 7feb7bfff640 1 -- 172.21.15.112:0/2008756590 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2786+0+0 (secure 0 0 0) 0x7feb881064f0 con 0x7feb94105380 2024-09-21T03:22:28.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.768+0000 7feb92ffd640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feb7c0b64d0 0x7feb7c0b89e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb84005fd0 tx=0x7feb84009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:28.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.770+0000 7feb7bfff640 1 -- 172.21.15.112:0/2008756590 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feb880d27d0 con 0x7feb94105380 2024-09-21T03:22:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:28.957+0000 7feb99602640 1 -- 172.21.15.112:0/2008756590 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7feb94063800 con 0x7feb7c0b64d0 2024-09-21T03:22:29.623 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:29 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi203 2024-09-21T03:22:29.623 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:29 smithi112 ceph-mon[31442]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:29.624 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:29 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-21T03:22:29.624 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:29 smithi112 ceph-mon[31442]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:22:29.624 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:29 smithi112 ceph-mon[31442]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:22:29.624 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:29.624 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:29.624 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:29.624 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:29.624 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:29.624 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:29 smithi112 ceph-mon[31442]: from='osd.2 ' entity='osd.2' 2024-09-21T03:22:29.660 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2[39869]: 2024-09-21T03:22:29.242+0000 7f721ce50640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T03:22:29.660 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2[39869]: 2024-09-21T03:22:29.382+0000 7f7214c46640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:22:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi203 2024-09-21T03:22:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-mon[34723]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-21T03:22:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-mon[34723]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:22:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-mon[34723]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:22:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:29 smithi203 ceph-mon[34723]: from='osd.2 ' entity='osd.2' 2024-09-21T03:22:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:29 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi203 2024-09-21T03:22:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:29 smithi187 ceph-mon[34603]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:29 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-21T03:22:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:29 smithi187 ceph-mon[34603]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:22:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:29 smithi187 ceph-mon[34603]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:22:29.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:29.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:29.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:29.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:29.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:29.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:29 smithi187 ceph-mon[34603]: from='osd.2 ' entity='osd.2' 2024-09-21T03:22:30.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:30 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:22:30.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:30 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:22:30.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:30 smithi112 ceph-mon[31442]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:30.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:30 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:30.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:30 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:22:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:30 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:22:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:30 smithi203 ceph-mon[34723]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:30 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:30 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:22:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:30 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:22:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:30 smithi187 ceph-mon[34603]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:30 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:31 smithi203 ceph-mon[34723]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:31 smithi203 ceph-mon[34723]: osd.2 [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217] boot 2024-09-21T03:22:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:31 smithi203 ceph-mon[34723]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:22:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:31 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:31 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:22:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:31 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1483159130' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "495ece6a-e62f-4deb-8600-ece833469091"} : dispatch 2024-09-21T03:22:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:31 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:22:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:31 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1483159130' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "495ece6a-e62f-4deb-8600-ece833469091"}]': finished 2024-09-21T03:22:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:31 smithi203 ceph-mon[34723]: osdmap e19: 4 total, 3 up, 4 in 2024-09-21T03:22:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:31 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:31 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:22:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:31 smithi187 ceph-mon[34603]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:31 smithi187 ceph-mon[34603]: osd.2 [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217] boot 2024-09-21T03:22:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:31 smithi187 ceph-mon[34603]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:22:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:31 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:31.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:31 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:22:31.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:31 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1483159130' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "495ece6a-e62f-4deb-8600-ece833469091"} : dispatch 2024-09-21T03:22:31.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:31 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:22:31.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:31 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1483159130' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "495ece6a-e62f-4deb-8600-ece833469091"}]': finished 2024-09-21T03:22:31.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:31 smithi187 ceph-mon[34603]: osdmap e19: 4 total, 3 up, 4 in 2024-09-21T03:22:31.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:31 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:31.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:31 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:22:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:31 smithi112 ceph-mon[31442]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:22:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:31 smithi112 ceph-mon[31442]: osd.2 [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217] boot 2024-09-21T03:22:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:31 smithi112 ceph-mon[31442]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T03:22:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:31 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:22:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:31 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:22:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:31 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1483159130' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "495ece6a-e62f-4deb-8600-ece833469091"} : dispatch 2024-09-21T03:22:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:31 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:22:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:31 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1483159130' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "495ece6a-e62f-4deb-8600-ece833469091"}]': finished 2024-09-21T03:22:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:31 smithi112 ceph-mon[31442]: osdmap e19: 4 total, 3 up, 4 in 2024-09-21T03:22:31.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:31 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:31.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:31 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:22:32.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2148712631' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:32.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 ceph-mon[34723]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:32.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:22:32.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 ceph-mon[34723]: osdmap e20: 4 total, 3 up, 4 in 2024-09-21T03:22:32.751 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:32.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2148712631' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:32.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 ceph-mon[31442]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:32.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:22:32.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 ceph-mon[31442]: osdmap e20: 4 total, 3 up, 4 in 2024-09-21T03:22:32.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:32.788 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52939]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:22:32.788 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52939]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:32.788 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52939]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:32.788 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52939]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:32.788 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52942]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:22:32.789 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52942]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:32.789 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52942]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:32.789 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52942]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:32.908 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42801]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:22:32.908 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42801]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:32.908 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42801]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:32.909 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42801]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:32.909 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42804]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:22:32.909 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42804]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:32.909 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42804]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:32.909 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42804]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:32.909 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2148712631' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:32.909 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 ceph-mon[34603]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:32.909 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:22:32.909 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 ceph-mon[34603]: osdmap e20: 4 total, 3 up, 4 in 2024-09-21T03:22:32.909 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:33.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43816]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:22:33.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43816]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:33.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43816]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:33.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43816]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:33.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43819]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:22:33.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43819]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:33.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43819]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:33.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43819]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:33.161 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43810]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:22:33.161 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43810]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:33.161 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43810]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:33.162 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43810]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:33.162 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43813]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:22:33.162 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43813]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:33.162 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43813]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:33.162 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:22:32 smithi203 sudo[43813]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:33.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42807]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:22:33.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42807]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:33.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42807]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:33.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42807]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:33.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42810]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:22:33.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42810]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:33.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42810]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:33.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:32 smithi187 sudo[42810]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52945]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:22:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52945]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52945]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52945]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52948]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:22:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52948]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:22:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52948]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:22:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:32 smithi112 sudo[52948]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:22:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:22:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:22:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:22:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:22:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:22:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:22:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:22:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:33.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:22:33.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:22:33.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:33.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:33.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:33.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:22:33.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:22:33.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:22:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:22:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:22:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:22:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:22:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:22:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:22:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:22:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:22:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T03:22:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:34.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:34.667+0000 7feb7bfff640 1 -- 172.21.15.112:0/2008756590 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7feb880ca010 con 0x7feb94105380 2024-09-21T03:22:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:34 smithi187 ceph-mon[34603]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:35.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:34 smithi112 ceph-mon[31442]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:35.160 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:34 smithi203 ceph-mon[34723]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:35.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:35 smithi187 ceph-mon[34603]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T03:22:36.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:35 smithi112 ceph-mon[31442]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T03:22:36.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:35 smithi203 ceph-mon[34723]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T03:22:36.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:36 smithi187 ceph-mon[34603]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:37.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:36 smithi112 ceph-mon[31442]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:37.160 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:36 smithi203 ceph-mon[34723]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:39.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:39 smithi112 ceph-mon[31442]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:39.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:39 smithi203 ceph-mon[34723]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:39.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:39 smithi187 ceph-mon[34603]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:40.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:40 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:22:40.527 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:40 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:40.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:40 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:22:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:40 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:40 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:22:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:40 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:41.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:41 smithi112 ceph-mon[31442]: Deploying daemon osd.3 on smithi112 2024-09-21T03:22:41.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:41 smithi112 ceph-mon[31442]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:41.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:41 smithi203 ceph-mon[34723]: Deploying daemon osd.3 on smithi112 2024-09-21T03:22:41.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:41 smithi203 ceph-mon[34723]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:41.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:41 smithi187 ceph-mon[34603]: Deploying daemon osd.3 on smithi112 2024-09-21T03:22:41.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:41 smithi187 ceph-mon[34603]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:43 smithi203 ceph-mon[34723]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:43.682 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:43 smithi187 ceph-mon[34603]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:43 smithi112 ceph-mon[31442]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:45.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:45 smithi112 ceph-mon[31442]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:45.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:45 smithi203 ceph-mon[34723]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:45.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:45 smithi187 ceph-mon[34603]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:46.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:46 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:46.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:46 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:46.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:46 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:46.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:46 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:46.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:46 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:46.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:46 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:46.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:46 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:46.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:46 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:46.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:46 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:47.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:47 smithi112 ceph-mon[31442]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:47.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:47 smithi203 ceph-mon[34723]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:47 smithi187 ceph-mon[34603]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:49.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.311+0000 7feb7bfff640 1 -- 172.21.15.112:0/2008756590 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7feb94063800 con 0x7feb7c0b64d0 2024-09-21T03:22:49.313 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 3 on host 'smithi112' 2024-09-21T03:22:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.313+0000 7feb99602640 1 -- 172.21.15.112:0/2008756590 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feb7c0b64d0 msgr2=0x7feb7c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.313+0000 7feb99602640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feb7c0b64d0 0x7feb7c0b89e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb84005fd0 tx=0x7feb84009500 comp rx=0 tx=0).stop 2024-09-21T03:22:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.313+0000 7feb99602640 1 -- 172.21.15.112:0/2008756590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb94105380 msgr2=0x7feb9419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.313+0000 7feb99602640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb94105380 0x7feb9419aaa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feb88006010 tx=0x7feb88004060 comp rx=0 tx=0).stop 2024-09-21T03:22:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.313+0000 7feb99602640 1 -- 172.21.15.112:0/2008756590 shutdown_connections 2024-09-21T03:22:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.313+0000 7feb99602640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feb7c0b64d0 0x7feb7c0b89e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.313+0000 7feb99602640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb9410de90 0x7feb9419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:49.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.313+0000 7feb99602640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb94105380 0x7feb9419aaa0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:49.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.313+0000 7feb99602640 1 --2- 172.21.15.112:0/2008756590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94104a30 0x7feb9419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:49.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.313+0000 7feb99602640 1 -- 172.21.15.112:0/2008756590 >> 172.21.15.112:0/2008756590 conn(0x7feb94079770 msgr2=0x7feb94100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:49.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.314+0000 7feb99602640 1 -- 172.21.15.112:0/2008756590 shutdown_connections 2024-09-21T03:22:49.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:22:49.314+0000 7feb99602640 1 -- 172.21.15.112:0/2008756590 wait complete. 2024-09-21T03:22:49.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:49 smithi112 ceph-mon[31442]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:49.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:49 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:49 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:49 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:49.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:49 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:49.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:49 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:49 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:49.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:49 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:49 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:49 smithi203 ceph-mon[34723]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:49 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:49 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:49 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:49 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:49 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:49 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:49 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:49 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:49 smithi187 ceph-mon[34603]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:49.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:49 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:49 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:49 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:49.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:49 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:49.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:49 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:49 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:22:49.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:49 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:49 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:49.947 DEBUG:teuthology.orchestra.run.smithi112:osd.3> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T03:22:49.949 INFO:tasks.cephadm:Deploying osd.4 on smithi187 with /dev/vg_nvme/lv_3... 2024-09-21T03:22:49.949 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:22:50.203 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:22:51.658 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:51 smithi187 ceph-mon[34603]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:51.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:51 smithi203 ceph-mon[34723]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:51.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:51 smithi112 ceph-mon[31442]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:52.350 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:22:52 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3[54831]: 2024-09-21T03:22:52.263+0000 7fc47066b640 -1 osd.3 0 log_to_monitors true 2024-09-21T03:22:52.608 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:52 smithi187 ceph-mon[34603]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:22:52.633 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:52 smithi112 ceph-mon[31442]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:22:52.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:52 smithi203 ceph-mon[34723]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:22:52.699 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:22:52.727 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi187:vg_nvme/lv_3 2024-09-21T03:22:52.979 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:22:53.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.435+0000 7f2673f22640 1 -- 172.21.15.187:0/1754691792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f266c1094a0 msgr2=0x7f266c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.435+0000 7f2673f22640 1 --2- 172.21.15.187:0/1754691792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f266c1094a0 0x7f266c1098b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f265c009900 tx=0x7f265c02a410 comp rx=0 tx=0).stop 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.435+0000 7f2673f22640 1 -- 172.21.15.187:0/1754691792 shutdown_connections 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.435+0000 7f2673f22640 1 --2- 172.21.15.187:0/1754691792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f266c109df0 0x7f266c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.435+0000 7f2673f22640 1 --2- 172.21.15.187:0/1754691792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f266c1094a0 0x7f266c1098b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.435+0000 7f2673f22640 1 --2- 172.21.15.187:0/1754691792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c103b60 0x7f266c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.435+0000 7f2673f22640 1 -- 172.21.15.187:0/1754691792 >> 172.21.15.187:0/1754691792 conn(0x7f266c07b850 msgr2=0x7f266c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.435+0000 7f2673f22640 1 -- 172.21.15.187:0/1754691792 shutdown_connections 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.435+0000 7f2673f22640 1 -- 172.21.15.187:0/1754691792 wait complete. 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f2673f22640 1 Processor -- start 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f2673f22640 1 -- start start 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f2673f22640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c103b60 0x7f266c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f2673f22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f266c1094a0 0x7f266c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:53.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f2673f22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f266c109df0 0x7f266c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:53.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f2673f22640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f266c113ab0 con 0x7f266c109df0 2024-09-21T03:22:53.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f2673f22640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f266c113930 con 0x7f266c1094a0 2024-09-21T03:22:53.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f2673f22640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f266c113c30 con 0x7f266c103b60 2024-09-21T03:22:53.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f26714bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f266c1094a0 0x7f266c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:53.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f26714bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f266c1094a0 0x7f266c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:56516/0 (socket says 172.21.15.187:56516) 2024-09-21T03:22:53.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f26714bd640 1 -- 172.21.15.187:0/2724438885 learned_addr learned my addr 172.21.15.187:0/2724438885 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T03:22:53.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f26724bf640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f266c109df0 0x7f266c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:53.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f2671cbe640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c103b60 0x7f266c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:53.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f26714bd640 1 -- 172.21.15.187:0/2724438885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c103b60 msgr2=0x7f266c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:53.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f26714bd640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c103b60 0x7f266c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:53.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f26714bd640 1 -- 172.21.15.187:0/2724438885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f266c109df0 msgr2=0x7f266c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f26714bd640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f266c109df0 0x7f266c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.436+0000 7f26714bd640 1 -- 172.21.15.187:0/2724438885 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c1a87a0 con 0x7f266c1094a0 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.437+0000 7f26714bd640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f266c1094a0 0x7f266c075c50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f265c02a930 tx=0x7f265c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.437+0000 7f2662ffd640 1 -- 172.21.15.187:0/2724438885 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c03f030 con 0x7f266c1094a0 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.437+0000 7f2662ffd640 1 -- 172.21.15.187:0/2724438885 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f265c003cb0 con 0x7f266c1094a0 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.437+0000 7f2673f22640 1 -- 172.21.15.187:0/2724438885 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266c1a8a30 con 0x7f266c1094a0 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.437+0000 7f2662ffd640 1 -- 172.21.15.187:0/2724438885 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c035070 con 0x7f266c1094a0 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.437+0000 7f2673f22640 1 -- 172.21.15.187:0/2724438885 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266c1a8f10 con 0x7f266c1094a0 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.438+0000 7f2673f22640 1 -- 172.21.15.187:0/2724438885 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266c103f70 con 0x7f266c1094a0 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.439+0000 7f2662ffd640 1 -- 172.21.15.187:0/2724438885 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f265c042030 con 0x7f266c1094a0 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.440+0000 7f2662ffd640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f26580b6560 0x7f26580b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.440+0000 7f2662ffd640 1 -- 172.21.15.187:0/2724438885 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3604+0+0 (secure 0 0 0) 0x7f265c1066b0 con 0x7f266c1094a0 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.440+0000 7f2671cbe640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f26580b6560 0x7f26580b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:22:53.451 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.441+0000 7f2671cbe640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f26580b6560 0x7f26580b8a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26680058c0 tx=0x7f266800a250 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:22:53.452 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.445+0000 7f2662ffd640 1 -- 172.21.15.187:0/2724438885 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f265c0d2580 con 0x7f266c1094a0 2024-09-21T03:22:53.632 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:22:53.630+0000 7f2673f22640 1 -- 172.21.15.187:0/2724438885 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f266c063800 con 0x7f26580b6560 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:53.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:53.634 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:22:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:22:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:22:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:53.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:53.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:53.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:53.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:53.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:22:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:22:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:22:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:53.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:22:53.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:53.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:53.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:22:53.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:22:54.325 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3[54831]: 2024-09-21T03:22:54.281+0000 7fc46c5ed640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T03:22:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi112 2024-09-21T03:22:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi112 to 8544M 2024-09-21T03:22:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:22:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:22:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:54.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:54.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' 2024-09-21T03:22:54.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:22:54.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:54.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:54.663 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi112 2024-09-21T03:22:54.663 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi112 to 8544M 2024-09-21T03:22:54.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:22:54.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:22:54.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:54.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:54.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:54.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:54.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:54.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' 2024-09-21T03:22:54.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:22:54.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:54.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:54.787 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3[54831]: 2024-09-21T03:22:54.460+0000 7fc4643e3640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:22:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi112 2024-09-21T03:22:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi112 to 8544M 2024-09-21T03:22:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:22:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:22:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:22:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:22:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:22:54.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: from='osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739]' entity='osd.3' 2024-09-21T03:22:54.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:22:54.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:54.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:55 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:22:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:55 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:22:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:55 smithi112 ceph-mon[31442]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:55 smithi112 ceph-mon[31442]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:55 smithi112 ceph-mon[31442]: osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739] boot 2024-09-21T03:22:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:55 smithi112 ceph-mon[31442]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:22:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:55 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:55.568 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:55 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:22:55.568 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:55 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:22:55.568 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:55 smithi187 ceph-mon[34603]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:55.568 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:55 smithi187 ceph-mon[34603]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:55.569 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:55 smithi187 ceph-mon[34603]: osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739] boot 2024-09-21T03:22:55.569 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:55 smithi187 ceph-mon[34603]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:22:55.569 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:55 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:55.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:55 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:22:55.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:55 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:22:55.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:55 smithi203 ceph-mon[34723]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:22:55.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:55 smithi203 ceph-mon[34723]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:22:55.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:55 smithi203 ceph-mon[34723]: osd.3 [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739] boot 2024-09-21T03:22:55.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:55 smithi203 ceph-mon[34723]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:22:55.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:55 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:22:56.539 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:56 smithi187 ceph-mon[34603]: from='client.? 172.21.15.187:0/2964881905' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c46289b-1c19-4c58-bd00-5e96505df718"} : dispatch 2024-09-21T03:22:56.539 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:56 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c46289b-1c19-4c58-bd00-5e96505df718"} : dispatch 2024-09-21T03:22:56.539 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:56 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c46289b-1c19-4c58-bd00-5e96505df718"}]': finished 2024-09-21T03:22:56.539 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:56 smithi187 ceph-mon[34603]: osdmap e26: 5 total, 4 up, 5 in 2024-09-21T03:22:56.539 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:56 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:22:56.539 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:56 smithi187 ceph-mon[34603]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:22:56.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:56 smithi203 ceph-mon[34723]: from='client.? 172.21.15.187:0/2964881905' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c46289b-1c19-4c58-bd00-5e96505df718"} : dispatch 2024-09-21T03:22:56.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:56 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c46289b-1c19-4c58-bd00-5e96505df718"} : dispatch 2024-09-21T03:22:56.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:56 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c46289b-1c19-4c58-bd00-5e96505df718"}]': finished 2024-09-21T03:22:56.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:56 smithi203 ceph-mon[34723]: osdmap e26: 5 total, 4 up, 5 in 2024-09-21T03:22:56.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:56 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:22:56.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:56 smithi203 ceph-mon[34723]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:22:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:56 smithi112 ceph-mon[31442]: from='client.? 172.21.15.187:0/2964881905' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c46289b-1c19-4c58-bd00-5e96505df718"} : dispatch 2024-09-21T03:22:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:56 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c46289b-1c19-4c58-bd00-5e96505df718"} : dispatch 2024-09-21T03:22:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:56 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c46289b-1c19-4c58-bd00-5e96505df718"}]': finished 2024-09-21T03:22:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:56 smithi112 ceph-mon[31442]: osdmap e26: 5 total, 4 up, 5 in 2024-09-21T03:22:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:56 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:22:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:56 smithi112 ceph-mon[31442]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:22:57.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:57 smithi203 ceph-mon[34723]: from='client.? 172.21.15.187:0/1947283355' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:57 smithi203 ceph-mon[34723]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:22:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:22:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:57 smithi187 ceph-mon[34603]: from='client.? 172.21.15.187:0/1947283355' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:57 smithi187 ceph-mon[34603]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:22:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:22:57.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:57 smithi112 ceph-mon[31442]: from='client.? 172.21.15.187:0/1947283355' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:22:57.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:57 smithi112 ceph-mon[31442]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:22:57.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:22:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:22:58 smithi203 ceph-mon[34723]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:22:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:22:58 smithi187 ceph-mon[34603]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:22:58.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:22:58 smithi112 ceph-mon[31442]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:01.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:01 smithi187 ceph-mon[34603]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:01.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:01 smithi203 ceph-mon[34723]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:01.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:01 smithi112 ceph-mon[31442]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:03.656 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:03 smithi187 ceph-mon[34603]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:23:03.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:03 smithi203 ceph-mon[34723]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:23:03.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:03 smithi112 ceph-mon[31442]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T03:23:04.617 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:23:04.617 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:23:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:23:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:05.620 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:05 smithi187 ceph-mon[34603]: Deploying daemon osd.4 on smithi187 2024-09-21T03:23:05.620 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:05 smithi187 ceph-mon[34603]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T03:23:05.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:05 smithi203 ceph-mon[34723]: Deploying daemon osd.4 on smithi187 2024-09-21T03:23:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:05 smithi203 ceph-mon[34723]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T03:23:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:05 smithi112 ceph-mon[31442]: Deploying daemon osd.4 on smithi187 2024-09-21T03:23:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:05 smithi112 ceph-mon[31442]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T03:23:07.516 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:07 smithi187 ceph-mon[34603]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:23:07.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:07 smithi203 ceph-mon[34723]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:23:07.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:07 smithi112 ceph-mon[31442]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T03:23:09.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:09 smithi203 ceph-mon[34723]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:23:09.669 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:09 smithi187 ceph-mon[34603]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:23:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:09 smithi112 ceph-mon[31442]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:23:10.640 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:10.640 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:10.640 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:10.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:11.643 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:11 smithi187 ceph-mon[34603]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:23:11.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:11 smithi203 ceph-mon[34723]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:23:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:11 smithi112 ceph-mon[31442]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:23:13.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:13 smithi203 ceph-mon[34723]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:23:13.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:13.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:13.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:13.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:13.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:13 smithi187 ceph-mon[34603]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:23:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:13.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:13 smithi112 ceph-mon[31442]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:23:13.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:13.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:13.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:13.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:13.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:14.106 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.104+0000 7f2662ffd640 1 -- 172.21.15.187:0/2724438885 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f266c063800 con 0x7f26580b6560 2024-09-21T03:23:14.107 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 4 on host 'smithi187' 2024-09-21T03:23:14.109 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 -- 172.21.15.187:0/2724438885 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f26580b6560 msgr2=0x7f26580b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:14.109 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f26580b6560 0x7f26580b8a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26680058c0 tx=0x7f266800a250 comp rx=0 tx=0).stop 2024-09-21T03:23:14.109 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 -- 172.21.15.187:0/2724438885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f266c1094a0 msgr2=0x7f266c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:14.109 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f266c1094a0 0x7f266c075c50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f265c02a930 tx=0x7f265c004270 comp rx=0 tx=0).stop 2024-09-21T03:23:14.109 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 -- 172.21.15.187:0/2724438885 shutdown_connections 2024-09-21T03:23:14.109 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f26580b6560 0x7f26580b8a70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:14.110 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f266c109df0 0x7f266c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:14.110 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f266c1094a0 0x7f266c075c50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:14.110 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 --2- 172.21.15.187:0/2724438885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c103b60 0x7f266c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:14.110 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 -- 172.21.15.187:0/2724438885 >> 172.21.15.187:0/2724438885 conn(0x7f266c07b850 msgr2=0x7f266c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:14.110 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 -- 172.21.15.187:0/2724438885 shutdown_connections 2024-09-21T03:23:14.110 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:23:14.107+0000 7f2673f22640 1 -- 172.21.15.187:0/2724438885 wait complete. 2024-09-21T03:23:14.658 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:14 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:14.658 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:14 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:14.658 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:14 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:14.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:14 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:14.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:14 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:14.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:14 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:14.692 DEBUG:teuthology.orchestra.run.smithi187:osd.4> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T03:23:14.694 INFO:tasks.cephadm:Deploying osd.5 on smithi203 with /dev/vg_nvme/lv_3... 2024-09-21T03:23:14.694 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:23:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:14 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:14 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:14 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:14.947 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:23:15.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:15 smithi203 ceph-mon[34723]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:15 smithi187 ceph-mon[34603]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:15.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:15 smithi112 ceph-mon[31442]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:16.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:16 smithi203 ceph-mon[34723]: from='osd.4 [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:23:16.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:16 smithi203 ceph-mon[34723]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:23:16.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:16 smithi187 ceph-mon[34603]: from='osd.4 [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:23:16.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:16 smithi187 ceph-mon[34603]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:23:16.680 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:23:16 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4[45392]: 2024-09-21T03:23:16.295+0000 7f5df182c640 -1 osd.4 0 log_to_monitors true 2024-09-21T03:23:16.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:16 smithi112 ceph-mon[31442]: from='osd.4 [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:23:16.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:16 smithi112 ceph-mon[31442]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:23:17.413 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:17 smithi203 ceph-mon[34723]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:17.413 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:17 smithi203 ceph-mon[34723]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:23:17.413 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:17 smithi203 ceph-mon[34723]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:23:17.413 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:17 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:17.413 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:17 smithi203 ceph-mon[34723]: from='osd.4 [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:23:17.413 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:17 smithi203 ceph-mon[34723]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:23:17.415 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:23:17.441 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi203:vg_nvme/lv_3 2024-09-21T03:23:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:17 smithi187 ceph-mon[34603]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:17 smithi187 ceph-mon[34603]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:23:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:17 smithi187 ceph-mon[34603]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:23:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:17 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:17.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:17 smithi187 ceph-mon[34603]: from='osd.4 [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:23:17.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:17 smithi187 ceph-mon[34603]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:23:17.688 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:23:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:17 smithi112 ceph-mon[31442]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:17 smithi112 ceph-mon[31442]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:23:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:17 smithi112 ceph-mon[31442]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:23:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:17 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:17 smithi112 ceph-mon[31442]: from='osd.4 [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:23:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:17 smithi112 ceph-mon[31442]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:23:18.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.223+0000 7f2fba04c640 1 -- 172.21.15.203:0/2190126079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fb4108420 msgr2=0x7f2fb4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:18.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.223+0000 7f2fba04c640 1 --2- 172.21.15.203:0/2190126079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fb4108420 0x7f2fb4108830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009900 tx=0x7f2fa802a410 comp rx=0 tx=0).stop 2024-09-21T03:23:18.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.223+0000 7f2fba04c640 1 -- 172.21.15.203:0/2190126079 shutdown_connections 2024-09-21T03:23:18.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.223+0000 7f2fba04c640 1 --2- 172.21.15.203:0/2190126079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4108d70 0x7f2fb410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:18.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.223+0000 7f2fba04c640 1 --2- 172.21.15.203:0/2190126079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fb4108420 0x7f2fb4108830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:18.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.223+0000 7f2fba04c640 1 --2- 172.21.15.203:0/2190126079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4102ae0 0x7f2fb4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:18.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.223+0000 7f2fba04c640 1 -- 172.21.15.203:0/2190126079 >> 172.21.15.203:0/2190126079 conn(0x7f2fb4079770 msgr2=0x7f2fb4079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:18.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.223+0000 7f2fba04c640 1 -- 172.21.15.203:0/2190126079 shutdown_connections 2024-09-21T03:23:18.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.223+0000 7f2fba04c640 1 -- 172.21.15.203:0/2190126079 wait complete. 2024-09-21T03:23:18.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fba04c640 1 Processor -- start 2024-09-21T03:23:18.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fba04c640 1 -- start start 2024-09-21T03:23:18.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fba04c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4102ae0 0x7f2fb4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:18.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fba04c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4108420 0x7f2fb41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:18.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fba04c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fb4108d70 0x7f2fb419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fba04c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4111930 con 0x7f2fb4108420 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fba04c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb41117b0 con 0x7f2fb4102ae0 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fba04c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4111ab0 con 0x7f2fb4108d70 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fb37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4102ae0 0x7f2fb4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fb3fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fb4108d70 0x7f2fb419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fb37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4102ae0 0x7f2fb4198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.203:49888/0 (socket says 172.21.15.203:49888) 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fb37fe640 1 -- 172.21.15.203:0/652241968 learned_addr learned my addr 172.21.15.203:0/652241968 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.224+0000 7f2fb2ffd640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4108420 0x7f2fb41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.225+0000 7f2fb37fe640 1 -- 172.21.15.203:0/652241968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fb4108d70 msgr2=0x7f2fb419dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.225+0000 7f2fb37fe640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fb4108d70 0x7f2fb419dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.225+0000 7f2fb37fe640 1 -- 172.21.15.203:0/652241968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4108420 msgr2=0x7f2fb41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.225+0000 7f2fb37fe640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4108420 0x7f2fb41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:18.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.225+0000 7f2fb37fe640 1 -- 172.21.15.203:0/652241968 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb419e3b0 con 0x7f2fb4102ae0 2024-09-21T03:23:18.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.225+0000 7f2fb37fe640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4102ae0 0x7f2fb4198440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4012f90 tx=0x7f2fa4010490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:18.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.225+0000 7f2fb0ff9640 1 -- 172.21.15.203:0/652241968 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4004b20 con 0x7f2fb4102ae0 2024-09-21T03:23:18.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.225+0000 7f2fba04c640 1 -- 172.21.15.203:0/652241968 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb419e640 con 0x7f2fb4102ae0 2024-09-21T03:23:18.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.225+0000 7f2fb0ff9640 1 -- 172.21.15.203:0/652241968 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa4004cc0 con 0x7f2fb4102ae0 2024-09-21T03:23:18.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.225+0000 7f2fb0ff9640 1 -- 172.21.15.203:0/652241968 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400bdc0 con 0x7f2fb4102ae0 2024-09-21T03:23:18.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.225+0000 7f2fba04c640 1 -- 172.21.15.203:0/652241968 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb41a4e00 con 0x7f2fb4102ae0 2024-09-21T03:23:18.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.226+0000 7f2fba04c640 1 -- 172.21.15.203:0/652241968 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb4103d60 con 0x7f2fb4102ae0 2024-09-21T03:23:18.229 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.228+0000 7f2fb0ff9640 1 -- 172.21.15.203:0/652241968 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f2fa401f020 con 0x7f2fb4102ae0 2024-09-21T03:23:18.230 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.228+0000 7f2fb0ff9640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f9c0b6560 0x7f2f9c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:18.230 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.228+0000 7f2fb0ff9640 1 -- 172.21.15.203:0/652241968 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4036+0+0 (secure 0 0 0) 0x7f2fa40eace0 con 0x7f2fb4102ae0 2024-09-21T03:23:18.230 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.229+0000 7f2fb2ffd640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f9c0b6560 0x7f2f9c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:18.230 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.229+0000 7f2fb2ffd640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f9c0b6560 0x7f2f9c0b8a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009230 tx=0x7f2fa8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:18.233 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.232+0000 7f2fb0ff9640 1 -- 172.21.15.203:0/652241968 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2fa4014040 con 0x7f2fb4102ae0 2024-09-21T03:23:18.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:23:18.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:23:18.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:18.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:18.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:18.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:18.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:18.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:18.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:18 smithi203 ceph-mon[34723]: from='osd.4 ' entity='osd.4' 2024-09-21T03:23:18.425 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:18.424+0000 7f2fba04c640 1 -- 172.21.15.203:0/652241968 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2fb4063800 con 0x7f2f9c0b6560 2024-09-21T03:23:18.679 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4[45392]: 2024-09-21T03:23:18.314+0000 7f5ded7ae640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T03:23:18.679 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4[45392]: 2024-09-21T03:23:18.396+0000 7f5de55a4640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:23:18.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:23:18.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:23:18.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:18.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:18.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:18.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:18.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:18.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:18.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:18 smithi187 ceph-mon[34603]: from='osd.4 ' entity='osd.4' 2024-09-21T03:23:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:23:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:23:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:18.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:18.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:18.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:18 smithi112 ceph-mon[31442]: from='osd.4 ' entity='osd.4' 2024-09-21T03:23:19.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:19 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:23:19.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:19 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:23:19.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:19 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi187 2024-09-21T03:23:19.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:19 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi187 to 8544M 2024-09-21T03:23:19.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:19 smithi203 ceph-mon[34723]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:19.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:19.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:19.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:19.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:19.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:19 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:23:19.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:19 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:23:19.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:19 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi187 2024-09-21T03:23:19.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:19 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi187 to 8544M 2024-09-21T03:23:19.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:19 smithi187 ceph-mon[34603]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:19.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:19.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:19.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:19.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:19 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:23:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:19 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:23:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:19 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi187 2024-09-21T03:23:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:19 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi187 to 8544M 2024-09-21T03:23:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:19 smithi112 ceph-mon[31442]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:20.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:20 smithi203 ceph-mon[34723]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:20.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:20 smithi203 ceph-mon[34723]: osd.4 [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939] boot 2024-09-21T03:23:20.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:20 smithi203 ceph-mon[34723]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:23:20.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:20 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:20.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:20 smithi203 ceph-mon[34723]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:20.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:20 smithi187 ceph-mon[34603]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:20.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:20 smithi187 ceph-mon[34603]: osd.4 [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939] boot 2024-09-21T03:23:20.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:20 smithi187 ceph-mon[34603]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:23:20.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:20 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:20.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:20 smithi187 ceph-mon[34603]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:20 smithi112 ceph-mon[31442]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:20 smithi112 ceph-mon[31442]: osd.4 [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939] boot 2024-09-21T03:23:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:20 smithi112 ceph-mon[31442]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:23:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:20 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:23:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:20 smithi112 ceph-mon[31442]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:23:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:21 smithi203 ceph-mon[34723]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:23:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:21 smithi203 ceph-mon[34723]: from='client.? 172.21.15.203:0/1611375428' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f4727682-1223-49ae-a893-2559f553b8b8"} : dispatch 2024-09-21T03:23:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:21 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f4727682-1223-49ae-a893-2559f553b8b8"} : dispatch 2024-09-21T03:23:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:21 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4727682-1223-49ae-a893-2559f553b8b8"}]': finished 2024-09-21T03:23:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:21 smithi203 ceph-mon[34723]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:23:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:21 smithi203 ceph-mon[34723]: from='client.? 172.21.15.203:0/545132017' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:21 smithi187 ceph-mon[34603]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:23:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:21 smithi187 ceph-mon[34603]: from='client.? 172.21.15.203:0/1611375428' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f4727682-1223-49ae-a893-2559f553b8b8"} : dispatch 2024-09-21T03:23:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:21 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f4727682-1223-49ae-a893-2559f553b8b8"} : dispatch 2024-09-21T03:23:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:21 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4727682-1223-49ae-a893-2559f553b8b8"}]': finished 2024-09-21T03:23:21.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:21 smithi187 ceph-mon[34603]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:23:21.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:21.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:21 smithi187 ceph-mon[34603]: from='client.? 172.21.15.203:0/545132017' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:21 smithi112 ceph-mon[31442]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:23:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:21 smithi112 ceph-mon[31442]: from='client.? 172.21.15.203:0/1611375428' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f4727682-1223-49ae-a893-2559f553b8b8"} : dispatch 2024-09-21T03:23:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:21 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f4727682-1223-49ae-a893-2559f553b8b8"} : dispatch 2024-09-21T03:23:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:21 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4727682-1223-49ae-a893-2559f553b8b8"}]': finished 2024-09-21T03:23:21.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:21 smithi112 ceph-mon[31442]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:23:21.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:21.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:21 smithi112 ceph-mon[31442]: from='client.? 172.21.15.203:0/545132017' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:22.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:22 smithi203 ceph-mon[34723]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:22.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:22 smithi187 ceph-mon[34603]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:22 smithi112 ceph-mon[31442]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:25.613 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:25 smithi203 ceph-mon[34723]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:25 smithi187 ceph-mon[34603]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:25 smithi112 ceph-mon[31442]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:27.562 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:27 smithi203 ceph-mon[34723]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:27 smithi187 ceph-mon[34603]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:27.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:27 smithi112 ceph-mon[31442]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:29.648 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:29 smithi203 ceph-mon[34723]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:29.648 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:23:29.648 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:29 smithi187 ceph-mon[34603]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:23:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:29.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:29 smithi112 ceph-mon[31442]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:29.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:23:29.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:30.539 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:30 smithi203 ceph-mon[34723]: Deploying daemon osd.5 on smithi203 2024-09-21T03:23:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:30 smithi187 ceph-mon[34603]: Deploying daemon osd.5 on smithi203 2024-09-21T03:23:30.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:30 smithi112 ceph-mon[31442]: Deploying daemon osd.5 on smithi203 2024-09-21T03:23:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:31 smithi203 ceph-mon[34723]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:31 smithi187 ceph-mon[34603]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:31 smithi112 ceph-mon[31442]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:33.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:33 smithi203 ceph-mon[34723]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:33.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:33 smithi187 ceph-mon[34603]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:33.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:33 smithi112 ceph-mon[31442]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:35 smithi203 ceph-mon[34723]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:35 smithi187 ceph-mon[34603]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:35 smithi112 ceph-mon[31442]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:37.551 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:37 smithi203 ceph-mon[34723]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:37.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:37 smithi187 ceph-mon[34603]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:37.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:37 smithi112 ceph-mon[31442]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:38 smithi203 ceph-mon[34723]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:38.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:38.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:38.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:38.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:38 smithi187 ceph-mon[34603]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:39.021 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.019+0000 7f2fb0ff9640 1 -- 172.21.15.203:0/652241968 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2fb4063800 con 0x7f2f9c0b6560 2024-09-21T03:23:39.022 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 5 on host 'smithi203' 2024-09-21T03:23:39.024 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 -- 172.21.15.203:0/652241968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f9c0b6560 msgr2=0x7f2f9c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:39.024 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f9c0b6560 0x7f2f9c0b8a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009230 tx=0x7f2fa8031400 comp rx=0 tx=0).stop 2024-09-21T03:23:39.024 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 -- 172.21.15.203:0/652241968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4102ae0 msgr2=0x7f2fb4198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:39.024 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4102ae0 0x7f2fb4198440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4012f90 tx=0x7f2fa4010490 comp rx=0 tx=0).stop 2024-09-21T03:23:39.024 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 -- 172.21.15.203:0/652241968 shutdown_connections 2024-09-21T03:23:39.024 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f9c0b6560 0x7f2f9c0b8a70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:39.025 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fb4108d70 0x7f2fb419dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:39.025 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fb4108420 0x7f2fb41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:39.025 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 --2- 172.21.15.203:0/652241968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4102ae0 0x7f2fb4198440 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:39.025 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 -- 172.21.15.203:0/652241968 >> 172.21.15.203:0/652241968 conn(0x7f2fb4079770 msgr2=0x7f2fb4104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:39.025 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 -- 172.21.15.203:0/652241968 shutdown_connections 2024-09-21T03:23:39.025 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:23:39.021+0000 7f2fba04c640 1 -- 172.21.15.203:0/652241968 wait complete. 2024-09-21T03:23:39.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:39.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:39.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:39.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:39.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:39.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:38 smithi112 ceph-mon[31442]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:39.694 DEBUG:teuthology.orchestra.run.smithi203:osd.5> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T03:23:39.696 INFO:tasks.cephadm:Deploying osd.6 on smithi112 with /dev/vg_nvme/lv_2... 2024-09-21T03:23:39.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:23:39.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:39.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:39.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:39.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:39.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:39.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:39.953 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:40.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:23:40.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:40.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:40.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:40 smithi203 ceph-mon[34723]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:40.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:40 smithi187 ceph-mon[34603]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:41.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:40 smithi112 ceph-mon[31442]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:41.659 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:23:41 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:23:41.351+0000 7f068b5b5640 -1 osd.5 0 log_to_monitors true 2024-09-21T03:23:41.909 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:41 smithi112 ceph-mon[31442]: from='osd.5 [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:23:41.909 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:41 smithi112 ceph-mon[31442]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:23:41.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:41 smithi203 ceph-mon[34723]: from='osd.5 [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:23:41.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:41 smithi203 ceph-mon[34723]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:23:41.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:41 smithi187 ceph-mon[34603]: from='osd.5 [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:23:41.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:41 smithi187 ceph-mon[34603]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:23:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:23:42.667 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-09-21T03:23:42.921 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:23:42.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:23:42.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='osd.5 [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:23:42.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi203 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi203 to 8544M 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:42.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:43.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:23:43.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='osd.5 [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:23:43.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:23:43.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:43.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:23:43.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:43.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi203 2024-09-21T03:23:43.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:43.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:43.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:43.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:43.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi203 to 8544M 2024-09-21T03:23:43.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:43.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:43.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:43.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:43.160 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:23:42.674+0000 7f0687537640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T03:23:43.161 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:23:42.685+0000 7f0683335640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='osd.5 [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi203 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:23:43.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi203 to 8544M 2024-09-21T03:23:43.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:43.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:43.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:23:43.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:23:43.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.426+0000 7fa221ad8640 1 -- 172.21.15.112:0/2006415510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa21c1083d0 msgr2=0x7fa21c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:43.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.426+0000 7fa221ad8640 1 --2- 172.21.15.112:0/2006415510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa21c1083d0 0x7fa21c1087e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa210009900 tx=0x7fa21002a410 comp rx=0 tx=0).stop 2024-09-21T03:23:43.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.426+0000 7fa221ad8640 1 -- 172.21.15.112:0/2006415510 shutdown_connections 2024-09-21T03:23:43.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.426+0000 7fa221ad8640 1 --2- 172.21.15.112:0/2006415510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa21c108d20 0x7fa21c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:43.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.426+0000 7fa221ad8640 1 --2- 172.21.15.112:0/2006415510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa21c1083d0 0x7fa21c1087e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:43.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.426+0000 7fa221ad8640 1 --2- 172.21.15.112:0/2006415510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa21c102ae0 0x7fa21c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:43.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.426+0000 7fa221ad8640 1 -- 172.21.15.112:0/2006415510 >> 172.21.15.112:0/2006415510 conn(0x7fa21c079790 msgr2=0x7fa21c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:23:43.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.426+0000 7fa221ad8640 1 -- 172.21.15.112:0/2006415510 shutdown_connections 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.426+0000 7fa221ad8640 1 -- 172.21.15.112:0/2006415510 wait complete. 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.426+0000 7fa221ad8640 1 Processor -- start 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa221ad8640 1 -- start start 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa221ad8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa21c102ae0 0x7fa21c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa221ad8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa21c1083d0 0x7fa21c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa221ad8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa21c108d20 0x7fa21c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa221ad8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c1129e0 con 0x7fa21c1083d0 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa221ad8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c112860 con 0x7fa21c102ae0 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa221ad8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c112b60 con 0x7fa21c108d20 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa21c1083d0 0x7fa21c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa21c1083d0 0x7fa21c1999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43556/0 (socket says 172.21.15.112:43556) 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21affd640 1 -- 172.21.15.112:0/2914199698 learned_addr learned my addr 172.21.15.112:0/2914199698 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:23:43.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21b7fe640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa21c102ae0 0x7fa21c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21bfff640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa21c108d20 0x7fa21c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21affd640 1 -- 172.21.15.112:0/2914199698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa21c108d20 msgr2=0x7fa21c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21affd640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa21c108d20 0x7fa21c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21affd640 1 -- 172.21.15.112:0/2914199698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa21c102ae0 msgr2=0x7fa21c199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21affd640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa21c102ae0 0x7fa21c199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21affd640 1 -- 172.21.15.112:0/2914199698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21c19f3a0 con 0x7fa21c1083d0 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21b7fe640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa21c102ae0 0x7fa21c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.427+0000 7fa21bfff640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa21c108d20 0x7fa21c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.428+0000 7fa21affd640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa21c1083d0 0x7fa21c1999d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa210006010 tx=0x7fa210004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.428+0000 7fa218ff9640 1 -- 172.21.15.112:0/2914199698 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21003f030 con 0x7fa21c1083d0 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.428+0000 7fa221ad8640 1 -- 172.21.15.112:0/2914199698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21c19f630 con 0x7fa21c1083d0 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.428+0000 7fa218ff9640 1 -- 172.21.15.112:0/2914199698 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa210003aa0 con 0x7fa21c1083d0 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.428+0000 7fa218ff9640 1 -- 172.21.15.112:0/2914199698 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210035070 con 0x7fa21c1083d0 2024-09-21T03:23:43.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.428+0000 7fa221ad8640 1 -- 172.21.15.112:0/2914199698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21c1a5df0 con 0x7fa21c1083d0 2024-09-21T03:23:43.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.429+0000 7fa221ad8640 1 -- 172.21.15.112:0/2914199698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21c103d10 con 0x7fa21c1083d0 2024-09-21T03:23:43.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.430+0000 7fa218ff9640 1 -- 172.21.15.112:0/2914199698 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fa210042030 con 0x7fa21c1083d0 2024-09-21T03:23:43.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.431+0000 7fa218ff9640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2040b6560 0x7fa2040b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:23:43.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.431+0000 7fa21b7fe640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2040b6560 0x7fa2040b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:23:43.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.431+0000 7fa218ff9640 1 -- 172.21.15.112:0/2914199698 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4468+0+0 (secure 0 0 0) 0x7fa210106e10 con 0x7fa21c1083d0 2024-09-21T03:23:43.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.431+0000 7fa21b7fe640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2040b6560 0x7fa2040b8a70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005fd0 tx=0x7fa20c00b5f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:23:43.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.434+0000 7fa218ff9640 1 -- 172.21.15.112:0/2914199698 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa2100ce540 con 0x7fa21c1083d0 2024-09-21T03:23:43.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:23:43.631+0000 7fa221ad8640 1 -- 172.21.15.112:0/2914199698 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa21c063800 con 0x7fa2040b6560 2024-09-21T03:23:44.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:43 smithi112 ceph-mon[31442]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:23:44.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:43 smithi112 ceph-mon[31442]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:23:44.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:43 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:44.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:43 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:44.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:43 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:44.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:43 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:44.002 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:43 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:44.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:43 smithi203 ceph-mon[34723]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:23:44.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:43 smithi203 ceph-mon[34723]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:23:44.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:43 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:44.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:43 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:44.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:43 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:44.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:43 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:44.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:43 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:44.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:43 smithi187 ceph-mon[34603]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:23:44.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:43 smithi187 ceph-mon[34603]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:23:44.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:43 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:44.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:43 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:44.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:43 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:23:44.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:43 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:23:44.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:43 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:44.990 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:44 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:23:44.990 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:44 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:23:44.990 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:44 smithi112 ceph-mon[31442]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:44.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:44 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:44.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:44 smithi112 ceph-mon[31442]: osd.5 [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741] boot 2024-09-21T03:23:44.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:44 smithi112 ceph-mon[31442]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:23:44.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:44 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:44.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:44 smithi112 ceph-mon[31442]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:44.991 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:44 smithi112 ceph-mon[31442]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:23:45.105 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:44 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:23:45.105 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:44 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:23:45.105 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:44 smithi187 ceph-mon[34603]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:45.105 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:44 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:45.105 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:44 smithi187 ceph-mon[34603]: osd.5 [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741] boot 2024-09-21T03:23:45.105 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:44 smithi187 ceph-mon[34603]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:23:45.105 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:44 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:45.106 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:44 smithi187 ceph-mon[34603]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:45.106 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:44 smithi187 ceph-mon[34603]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:23:45.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:44 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:23:45.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:44 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:23:45.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:44 smithi203 ceph-mon[34723]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:23:45.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:44 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:45.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:44 smithi203 ceph-mon[34723]: osd.5 [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741] boot 2024-09-21T03:23:45.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:44 smithi203 ceph-mon[34723]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:23:45.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:44 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:23:45.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:44 smithi203 ceph-mon[34723]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:23:45.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:44 smithi203 ceph-mon[34723]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:23:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:46 smithi203 ceph-mon[34723]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:23:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:46 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1449402603' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a3b2e075-188b-4640-827b-735e57c61d82"} : dispatch 2024-09-21T03:23:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:46 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1449402603' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3b2e075-188b-4640-827b-735e57c61d82"}]': finished 2024-09-21T03:23:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:46 smithi203 ceph-mon[34723]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:23:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:46 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:23:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:46 smithi203 ceph-mon[34723]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:23:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:46 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/641214273' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:46.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:46 smithi187 ceph-mon[34603]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:23:46.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:46 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1449402603' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a3b2e075-188b-4640-827b-735e57c61d82"} : dispatch 2024-09-21T03:23:46.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:46 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1449402603' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3b2e075-188b-4640-827b-735e57c61d82"}]': finished 2024-09-21T03:23:46.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:46 smithi187 ceph-mon[34603]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:23:46.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:46 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:23:46.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:46 smithi187 ceph-mon[34603]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:23:46.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:46 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/641214273' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:46 smithi112 ceph-mon[31442]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:23:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:46 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1449402603' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a3b2e075-188b-4640-827b-735e57c61d82"} : dispatch 2024-09-21T03:23:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:46 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1449402603' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3b2e075-188b-4640-827b-735e57c61d82"}]': finished 2024-09-21T03:23:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:46 smithi112 ceph-mon[31442]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:23:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:46 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:23:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:46 smithi112 ceph-mon[31442]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:23:47.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:46 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/641214273' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:23:49.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:49 smithi203 ceph-mon[34723]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-21T03:23:49.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:49 smithi187 ceph-mon[34603]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-21T03:23:49.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:49 smithi112 ceph-mon[31442]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-21T03:23:51.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:51 smithi203 ceph-mon[34723]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:23:51.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:51 smithi187 ceph-mon[34603]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:23:51.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:51 smithi112 ceph-mon[31442]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:23:53.465 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:53 smithi112 ceph-mon[31442]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-21T03:23:53.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:53 smithi203 ceph-mon[34723]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-21T03:23:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:53 smithi187 ceph-mon[34603]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-21T03:23:55.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:55 smithi203 ceph-mon[34723]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T03:23:55.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:55 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:23:55.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:55 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:55.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:55 smithi187 ceph-mon[34603]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T03:23:55.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:55 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:23:55.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:55 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:55.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:55 smithi112 ceph-mon[31442]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T03:23:55.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:55 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:23:55.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:55 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:23:56.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:56 smithi187 ceph-mon[34603]: Deploying daemon osd.6 on smithi112 2024-09-21T03:23:56.552 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:56 smithi112 ceph-mon[31442]: Deploying daemon osd.6 on smithi112 2024-09-21T03:23:56.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:56 smithi203 ceph-mon[34723]: Deploying daemon osd.6 on smithi112 2024-09-21T03:23:57.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:57 smithi112 ceph-mon[31442]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:23:57.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:57 smithi203 ceph-mon[34723]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:23:58.040 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:57 smithi187 ceph-mon[34603]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T03:23:59.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:23:59 smithi203 ceph-mon[34723]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:23:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:23:59 smithi187 ceph-mon[34603]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:23:59.770 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:23:59 smithi112 ceph-mon[31442]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:24:01.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:01 smithi203 ceph-mon[34723]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:01.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:01 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:01.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:01 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:01.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:01 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:01.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:01 smithi187 ceph-mon[34603]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:01.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:01 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:01.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:01 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:01.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:01 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:01.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:01 smithi112 ceph-mon[31442]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:01.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:01 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:01.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:01 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:01.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:01 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:03.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:03 smithi112 ceph-mon[31442]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:03.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:03 smithi203 ceph-mon[34723]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:03 smithi187 ceph-mon[34603]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:04.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:04.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:04.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:04.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:04.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:04 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:04.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:04 smithi112 ceph-mon[31442]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:04.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.531+0000 7fa218ff9640 1 -- 172.21.15.112:0/2914199698 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa21c063800 con 0x7fa2040b6560 2024-09-21T03:24:04.533 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 6 on host 'smithi112' 2024-09-21T03:24:04.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.533+0000 7fa221ad8640 1 -- 172.21.15.112:0/2914199698 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2040b6560 msgr2=0x7fa2040b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:04.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.533+0000 7fa221ad8640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2040b6560 0x7fa2040b8a70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005fd0 tx=0x7fa20c00b5f0 comp rx=0 tx=0).stop 2024-09-21T03:24:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.533+0000 7fa221ad8640 1 -- 172.21.15.112:0/2914199698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa21c1083d0 msgr2=0x7fa21c1999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.533+0000 7fa221ad8640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa21c1083d0 0x7fa21c1999d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa210006010 tx=0x7fa210004060 comp rx=0 tx=0).stop 2024-09-21T03:24:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.533+0000 7fa221ad8640 1 -- 172.21.15.112:0/2914199698 shutdown_connections 2024-09-21T03:24:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.533+0000 7fa221ad8640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2040b6560 0x7fa2040b8a70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.533+0000 7fa221ad8640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa21c108d20 0x7fa21c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.533+0000 7fa221ad8640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa21c1083d0 0x7fa21c1999d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.534+0000 7fa221ad8640 1 --2- 172.21.15.112:0/2914199698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa21c102ae0 0x7fa21c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.534+0000 7fa221ad8640 1 -- 172.21.15.112:0/2914199698 >> 172.21.15.112:0/2914199698 conn(0x7fa21c079790 msgr2=0x7fa21c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.534+0000 7fa221ad8640 1 -- 172.21.15.112:0/2914199698 shutdown_connections 2024-09-21T03:24:04.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:04.534+0000 7fa221ad8640 1 -- 172.21.15.112:0/2914199698 wait complete. 2024-09-21T03:24:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:04 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:04 smithi203 ceph-mon[34723]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:04 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:04 smithi187 ceph-mon[34603]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:05.196 DEBUG:teuthology.orchestra.run.smithi112:osd.6> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T03:24:05.198 INFO:tasks.cephadm:Deploying osd.7 on smithi187 with /dev/vg_nvme/lv_2... 2024-09-21T03:24:05.198 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:24:05.453 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:24:05.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:05 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:05.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:05 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:05.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:05 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:05.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:05 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:05.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:05 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:05.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:05 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:05.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:05 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:05.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:05 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:05 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:06.606 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:06 smithi112 ceph-mon[31442]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:06.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:06 smithi203 ceph-mon[34723]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:06.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:06 smithi187 ceph-mon[34603]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:07.287 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:24:07 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:24:07.007+0000 7f32cff6e640 -1 osd.6 0 log_to_monitors true 2024-09-21T03:24:07.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:07 smithi112 ceph-mon[31442]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:24:07.649 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:07 smithi187 ceph-mon[34603]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:24:07.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:07 smithi203 ceph-mon[34723]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:24:07.983 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:24:08.009 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi187:vg_nvme/lv_2 2024-09-21T03:24:08.262 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:24:08.647 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi112 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi112 to 5696M 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:08.648 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:08.649 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.649 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:08 smithi187 ceph-mon[34603]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:08.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.694+0000 7f58f6f11640 1 -- 172.21.15.187:0/658341634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f0103b60 msgr2=0x7f58f0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:08.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.694+0000 7f58f6f11640 1 --2- 172.21.15.187:0/658341634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f0103b60 0x7f58f0103f70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009900 tx=0x7f58e002a410 comp rx=0 tx=0).stop 2024-09-21T03:24:08.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.694+0000 7f58f6f11640 1 -- 172.21.15.187:0/658341634 shutdown_connections 2024-09-21T03:24:08.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.694+0000 7f58f6f11640 1 --2- 172.21.15.187:0/658341634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0109df0 0x7f58f0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:08.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.694+0000 7f58f6f11640 1 --2- 172.21.15.187:0/658341634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f01094a0 0x7f58f01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:08.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.694+0000 7f58f6f11640 1 --2- 172.21.15.187:0/658341634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f0103b60 0x7f58f0103f70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:08.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.694+0000 7f58f6f11640 1 -- 172.21.15.187:0/658341634 >> 172.21.15.187:0/658341634 conn(0x7f58f007b850 msgr2=0x7f58f007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:08.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.694+0000 7f58f6f11640 1 -- 172.21.15.187:0/658341634 shutdown_connections 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.694+0000 7f58f6f11640 1 -- 172.21.15.187:0/658341634 wait complete. 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f6f11640 1 Processor -- start 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f6f11640 1 -- start start 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f6f11640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f0103b60 0x7f58f019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f6f11640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f01094a0 0x7f58f019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f6f11640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0109df0 0x7f58f019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f6f11640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0113ab0 con 0x7f58f0109df0 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f6f11640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0113930 con 0x7f58f0103b60 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f6f11640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0113c30 con 0x7f58f01094a0 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f4cad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f0103b60 0x7f58f019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f4cad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f0103b60 0x7f58f019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:37010/0 (socket says 172.21.15.187:37010) 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f4cad640 1 -- 172.21.15.187:0/3688022532 learned_addr learned my addr 172.21.15.187:0/3688022532 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58effff640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f01094a0 0x7f58f019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:08.710 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f54ae640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0109df0 0x7f58f019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f4cad640 1 -- 172.21.15.187:0/3688022532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f01094a0 msgr2=0x7f58f019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f4cad640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f01094a0 0x7f58f019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.695+0000 7f58f4cad640 1 -- 172.21.15.187:0/3688022532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0109df0 msgr2=0x7f58f019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.696+0000 7f58f4cad640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0109df0 0x7f58f019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.696+0000 7f58f4cad640 1 -- 172.21.15.187:0/3688022532 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f01a03d0 con 0x7f58f0103b60 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.696+0000 7f58f4cad640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f0103b60 0x7f58f019a460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58e0005940 tx=0x7f58e00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.696+0000 7f58edffb640 1 -- 172.21.15.187:0/3688022532 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e003f030 con 0x7f58f0103b60 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.696+0000 7f58edffb640 1 -- 172.21.15.187:0/3688022532 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e0003cb0 con 0x7f58f0103b60 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.696+0000 7f58edffb640 1 -- 172.21.15.187:0/3688022532 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0035070 con 0x7f58f0103b60 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.696+0000 7f58f6f11640 1 -- 172.21.15.187:0/3688022532 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f01a0660 con 0x7f58f0103b60 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.696+0000 7f58f6f11640 1 -- 172.21.15.187:0/3688022532 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01a6e20 con 0x7f58f0103b60 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.697+0000 7f58f6f11640 1 -- 172.21.15.187:0/3688022532 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f0103f70 con 0x7f58f0103b60 2024-09-21T03:24:08.711 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.699+0000 7f58edffb640 1 -- 172.21.15.187:0/3688022532 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f58e0042030 con 0x7f58f0103b60 2024-09-21T03:24:08.712 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.699+0000 7f58edffb640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58d80b6560 0x7f58d80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:08.712 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.700+0000 7f58edffb640 1 -- 172.21.15.187:0/3688022532 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4900+0+0 (secure 0 0 0) 0x7f58e0107080 con 0x7f58f0103b60 2024-09-21T03:24:08.712 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.700+0000 7f58effff640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58d80b6560 0x7f58d80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:08.712 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.700+0000 7f58effff640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58d80b6560 0x7f58d80b8a70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f58f0109200 tx=0x7f58e400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:08.712 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.703+0000 7f58edffb640 1 -- 172.21.15.187:0/3688022532 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f58e00d2a40 con 0x7f58f0103b60 2024-09-21T03:24:08.787 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:24:08.423+0000 7f32cbef0640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T03:24:08.787 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:24:08.432+0000 7f32c7cee640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:24:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:24:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:24:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:24:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi112 2024-09-21T03:24:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:08.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:08.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:08.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi112 to 5696M 2024-09-21T03:24:08.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:08.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:08.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:08 smithi112 ceph-mon[31442]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:08.891 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:08.889+0000 7f58f6f11640 1 -- 172.21.15.187:0/3688022532 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f58f010bbf0 con 0x7f58d80b6560 2024-09-21T03:24:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:24:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:24:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:24:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi112 2024-09-21T03:24:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi112 to 5696M 2024-09-21T03:24:08.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:08.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:08.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:08.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:08 smithi203 ceph-mon[34723]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:09.586 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:09 smithi187 ceph-mon[34603]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:24:09.587 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:09 smithi187 ceph-mon[34603]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:24:09.587 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:09.587 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:09.587 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:09 smithi187 ceph-mon[34603]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:09.587 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:09.587 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:09.587 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:09 smithi112 ceph-mon[31442]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:24:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:09 smithi112 ceph-mon[31442]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:24:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:09 smithi112 ceph-mon[31442]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:09 smithi203 ceph-mon[34723]: from='osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:24:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:09 smithi203 ceph-mon[34723]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:24:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:09 smithi203 ceph-mon[34723]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:10.502 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:10 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:24:10.502 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:10 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:24:10.502 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:10 smithi187 ceph-mon[34603]: osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356] boot 2024-09-21T03:24:10.502 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:10 smithi187 ceph-mon[34603]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:24:10.502 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:10.502 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:10 smithi187 ceph-mon[34603]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:10 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:24:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:10 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:24:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:10 smithi112 ceph-mon[31442]: osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356] boot 2024-09-21T03:24:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:10 smithi112 ceph-mon[31442]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:24:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:10 smithi112 ceph-mon[31442]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:10.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:10 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:24:10.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:10 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:24:10.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:10 smithi203 ceph-mon[34723]: osd.6 [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356] boot 2024-09-21T03:24:10.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:10 smithi203 ceph-mon[34723]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:24:10.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:24:10.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:10 smithi203 ceph-mon[34723]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:24:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:11 smithi112 ceph-mon[31442]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:24:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:11 smithi112 ceph-mon[31442]: from='client.? 172.21.15.187:0/2590347239' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "530f9be1-24fa-4f16-8225-fddebd4aba99"} : dispatch 2024-09-21T03:24:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:11 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "530f9be1-24fa-4f16-8225-fddebd4aba99"} : dispatch 2024-09-21T03:24:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:11 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "530f9be1-24fa-4f16-8225-fddebd4aba99"}]': finished 2024-09-21T03:24:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:11 smithi112 ceph-mon[31442]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:24:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:11.844 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:11 smithi187 ceph-mon[34603]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:24:11.844 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:11 smithi187 ceph-mon[34603]: from='client.? 172.21.15.187:0/2590347239' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "530f9be1-24fa-4f16-8225-fddebd4aba99"} : dispatch 2024-09-21T03:24:11.844 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:11 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "530f9be1-24fa-4f16-8225-fddebd4aba99"} : dispatch 2024-09-21T03:24:11.844 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:11 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "530f9be1-24fa-4f16-8225-fddebd4aba99"}]': finished 2024-09-21T03:24:11.844 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:11 smithi187 ceph-mon[34603]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:24:11.844 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:11.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:11 smithi203 ceph-mon[34723]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:24:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:11 smithi203 ceph-mon[34723]: from='client.? 172.21.15.187:0/2590347239' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "530f9be1-24fa-4f16-8225-fddebd4aba99"} : dispatch 2024-09-21T03:24:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:11 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "530f9be1-24fa-4f16-8225-fddebd4aba99"} : dispatch 2024-09-21T03:24:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:11 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "530f9be1-24fa-4f16-8225-fddebd4aba99"}]': finished 2024-09-21T03:24:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:11 smithi203 ceph-mon[34723]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:24:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:12.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:12 smithi112 ceph-mon[31442]: from='client.? 172.21.15.187:0/2471113378' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:12.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:12 smithi112 ceph-mon[31442]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:12.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:12 smithi203 ceph-mon[34723]: from='client.? 172.21.15.187:0/2471113378' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:12.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:12 smithi203 ceph-mon[34723]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:12.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:12 smithi187 ceph-mon[34603]: from='client.? 172.21.15.187:0/2471113378' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:12.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:12 smithi187 ceph-mon[34603]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:15.614 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:15 smithi203 ceph-mon[34723]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:15.633 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:15 smithi187 ceph-mon[34603]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:15.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:15 smithi112 ceph-mon[31442]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:17.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:17 smithi203 ceph-mon[34723]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:17 smithi187 ceph-mon[34603]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:17 smithi112 ceph-mon[31442]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:19.511 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:19 smithi187 ceph-mon[34603]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:19.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:19 smithi203 ceph-mon[34723]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:19.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:19 smithi112 ceph-mon[31442]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:20.589 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:20 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:24:20.589 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:20 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:20.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:20 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:24:20.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:20 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:20 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:24:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:20 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:21.612 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:21 smithi187 ceph-mon[34603]: Deploying daemon osd.7 on smithi187 2024-09-21T03:24:21.612 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:21 smithi187 ceph-mon[34603]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:21.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:21 smithi203 ceph-mon[34723]: Deploying daemon osd.7 on smithi187 2024-09-21T03:24:21.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:21 smithi203 ceph-mon[34723]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:21 smithi112 ceph-mon[31442]: Deploying daemon osd.7 on smithi187 2024-09-21T03:24:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:21 smithi112 ceph-mon[31442]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:23 smithi203 ceph-mon[34723]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:23.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:23 smithi187 ceph-mon[34603]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:23.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:23 smithi112 ceph-mon[31442]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:25.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:25 smithi187 ceph-mon[34603]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:25.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:25 smithi203 ceph-mon[34723]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:25 smithi112 ceph-mon[31442]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:26.407 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:26 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:26.408 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:26 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:26.408 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:26 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:26.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:26 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:26 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:26 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:26 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:26 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:26 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:27.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:27 smithi203 ceph-mon[34723]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:27 smithi187 ceph-mon[34603]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:27.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:27 smithi112 ceph-mon[31442]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:29.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:29 smithi187 ceph-mon[34603]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:29.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:29.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:29.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:29.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:29.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:29.505 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 7 on host 'smithi187' 2024-09-21T03:24:29.505 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.504+0000 7f58edffb640 1 -- 172.21.15.187:0/3688022532 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f58f010bbf0 con 0x7f58d80b6560 2024-09-21T03:24:29.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.506+0000 7f58f6f11640 1 -- 172.21.15.187:0/3688022532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58d80b6560 msgr2=0x7f58d80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:29.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.506+0000 7f58f6f11640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58d80b6560 0x7f58d80b8a70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f58f0109200 tx=0x7f58e400b390 comp rx=0 tx=0).stop 2024-09-21T03:24:29.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.506+0000 7f58f6f11640 1 -- 172.21.15.187:0/3688022532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f0103b60 msgr2=0x7f58f019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:29.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.506+0000 7f58f6f11640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f0103b60 0x7f58f019a460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58e0005940 tx=0x7f58e00042f0 comp rx=0 tx=0).stop 2024-09-21T03:24:29.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.506+0000 7f58f6f11640 1 -- 172.21.15.187:0/3688022532 shutdown_connections 2024-09-21T03:24:29.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.506+0000 7f58f6f11640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58d80b6560 0x7f58d80b8a70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:29.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.506+0000 7f58f6f11640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f0109df0 0x7f58f019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:29.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.506+0000 7f58f6f11640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f01094a0 0x7f58f019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:29.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.506+0000 7f58f6f11640 1 --2- 172.21.15.187:0/3688022532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f0103b60 0x7f58f019a460 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:29.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.506+0000 7f58f6f11640 1 -- 172.21.15.187:0/3688022532 >> 172.21.15.187:0/3688022532 conn(0x7f58f007b850 msgr2=0x7f58f0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:29.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.507+0000 7f58f6f11640 1 -- 172.21.15.187:0/3688022532 shutdown_connections 2024-09-21T03:24:29.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:24:29.507+0000 7f58f6f11640 1 -- 172.21.15.187:0/3688022532 wait complete. 2024-09-21T03:24:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:29 smithi203 ceph-mon[34723]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:29.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:29 smithi112 ceph-mon[31442]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:29.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:29.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:29.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:29.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:29.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:30.115 DEBUG:teuthology.orchestra.run.smithi187:osd.7> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T03:24:30.117 INFO:tasks.cephadm:Deploying osd.8 on smithi203 with /dev/vg_nvme/lv_2... 2024-09-21T03:24:30.118 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:24:30.376 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:24:30.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:30 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:30.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:30 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:30.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:30 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:30 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:30 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:30.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:30 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:30.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:30 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:30.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:30 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:30.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:30 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:31.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:31 smithi203 ceph-mon[34723]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:31.467 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:31 smithi187 ceph-mon[34603]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:31 smithi112 ceph-mon[31442]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:32.397 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:24:32 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:24:32.100+0000 7fdfcff91640 -1 osd.7 0 log_to_monitors true 2024-09-21T03:24:32.477 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:32 smithi203 ceph-mon[34723]: from='osd.7 [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:24:32.477 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:32 smithi203 ceph-mon[34723]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:24:32.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:32 smithi187 ceph-mon[34603]: from='osd.7 [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:24:32.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:32 smithi187 ceph-mon[34603]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:24:32.781 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:24:32.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:32 smithi112 ceph-mon[31442]: from='osd.7 [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:24:32.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:32 smithi112 ceph-mon[31442]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:24:32.807 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi203:vg_nvme/lv_2 2024-09-21T03:24:33.063 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:24:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:24:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='osd.7 [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:24:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:24:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:24:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi187 2024-09-21T03:24:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:33.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:33.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:33.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi187 to 5696M 2024-09-21T03:24:33.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:33.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:33.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:33.484 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:33.495 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.493+0000 7fc6d50d8640 1 -- 172.21.15.203:0/790363166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d0103b60 msgr2=0x7fc6d0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:33.495 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.493+0000 7fc6d50d8640 1 --2- 172.21.15.203:0/790363166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d0103b60 0x7fc6d0103f70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009900 tx=0x7fc6c002a410 comp rx=0 tx=0).stop 2024-09-21T03:24:33.495 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.493+0000 7fc6d50d8640 1 -- 172.21.15.203:0/790363166 shutdown_connections 2024-09-21T03:24:33.495 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.493+0000 7fc6d50d8640 1 --2- 172.21.15.203:0/790363166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d0109df0 0x7fc6d0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:33.495 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.493+0000 7fc6d50d8640 1 --2- 172.21.15.203:0/790363166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d01094a0 0x7fc6d01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:33.495 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.493+0000 7fc6d50d8640 1 --2- 172.21.15.203:0/790363166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d0103b60 0x7fc6d0103f70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:33.495 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.493+0000 7fc6d50d8640 1 -- 172.21.15.203:0/790363166 >> 172.21.15.203:0/790363166 conn(0x7fc6d007b850 msgr2=0x7fc6d007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:33.495 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.493+0000 7fc6d50d8640 1 -- 172.21.15.203:0/790363166 shutdown_connections 2024-09-21T03:24:33.496 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.493+0000 7fc6d50d8640 1 -- 172.21.15.203:0/790363166 wait complete. 2024-09-21T03:24:33.497 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.494+0000 7fc6d50d8640 1 Processor -- start 2024-09-21T03:24:33.497 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.494+0000 7fc6d50d8640 1 -- start start 2024-09-21T03:24:33.497 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6d50d8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d0103b60 0x7fc6d019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6d50d8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d01094a0 0x7fc6d019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6d50d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d0109df0 0x7fc6d019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6d50d8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0113ab0 con 0x7fc6d0109df0 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6d50d8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0113930 con 0x7fc6d01094a0 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6d50d8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0113c30 con 0x7fc6d0103b60 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6ced9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d0103b60 0x7fc6d019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6ced9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d0103b60 0x7fc6d019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:54580/0 (socket says 172.21.15.203:54580) 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6ced9d640 1 -- 172.21.15.203:0/1972442097 learned_addr learned my addr 172.21.15.203:0/1972442097 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6ce59c640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d01094a0 0x7fc6d019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6cf59e640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d0109df0 0x7fc6d019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6ced9d640 1 -- 172.21.15.203:0/1972442097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d01094a0 msgr2=0x7fc6d019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6ced9d640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d01094a0 0x7fc6d019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:33.498 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6ced9d640 1 -- 172.21.15.203:0/1972442097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d0109df0 msgr2=0x7fc6d019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:33.499 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6ced9d640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d0109df0 0x7fc6d019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:33.499 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6ced9d640 1 -- 172.21.15.203:0/1972442097 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d01a0490 con 0x7fc6d0103b60 2024-09-21T03:24:33.499 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.495+0000 7fc6ced9d640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d0103b60 0x7fc6d019a520 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0005940 tx=0x7fc6c0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:33.499 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.496+0000 7fc6b7fff640 1 -- 172.21.15.203:0/1972442097 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c003f030 con 0x7fc6d0103b60 2024-09-21T03:24:33.499 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.496+0000 7fc6b7fff640 1 -- 172.21.15.203:0/1972442097 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6c0003aa0 con 0x7fc6d0103b60 2024-09-21T03:24:33.499 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.496+0000 7fc6d50d8640 1 -- 172.21.15.203:0/1972442097 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d01a0720 con 0x7fc6d0103b60 2024-09-21T03:24:33.499 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.496+0000 7fc6b7fff640 1 -- 172.21.15.203:0/1972442097 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0035070 con 0x7fc6d0103b60 2024-09-21T03:24:33.499 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.496+0000 7fc6d50d8640 1 -- 172.21.15.203:0/1972442097 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d01a6ee0 con 0x7fc6d0103b60 2024-09-21T03:24:33.499 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.497+0000 7fc6d50d8640 1 -- 172.21.15.203:0/1972442097 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d0103f70 con 0x7fc6d0103b60 2024-09-21T03:24:33.500 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.498+0000 7fc6b7fff640 1 -- 172.21.15.203:0/1972442097 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fc6c0042030 con 0x7fc6d0103b60 2024-09-21T03:24:33.501 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.499+0000 7fc6b7fff640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6b80b6560 0x7fc6b80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:33.501 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.499+0000 7fc6b7fff640 1 -- 172.21.15.203:0/1972442097 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5332+0+0 (secure 0 0 0) 0x7fc6c0107120 con 0x7fc6d0103b60 2024-09-21T03:24:33.501 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.499+0000 7fc6ce59c640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6b80b6560 0x7fc6b80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:33.501 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.500+0000 7fc6ce59c640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6b80b6560 0x7fc6b80b8a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc6d01064c0 tx=0x7fc6c4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:33.504 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.502+0000 7fc6b7fff640 1 -- 172.21.15.203:0/1972442097 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc6c00d2930 con 0x7fc6d0103b60 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='osd.7 [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi187 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:33.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi187 to 5696M 2024-09-21T03:24:33.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:33.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:33.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:33.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:33.692 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:33.690+0000 7fc6d50d8640 1 -- 172.21.15.203:0/1972442097 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc6d0063800 con 0x7fc6b80b6560 2024-09-21T03:24:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:24:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='osd.7 [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:24:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:24:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:24:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi187 2024-09-21T03:24:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:33.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:33.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:33.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi187 to 5696M 2024-09-21T03:24:33.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:33.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:33.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:33.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:34 smithi203 ceph-mon[34723]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:24:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:34 smithi203 ceph-mon[34723]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:24:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:34 smithi203 ceph-mon[34723]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:34.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:34 smithi203 ceph-mon[34723]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:34.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:34 smithi203 ceph-mon[34723]: from='osd.7 ' entity='osd.7' 2024-09-21T03:24:34.679 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:24:34.313+0000 7fdfcbf13640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T03:24:34.679 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:24:34.486+0000 7fdfc3d09640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:24:34.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-mon[34603]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:24:34.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-mon[34603]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:24:34.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:34.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:34.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-mon[34603]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:34.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:34.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:34.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:34.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-mon[34603]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:34.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:34 smithi187 ceph-mon[34603]: from='osd.7 ' entity='osd.7' 2024-09-21T03:24:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:34 smithi112 ceph-mon[31442]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:24:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:34 smithi112 ceph-mon[31442]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:24:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:34 smithi112 ceph-mon[31442]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:34 smithi112 ceph-mon[31442]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:34.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:34 smithi112 ceph-mon[31442]: from='osd.7 ' entity='osd.7' 2024-09-21T03:24:35.585 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:35 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:24:35.585 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:35 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:24:35.585 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:35.585 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:35 smithi203 ceph-mon[34723]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T03:24:35.585 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:35.585 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:35 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:24:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:35 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:24:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:35 smithi187 ceph-mon[34603]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T03:24:35.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:35.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:35 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:24:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:35 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:24:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:35 smithi112 ceph-mon[31442]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T03:24:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:36.560 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:36 smithi203 ceph-mon[34723]: osd.7 [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172] boot 2024-09-21T03:24:36.560 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:36 smithi203 ceph-mon[34723]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:24:36.560 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:36.560 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:36 smithi203 ceph-mon[34723]: from='client.? 172.21.15.203:0/2470969686' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c3869ee-6cae-409d-98bb-9ee286713295"} : dispatch 2024-09-21T03:24:36.560 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:36 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c3869ee-6cae-409d-98bb-9ee286713295"} : dispatch 2024-09-21T03:24:36.560 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:36 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c3869ee-6cae-409d-98bb-9ee286713295"}]': finished 2024-09-21T03:24:36.560 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:36 smithi203 ceph-mon[34723]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T03:24:36.560 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:36 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:36.560 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:36 smithi203 ceph-mon[34723]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:36 smithi187 ceph-mon[34603]: osd.7 [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172] boot 2024-09-21T03:24:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:36 smithi187 ceph-mon[34603]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:24:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:36 smithi187 ceph-mon[34603]: from='client.? 172.21.15.203:0/2470969686' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c3869ee-6cae-409d-98bb-9ee286713295"} : dispatch 2024-09-21T03:24:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:36 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c3869ee-6cae-409d-98bb-9ee286713295"} : dispatch 2024-09-21T03:24:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:36 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c3869ee-6cae-409d-98bb-9ee286713295"}]': finished 2024-09-21T03:24:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:36 smithi187 ceph-mon[34603]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T03:24:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:36 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:36.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:36 smithi187 ceph-mon[34603]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:36 smithi112 ceph-mon[31442]: osd.7 [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172] boot 2024-09-21T03:24:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:36 smithi112 ceph-mon[31442]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:24:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:24:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:36 smithi112 ceph-mon[31442]: from='client.? 172.21.15.203:0/2470969686' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c3869ee-6cae-409d-98bb-9ee286713295"} : dispatch 2024-09-21T03:24:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:36 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c3869ee-6cae-409d-98bb-9ee286713295"} : dispatch 2024-09-21T03:24:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:36 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c3869ee-6cae-409d-98bb-9ee286713295"}]': finished 2024-09-21T03:24:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:36 smithi112 ceph-mon[31442]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T03:24:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:36 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:36 smithi112 ceph-mon[31442]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:24:37.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:37 smithi187 ceph-mon[34603]: from='client.? 172.21.15.203:0/1946986855' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:37.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:37 smithi187 ceph-mon[34603]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:24:37.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:37 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:37.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:37 smithi112 ceph-mon[31442]: from='client.? 172.21.15.203:0/1946986855' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:37.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:37 smithi112 ceph-mon[31442]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:24:37.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:37 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:37.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:37 smithi203 ceph-mon[34723]: from='client.? 172.21.15.203:0/1946986855' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:24:37.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:37 smithi203 ceph-mon[34723]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:24:37.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:37 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:38.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:38 smithi187 ceph-mon[34603]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:38 smithi112 ceph-mon[31442]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:38.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:38 smithi203 ceph-mon[34723]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:41.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:41 smithi203 ceph-mon[34723]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:41.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:41 smithi187 ceph-mon[34603]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:41.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:41 smithi112 ceph-mon[31442]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:43.377 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:43 smithi203 ceph-mon[34723]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-09-21T03:24:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:43 smithi187 ceph-mon[34603]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-09-21T03:24:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:43 smithi112 ceph-mon[31442]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-09-21T03:24:45.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:45 smithi203 ceph-mon[34723]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-21T03:24:45.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:45 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:24:45.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:45 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:45.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:45 smithi187 ceph-mon[34603]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-21T03:24:45.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:45 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:24:45.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:45 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:45 smithi112 ceph-mon[31442]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-21T03:24:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:45 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:24:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:45 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:46.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:46 smithi203 ceph-mon[34723]: Deploying daemon osd.8 on smithi203 2024-09-21T03:24:46.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:46 smithi187 ceph-mon[34603]: Deploying daemon osd.8 on smithi203 2024-09-21T03:24:46.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:46 smithi112 ceph-mon[31442]: Deploying daemon osd.8 on smithi203 2024-09-21T03:24:47.408 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:47 smithi203 ceph-mon[34723]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T03:24:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:47 smithi187 ceph-mon[34603]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T03:24:47.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:47 smithi112 ceph-mon[31442]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T03:24:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:49 smithi203 ceph-mon[34723]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:24:49.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:49 smithi187 ceph-mon[34603]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:24:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:49 smithi112 ceph-mon[31442]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:24:50.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:50 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:50.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:50 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:50.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:50 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:50.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:50 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:50.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:50 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:50.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:50 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:50 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:50 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:50 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:51.639 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:51 smithi203 ceph-mon[34723]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:24:51.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:51 smithi187 ceph-mon[34603]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:24:51.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:51 smithi112 ceph-mon[31442]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:24:53.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:53 smithi203 ceph-mon[34723]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:24:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:53 smithi187 ceph-mon[34603]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:24:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:53.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:53 smithi112 ceph-mon[31442]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:24:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:53.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:54.046 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.042+0000 7fc6b7fff640 1 -- 172.21.15.203:0/1972442097 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc6d0063800 con 0x7fc6b80b6560 2024-09-21T03:24:54.046 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 8 on host 'smithi203' 2024-09-21T03:24:54.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 -- 172.21.15.203:0/1972442097 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6b80b6560 msgr2=0x7fc6b80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:54.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6b80b6560 0x7fc6b80b8a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc6d01064c0 tx=0x7fc6c4005f90 comp rx=0 tx=0).stop 2024-09-21T03:24:54.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 -- 172.21.15.203:0/1972442097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d0103b60 msgr2=0x7fc6d019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:54.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d0103b60 0x7fc6d019a520 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0005940 tx=0x7fc6c0004270 comp rx=0 tx=0).stop 2024-09-21T03:24:54.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 -- 172.21.15.203:0/1972442097 shutdown_connections 2024-09-21T03:24:54.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6b80b6560 0x7fc6b80b8a70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:54.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d0109df0 0x7fc6d019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:54.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d01094a0 0x7fc6d019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:54.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 --2- 172.21.15.203:0/1972442097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d0103b60 0x7fc6d019a520 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:54.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 -- 172.21.15.203:0/1972442097 >> 172.21.15.203:0/1972442097 conn(0x7fc6d007b850 msgr2=0x7fc6d01075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:54.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 -- 172.21.15.203:0/1972442097 shutdown_connections 2024-09-21T03:24:54.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:24:54.045+0000 7fc6d50d8640 1 -- 172.21.15.203:0/1972442097 wait complete. 2024-09-21T03:24:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:54.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:54.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:54.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:54.689 DEBUG:teuthology.orchestra.run.smithi203:osd.8> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T03:24:54.692 INFO:tasks.cephadm:Deploying osd.9 on smithi112 with /dev/vg_nvme/lv_1... 2024-09-21T03:24:54.692 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:24:54.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:24:54.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:54.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:54.953 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:55.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:55 smithi203 ceph-mon[34723]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:55.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:55 smithi187 ceph-mon[34603]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:55.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:55 smithi112 ceph-mon[31442]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:56.911 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:24:56 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:24:56.581+0000 7f5bc689e640 -1 osd.8 0 log_to_monitors true 2024-09-21T03:24:57.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:57 smithi112 ceph-mon[31442]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:57.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:57 smithi112 ceph-mon[31442]: from='osd.8 [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:24:57.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:57 smithi112 ceph-mon[31442]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:24:57.652 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:24:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:57 smithi203 ceph-mon[34723]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:57 smithi203 ceph-mon[34723]: from='osd.8 [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:24:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:57 smithi203 ceph-mon[34723]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:24:57.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-09-21T03:24:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:57 smithi187 ceph-mon[34603]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:57 smithi187 ceph-mon[34603]: from='osd.8 [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:24:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:57 smithi187 ceph-mon[34603]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:24:57.961 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:24:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.395+0000 7fdca5648640 1 -- 172.21.15.112:0/2912882853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0109e00 msgr2=0x7fdca0110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.395+0000 7fdca5648640 1 --2- 172.21.15.112:0/2912882853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0109e00 0x7fdca0110630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009ca0 tx=0x7fdc9802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.395+0000 7fdca5648640 1 -- 172.21.15.112:0/2912882853 shutdown_connections 2024-09-21T03:24:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.395+0000 7fdca5648640 1 --2- 172.21.15.112:0/2912882853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0109e00 0x7fdca0110630 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.395+0000 7fdca5648640 1 --2- 172.21.15.112:0/2912882853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdca01094b0 0x7fdca01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.395+0000 7fdca5648640 1 --2- 172.21.15.112:0/2912882853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdca0101f70 0x7fdca0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.395+0000 7fdca5648640 1 -- 172.21.15.112:0/2912882853 >> 172.21.15.112:0/2912882853 conn(0x7fdca0077c60 msgr2=0x7fdca00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:24:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.395+0000 7fdca5648640 1 -- 172.21.15.112:0/2912882853 shutdown_connections 2024-09-21T03:24:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.395+0000 7fdca5648640 1 -- 172.21.15.112:0/2912882853 wait complete. 2024-09-21T03:24:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.396+0000 7fdca5648640 1 Processor -- start 2024-09-21T03:24:58.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.396+0000 7fdca5648640 1 -- start start 2024-09-21T03:24:58.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.396+0000 7fdca5648640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0101f70 0x7fdca0102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.396+0000 7fdca5648640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdca01094b0 0x7fdca0103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.396+0000 7fdca5648640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdca0109e00 0x7fdca0103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.396+0000 7fdca5648640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0113ac0 con 0x7fdca0101f70 2024-09-21T03:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.396+0000 7fdca5648640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0113940 con 0x7fdca01094b0 2024-09-21T03:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.396+0000 7fdca5648640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0113c40 con 0x7fdca0109e00 2024-09-21T03:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.397+0000 7fdc9effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0101f70 0x7fdca0102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.397+0000 7fdc9effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0101f70 0x7fdca0102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38994/0 (socket says 172.21.15.112:38994) 2024-09-21T03:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.397+0000 7fdc9effd640 1 -- 172.21.15.112:0/2461535890 learned_addr learned my addr 172.21.15.112:0/2461535890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.397+0000 7fdc9f7fe640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdca0109e00 0x7fdca0103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.397+0000 7fdc9e7fc640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdca01094b0 0x7fdca0103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.397+0000 7fdc9effd640 1 -- 172.21.15.112:0/2461535890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdca0109e00 msgr2=0x7fdca0103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.397+0000 7fdc9effd640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdca0109e00 0x7fdca0103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.397+0000 7fdc9effd640 1 -- 172.21.15.112:0/2461535890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdca01094b0 msgr2=0x7fdca0103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:24:58.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.397+0000 7fdc9effd640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdca01094b0 0x7fdca0103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:24:58.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.397+0000 7fdc9effd640 1 -- 172.21.15.112:0/2461535890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca01a86f0 con 0x7fdca0101f70 2024-09-21T03:24:58.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.397+0000 7fdc9effd640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0101f70 0x7fdca0102f10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000d950 tx=0x7fdc9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:58.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.401+0000 7fdc87fff640 1 -- 172.21.15.112:0/2461535890 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90017070 con 0x7fdca0101f70 2024-09-21T03:24:58.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.401+0000 7fdca5648640 1 -- 172.21.15.112:0/2461535890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca01a89e0 con 0x7fdca0101f70 2024-09-21T03:24:58.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.402+0000 7fdca5648640 1 -- 172.21.15.112:0/2461535890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca01a8f70 con 0x7fdca0101f70 2024-09-21T03:24:58.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.402+0000 7fdc87fff640 1 -- 172.21.15.112:0/2461535890 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc90009c30 con 0x7fdca0101f70 2024-09-21T03:24:58.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.402+0000 7fdc87fff640 1 -- 172.21.15.112:0/2461535890 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90020060 con 0x7fdca0101f70 2024-09-21T03:24:58.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.403+0000 7fdca5648640 1 -- 172.21.15.112:0/2461535890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8c005290 con 0x7fdca0101f70 2024-09-21T03:24:58.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.405+0000 7fdc87fff640 1 -- 172.21.15.112:0/2461535890 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fdc90005030 con 0x7fdca0101f70 2024-09-21T03:24:58.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.406+0000 7fdc87fff640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc880b6560 0x7fdc880b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:24:58.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.406+0000 7fdc9e7fc640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc880b6560 0x7fdc880b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:24:58.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.406+0000 7fdc87fff640 1 -- 172.21.15.112:0/2461535890 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5764+0+0 (secure 0 0 0) 0x7fdc900e6250 con 0x7fdca0101f70 2024-09-21T03:24:58.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.407+0000 7fdc9e7fc640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc880b6560 0x7fdc880b8a70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdca0107070 tx=0x7fdc9400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:24:58.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.409+0000 7fdc87fff640 1 -- 172.21.15.112:0/2461535890 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdc90009de0 con 0x7fdca0101f70 2024-09-21T03:24:58.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:24:58.597+0000 7fdca5648640 1 -- 172.21.15.112:0/2461535890 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdc8c000bc0 con 0x7fdc880b6560 2024-09-21T03:24:58.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:24:58.400+0000 7f5bc2820640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T03:24:58.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:24:58.412+0000 7f5bbe61e640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:24:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:24:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:24:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='osd.8 [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:24:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:24:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:58.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:58.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:58.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:58.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:24:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:24:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='osd.8 [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:24:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:24:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:24:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:24:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='osd.8 [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:24:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:24:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:58.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:24:58.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:58.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:58.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:24:58.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:24:59.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:59 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi203 2024-09-21T03:24:59.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:59 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi203 to 5696M 2024-09-21T03:24:59.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:59 smithi112 ceph-mon[31442]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:59.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:59 smithi112 ceph-mon[31442]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:24:59.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:59 smithi112 ceph-mon[31442]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:24:59.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:59 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:59.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:59 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:59.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:59 smithi112 ceph-mon[31442]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:59.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:59 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:59.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:59 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:59.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:24:59 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:59 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi203 2024-09-21T03:24:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:59 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi203 to 5696M 2024-09-21T03:24:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:59 smithi203 ceph-mon[34723]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:59 smithi203 ceph-mon[34723]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:24:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:59 smithi203 ceph-mon[34723]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:24:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:59 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:59 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:59 smithi203 ceph-mon[34723]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:59 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:59 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:24:59 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:24:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:59 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi203 2024-09-21T03:24:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:59 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi203 to 5696M 2024-09-21T03:24:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:59 smithi187 ceph-mon[34603]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:24:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:59 smithi187 ceph-mon[34603]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:24:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:59 smithi187 ceph-mon[34603]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T03:24:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:59 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:59.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:59 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:24:59.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:59 smithi187 ceph-mon[34603]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:24:59.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:59 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:24:59.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:59 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:24:59.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:24:59 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:00 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:25:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:00 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:25:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:00 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:25:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:00 smithi203 ceph-mon[34723]: osd.8 [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460] boot 2024-09-21T03:25:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:00 smithi203 ceph-mon[34723]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:25:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:00 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:25:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:00 smithi203 ceph-mon[34723]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:25:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:00 smithi203 ceph-mon[34723]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:25:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:00 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:25:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:00 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:25:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:00 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:25:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:00 smithi187 ceph-mon[34603]: osd.8 [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460] boot 2024-09-21T03:25:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:00 smithi187 ceph-mon[34603]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:25:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:00 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:25:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:00 smithi187 ceph-mon[34603]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:25:00.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:00 smithi187 ceph-mon[34603]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:25:00.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:00 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:25:00.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:00 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:25:00.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:00 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:25:00.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:00 smithi112 ceph-mon[31442]: osd.8 [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460] boot 2024-09-21T03:25:00.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:00 smithi112 ceph-mon[31442]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:25:00.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:00 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:25:00.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:00 smithi112 ceph-mon[31442]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:25:00.720 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:00 smithi112 ceph-mon[31442]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:25:01.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:01 smithi203 ceph-mon[34723]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:25:01.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:01 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/948091459' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c47f58d2-8b8a-4f98-b14d-879143b50781"} : dispatch 2024-09-21T03:25:01.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:01 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/948091459' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c47f58d2-8b8a-4f98-b14d-879143b50781"}]': finished 2024-09-21T03:25:01.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:01 smithi203 ceph-mon[34723]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:25:01.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:01 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:01.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:01 smithi187 ceph-mon[34603]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:25:01.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:01 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/948091459' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c47f58d2-8b8a-4f98-b14d-879143b50781"} : dispatch 2024-09-21T03:25:01.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:01 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/948091459' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c47f58d2-8b8a-4f98-b14d-879143b50781"}]': finished 2024-09-21T03:25:01.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:01 smithi187 ceph-mon[34603]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:25:01.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:01 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:02.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:01 smithi112 ceph-mon[31442]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T03:25:02.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:01 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/948091459' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c47f58d2-8b8a-4f98-b14d-879143b50781"} : dispatch 2024-09-21T03:25:02.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:01 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/948091459' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c47f58d2-8b8a-4f98-b14d-879143b50781"}]': finished 2024-09-21T03:25:02.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:01 smithi112 ceph-mon[31442]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T03:25:02.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:01 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:02.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:02 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1928070411' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:02.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:02 smithi203 ceph-mon[34723]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-21T03:25:02.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:02 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1928070411' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:02.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:02 smithi187 ceph-mon[34603]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-21T03:25:03.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:02 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1928070411' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:03.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:02 smithi112 ceph-mon[31442]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-21T03:25:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:05 smithi203 ceph-mon[34723]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-21T03:25:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:05 smithi187 ceph-mon[34603]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-21T03:25:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:05 smithi112 ceph-mon[31442]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-21T03:25:07.629 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:07 smithi112 ceph-mon[31442]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-21T03:25:07.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:07 smithi203 ceph-mon[34723]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-21T03:25:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:07 smithi187 ceph-mon[34603]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-21T03:25:09.614 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:09 smithi112 ceph-mon[31442]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:25:09.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:09 smithi203 ceph-mon[34723]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:25:09.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:09 smithi187 ceph-mon[34603]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:25:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:25:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:10.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:25:10.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T03:25:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:11.557 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:11 smithi112 ceph-mon[31442]: Deploying daemon osd.9 on smithi112 2024-09-21T03:25:11.557 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:11 smithi112 ceph-mon[31442]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T03:25:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:11 smithi203 ceph-mon[34723]: Deploying daemon osd.9 on smithi112 2024-09-21T03:25:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:11 smithi203 ceph-mon[34723]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T03:25:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:11 smithi187 ceph-mon[34603]: Deploying daemon osd.9 on smithi112 2024-09-21T03:25:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:11 smithi187 ceph-mon[34603]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T03:25:13.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:13 smithi203 ceph-mon[34723]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:25:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:13 smithi187 ceph-mon[34603]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:25:13.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:13 smithi112 ceph-mon[31442]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T03:25:15.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:15 smithi203 ceph-mon[34723]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:15.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:15 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:15.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:15 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:15.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:15 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:15 smithi187 ceph-mon[34603]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:15 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:15 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:15 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:15.752 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:15 smithi112 ceph-mon[31442]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:15.752 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:15 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:15.752 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:15 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:15.752 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:15 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:17.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:17 smithi203 ceph-mon[34723]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:17 smithi187 ceph-mon[34603]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:17.681 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:17 smithi112 ceph-mon[31442]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:18.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.886+0000 7fdc87fff640 1 -- 172.21.15.112:0/2461535890 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdc8c000bc0 con 0x7fdc880b6560 2024-09-21T03:25:18.888 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 9 on host 'smithi112' 2024-09-21T03:25:18.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 -- 172.21.15.112:0/2461535890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc880b6560 msgr2=0x7fdc880b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc880b6560 0x7fdc880b8a70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdca0107070 tx=0x7fdc9400b390 comp rx=0 tx=0).stop 2024-09-21T03:25:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 -- 172.21.15.112:0/2461535890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0101f70 msgr2=0x7fdca0102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0101f70 0x7fdca0102f10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000d950 tx=0x7fdc9000de10 comp rx=0 tx=0).stop 2024-09-21T03:25:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 -- 172.21.15.112:0/2461535890 shutdown_connections 2024-09-21T03:25:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc880b6560 0x7fdc880b8a70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdca0109e00 0x7fdca0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdca01094b0 0x7fdca0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 --2- 172.21.15.112:0/2461535890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0101f70 0x7fdca0102f10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 -- 172.21.15.112:0/2461535890 >> 172.21.15.112:0/2461535890 conn(0x7fdca0077c60 msgr2=0x7fdca0105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 -- 172.21.15.112:0/2461535890 shutdown_connections 2024-09-21T03:25:18.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:25:18.889+0000 7fdca5648640 1 -- 172.21.15.112:0/2461535890 wait complete. 2024-09-21T03:25:19.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:19 smithi112 ceph-mon[31442]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:19.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:19.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:19.192 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.192 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:19.192 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.192 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:19 smithi203 ceph-mon[34723]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:19.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:19 smithi187 ceph-mon[34603]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:19.520 DEBUG:teuthology.orchestra.run.smithi112:osd.9> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T03:25:19.522 INFO:tasks.cephadm:Deploying osd.10 on smithi187 with /dev/vg_nvme/lv_1... 2024-09-21T03:25:19.522 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:25:19.778 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:25:21.370 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:25:21 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:25:21.151+0000 7f20995db640 -1 osd.9 0 log_to_monitors true 2024-09-21T03:25:21.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:21 smithi187 ceph-mon[34603]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:21.560 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:21 smithi187 ceph-mon[34603]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:25:21.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:21 smithi203 ceph-mon[34723]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:21 smithi203 ceph-mon[34723]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:25:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:21 smithi112 ceph-mon[31442]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:21 smithi112 ceph-mon[31442]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T03:25:22.248 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-09-21T03:25:22.274 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi187:vg_nvme/lv_1 2024-09-21T03:25:22.537 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:25:22.569 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:22 smithi187 ceph-mon[34603]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:25:22.569 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:22 smithi187 ceph-mon[34603]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:25:22.569 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:22 smithi187 ceph-mon[34603]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:25:22.569 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:22 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:22.569 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:22 smithi187 ceph-mon[34603]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:25:22.569 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:22 smithi187 ceph-mon[34603]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:25:22.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:22 smithi203 ceph-mon[34723]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:25:22.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:22 smithi203 ceph-mon[34723]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:25:22.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:22 smithi203 ceph-mon[34723]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:25:22.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:22 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:22.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:22 smithi203 ceph-mon[34723]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:25:22.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:22 smithi203 ceph-mon[34723]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:25:22.787 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:25:22 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:25:22.372+0000 7f209555d640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T03:25:22.787 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:25:22 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:25:22.385+0000 7f209135b640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:25:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:22 smithi112 ceph-mon[31442]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T03:25:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:22 smithi112 ceph-mon[31442]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T03:25:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:22 smithi112 ceph-mon[31442]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]} : dispatch 2024-09-21T03:25:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:22 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:22 smithi112 ceph-mon[31442]: from='osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-09-21T03:25:22.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:22 smithi112 ceph-mon[31442]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T03:25:22.981 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.975+0000 7ff5427a7640 1 -- 172.21.15.187:0/1139242924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff53c1083d0 msgr2=0x7ff53c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:22.981 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.975+0000 7ff5427a7640 1 --2- 172.21.15.187:0/1139242924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff53c1083d0 0x7ff53c1087e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff524009900 tx=0x7ff52402a410 comp rx=0 tx=0).stop 2024-09-21T03:25:22.981 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.975+0000 7ff5427a7640 1 -- 172.21.15.187:0/1139242924 shutdown_connections 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.975+0000 7ff5427a7640 1 --2- 172.21.15.187:0/1139242924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53c108d20 0x7ff53c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.975+0000 7ff5427a7640 1 --2- 172.21.15.187:0/1139242924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff53c1083d0 0x7ff53c1087e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.975+0000 7ff5427a7640 1 --2- 172.21.15.187:0/1139242924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff53c102ae0 0x7ff53c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.975+0000 7ff5427a7640 1 -- 172.21.15.187:0/1139242924 >> 172.21.15.187:0/1139242924 conn(0x7ff53c0797b0 msgr2=0x7ff53c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.975+0000 7ff5427a7640 1 -- 172.21.15.187:0/1139242924 shutdown_connections 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.975+0000 7ff5427a7640 1 -- 172.21.15.187:0/1139242924 wait complete. 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff5427a7640 1 Processor -- start 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff5427a7640 1 -- start start 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff5427a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff53c102ae0 0x7ff53c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff5427a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff53c1083d0 0x7ff53c19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff5427a7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53c108d20 0x7ff53c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff5427a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c1129e0 con 0x7ff53c1083d0 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff5427a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c112860 con 0x7ff53c102ae0 2024-09-21T03:25:22.982 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff5427a7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c112b60 con 0x7ff53c108d20 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff53bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff53c102ae0 0x7ff53c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff53bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff53c102ae0 0x7ff53c19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:55512/0 (socket says 172.21.15.187:55512) 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff53bfff640 1 -- 172.21.15.187:0/672918132 learned_addr learned my addr 172.21.15.187:0/672918132 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.976+0000 7ff540d44640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53c108d20 0x7ff53c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff53b7fe640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff53c1083d0 0x7ff53c19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff53bfff640 1 -- 172.21.15.187:0/672918132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53c108d20 msgr2=0x7ff53c19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff53bfff640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53c108d20 0x7ff53c19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff53bfff640 1 -- 172.21.15.187:0/672918132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff53c1083d0 msgr2=0x7ff53c19b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff53bfff640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff53c1083d0 0x7ff53c19b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff53bfff640 1 -- 172.21.15.187:0/672918132 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53c1a6540 con 0x7ff53c102ae0 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff53bfff640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff53c102ae0 0x7ff53c19b2b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00d950 tx=0x7ff52c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff5397fa640 1 -- 172.21.15.187:0/672918132 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c017030 con 0x7ff53c102ae0 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff5427a7640 1 -- 172.21.15.187:0/672918132 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53c1a6830 con 0x7ff53c102ae0 2024-09-21T03:25:22.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff5397fa640 1 -- 172.21.15.187:0/672918132 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff52c004020 con 0x7ff53c102ae0 2024-09-21T03:25:22.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff5397fa640 1 -- 172.21.15.187:0/672918132 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0047a0 con 0x7ff53c102ae0 2024-09-21T03:25:22.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.977+0000 7ff5427a7640 1 -- 172.21.15.187:0/672918132 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53c1a6d40 con 0x7ff53c102ae0 2024-09-21T03:25:22.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.978+0000 7ff5427a7640 1 -- 172.21.15.187:0/672918132 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff53c102ef0 con 0x7ff53c102ae0 2024-09-21T03:25:22.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.980+0000 7ff5397fa640 1 -- 172.21.15.187:0/672918132 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7ff52c009a30 con 0x7ff53c102ae0 2024-09-21T03:25:22.984 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.981+0000 7ff5397fa640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff5280b6590 0x7ff5280b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:22.985 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.981+0000 7ff5397fa640 1 -- 172.21.15.187:0/672918132 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6196+0+0 (secure 0 0 0) 0x7ff52c0e6c20 con 0x7ff53c102ae0 2024-09-21T03:25:22.985 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.981+0000 7ff53b7fe640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff5280b6590 0x7ff5280b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:22.985 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.982+0000 7ff53b7fe640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff5280b6590 0x7ff5280b8aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff524009230 tx=0x7ff524031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:22.986 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:22.984+0000 7ff5397fa640 1 -- 172.21.15.187:0/672918132 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff52c0adcf0 con 0x7ff53c102ae0 2024-09-21T03:25:23.199 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:23.197+0000 7ff5427a7640 1 -- 172.21.15.187:0/672918132 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff53c063800 con 0x7ff5280b6590 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:23.574 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:23.575 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.575 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:23.575 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:23.575 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:23.575 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:23.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:23.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:23.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:23.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:23.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:23.791 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:23.792 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:23.793 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:23.793 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:23.793 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:24.563 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:24 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:25:24.563 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:24 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:25:24.563 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:24 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi112 2024-09-21T03:25:24.563 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:24 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi112 to 4272M 2024-09-21T03:25:24.563 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:24 smithi187 ceph-mon[34603]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:24.563 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:24 smithi187 ceph-mon[34603]: osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143] boot 2024-09-21T03:25:24.564 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:24 smithi187 ceph-mon[34603]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:25:24.564 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:24 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:24.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:24 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:25:24.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:24 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:25:24.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:24 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi112 2024-09-21T03:25:24.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:24 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi112 to 4272M 2024-09-21T03:25:24.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:24 smithi203 ceph-mon[34723]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:24.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:24 smithi203 ceph-mon[34723]: osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143] boot 2024-09-21T03:25:24.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:24 smithi203 ceph-mon[34723]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:25:24.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:24 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:24 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:25:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:24 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:25:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:24 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi112 2024-09-21T03:25:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:24 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi112 to 4272M 2024-09-21T03:25:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:24 smithi112 ceph-mon[31442]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:24 smithi112 ceph-mon[31442]: osd.9 [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143] boot 2024-09-21T03:25:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:24 smithi112 ceph-mon[31442]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T03:25:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:24 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:25:25.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:25 smithi203 ceph-mon[34723]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:25.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:25 smithi203 ceph-mon[34723]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:25:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:25 smithi187 ceph-mon[34603]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:25.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:25 smithi187 ceph-mon[34603]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:25:25.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:25 smithi112 ceph-mon[31442]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T03:25:25.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:25 smithi112 ceph-mon[31442]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T03:25:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:26 smithi203 ceph-mon[34723]: from='client.? 172.21.15.187:0/377813817' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "54812bd2-e71b-40eb-8411-d281bdbe0821"} : dispatch 2024-09-21T03:25:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:26 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "54812bd2-e71b-40eb-8411-d281bdbe0821"} : dispatch 2024-09-21T03:25:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:26 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54812bd2-e71b-40eb-8411-d281bdbe0821"}]': finished 2024-09-21T03:25:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:26 smithi203 ceph-mon[34723]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:25:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:26 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:26 smithi203 ceph-mon[34723]: from='client.? 172.21.15.187:0/2600303437' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:26 smithi187 ceph-mon[34603]: from='client.? 172.21.15.187:0/377813817' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "54812bd2-e71b-40eb-8411-d281bdbe0821"} : dispatch 2024-09-21T03:25:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:26 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "54812bd2-e71b-40eb-8411-d281bdbe0821"} : dispatch 2024-09-21T03:25:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:26 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54812bd2-e71b-40eb-8411-d281bdbe0821"}]': finished 2024-09-21T03:25:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:26 smithi187 ceph-mon[34603]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:25:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:26 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:26 smithi187 ceph-mon[34603]: from='client.? 172.21.15.187:0/2600303437' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:26 smithi112 ceph-mon[31442]: from='client.? 172.21.15.187:0/377813817' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "54812bd2-e71b-40eb-8411-d281bdbe0821"} : dispatch 2024-09-21T03:25:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:26 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "54812bd2-e71b-40eb-8411-d281bdbe0821"} : dispatch 2024-09-21T03:25:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:26 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54812bd2-e71b-40eb-8411-d281bdbe0821"}]': finished 2024-09-21T03:25:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:26 smithi112 ceph-mon[31442]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T03:25:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:26 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:26 smithi112 ceph-mon[31442]: from='client.? 172.21.15.187:0/2600303437' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:27.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:27 smithi203 ceph-mon[34723]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:27 smithi187 ceph-mon[34603]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:27.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:27 smithi112 ceph-mon[31442]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:28.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:28 smithi112 ceph-mon[31442]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:28.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:28 smithi203 ceph-mon[34723]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:28.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:28 smithi187 ceph-mon[34603]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:31.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:31 smithi187 ceph-mon[34603]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:31 smithi203 ceph-mon[34723]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:31.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:31 smithi112 ceph-mon[31442]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:33.635 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:33 smithi187 ceph-mon[34603]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:33.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:33 smithi203 ceph-mon[34723]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:33.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:33 smithi112 ceph-mon[31442]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:34.653 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:25:34.653 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:34 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:34.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:25:34.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:34 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T03:25:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:34 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:35 smithi203 ceph-mon[34723]: Deploying daemon osd.10 on smithi187 2024-09-21T03:25:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:35 smithi203 ceph-mon[34723]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:35 smithi187 ceph-mon[34603]: Deploying daemon osd.10 on smithi187 2024-09-21T03:25:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:35 smithi187 ceph-mon[34603]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:35 smithi112 ceph-mon[31442]: Deploying daemon osd.10 on smithi187 2024-09-21T03:25:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:35 smithi112 ceph-mon[31442]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:37.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:37 smithi187 ceph-mon[34603]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:37.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:37 smithi203 ceph-mon[34723]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:37.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:37 smithi112 ceph-mon[31442]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:39.608 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:39 smithi187 ceph-mon[34603]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:39.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:39 smithi203 ceph-mon[34723]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:39 smithi112 ceph-mon[31442]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:40.495 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:40 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:40.495 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:40 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:40.495 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:40 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:40 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:40 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:40 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:40 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:40 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:40 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:41.641 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:41 smithi187 ceph-mon[34603]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:41.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:41 smithi203 ceph-mon[34723]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:41.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:41 smithi112 ceph-mon[31442]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:43.408 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:43 smithi187 ceph-mon[34603]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:43.409 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:43 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:43.409 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:43 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:43.409 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:43 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:43.409 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:43 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:43.409 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:43 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:43.455 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.453+0000 7ff5397fa640 1 -- 172.21.15.187:0/672918132 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff53c063800 con 0x7ff5280b6590 2024-09-21T03:25:43.456 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 10 on host 'smithi187' 2024-09-21T03:25:43.457 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.455+0000 7ff5427a7640 1 -- 172.21.15.187:0/672918132 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff5280b6590 msgr2=0x7ff5280b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:43.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.455+0000 7ff5427a7640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff5280b6590 0x7ff5280b8aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff524009230 tx=0x7ff524031400 comp rx=0 tx=0).stop 2024-09-21T03:25:43.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.455+0000 7ff5427a7640 1 -- 172.21.15.187:0/672918132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff53c102ae0 msgr2=0x7ff53c19b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:43.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.455+0000 7ff5427a7640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff53c102ae0 0x7ff53c19b2b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00d950 tx=0x7ff52c00de10 comp rx=0 tx=0).stop 2024-09-21T03:25:43.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.455+0000 7ff5427a7640 1 -- 172.21.15.187:0/672918132 shutdown_connections 2024-09-21T03:25:43.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.455+0000 7ff5427a7640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff5280b6590 0x7ff5280b8aa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:43.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.455+0000 7ff5427a7640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53c108d20 0x7ff53c19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:43.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.455+0000 7ff5427a7640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff53c1083d0 0x7ff53c19b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:43.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.455+0000 7ff5427a7640 1 --2- 172.21.15.187:0/672918132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff53c102ae0 0x7ff53c19b2b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:43.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.455+0000 7ff5427a7640 1 -- 172.21.15.187:0/672918132 >> 172.21.15.187:0/672918132 conn(0x7ff53c0797b0 msgr2=0x7ff53c106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:43.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.456+0000 7ff5427a7640 1 -- 172.21.15.187:0/672918132 shutdown_connections 2024-09-21T03:25:43.458 INFO:teuthology.orchestra.run.smithi187.stderr:2024-09-21T03:25:43.456+0000 7ff5427a7640 1 -- 172.21.15.187:0/672918132 wait complete. 2024-09-21T03:25:43.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:43 smithi203 ceph-mon[34723]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:43 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:43 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:43 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:43 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:43 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:43 smithi112 ceph-mon[31442]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:43 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:43 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:43 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:43 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:43 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:44.098 DEBUG:teuthology.orchestra.run.smithi187:osd.10> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T03:25:44.100 INFO:tasks.cephadm:Deploying osd.11 on smithi203 with /dev/vg_nvme/lv_1... 2024-09-21T03:25:44.101 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:25:44.359 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:25:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:44 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:44 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:44 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:44 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:44 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:44 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:44 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:25:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:44 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:44 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:45.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:45 smithi187 ceph-mon[34603]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:45.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:45 smithi203 ceph-mon[34723]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:45 smithi112 ceph-mon[31442]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:45.929 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:25:45 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:25:45.437+0000 7f232a14a640 -1 osd.10 0 log_to_monitors true 2024-09-21T03:25:46.653 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:46 smithi187 ceph-mon[34603]: from='osd.10 [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:25:46.653 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:46 smithi187 ceph-mon[34603]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:25:46.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:46 smithi203 ceph-mon[34723]: from='osd.10 [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:25:46.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:46 smithi203 ceph-mon[34723]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:25:46.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:46 smithi112 ceph-mon[31442]: from='osd.10 [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:25:46.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:46 smithi112 ceph-mon[31442]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T03:25:46.869 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-21T03:25:46.894 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi203:vg_nvme/lv_1 2024-09-21T03:25:47.159 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:25:47.544 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:47.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:25:47.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:25:47.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:47.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='osd.10 [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:25:47.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:25:47.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.545 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.546 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.546 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:47.546 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:47.546 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.604+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/1413595888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d4102a60 msgr2=0x7fb2d4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:47.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.604+0000 7fb2d9ae3640 1 --2- 172.21.15.203:0/1413595888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d4102a60 0x7fb2d4102e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009900 tx=0x7fb2c402a410 comp rx=0 tx=0).stop 2024-09-21T03:25:47.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.604+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/1413595888 shutdown_connections 2024-09-21T03:25:47.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.604+0000 7fb2d9ae3640 1 --2- 172.21.15.203:0/1413595888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4108cf0 0x7fb2d410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:47.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.604+0000 7fb2d9ae3640 1 --2- 172.21.15.203:0/1413595888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d41083a0 0x7fb2d41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:47.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.604+0000 7fb2d9ae3640 1 --2- 172.21.15.203:0/1413595888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d4102a60 0x7fb2d4102e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:47.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.604+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/1413595888 >> 172.21.15.203:0/1413595888 conn(0x7fb2d4079730 msgr2=0x7fb2d4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:25:47.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.604+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/1413595888 shutdown_connections 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.604+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/1413595888 wait complete. 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.605+0000 7fb2d9ae3640 1 Processor -- start 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.605+0000 7fb2d9ae3640 1 -- start start 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.605+0000 7fb2d9ae3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4102a60 0x7fb2d419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.605+0000 7fb2d9ae3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d41083a0 0x7fb2d419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.605+0000 7fb2d9ae3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4108cf0 0x7fb2d419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.605+0000 7fb2d9ae3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d4112960 con 0x7fb2d4108cf0 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.605+0000 7fb2d9ae3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d41127e0 con 0x7fb2d4102a60 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.605+0000 7fb2d9ae3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d4112ae0 con 0x7fb2d41083a0 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.605+0000 7fb2cbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d41083a0 0x7fb2d419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.605+0000 7fb2cbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d41083a0 0x7fb2d419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48018/0 (socket says 172.21.15.203:48018) 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.605+0000 7fb2cbfff640 1 -- 172.21.15.203:0/3924181901 learned_addr learned my addr 172.21.15.203:0/3924181901 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2d37fe640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4102a60 0x7fb2d419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:47.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2d3fff640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4108cf0 0x7fb2d419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2cbfff640 1 -- 172.21.15.203:0/3924181901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4102a60 msgr2=0x7fb2d419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2cbfff640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4102a60 0x7fb2d419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2cbfff640 1 -- 172.21.15.203:0/3924181901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4108cf0 msgr2=0x7fb2d419bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2cbfff640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4108cf0 0x7fb2d419bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2cbfff640 1 -- 172.21.15.203:0/3924181901 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d41a6470 con 0x7fb2d41083a0 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2d3fff640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4108cf0 0x7fb2d419bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2cbfff640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d41083a0 0x7fb2d419b7b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c970 tx=0x7fb2bc00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2d17fa640 1 -- 172.21.15.203:0/3924181901 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc014070 con 0x7fb2d41083a0 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/3924181901 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d41a6760 con 0x7fb2d41083a0 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2d17fa640 1 -- 172.21.15.203:0/3924181901 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2bc004440 con 0x7fb2d41083a0 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2d17fa640 1 -- 172.21.15.203:0/3924181901 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc003e50 con 0x7fb2d41083a0 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.606+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/3924181901 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d41a6cf0 con 0x7fb2d41083a0 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.607+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/3924181901 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d4102e70 con 0x7fb2d41083a0 2024-09-21T03:25:47.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.609+0000 7fb2d17fa640 1 -- 172.21.15.203:0/3924181901 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fb2bc017070 con 0x7fb2d41083a0 2024-09-21T03:25:47.630 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.609+0000 7fb2d17fa640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb2c00b6730 0x7fb2c00b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:25:47.630 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.609+0000 7fb2d17fa640 1 -- 172.21.15.203:0/3924181901 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6629+0+0 (secure 0 0 0) 0x7fb2bc0e7070 con 0x7fb2d41083a0 2024-09-21T03:25:47.630 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.609+0000 7fb2d37fe640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb2c00b6730 0x7fb2c00b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:25:47.630 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.610+0000 7fb2d37fe640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb2c00b6730 0x7fb2c00b8c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4000c00 tx=0x7fb2c4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:25:47.630 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.612+0000 7fb2d17fa640 1 -- 172.21.15.203:0/3924181901 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb2bc01e030 con 0x7fb2d41083a0 2024-09-21T03:25:47.679 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:25:47.422+0000 7f23260cc640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T03:25:47.679 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:25:47.435+0000 7f2321eca640 -1 osd.10 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:25:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='osd.10 [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:47.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T03:25:47.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T03:25:47.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:47.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='osd.10 [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:25:47.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]} : dispatch 2024-09-21T03:25:47.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:25:47.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:47.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:25:47.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:25:47.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:25:47.801+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/3924181901 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb2d4063800 con 0x7fb2c00b6730 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi187 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi187 to 4272M 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: from='client.24553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:48.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: osd.10 [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149] boot 2024-09-21T03:25:48.435 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:25:48.435 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:48 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi187 2024-09-21T03:25:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi187 to 4272M 2024-09-21T03:25:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:25:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:25:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: from='client.24553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:48.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:48.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:48.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:48.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: osd.10 [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149] boot 2024-09-21T03:25:48.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:25:48.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:48 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:48.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi187 2024-09-21T03:25:48.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi187 to 4272M 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: from='client.24553 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: osd.10 [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149] boot 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T03:25:48.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:48 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T03:25:49.446 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:49 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:25:49.446 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:49 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:25:49.446 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:49 smithi203 ceph-mon[34723]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:25:49.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:49 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:25:49.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:49 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:25:49.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:49 smithi187 ceph-mon[34603]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:25:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:49 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:25:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:49 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:25:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:49 smithi112 ceph-mon[31442]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T03:25:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:50 smithi112 ceph-mon[31442]: from='client.? 172.21.15.203:0/1484242856' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4157bc4f-e275-4f4d-8d38-f934f9f61a95"} : dispatch 2024-09-21T03:25:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:50 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4157bc4f-e275-4f4d-8d38-f934f9f61a95"} : dispatch 2024-09-21T03:25:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:50 smithi112 ceph-mon[31442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4157bc4f-e275-4f4d-8d38-f934f9f61a95"}]': finished 2024-09-21T03:25:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:50 smithi112 ceph-mon[31442]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T03:25:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:50 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:25:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:50 smithi112 ceph-mon[31442]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:25:50.872 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:50 smithi203 ceph-mon[34723]: from='client.? 172.21.15.203:0/1484242856' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4157bc4f-e275-4f4d-8d38-f934f9f61a95"} : dispatch 2024-09-21T03:25:50.872 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:50 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4157bc4f-e275-4f4d-8d38-f934f9f61a95"} : dispatch 2024-09-21T03:25:50.872 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:50 smithi203 ceph-mon[34723]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4157bc4f-e275-4f4d-8d38-f934f9f61a95"}]': finished 2024-09-21T03:25:50.872 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:50 smithi203 ceph-mon[34723]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T03:25:50.872 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:50 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:25:50.872 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:50 smithi203 ceph-mon[34723]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:25:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:50 smithi187 ceph-mon[34603]: from='client.? 172.21.15.203:0/1484242856' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4157bc4f-e275-4f4d-8d38-f934f9f61a95"} : dispatch 2024-09-21T03:25:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:50 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4157bc4f-e275-4f4d-8d38-f934f9f61a95"} : dispatch 2024-09-21T03:25:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:50 smithi187 ceph-mon[34603]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4157bc4f-e275-4f4d-8d38-f934f9f61a95"}]': finished 2024-09-21T03:25:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:50 smithi187 ceph-mon[34603]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T03:25:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:50 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:25:50.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:50 smithi187 ceph-mon[34603]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:25:51.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:51 smithi112 ceph-mon[31442]: from='client.? 172.21.15.203:0/729038314' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:51.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:51 smithi203 ceph-mon[34723]: from='client.? 172.21.15.203:0/729038314' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:51.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:51 smithi187 ceph-mon[34603]: from='client.? 172.21.15.203:0/729038314' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:25:52.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:52 smithi112 ceph-mon[31442]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T03:25:52.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:52 smithi203 ceph-mon[34723]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T03:25:52.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:52 smithi187 ceph-mon[34603]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T03:25:55.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:55 smithi203 ceph-mon[34723]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:25:55.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:55 smithi187 ceph-mon[34603]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:25:55.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:55 smithi112 ceph-mon[31442]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:25:57.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:57 smithi203 ceph-mon[34723]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:25:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:57 smithi187 ceph-mon[34603]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:25:57.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:57 smithi112 ceph-mon[31442]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T03:25:59.655 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:59 smithi203 ceph-mon[34723]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:25:59.655 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:59 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:25:59.655 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:25:59 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:59 smithi187 ceph-mon[34603]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:25:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:59 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:25:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:25:59 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:25:59.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:59 smithi112 ceph-mon[31442]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T03:25:59.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:59 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T03:25:59.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:25:59 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:00.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:00 smithi203 ceph-mon[34723]: Deploying daemon osd.11 on smithi203 2024-09-21T03:26:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:00 smithi187 ceph-mon[34603]: Deploying daemon osd.11 on smithi203 2024-09-21T03:26:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:00 smithi112 ceph-mon[31442]: Deploying daemon osd.11 on smithi203 2024-09-21T03:26:01.569 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:01 smithi203 ceph-mon[34723]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T03:26:01.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:01 smithi187 ceph-mon[34603]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T03:26:01.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:01 smithi112 ceph-mon[31442]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T03:26:03.494 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:03 smithi203 ceph-mon[34723]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:03 smithi187 ceph-mon[34603]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:03.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:03 smithi112 ceph-mon[31442]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:26:05.508 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:05 smithi203 ceph-mon[34723]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:05.508 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:05 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:05.508 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:05 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:05.508 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:05 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:05 smithi187 ceph-mon[34603]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:05 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:05.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:05 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:05.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:05 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:05 smithi112 ceph-mon[31442]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:05 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:05 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:05 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:07.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:07 smithi203 ceph-mon[34723]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:07 smithi187 ceph-mon[34603]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:07.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:07 smithi112 ceph-mon[31442]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:08.343 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.340+0000 7fb2d17fa640 1 -- 172.21.15.203:0/3924181901 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb2d4063800 con 0x7fb2c00b6730 2024-09-21T03:26:08.343 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 11 on host 'smithi203' 2024-09-21T03:26:08.345 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/3924181901 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb2c00b6730 msgr2=0x7fb2c00b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb2c00b6730 0x7fb2c00b8c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4000c00 tx=0x7fb2c4031400 comp rx=0 tx=0).stop 2024-09-21T03:26:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/3924181901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d41083a0 msgr2=0x7fb2d419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d41083a0 0x7fb2d419b7b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c970 tx=0x7fb2bc00ce30 comp rx=0 tx=0).stop 2024-09-21T03:26:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/3924181901 shutdown_connections 2024-09-21T03:26:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb2c00b6730 0x7fb2c00b8c40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4108cf0 0x7fb2d419bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d41083a0 0x7fb2d419b7b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 --2- 172.21.15.203:0/3924181901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4102a60 0x7fb2d419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/3924181901 >> 172.21.15.203:0/3924181901 conn(0x7fb2d4079730 msgr2=0x7fb2d41013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/3924181901 shutdown_connections 2024-09-21T03:26:08.346 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:08.343+0000 7fb2d9ae3640 1 -- 172.21.15.203:0/3924181901 wait complete. 2024-09-21T03:26:08.628 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.628 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.628 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:08.628 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:08.628 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.628 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:08 smithi203 ceph-mon[34723]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:08.628 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:08.629 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.629 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:08.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:08.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:08 smithi187 ceph-mon[34603]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:08.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:08.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.947 DEBUG:teuthology.orchestra.run.smithi203:osd.11> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T03:26:08.949 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T03:26:08.949 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:26:08.978 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.978 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.978 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:08.979 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:08.979 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.979 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:08 smithi112 ceph-mon[31442]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:08.979 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:08.979 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:08.979 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:09.212 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:09.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.663+0000 7f14e4f24640 1 -- 172.21.15.112:0/3280394702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e0102a60 msgr2=0x7f14e0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.663+0000 7f14e4f24640 1 --2- 172.21.15.112:0/3280394702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e0102a60 0x7f14e0102e70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009900 tx=0x7f14d402a410 comp rx=0 tx=0).stop 2024-09-21T03:26:09.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.663+0000 7f14e4f24640 1 -- 172.21.15.112:0/3280394702 shutdown_connections 2024-09-21T03:26:09.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.663+0000 7f14e4f24640 1 --2- 172.21.15.112:0/3280394702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14e0108cf0 0x7f14e010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.663+0000 7f14e4f24640 1 --2- 172.21.15.112:0/3280394702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14e01083a0 0x7f14e01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.663+0000 7f14e4f24640 1 --2- 172.21.15.112:0/3280394702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e0102a60 0x7f14e0102e70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.663+0000 7f14e4f24640 1 -- 172.21.15.112:0/3280394702 >> 172.21.15.112:0/3280394702 conn(0x7f14e0079730 msgr2=0x7f14e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.663+0000 7f14e4f24640 1 -- 172.21.15.112:0/3280394702 shutdown_connections 2024-09-21T03:26:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.663+0000 7f14e4f24640 1 -- 172.21.15.112:0/3280394702 wait complete. 2024-09-21T03:26:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.664+0000 7f14e4f24640 1 Processor -- start 2024-09-21T03:26:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.664+0000 7f14e4f24640 1 -- start start 2024-09-21T03:26:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.664+0000 7f14e4f24640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14e0102a60 0x7f14e019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.664+0000 7f14e4f24640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e01083a0 0x7f14e019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.664+0000 7f14e4f24640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14e0108cf0 0x7f14e019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.664+0000 7f14e4f24640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0112960 con 0x7f14e01083a0 2024-09-21T03:26:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.664+0000 7f14e4f24640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01127e0 con 0x7f14e0108cf0 2024-09-21T03:26:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.664+0000 7f14e4f24640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0112ae0 con 0x7f14e0102a60 2024-09-21T03:26:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.664+0000 7f14ddd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e01083a0 0x7f14e019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.664+0000 7f14ddd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e01083a0 0x7f14e019b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36744/0 (socket says 172.21.15.112:36744) 2024-09-21T03:26:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.664+0000 7f14ddd9b640 1 -- 172.21.15.112:0/2524446029 learned_addr learned my addr 172.21.15.112:0/2524446029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14de59c640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14e0102a60 0x7f14e019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14ded9d640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14e0108cf0 0x7f14e019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14ddd9b640 1 -- 172.21.15.112:0/2524446029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14e0102a60 msgr2=0x7f14e019b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14ddd9b640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14e0102a60 0x7f14e019b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14ddd9b640 1 -- 172.21.15.112:0/2524446029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14e0108cf0 msgr2=0x7f14e019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14ddd9b640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14e0108cf0 0x7f14e019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14ddd9b640 1 -- 172.21.15.112:0/2524446029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e01a64e0 con 0x7f14e01083a0 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14ddd9b640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e01083a0 0x7f14e019b790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f14d0005fd0 tx=0x7f14d000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14cf7fe640 1 -- 172.21.15.112:0/2524446029 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0017070 con 0x7f14e01083a0 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14cf7fe640 1 -- 172.21.15.112:0/2524446029 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d0009c30 con 0x7f14e01083a0 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14e4f24640 1 -- 172.21.15.112:0/2524446029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e01a67d0 con 0x7f14e01083a0 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14cf7fe640 1 -- 172.21.15.112:0/2524446029 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0003d30 con 0x7f14e01083a0 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.665+0000 7f14e4f24640 1 -- 172.21.15.112:0/2524446029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e01a6d10 con 0x7f14e01083a0 2024-09-21T03:26:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.666+0000 7f14e4f24640 1 -- 172.21.15.112:0/2524446029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e0102e70 con 0x7f14e01083a0 2024-09-21T03:26:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.668+0000 7f14cf7fe640 1 -- 172.21.15.112:0/2524446029 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f14d0003ed0 con 0x7f14e01083a0 2024-09-21T03:26:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.668+0000 7f14cf7fe640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14c80b6590 0x7f14c80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.669+0000 7f14de59c640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14c80b6590 0x7f14c80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.669+0000 7f14cf7fe640 1 -- 172.21.15.112:0/2524446029 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 7024+0+0 (secure 0 0 0) 0x7f14d00e7210 con 0x7f14e01083a0 2024-09-21T03:26:09.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.669+0000 7f14de59c640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14c80b6590 0x7f14c80b8aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f14d4000c00 tx=0x7f14d4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:09.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.672+0000 7f14cf7fe640 1 -- 172.21.15.112:0/2524446029 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14d0020030 con 0x7f14e01083a0 2024-09-21T03:26:09.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.853+0000 7f14e4f24640 1 -- 172.21.15.112:0/2524446029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f14e006b270 con 0x7f14e01083a0 2024-09-21T03:26:09.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.854+0000 7f14cf7fe640 1 -- 172.21.15.112:0/2524446029 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v65) ==== 74+0+133 (secure 0 0 0) 0x7f14e006b270 con 0x7f14e01083a0 2024-09-21T03:26:09.856 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 -- 172.21.15.112:0/2524446029 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14c80b6590 msgr2=0x7f14c80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14c80b6590 0x7f14c80b8aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f14d4000c00 tx=0x7f14d4031400 comp rx=0 tx=0).stop 2024-09-21T03:26:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 -- 172.21.15.112:0/2524446029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e01083a0 msgr2=0x7f14e019b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e01083a0 0x7f14e019b790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f14d0005fd0 tx=0x7f14d000dde0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 -- 172.21.15.112:0/2524446029 shutdown_connections 2024-09-21T03:26:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14c80b6590 0x7f14c80b8aa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14e0108cf0 0x7f14e019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14e01083a0 0x7f14e019b790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 --2- 172.21.15.112:0/2524446029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14e0102a60 0x7f14e019b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 -- 172.21.15.112:0/2524446029 >> 172.21.15.112:0/2524446029 conn(0x7f14e0079730 msgr2=0x7f14e0101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 -- 172.21.15.112:0/2524446029 shutdown_connections 2024-09-21T03:26:09.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:09.856+0000 7f14e4f24640 1 -- 172.21.15.112:0/2524446029 wait complete. 2024-09-21T03:26:10.094 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:09 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2524446029' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:26:10.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:09 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2524446029' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:26:10.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:09 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2524446029' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:26:10.565 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":65,"num_osds":12,"num_up_osds":11,"osd_up_since":1726889148,"num_in_osds":12,"osd_in_since":1726889150,"num_remapped_pgs":0} 2024-09-21T03:26:11.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:10 smithi203 ceph-mon[34723]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:11.099 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:10 smithi203 ceph-mon[34723]: from='osd.11 [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:26:11.100 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:10 smithi203 ceph-mon[34723]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:26:11.100 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:26:10 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:26:10.670+0000 7f7036e51640 -1 osd.11 0 log_to_monitors true 2024-09-21T03:26:11.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:10 smithi187 ceph-mon[34603]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:11.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:10 smithi187 ceph-mon[34603]: from='osd.11 [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:26:11.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:10 smithi187 ceph-mon[34603]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:26:11.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:10 smithi112 ceph-mon[31442]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:11.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:10 smithi112 ceph-mon[31442]: from='osd.11 [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:26:11.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:10 smithi112 ceph-mon[31442]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T03:26:11.566 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:26:11.829 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='osd.11 [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:12.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:12.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:26:12.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='osd.11 [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:12.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T03:26:12.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='osd.11 [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:26:12.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T03:26:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]} : dispatch 2024-09-21T03:26:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:26:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:12.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:12.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:12.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.612+0000 7f6e7096d640 1 -- 172.21.15.112:0/3773259118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c103b60 msgr2=0x7f6e6c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:12.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.612+0000 7f6e7096d640 1 --2- 172.21.15.112:0/3773259118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c103b60 0x7f6e6c103f70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009900 tx=0x7f6e5c02a410 comp rx=0 tx=0).stop 2024-09-21T03:26:12.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.612+0000 7f6e7096d640 1 -- 172.21.15.112:0/3773259118 shutdown_connections 2024-09-21T03:26:12.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.612+0000 7f6e7096d640 1 --2- 172.21.15.112:0/3773259118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e6c109df0 0x7f6e6c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:12.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.612+0000 7f6e7096d640 1 --2- 172.21.15.112:0/3773259118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e6c1094a0 0x7f6e6c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.612+0000 7f6e7096d640 1 --2- 172.21.15.112:0/3773259118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c103b60 0x7f6e6c103f70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.612+0000 7f6e7096d640 1 -- 172.21.15.112:0/3773259118 >> 172.21.15.112:0/3773259118 conn(0x7f6e6c07b850 msgr2=0x7f6e6c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.612+0000 7f6e7096d640 1 -- 172.21.15.112:0/3773259118 shutdown_connections 2024-09-21T03:26:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.612+0000 7f6e7096d640 1 -- 172.21.15.112:0/3773259118 wait complete. 2024-09-21T03:26:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e7096d640 1 Processor -- start 2024-09-21T03:26:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e7096d640 1 -- start start 2024-09-21T03:26:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e7096d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c103b60 0x7f6e6c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e7096d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e6c1094a0 0x7f6e6c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e7096d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e6c109df0 0x7f6e6c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e7096d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c113ab0 con 0x7f6e6c103b60 2024-09-21T03:26:12.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e7096d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c113930 con 0x7f6e6c1094a0 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e7096d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c113c30 con 0x7f6e6c109df0 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e6a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c103b60 0x7f6e6c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e6a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c103b60 0x7f6e6c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36756/0 (socket says 172.21.15.112:36756) 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e6a59c640 1 -- 172.21.15.112:0/161062001 learned_addr learned my addr 172.21.15.112:0/161062001 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e6ad9d640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e6c109df0 0x7f6e6c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e69d9b640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e6c1094a0 0x7f6e6c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e6a59c640 1 -- 172.21.15.112:0/161062001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e6c109df0 msgr2=0x7f6e6c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e6a59c640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e6c109df0 0x7f6e6c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e6a59c640 1 -- 172.21.15.112:0/161062001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e6c1094a0 msgr2=0x7f6e6c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e6a59c640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e6c1094a0 0x7f6e6c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.613+0000 7f6e6a59c640 1 -- 172.21.15.112:0/161062001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c1a03f0 con 0x7f6e6c103b60 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.614+0000 7f6e6a59c640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c103b60 0x7f6e6c19a480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c005940 tx=0x7f6e5c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:12.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.614+0000 7f6e5b7fe640 1 -- 172.21.15.112:0/161062001 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c03f030 con 0x7f6e6c103b60 2024-09-21T03:26:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.614+0000 7f6e7096d640 1 -- 172.21.15.112:0/161062001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c1a0680 con 0x7f6e6c103b60 2024-09-21T03:26:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.614+0000 7f6e5b7fe640 1 -- 172.21.15.112:0/161062001 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e5c0044c0 con 0x7f6e6c103b60 2024-09-21T03:26:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.614+0000 7f6e5b7fe640 1 -- 172.21.15.112:0/161062001 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c035070 con 0x7f6e6c103b60 2024-09-21T03:26:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.614+0000 7f6e7096d640 1 -- 172.21.15.112:0/161062001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c1a6e40 con 0x7f6e6c103b60 2024-09-21T03:26:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.615+0000 7f6e7096d640 1 -- 172.21.15.112:0/161062001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c104de0 con 0x7f6e6c103b60 2024-09-21T03:26:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.616+0000 7f6e5b7fe640 1 -- 172.21.15.112:0/161062001 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f6e5c042030 con 0x7f6e6c103b60 2024-09-21T03:26:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.617+0000 7f6e5b7fe640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e540b6560 0x7f6e540b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.617+0000 7f6e69d9b640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e540b6560 0x7f6e540b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.617+0000 7f6e5b7fe640 1 -- 172.21.15.112:0/161062001 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 7062+0+0 (secure 0 0 0) 0x7f6e5c107270 con 0x7f6e6c103b60 2024-09-21T03:26:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.618+0000 7f6e69d9b640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e540b6560 0x7f6e540b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c106450 tx=0x7f6e60005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:12.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.622+0000 7f6e5b7fe640 1 -- 172.21.15.112:0/161062001 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e5c0cdfe0 con 0x7f6e6c103b60 2024-09-21T03:26:12.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.800+0000 7f6e7096d640 1 -- 172.21.15.112:0/161062001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6e6c10bbc0 con 0x7f6e6c103b60 2024-09-21T03:26:12.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.800+0000 7f6e5b7fe640 1 -- 172.21.15.112:0/161062001 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v67) ==== 74+0+133 (secure 0 0 0) 0x7f6e5c0d23c0 con 0x7f6e6c103b60 2024-09-21T03:26:12.802 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 -- 172.21.15.112:0/161062001 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e540b6560 msgr2=0x7f6e540b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e540b6560 0x7f6e540b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c106450 tx=0x7f6e60005f90 comp rx=0 tx=0).stop 2024-09-21T03:26:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 -- 172.21.15.112:0/161062001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c103b60 msgr2=0x7f6e6c19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c103b60 0x7f6e6c19a480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c005940 tx=0x7f6e5c0040c0 comp rx=0 tx=0).stop 2024-09-21T03:26:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 -- 172.21.15.112:0/161062001 shutdown_connections 2024-09-21T03:26:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e540b6560 0x7f6e540b8a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e6c109df0 0x7f6e6c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e6c1094a0 0x7f6e6c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 --2- 172.21.15.112:0/161062001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e6c103b60 0x7f6e6c19a480 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 -- 172.21.15.112:0/161062001 >> 172.21.15.112:0/161062001 conn(0x7f6e6c07b850 msgr2=0x7f6e6c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:12.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 -- 172.21.15.112:0/161062001 shutdown_connections 2024-09-21T03:26:12.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:12.802+0000 7f6e7096d640 1 -- 172.21.15.112:0/161062001 wait complete. 2024-09-21T03:26:13.161 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:26:12 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:26:12.823+0000 7f7032dd3640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T03:26:13.161 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:26:12 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:26:12.974+0000 7f702abc9640 -1 osd.11 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:26:13.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:12 smithi203 ceph-mon[34723]: Detected new or changed devices on smithi203 2024-09-21T03:26:13.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:12 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi203 to 4272M 2024-09-21T03:26:13.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:12 smithi203 ceph-mon[34723]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:26:13.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:12 smithi203 ceph-mon[34723]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:26:13.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:12 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:13.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:12 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:13.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:12 smithi203 ceph-mon[34723]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:13.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:12 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/161062001' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:26:13.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:12 smithi203 ceph-mon[34723]: from='osd.11 ' entity='osd.11' 2024-09-21T03:26:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:12 smithi112 ceph-mon[31442]: Detected new or changed devices on smithi203 2024-09-21T03:26:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:12 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi203 to 4272M 2024-09-21T03:26:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:12 smithi112 ceph-mon[31442]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:26:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:12 smithi112 ceph-mon[31442]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:26:13.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:12 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:13.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:12 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:13.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:12 smithi112 ceph-mon[31442]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:13.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:12 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/161062001' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:26:13.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:12 smithi112 ceph-mon[31442]: from='osd.11 ' entity='osd.11' 2024-09-21T03:26:13.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:12 smithi187 ceph-mon[34603]: Detected new or changed devices on smithi203 2024-09-21T03:26:13.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:12 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi203 to 4272M 2024-09-21T03:26:13.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:12 smithi187 ceph-mon[34603]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-21T03:26:13.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:12 smithi187 ceph-mon[34603]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T03:26:13.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:12 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:13.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:12 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:13.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:12 smithi187 ceph-mon[34603]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:13.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:12 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/161062001' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:26:13.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:12 smithi187 ceph-mon[34603]: from='osd.11 ' entity='osd.11' 2024-09-21T03:26:13.590 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":67,"num_osds":12,"num_up_osds":11,"osd_up_since":1726889148,"num_in_osds":12,"osd_in_since":1726889150,"num_remapped_pgs":0} 2024-09-21T03:26:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:13 smithi112 ceph-mon[31442]: purged_snaps scrub starts 2024-09-21T03:26:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:13 smithi112 ceph-mon[31442]: purged_snaps scrub ok 2024-09-21T03:26:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:14.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:13 smithi203 ceph-mon[34723]: purged_snaps scrub starts 2024-09-21T03:26:14.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:13 smithi203 ceph-mon[34723]: purged_snaps scrub ok 2024-09-21T03:26:14.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:14.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:14.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:13 smithi187 ceph-mon[34603]: purged_snaps scrub starts 2024-09-21T03:26:14.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:13 smithi187 ceph-mon[34603]: purged_snaps scrub ok 2024-09-21T03:26:14.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:14.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:14.592 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T03:26:14.850 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:15.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:14 smithi112 ceph-mon[31442]: osd.11 [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655] boot 2024-09-21T03:26:15.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:14 smithi112 ceph-mon[31442]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T03:26:15.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:14 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:15.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:14 smithi112 ceph-mon[31442]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.293+0000 7f01846fb640 1 -- 172.21.15.112:0/1871520408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c109df0 msgr2=0x7f017c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.293+0000 7f01846fb640 1 --2- 172.21.15.112:0/1871520408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c109df0 0x7f017c110620 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0174009ca0 tx=0x7f017402a8e0 comp rx=0 tx=0).stop 2024-09-21T03:26:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.293+0000 7f01846fb640 1 -- 172.21.15.112:0/1871520408 shutdown_connections 2024-09-21T03:26:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.293+0000 7f01846fb640 1 --2- 172.21.15.112:0/1871520408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c109df0 0x7f017c110620 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.293+0000 7f01846fb640 1 --2- 172.21.15.112:0/1871520408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f017c1094a0 0x7f017c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.293+0000 7f01846fb640 1 --2- 172.21.15.112:0/1871520408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f017c103b60 0x7f017c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.293+0000 7f01846fb640 1 -- 172.21.15.112:0/1871520408 >> 172.21.15.112:0/1871520408 conn(0x7f017c07b850 msgr2=0x7f017c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.293+0000 7f01846fb640 1 -- 172.21.15.112:0/1871520408 shutdown_connections 2024-09-21T03:26:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.293+0000 7f01846fb640 1 -- 172.21.15.112:0/1871520408 wait complete. 2024-09-21T03:26:15.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.294+0000 7f01846fb640 1 Processor -- start 2024-09-21T03:26:15.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.294+0000 7f01846fb640 1 -- start start 2024-09-21T03:26:15.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.294+0000 7f01846fb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f017c103b60 0x7f017c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:15.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.294+0000 7f01846fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c1094a0 0x7f017c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:15.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.294+0000 7f01846fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f017c109df0 0x7f017c19fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:15.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.294+0000 7f01846fb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f017c113b70 con 0x7f017c1094a0 2024-09-21T03:26:15.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.294+0000 7f01846fb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f017c1139f0 con 0x7f017c109df0 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.294+0000 7f01846fb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f017c113cf0 con 0x7f017c103b60 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.294+0000 7f0181c96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c1094a0 0x7f017c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0181c96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c1094a0 0x7f017c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45594/0 (socket says 172.21.15.112:45594) 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0181c96640 1 -- 172.21.15.112:0/2798054226 learned_addr learned my addr 172.21.15.112:0/2798054226 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0182497640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f017c103b60 0x7f017c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0182c98640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f017c109df0 0x7f017c19fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0181c96640 1 -- 172.21.15.112:0/2798054226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f017c103b60 msgr2=0x7f017c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0181c96640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f017c103b60 0x7f017c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0181c96640 1 -- 172.21.15.112:0/2798054226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f017c109df0 msgr2=0x7f017c19fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0181c96640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f017c109df0 0x7f017c19fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0181c96640 1 -- 172.21.15.112:0/2798054226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c1a0610 con 0x7f017c1094a0 2024-09-21T03:26:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0182497640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f017c103b60 0x7f017c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0182c98640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f017c109df0 0x7f017c19fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:26:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f0181c96640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c1094a0 0x7f017c19ac40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0178006fd0 tx=0x7f017800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f016f7fe640 1 -- 172.21.15.112:0/2798054226 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178011830 con 0x7f017c1094a0 2024-09-21T03:26:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f01846fb640 1 -- 172.21.15.112:0/2798054226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c1a0900 con 0x7f017c1094a0 2024-09-21T03:26:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f016f7fe640 1 -- 172.21.15.112:0/2798054226 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0178016070 con 0x7f017c1094a0 2024-09-21T03:26:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f016f7fe640 1 -- 172.21.15.112:0/2798054226 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178010e00 con 0x7f017c1094a0 2024-09-21T03:26:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.295+0000 7f01846fb640 1 -- 172.21.15.112:0/2798054226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c1a7060 con 0x7f017c1094a0 2024-09-21T03:26:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.296+0000 7f01846fb640 1 -- 172.21.15.112:0/2798054226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017c103ff0 con 0x7f017c1094a0 2024-09-21T03:26:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.298+0000 7f016f7fe640 1 -- 172.21.15.112:0/2798054226 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f01780119d0 con 0x7f017c1094a0 2024-09-21T03:26:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.299+0000 7f016f7fe640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f01680b6560 0x7f01680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.299+0000 7f0182497640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f01680b6560 0x7f01680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.299+0000 7f016f7fe640 1 -- 172.21.15.112:0/2798054226 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f01780e6c50 con 0x7f017c1094a0 2024-09-21T03:26:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.299+0000 7f0182497640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f01680b6560 0x7f01680b8a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0170009930 tx=0x7f0170008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.302+0000 7f016f7fe640 1 -- 172.21.15.112:0/2798054226 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01780ad8b0 con 0x7f017c1094a0 2024-09-21T03:26:15.312 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:14 smithi203 ceph-mon[34723]: osd.11 [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655] boot 2024-09-21T03:26:15.312 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:14 smithi203 ceph-mon[34723]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T03:26:15.312 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:14 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:15.312 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:14 smithi203 ceph-mon[34723]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:15.330 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:14 smithi187 ceph-mon[34603]: osd.11 [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655] boot 2024-09-21T03:26:15.330 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:14 smithi187 ceph-mon[34603]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T03:26:15.330 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:14 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T03:26:15.330 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:14 smithi187 ceph-mon[34603]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T03:26:15.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.482+0000 7f01846fb640 1 -- 172.21.15.112:0/2798054226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f017c10bc20 con 0x7f017c1094a0 2024-09-21T03:26:15.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.482+0000 7f016f7fe640 1 -- 172.21.15.112:0/2798054226 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7f01780b1c90 con 0x7f017c1094a0 2024-09-21T03:26:15.485 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:15.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.484+0000 7f01846fb640 1 -- 172.21.15.112:0/2798054226 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f01680b6560 msgr2=0x7f01680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:15.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.484+0000 7f01846fb640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f01680b6560 0x7f01680b8a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0170009930 tx=0x7f0170008040 comp rx=0 tx=0).stop 2024-09-21T03:26:15.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.484+0000 7f01846fb640 1 -- 172.21.15.112:0/2798054226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c1094a0 msgr2=0x7f017c19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.484+0000 7f01846fb640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c1094a0 0x7f017c19ac40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0178006fd0 tx=0x7f017800b8e0 comp rx=0 tx=0).stop 2024-09-21T03:26:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.484+0000 7f01846fb640 1 -- 172.21.15.112:0/2798054226 shutdown_connections 2024-09-21T03:26:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.484+0000 7f01846fb640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f01680b6560 0x7f01680b8a70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.484+0000 7f01846fb640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f017c109df0 0x7f017c19fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.484+0000 7f01846fb640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f017c1094a0 0x7f017c19ac40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.484+0000 7f01846fb640 1 --2- 172.21.15.112:0/2798054226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f017c103b60 0x7f017c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.484+0000 7f01846fb640 1 -- 172.21.15.112:0/2798054226 >> 172.21.15.112:0/2798054226 conn(0x7f017c07b850 msgr2=0x7f017c107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.485+0000 7f01846fb640 1 -- 172.21.15.112:0/2798054226 shutdown_connections 2024-09-21T03:26:15.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:15.485+0000 7f01846fb640 1 -- 172.21.15.112:0/2798054226 wait complete. 2024-09-21T03:26:16.177 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:15 smithi112 ceph-mon[31442]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:26:16.177 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:15 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2798054226' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:26:16.207 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":69,"num_osds":12,"num_up_osds":12,"osd_up_since":1726889173,"num_in_osds":12,"osd_in_since":1726889150,"num_remapped_pgs":0} 2024-09-21T03:26:16.207 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:26:16.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:15 smithi203 ceph-mon[34723]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:26:16.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:15 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2798054226' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:26:16.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:15 smithi187 ceph-mon[34603]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T03:26:16.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:15 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2798054226' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T03:26:16.453 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.883+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1273745249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f981083e0 msgr2=0x7f5f981087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.883+0000 7f5f9d8de640 1 --2- 172.21.15.112:0/1273745249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f981083e0 0x7f5f981087f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009ca0 tx=0x7f5f8c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.883+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1273745249 shutdown_connections 2024-09-21T03:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.883+0000 7f5f9d8de640 1 --2- 172.21.15.112:0/1273745249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f98108d30 0x7f5f9810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.883+0000 7f5f9d8de640 1 --2- 172.21.15.112:0/1273745249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f981083e0 0x7f5f981087f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.883+0000 7f5f9d8de640 1 --2- 172.21.15.112:0/1273745249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f98102aa0 0x7f5f98102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.883+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1273745249 >> 172.21.15.112:0/1273745249 conn(0x7f5f98079750 msgr2=0x7f5f98079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.883+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1273745249 shutdown_connections 2024-09-21T03:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.883+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1273745249 wait complete. 2024-09-21T03:26:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.883+0000 7f5f9d8de640 1 Processor -- start 2024-09-21T03:26:16.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f9d8de640 1 -- start start 2024-09-21T03:26:16.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f9d8de640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f98102aa0 0x7f5f98110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f9d8de640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f981083e0 0x7f5f98110580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f9d8de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98108d30 0x7f5f98110ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f9d8de640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98113b10 con 0x7f5f98108d30 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f9d8de640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98113990 con 0x7f5f981083e0 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f9d8de640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98113c90 con 0x7f5f98102aa0 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98108d30 0x7f5f98110ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98108d30 0x7f5f98110ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45612/0 (socket says 172.21.15.112:45612) 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f977fe640 1 -- 172.21.15.112:0/1999217077 learned_addr learned my addr 172.21.15.112:0/1999217077 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f96ffd640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f98102aa0 0x7f5f98110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f967fc640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f981083e0 0x7f5f98110580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f977fe640 1 -- 172.21.15.112:0/1999217077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f98102aa0 msgr2=0x7f5f98110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f977fe640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f98102aa0 0x7f5f98110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f977fe640 1 -- 172.21.15.112:0/1999217077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f981083e0 msgr2=0x7f5f98110580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f977fe640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f981083e0 0x7f5f98110580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:16.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f977fe640 1 -- 172.21.15.112:0/1999217077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f981a8780 con 0x7f5f98108d30 2024-09-21T03:26:16.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.884+0000 7f5f977fe640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98108d30 0x7f5f98110ac0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000d750 tx=0x7f5f9000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:16.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.885+0000 7f5f9c8dc640 1 -- 172.21.15.112:0/1999217077 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90017070 con 0x7f5f98108d30 2024-09-21T03:26:16.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.885+0000 7f5f9c8dc640 1 -- 172.21.15.112:0/1999217077 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f900054d0 con 0x7f5f98108d30 2024-09-21T03:26:16.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.885+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1999217077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f981a8a70 con 0x7f5f98108d30 2024-09-21T03:26:16.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.885+0000 7f5f9c8dc640 1 -- 172.21.15.112:0/1999217077 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90004c60 con 0x7f5f98108d30 2024-09-21T03:26:16.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.885+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1999217077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f981a8fd0 con 0x7f5f98108d30 2024-09-21T03:26:16.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.886+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1999217077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f98102eb0 con 0x7f5f98108d30 2024-09-21T03:26:16.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.887+0000 7f5f9c8dc640 1 -- 172.21.15.112:0/1999217077 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f5f90009bd0 con 0x7f5f98108d30 2024-09-21T03:26:16.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.888+0000 7f5f9c8dc640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5f800b65e0 0x7f5f800b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:16.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.888+0000 7f5f96ffd640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5f800b65e0 0x7f5f800b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:16.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.888+0000 7f5f9c8dc640 1 -- 172.21.15.112:0/1999217077 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f5f900e6ce0 con 0x7f5f98108d30 2024-09-21T03:26:16.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.888+0000 7f5f96ffd640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5f800b65e0 0x7f5f800b8af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5f98111a90 tx=0x7f5f8800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:16.891+0000 7f5f9c8dc640 1 -- 172.21.15.112:0/1999217077 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5f900ad9c0 con 0x7f5f98108d30 2024-09-21T03:26:17.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.072+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1999217077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f9802e640 con 0x7f5f98108d30 2024-09-21T03:26:17.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.073+0000 7f5f9c8dc640 1 -- 172.21.15.112:0/1999217077 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+18821 (secure 0 0 0) 0x7f5f900b1da0 con 0x7f5f98108d30 2024-09-21T03:26:17.076 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:17.076 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":69,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:26:14.941225+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:26:17.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1999217077 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5f800b65e0 msgr2=0x7f5f800b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:17.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5f800b65e0 0x7f5f800b8af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5f98111a90 tx=0x7f5f8800b390 comp rx=0 tx=0).stop 2024-09-21T03:26:17.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1999217077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98108d30 msgr2=0x7f5f98110ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:17.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98108d30 0x7f5f98110ac0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000d750 tx=0x7f5f9000dc10 comp rx=0 tx=0).stop 2024-09-21T03:26:17.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1999217077 shutdown_connections 2024-09-21T03:26:17.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5f800b65e0 0x7f5f800b8af0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:17.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98108d30 0x7f5f98110ac0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:17.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f981083e0 0x7f5f98110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:17.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 --2- 172.21.15.112:0/1999217077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f98102aa0 0x7f5f98110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:17.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1999217077 >> 172.21.15.112:0/1999217077 conn(0x7f5f98079750 msgr2=0x7f5f98106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:17.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1999217077 shutdown_connections 2024-09-21T03:26:17.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:17.075+0000 7f5f9d8de640 1 -- 172.21.15.112:0/1999217077 wait complete. 2024-09-21T03:26:17.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:16 smithi112 ceph-mon[31442]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:17.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:16 smithi203 ceph-mon[34723]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:17.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:16 smithi187 ceph-mon[34603]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:17.783 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:22:30.533177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:26:17.784 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T03:26:18.029 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:18.265 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:17 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1999217077' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:26:18.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:17 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1999217077' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:26:18.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:17 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1999217077' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:26:18.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.469+0000 7fa322dbd640 1 -- 172.21.15.112:0/1029796438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c103b60 msgr2=0x7fa31c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:18.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.469+0000 7fa322dbd640 1 --2- 172.21.15.112:0/1029796438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c103b60 0x7fa31c103f70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa310009900 tx=0x7fa31002a410 comp rx=0 tx=0).stop 2024-09-21T03:26:18.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.469+0000 7fa322dbd640 1 -- 172.21.15.112:0/1029796438 shutdown_connections 2024-09-21T03:26:18.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.469+0000 7fa322dbd640 1 --2- 172.21.15.112:0/1029796438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa31c109df0 0x7fa31c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:18.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.469+0000 7fa322dbd640 1 --2- 172.21.15.112:0/1029796438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c1094a0 0x7fa31c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:18.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.469+0000 7fa322dbd640 1 --2- 172.21.15.112:0/1029796438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c103b60 0x7fa31c103f70 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa310009900 tx=0x7fa31002a410 comp rx=0 tx=0).stop 2024-09-21T03:26:18.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.469+0000 7fa322dbd640 1 -- 172.21.15.112:0/1029796438 >> 172.21.15.112:0/1029796438 conn(0x7fa31c07b850 msgr2=0x7fa31c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:18.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.469+0000 7fa322dbd640 1 -- 172.21.15.112:0/1029796438 shutdown_connections 2024-09-21T03:26:18.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.469+0000 7fa322dbd640 1 -- 172.21.15.112:0/1029796438 wait complete. 2024-09-21T03:26:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.469+0000 7fa322dbd640 1 Processor -- start 2024-09-21T03:26:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa322dbd640 1 -- start start 2024-09-21T03:26:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa322dbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c1094a0 0x7fa31c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa322dbd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa31c109df0 0x7fa31c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa322dbd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c19fd60 0x7fa31c1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa322dbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c113ab0 con 0x7fa31c1094a0 2024-09-21T03:26:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa322dbd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c113930 con 0x7fa31c109df0 2024-09-21T03:26:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa322dbd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c113c30 con 0x7fa31c19fd60 2024-09-21T03:26:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa320b59640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c1094a0 0x7fa31c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa320b59640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c1094a0 0x7fa31c19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45622/0 (socket says 172.21.15.112:45622) 2024-09-21T03:26:18.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa320b59640 1 -- 172.21.15.112:0/3381604363 learned_addr learned my addr 172.21.15.112:0/3381604363 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:18.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa31bfff640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa31c109df0 0x7fa31c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:18.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa32135a640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c19fd60 0x7fa31c1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:18.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa320b59640 1 -- 172.21.15.112:0/3381604363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c19fd60 msgr2=0x7fa31c1a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:18.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa320b59640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c19fd60 0x7fa31c1a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:18.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa320b59640 1 -- 172.21.15.112:0/3381604363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa31c109df0 msgr2=0x7fa31c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:18.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa320b59640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa31c109df0 0x7fa31c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:18.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.470+0000 7fa320b59640 1 -- 172.21.15.112:0/3381604363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c1a0940 con 0x7fa31c1094a0 2024-09-21T03:26:18.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.471+0000 7fa320b59640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c1094a0 0x7fa31c19a5e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa310006010 tx=0x7fa3100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:18.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.471+0000 7fa319ffb640 1 -- 172.21.15.112:0/3381604363 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31003f030 con 0x7fa31c1094a0 2024-09-21T03:26:18.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.471+0000 7fa319ffb640 1 -- 172.21.15.112:0/3381604363 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa310003cb0 con 0x7fa31c1094a0 2024-09-21T03:26:18.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.471+0000 7fa322dbd640 1 -- 172.21.15.112:0/3381604363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c1a6ee0 con 0x7fa31c1094a0 2024-09-21T03:26:18.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.471+0000 7fa319ffb640 1 -- 172.21.15.112:0/3381604363 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310035070 con 0x7fa31c1094a0 2024-09-21T03:26:18.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.471+0000 7fa322dbd640 1 -- 172.21.15.112:0/3381604363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c1a7280 con 0x7fa31c1094a0 2024-09-21T03:26:18.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.472+0000 7fa322dbd640 1 -- 172.21.15.112:0/3381604363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa31c103f70 con 0x7fa31c1094a0 2024-09-21T03:26:18.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.474+0000 7fa319ffb640 1 -- 172.21.15.112:0/3381604363 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fa310042030 con 0x7fa31c1094a0 2024-09-21T03:26:18.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.474+0000 7fa319ffb640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa30c0b6560 0x7fa30c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:18.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.474+0000 7fa31bfff640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa30c0b6560 0x7fa30c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:18.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.474+0000 7fa319ffb640 1 -- 172.21.15.112:0/3381604363 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7fa310107a80 con 0x7fa31c1094a0 2024-09-21T03:26:18.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.475+0000 7fa31bfff640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa30c0b6560 0x7fa30c0b8a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa31c1064c0 tx=0x7fa30800a250 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:18.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.477+0000 7fa319ffb640 1 -- 172.21.15.112:0/3381604363 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa310049020 con 0x7fa31c1094a0 2024-09-21T03:26:18.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.655+0000 7fa322dbd640 1 -- 172.21.15.112:0/3381604363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa31c02e640 con 0x7fa31c1094a0 2024-09-21T03:26:18.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.656+0000 7fa319ffb640 1 -- 172.21.15.112:0/3381604363 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7fa3100d5e20 con 0x7fa31c1094a0 2024-09-21T03:26:18.660 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-09-21T03:26:18.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.658+0000 7fa322dbd640 1 -- 172.21.15.112:0/3381604363 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa30c0b6560 msgr2=0x7fa30c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:18.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.658+0000 7fa322dbd640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa30c0b6560 0x7fa30c0b8a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa31c1064c0 tx=0x7fa30800a250 comp rx=0 tx=0).stop 2024-09-21T03:26:18.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.658+0000 7fa322dbd640 1 -- 172.21.15.112:0/3381604363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c1094a0 msgr2=0x7fa31c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:18.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.658+0000 7fa322dbd640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c1094a0 0x7fa31c19a5e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa310006010 tx=0x7fa3100042f0 comp rx=0 tx=0).stop 2024-09-21T03:26:18.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.659+0000 7fa322dbd640 1 -- 172.21.15.112:0/3381604363 shutdown_connections 2024-09-21T03:26:18.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.659+0000 7fa322dbd640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa30c0b6560 0x7fa30c0b8a70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:18.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.659+0000 7fa322dbd640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c19fd60 0x7fa31c1a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:18.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.659+0000 7fa322dbd640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa31c109df0 0x7fa31c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:18.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.659+0000 7fa322dbd640 1 --2- 172.21.15.112:0/3381604363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa31c1094a0 0x7fa31c19a5e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:18.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.659+0000 7fa322dbd640 1 -- 172.21.15.112:0/3381604363 >> 172.21.15.112:0/3381604363 conn(0x7fa31c07b850 msgr2=0x7fa31c1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:18.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.659+0000 7fa322dbd640 1 -- 172.21.15.112:0/3381604363 shutdown_connections 2024-09-21T03:26:18.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:18.659+0000 7fa322dbd640 1 -- 172.21.15.112:0/3381604363 wait complete. 2024-09-21T03:26:19.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:18 smithi112 ceph-mon[31442]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:19.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:18 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3381604363' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:26:19.279 INFO:tasks.cephadm:Adding mds.a on smithi112 2024-09-21T03:26:19.279 INFO:tasks.cephadm:Adding mds.d on smithi112 2024-09-21T03:26:19.279 INFO:tasks.cephadm:Adding mds.g on smithi112 2024-09-21T03:26:19.279 INFO:tasks.cephadm:Adding mds.j on smithi112 2024-09-21T03:26:19.280 INFO:tasks.cephadm:Adding mds.b on smithi187 2024-09-21T03:26:19.280 INFO:tasks.cephadm:Adding mds.e on smithi187 2024-09-21T03:26:19.280 INFO:tasks.cephadm:Adding mds.h on smithi187 2024-09-21T03:26:19.280 INFO:tasks.cephadm:Adding mds.k on smithi187 2024-09-21T03:26:19.280 INFO:tasks.cephadm:Adding mds.c on smithi203 2024-09-21T03:26:19.280 INFO:tasks.cephadm:Adding mds.f on smithi203 2024-09-21T03:26:19.280 INFO:tasks.cephadm:Adding mds.i on smithi203 2024-09-21T03:26:19.280 INFO:tasks.cephadm:Adding mds.l on smithi203 2024-09-21T03:26:19.281 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l' 2024-09-21T03:26:19.310 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:18 smithi203 ceph-mon[34723]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:19.310 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:18 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3381604363' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:26:19.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:18 smithi187 ceph-mon[34603]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:19.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:18 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3381604363' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:26:19.541 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:26:20.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.015+0000 7f7c705a7640 1 -- 172.21.15.203:0/1532251519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c681094a0 msgr2=0x7f7c681098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:20.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.015+0000 7f7c705a7640 1 --2- 172.21.15.203:0/1532251519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c681094a0 0x7f7c681098b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009900 tx=0x7f7c5802a410 comp rx=0 tx=0).stop 2024-09-21T03:26:20.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.015+0000 7f7c705a7640 1 -- 172.21.15.203:0/1532251519 shutdown_connections 2024-09-21T03:26:20.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.015+0000 7f7c705a7640 1 --2- 172.21.15.203:0/1532251519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c68109df0 0x7f7c68110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:20.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.015+0000 7f7c705a7640 1 --2- 172.21.15.203:0/1532251519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c681094a0 0x7f7c681098b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:20.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.015+0000 7f7c705a7640 1 --2- 172.21.15.203:0/1532251519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c68103b60 0x7f7c68103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:20.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.015+0000 7f7c705a7640 1 -- 172.21.15.203:0/1532251519 >> 172.21.15.203:0/1532251519 conn(0x7f7c6807b850 msgr2=0x7f7c6807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:20.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.015+0000 7f7c705a7640 1 -- 172.21.15.203:0/1532251519 shutdown_connections 2024-09-21T03:26:20.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.015+0000 7f7c705a7640 1 -- 172.21.15.203:0/1532251519 wait complete. 2024-09-21T03:26:20.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.015+0000 7f7c705a7640 1 Processor -- start 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c705a7640 1 -- start start 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c705a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c68103b60 0x7f7c6819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c705a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c681094a0 0x7f7c6819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c705a7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68109df0 0x7f7c6819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c705a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68113ab0 con 0x7f7c681094a0 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c705a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68113930 con 0x7f7c68103b60 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c705a7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68113c30 con 0x7f7c68109df0 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6eb44640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68109df0 0x7f7c6819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6eb44640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68109df0 0x7f7c6819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:52526/0 (socket says 172.21.15.203:52526) 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6eb44640 1 -- 172.21.15.203:0/400749296 learned_addr learned my addr 172.21.15.203:0/400749296 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6db42640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c681094a0 0x7f7c6819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6e343640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c68103b60 0x7f7c6819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:20.057 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6eb44640 1 -- 172.21.15.203:0/400749296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c68103b60 msgr2=0x7f7c6819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6eb44640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c68103b60 0x7f7c6819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6eb44640 1 -- 172.21.15.203:0/400749296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c681094a0 msgr2=0x7f7c6819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6eb44640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c681094a0 0x7f7c6819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6eb44640 1 -- 172.21.15.203:0/400749296 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c681a0410 con 0x7f7c68109df0 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6e343640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c68103b60 0x7f7c6819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6db42640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c681094a0 0x7f7c6819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.016+0000 7f7c6eb44640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68109df0 0x7f7c6819fce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000eaa0 tx=0x7f7c6000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.017+0000 7f7c5f7fe640 1 -- 172.21.15.203:0/400749296 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000cdb0 con 0x7f7c68109df0 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.017+0000 7f7c5f7fe640 1 -- 172.21.15.203:0/400749296 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c60005020 con 0x7f7c68109df0 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.017+0000 7f7c705a7640 1 -- 172.21.15.203:0/400749296 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c681a0700 con 0x7f7c68109df0 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.017+0000 7f7c5f7fe640 1 -- 172.21.15.203:0/400749296 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c600116d0 con 0x7f7c68109df0 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.017+0000 7f7c705a7640 1 -- 172.21.15.203:0/400749296 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c681a6e60 con 0x7f7c68109df0 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.018+0000 7f7c705a7640 1 -- 172.21.15.203:0/400749296 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c68103f70 con 0x7f7c68109df0 2024-09-21T03:26:20.058 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.020+0000 7f7c5f7fe640 1 -- 172.21.15.203:0/400749296 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f7c600051d0 con 0x7f7c68109df0 2024-09-21T03:26:20.059 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.021+0000 7f7c5f7fe640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7c540b6560 0x7f7c540b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:20.059 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.021+0000 7f7c5f7fe640 1 -- 172.21.15.203:0/400749296 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f7c600e7590 con 0x7f7c68109df0 2024-09-21T03:26:20.059 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.021+0000 7f7c6e343640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7c540b6560 0x7f7c540b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:20.059 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.021+0000 7f7c6e343640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7c540b6560 0x7f7c540b8a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009b40 tx=0x7f7c64005e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:20.059 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.024+0000 7f7c5f7fe640 1 -- 172.21.15.203:0/400749296 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c60016030 con 0x7f7c68109df0 2024-09-21T03:26:20.211 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.210+0000 7f7c705a7640 1 -- 172.21.15.203:0/400749296 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}) -- 0x7f7c68063900 con 0x7f7c540b6560 2024-09-21T03:26:20.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.218+0000 7f7c5f7fe640 1 -- 172.21.15.203:0/400749296 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7c68063900 con 0x7f7c540b6560 2024-09-21T03:26:20.220 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mds.all update... 2024-09-21T03:26:20.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.220+0000 7f7c705a7640 1 -- 172.21.15.203:0/400749296 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7c540b6560 msgr2=0x7f7c540b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:20.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.220+0000 7f7c705a7640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7c540b6560 0x7f7c540b8a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009b40 tx=0x7f7c64005e50 comp rx=0 tx=0).stop 2024-09-21T03:26:20.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.220+0000 7f7c705a7640 1 -- 172.21.15.203:0/400749296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68109df0 msgr2=0x7f7c6819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:20.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.220+0000 7f7c705a7640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68109df0 0x7f7c6819fce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000eaa0 tx=0x7f7c6000ef60 comp rx=0 tx=0).stop 2024-09-21T03:26:20.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.221+0000 7f7c705a7640 1 -- 172.21.15.203:0/400749296 shutdown_connections 2024-09-21T03:26:20.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.221+0000 7f7c705a7640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7c540b6560 0x7f7c540b8a70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:20.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.221+0000 7f7c705a7640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c68109df0 0x7f7c6819fce0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:20.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.221+0000 7f7c705a7640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c681094a0 0x7f7c6819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:20.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.221+0000 7f7c705a7640 1 --2- 172.21.15.203:0/400749296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c68103b60 0x7f7c6819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:20.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.221+0000 7f7c705a7640 1 -- 172.21.15.203:0/400749296 >> 172.21.15.203:0/400749296 conn(0x7f7c6807b850 msgr2=0x7f7c68107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:20.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.221+0000 7f7c705a7640 1 -- 172.21.15.203:0/400749296 shutdown_connections 2024-09-21T03:26:20.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-21T03:26:20.221+0000 7f7c705a7640 1 -- 172.21.15.203:0/400749296 wait complete. 2024-09-21T03:26:20.835 DEBUG:teuthology.orchestra.run.smithi112:mds.a> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T03:26:20.837 DEBUG:teuthology.orchestra.run.smithi112:mds.d> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T03:26:20.880 DEBUG:teuthology.orchestra.run.smithi112:mds.g> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T03:26:20.924 DEBUG:teuthology.orchestra.run.smithi112:mds.j> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T03:26:20.966 DEBUG:teuthology.orchestra.run.smithi187:mds.b> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T03:26:20.969 DEBUG:teuthology.orchestra.run.smithi187:mds.e> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T03:26:21.012 DEBUG:teuthology.orchestra.run.smithi187:mds.h> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T03:26:21.055 DEBUG:teuthology.orchestra.run.smithi187:mds.k> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T03:26:21.098 DEBUG:teuthology.orchestra.run.smithi203:mds.c> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T03:26:21.101 DEBUG:teuthology.orchestra.run.smithi203:mds.f> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T03:26:21.143 DEBUG:teuthology.orchestra.run.smithi203:mds.i> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T03:26:21.186 DEBUG:teuthology.orchestra.run.smithi203:mds.l> sudo journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T03:26:21.229 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T03:26:21.229 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:26:21.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:21 smithi203 ceph-mon[34723]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:26:21.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:21 smithi203 ceph-mon[34723]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-21T03:26:21.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:21.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:21.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:21 smithi203 ceph-mon[34723]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:21.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:21.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:21.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:21.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:21.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:21.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:21.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:21 smithi187 ceph-mon[34603]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:26:21.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:21 smithi187 ceph-mon[34603]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-21T03:26:21.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:21.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:21.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:21 smithi187 ceph-mon[34603]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:21.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:21.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:21.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:21.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:21.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:21.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:21.471 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:21 smithi112 ceph-mon[31442]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:26:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:21 smithi112 ceph-mon[31442]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-21T03:26:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:21 smithi112 ceph-mon[31442]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:21.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:21.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.882+0000 7f5084649640 1 -- 172.21.15.112:0/5829291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507c109df0 msgr2=0x7f507c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:21.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.882+0000 7f5084649640 1 --2- 172.21.15.112:0/5829291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507c109df0 0x7f507c110620 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5074009900 tx=0x7f507402a410 comp rx=0 tx=0).stop 2024-09-21T03:26:21.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.882+0000 7f5084649640 1 -- 172.21.15.112:0/5829291 shutdown_connections 2024-09-21T03:26:21.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.882+0000 7f5084649640 1 --2- 172.21.15.112:0/5829291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507c109df0 0x7f507c110620 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:21.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.882+0000 7f5084649640 1 --2- 172.21.15.112:0/5829291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f507c1094a0 0x7f507c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.882+0000 7f5084649640 1 --2- 172.21.15.112:0/5829291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f507c103b60 0x7f507c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.882+0000 7f5084649640 1 -- 172.21.15.112:0/5829291 >> 172.21.15.112:0/5829291 conn(0x7f507c07b850 msgr2=0x7f507c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.882+0000 7f5084649640 1 -- 172.21.15.112:0/5829291 shutdown_connections 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.882+0000 7f5084649640 1 -- 172.21.15.112:0/5829291 wait complete. 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f5084649640 1 Processor -- start 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f5084649640 1 -- start start 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f5084649640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f507c103b60 0x7f507c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f5084649640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507c1094a0 0x7f507c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f5084649640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f507c109df0 0x7f507c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f5084649640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f507c113ab0 con 0x7f507c1094a0 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f5084649640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f507c113930 con 0x7f507c103b60 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f5084649640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f507c113c30 con 0x7f507c109df0 2024-09-21T03:26:21.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f5081be4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507c1094a0 0x7f507c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f5081be4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507c1094a0 0x7f507c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45638/0 (socket says 172.21.15.112:45638) 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f5081be4640 1 -- 172.21.15.112:0/3754167330 learned_addr learned my addr 172.21.15.112:0/3754167330 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.883+0000 7f50823e5640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f507c103b60 0x7f507c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.884+0000 7f5082be6640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f507c109df0 0x7f507c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.884+0000 7f5081be4640 1 -- 172.21.15.112:0/3754167330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f507c109df0 msgr2=0x7f507c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.884+0000 7f5081be4640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f507c109df0 0x7f507c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.884+0000 7f5081be4640 1 -- 172.21.15.112:0/3754167330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f507c103b60 msgr2=0x7f507c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.884+0000 7f5081be4640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f507c103b60 0x7f507c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.884+0000 7f5081be4640 1 -- 172.21.15.112:0/3754167330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507c1a03b0 con 0x7f507c1094a0 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.884+0000 7f5082be6640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f507c109df0 0x7f507c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.884+0000 7f5081be4640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507c1094a0 0x7f507c19a9e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f507000d740 tx=0x7f507000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.884+0000 7f507b7fe640 1 -- 172.21.15.112:0/3754167330 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070017070 con 0x7f507c1094a0 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.884+0000 7f5084649640 1 -- 172.21.15.112:0/3754167330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507c1a06a0 con 0x7f507c1094a0 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.885+0000 7f5084649640 1 -- 172.21.15.112:0/3754167330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507c1a6e00 con 0x7f507c1094a0 2024-09-21T03:26:21.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.885+0000 7f507b7fe640 1 -- 172.21.15.112:0/3754167330 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f507000de50 con 0x7f507c1094a0 2024-09-21T03:26:21.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.885+0000 7f507b7fe640 1 -- 172.21.15.112:0/3754167330 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507000f620 con 0x7f507c1094a0 2024-09-21T03:26:21.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.886+0000 7f5084649640 1 -- 172.21.15.112:0/3754167330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f507c103f70 con 0x7f507c1094a0 2024-09-21T03:26:21.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.888+0000 7f507b7fe640 1 -- 172.21.15.112:0/3754167330 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f507001a070 con 0x7f507c1094a0 2024-09-21T03:26:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.889+0000 7f507b7fe640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f50640b65e0 0x7f50640b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.889+0000 7f50823e5640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f50640b65e0 0x7f50640b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.889+0000 7f507b7fe640 1 -- 172.21.15.112:0/3754167330 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f50700e68b0 con 0x7f507c1094a0 2024-09-21T03:26:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.889+0000 7f50823e5640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f50640b65e0 0x7f50640b8af0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f506c005fd0 tx=0x7f506c009760 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:21.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:21.892+0000 7f507b7fe640 1 -- 172.21.15.112:0/3754167330 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50700ad570 con 0x7f507c1094a0 2024-09-21T03:26:22.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.105+0000 7f5084649640 1 -- 172.21.15.112:0/3754167330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f507c19b570 con 0x7f507c1094a0 2024-09-21T03:26:22.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.106+0000 7f507b7fe640 1 -- 172.21.15.112:0/3754167330 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f50700b1950 con 0x7f507c1094a0 2024-09-21T03:26:22.108 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:22.108 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":1,"btime":"2024-09-21T03:18:41:875043+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T03:26:22.108 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 1 2024-09-21T03:26:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 -- 172.21.15.112:0/3754167330 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f50640b65e0 msgr2=0x7f50640b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f50640b65e0 0x7f50640b8af0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f506c005fd0 tx=0x7f506c009760 comp rx=0 tx=0).stop 2024-09-21T03:26:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 -- 172.21.15.112:0/3754167330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507c1094a0 msgr2=0x7f507c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507c1094a0 0x7f507c19a9e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f507000d740 tx=0x7f507000dc00 comp rx=0 tx=0).stop 2024-09-21T03:26:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 -- 172.21.15.112:0/3754167330 shutdown_connections 2024-09-21T03:26:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f50640b65e0 0x7f50640b8af0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f507c109df0 0x7f507c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f507c1094a0 0x7f507c19a9e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 --2- 172.21.15.112:0/3754167330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f507c103b60 0x7f507c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 -- 172.21.15.112:0/3754167330 >> 172.21.15.112:0/3754167330 conn(0x7f507c07b850 msgr2=0x7f507c1074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 -- 172.21.15.112:0/3754167330 shutdown_connections 2024-09-21T03:26:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:22.108+0000 7f5084649640 1 -- 172.21.15.112:0/3754167330 wait complete. 2024-09-21T03:26:22.515 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:22 smithi203 ceph-mon[34723]: Deploying daemon mds.l on smithi203 2024-09-21T03:26:22.515 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:22 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3754167330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:22.515 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:26:22 smithi203 systemd[1]: Starting Ceph mds.l for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:26:22.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:22 smithi112 ceph-mon[31442]: Deploying daemon mds.l on smithi203 2024-09-21T03:26:22.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:22 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3754167330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:22.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:22 smithi187 ceph-mon[34603]: Deploying daemon mds.l on smithi203 2024-09-21T03:26:22.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:22 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3754167330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:22.866 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:26:22 smithi203 podman[63522]: 2024-09-21 03:26:22.630141717 +0000 UTC m=+0.016531036 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:26:22.866 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:26:22 smithi203 podman[63522]: 2024-09-21 03:26:22.744941264 +0000 UTC m=+0.131330572 container create 5e23b10be529d244764870c4fa452421c85589bab3b4497da9d0af2de52a6a11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-l, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T03:26:23.229 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:26:22 smithi203 podman[63522]: 2024-09-21 03:26:22.955731249 +0000 UTC m=+0.342120555 container init 5e23b10be529d244764870c4fa452421c85589bab3b4497da9d0af2de52a6a11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-l, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:26:23.230 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:26:22 smithi203 podman[63522]: 2024-09-21 03:26:22.958714811 +0000 UTC m=+0.345104117 container start 5e23b10be529d244764870c4fa452421c85589bab3b4497da9d0af2de52a6a11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T03:26:23.230 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:26:22 smithi203 ceph-mds[63541]: -- 172.21.15.203:0/2264930835 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55beb606d860 con 0x55beba215600 2024-09-21T03:26:23.230 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:26:22 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-l[63537]: starting mds.l at 2024-09-21T03:26:23.230 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:26:23 smithi203 bash[63522]: 5e23b10be529d244764870c4fa452421c85589bab3b4497da9d0af2de52a6a11 2024-09-21T03:26:23.230 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:26:23 smithi203 systemd[1]: Started Ceph mds.l for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:23.504 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:23 smithi187 ceph-mon[34603]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:23.504 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:23 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.203:6832/4285434211,v1:172.21.15.203:6833/4285434211] up:boot 2024-09-21T03:26:23.504 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:23 smithi187 ceph-mon[34603]: fsmap 1 up:standby 2024-09-21T03:26:23.504 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:26:23.504 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:23.504 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:23.505 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:23.505 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:23.505 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:23.505 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:23 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:23.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:23 smithi112 ceph-mon[31442]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:23.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:23 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.203:6832/4285434211,v1:172.21.15.203:6833/4285434211] up:boot 2024-09-21T03:26:23.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:23 smithi112 ceph-mon[31442]: fsmap 1 up:standby 2024-09-21T03:26:23.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:26:23.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:23.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:23.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:23.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:23.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:23.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:23 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:23 smithi203 ceph-mon[34723]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:23 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.203:6832/4285434211,v1:172.21.15.203:6833/4285434211] up:boot 2024-09-21T03:26:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:23 smithi203 ceph-mon[34723]: fsmap 1 up:standby 2024-09-21T03:26:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:26:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:23.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:23 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:24.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:24 smithi187 ceph-mon[34603]: Deploying daemon mds.k on smithi187 2024-09-21T03:26:24.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:24 smithi112 ceph-mon[31442]: Deploying daemon mds.k on smithi187 2024-09-21T03:26:24.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:24 smithi203 ceph-mon[34723]: Deploying daemon mds.k on smithi187 2024-09-21T03:26:24.861 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:26:24.873 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:26:24 smithi187 systemd[1]: Starting Ceph mds.k for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:26:25.119 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:25.179 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:26:25 smithi187 podman[62532]: 2024-09-21 03:26:24.984648284 +0000 UTC m=+0.015401676 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:26:25.179 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:26:25 smithi187 podman[62532]: 2024-09-21 03:26:25.105096262 +0000 UTC m=+0.135849694 container create 3fbc25157879db5c90ca2f170f9fd194719943a67d21a1d5509096add98e7980 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-k, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:26:25.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:25 smithi112 ceph-mon[31442]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:25.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.586+0000 7f8273c5a640 1 -- 172.21.15.112:0/2044090619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f826c1083a0 msgr2=0x7f826c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:25.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.586+0000 7f8273c5a640 1 --2- 172.21.15.112:0/2044090619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f826c1083a0 0x7f826c1087b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f825c009900 tx=0x7f825c02a410 comp rx=0 tx=0).stop 2024-09-21T03:26:25.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.586+0000 7f8273c5a640 1 -- 172.21.15.112:0/2044090619 shutdown_connections 2024-09-21T03:26:25.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.586+0000 7f8273c5a640 1 --2- 172.21.15.112:0/2044090619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f826c108cf0 0x7f826c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:25.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.586+0000 7f8273c5a640 1 --2- 172.21.15.112:0/2044090619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f826c1083a0 0x7f826c1087b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:25.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.586+0000 7f8273c5a640 1 --2- 172.21.15.112:0/2044090619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826c102a60 0x7f826c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:25.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.586+0000 7f8273c5a640 1 -- 172.21.15.112:0/2044090619 >> 172.21.15.112:0/2044090619 conn(0x7f826c079730 msgr2=0x7f826c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:25.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.586+0000 7f8273c5a640 1 -- 172.21.15.112:0/2044090619 shutdown_connections 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.586+0000 7f8273c5a640 1 -- 172.21.15.112:0/2044090619 wait complete. 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f8273c5a640 1 Processor -- start 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f8273c5a640 1 -- start start 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f8273c5a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f826c102a60 0x7f826c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f8273c5a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826c1083a0 0x7f826c19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f8273c5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f826c108cf0 0x7f826c19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f8273c5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f826c112960 con 0x7f826c108cf0 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f8273c5a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1127e0 con 0x7f826c102a60 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f8273c5a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f826c112ae0 con 0x7f826c1083a0 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f82721f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f826c108cf0 0x7f826c19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f82721f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f826c108cf0 0x7f826c19bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56832/0 (socket says 172.21.15.112:56832) 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f82719f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f826c102a60 0x7f826c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f82721f7640 1 -- 172.21.15.112:0/545738448 learned_addr learned my addr 172.21.15.112:0/545738448 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:25.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.587+0000 7f82711f5640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826c1083a0 0x7f826c19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f82721f7640 1 -- 172.21.15.112:0/545738448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826c1083a0 msgr2=0x7f826c19b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f82721f7640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826c1083a0 0x7f826c19b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f82721f7640 1 -- 172.21.15.112:0/545738448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f826c102a60 msgr2=0x7f826c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f82721f7640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f826c102a60 0x7f826c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f82721f7640 1 -- 172.21.15.112:0/545738448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c1a6580 con 0x7f826c108cf0 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f82711f5640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826c1083a0 0x7f826c19b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f82719f6640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f826c102a60 0x7f826c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f82721f7640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f826c108cf0 0x7f826c19bd70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f826400eaa0 tx=0x7f826400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f8262ffd640 1 -- 172.21.15.112:0/545738448 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826400cdb0 con 0x7f826c108cf0 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f8273c5a640 1 -- 172.21.15.112:0/545738448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c1a6870 con 0x7f826c108cf0 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f8262ffd640 1 -- 172.21.15.112:0/545738448 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8264005020 con 0x7f826c108cf0 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f8262ffd640 1 -- 172.21.15.112:0/545738448 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82640116d0 con 0x7f826c108cf0 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.588+0000 7f8273c5a640 1 -- 172.21.15.112:0/545738448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c1a6e00 con 0x7f826c108cf0 2024-09-21T03:26:25.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.589+0000 7f8273c5a640 1 -- 172.21.15.112:0/545738448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826c102e70 con 0x7f826c108cf0 2024-09-21T03:26:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.591+0000 7f8262ffd640 1 -- 172.21.15.112:0/545738448 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f82640054a0 con 0x7f826c108cf0 2024-09-21T03:26:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.592+0000 7f8262ffd640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f82580b6590 0x7f82580b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.592+0000 7f82719f6640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f82580b6590 0x7f82580b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.592+0000 7f8262ffd640 1 -- 172.21.15.112:0/545738448 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f82640e7470 con 0x7f826c108cf0 2024-09-21T03:26:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.593+0000 7f82719f6640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f82580b6590 0x7f82580b8aa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f826c06b9c0 tx=0x7f8268005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:25.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.595+0000 7f8262ffd640 1 -- 172.21.15.112:0/545738448 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8264016030 con 0x7f826c108cf0 2024-09-21T03:26:25.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:25 smithi203 ceph-mon[34723]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:25 smithi187 ceph-mon[34603]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:25.679 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:26:25 smithi187 podman[62532]: 2024-09-21 03:26:25.324195375 +0000 UTC m=+0.354948765 container init 3fbc25157879db5c90ca2f170f9fd194719943a67d21a1d5509096add98e7980 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-k, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T03:26:25.680 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:26:25 smithi187 podman[62532]: 2024-09-21 03:26:25.326984903 +0000 UTC m=+0.357738294 container start 3fbc25157879db5c90ca2f170f9fd194719943a67d21a1d5509096add98e7980 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T03:26:25.680 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:26:25 smithi187 ceph-mds[62551]: -- 172.21.15.187:0/942471081 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5621b49c1c20 con 0x5621b8b69a80 2024-09-21T03:26:25.680 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:26:25 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-k[62547]: starting mds.k at 2024-09-21T03:26:25.681 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:26:25 smithi187 bash[62532]: 3fbc25157879db5c90ca2f170f9fd194719943a67d21a1d5509096add98e7980 2024-09-21T03:26:25.681 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:26:25 smithi187 systemd[1]: Started Ceph mds.k for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:25.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.830+0000 7f8273c5a640 1 -- 172.21.15.112:0/545738448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f826c06b270 con 0x7f826c108cf0 2024-09-21T03:26:25.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.831+0000 7f8262ffd640 1 -- 172.21.15.112:0/545738448 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f82640ae0d0 con 0x7f826c108cf0 2024-09-21T03:26:25.833 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:25.833 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"btime":"2024-09-21T03:26:22:997060+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:26:25.833 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-09-21T03:26:25.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 -- 172.21.15.112:0/545738448 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f82580b6590 msgr2=0x7f82580b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:25.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f82580b6590 0x7f82580b8aa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f826c06b9c0 tx=0x7f8268005f90 comp rx=0 tx=0).stop 2024-09-21T03:26:25.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 -- 172.21.15.112:0/545738448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f826c108cf0 msgr2=0x7f826c19bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:25.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f826c108cf0 0x7f826c19bd70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f826400eaa0 tx=0x7f826400ef60 comp rx=0 tx=0).stop 2024-09-21T03:26:25.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 -- 172.21.15.112:0/545738448 shutdown_connections 2024-09-21T03:26:25.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f82580b6590 0x7f82580b8aa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:25.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f826c108cf0 0x7f826c19bd70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:25.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826c1083a0 0x7f826c19b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:25.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 --2- 172.21.15.112:0/545738448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f826c102a60 0x7f826c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:25.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 -- 172.21.15.112:0/545738448 >> 172.21.15.112:0/545738448 conn(0x7f826c079730 msgr2=0x7f826c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:25.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 -- 172.21.15.112:0/545738448 shutdown_connections 2024-09-21T03:26:25.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:25.833+0000 7f8273c5a640 1 -- 172.21.15.112:0/545738448 wait complete. 2024-09-21T03:26:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:26.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: Deploying daemon mds.j on smithi112 2024-09-21T03:26:26.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/545738448' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:26.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.187:6832/3187781490,v1:172.21.15.187:6833/3187781490] up:boot 2024-09-21T03:26:26.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: fsmap 2 up:standby 2024-09-21T03:26:26.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:26:26.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:26 smithi112 ceph-mon[31442]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:26.833 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:26.833 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:26.833 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:26.833 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:26.834 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:26.834 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:26.834 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: Deploying daemon mds.j on smithi112 2024-09-21T03:26:26.834 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/545738448' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:26.834 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.187:6832/3187781490,v1:172.21.15.187:6833/3187781490] up:boot 2024-09-21T03:26:26.834 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: fsmap 2 up:standby 2024-09-21T03:26:26.834 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:26:26.834 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:26 smithi187 ceph-mon[34603]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: Deploying daemon mds.j on smithi112 2024-09-21T03:26:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/545738448' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.187:6832/3187781490,v1:172.21.15.187:6833/3187781490] up:boot 2024-09-21T03:26:26.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: fsmap 2 up:standby 2024-09-21T03:26:26.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:26:26.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:26 smithi203 ceph-mon[34723]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:28.205 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:26:27 smithi112 systemd[1]: Starting Ceph mds.j for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:26:28.479 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:26:28 smithi112 podman[73433]: 2024-09-21 03:26:28.105867974 +0000 UTC m=+0.016002590 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:26:28.480 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:26:28 smithi112 podman[73433]: 2024-09-21 03:26:28.205240284 +0000 UTC m=+0.115374928 container create 671905f460a214c77f98205e87d90c4906069b211fd172edbefbdeade1f84e08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-j, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T03:26:28.480 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:26:28 smithi112 podman[73433]: 2024-09-21 03:26:28.441759106 +0000 UTC m=+0.351893721 container init 671905f460a214c77f98205e87d90c4906069b211fd172edbefbdeade1f84e08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:26:28.480 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:26:28 smithi112 podman[73433]: 2024-09-21 03:26:28.444837328 +0000 UTC m=+0.354971930 container start 671905f460a214c77f98205e87d90c4906069b211fd172edbefbdeade1f84e08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-j, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T03:26:28.686 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:26:28.787 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:26:28 smithi112 ceph-mds[73453]: -- 172.21.15.112:0/3158447882 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56072f2d7860 con 0x56073347f600 2024-09-21T03:26:28.787 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:26:28 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-j[73449]: starting mds.j at 2024-09-21T03:26:28.787 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:26:28 smithi112 bash[73433]: 671905f460a214c77f98205e87d90c4906069b211fd172edbefbdeade1f84e08 2024-09-21T03:26:28.787 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:26:28 smithi112 systemd[1]: Started Ceph mds.j for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:29.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:29 smithi112 ceph-mon[31442]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:29.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:29.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:29.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:29.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:29.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:29.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:29 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:29.556 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:29 smithi203 ceph-mon[34723]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:29 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:29 smithi187 ceph-mon[34603]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:29 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:30.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.020+0000 7f16defe0640 1 -- 172.21.15.112:0/2460931655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d8102a70 msgr2=0x7f16d8102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:30.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.020+0000 7f16defe0640 1 --2- 172.21.15.112:0/2460931655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d8102a70 0x7f16d8102e80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009900 tx=0x7f16d402a410 comp rx=0 tx=0).stop 2024-09-21T03:26:30.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.020+0000 7f16defe0640 1 -- 172.21.15.112:0/2460931655 shutdown_connections 2024-09-21T03:26:30.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.020+0000 7f16defe0640 1 --2- 172.21.15.112:0/2460931655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16d8108d00 0x7f16d810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:30.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.020+0000 7f16defe0640 1 --2- 172.21.15.112:0/2460931655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d81083b0 0x7f16d81087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:30.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.020+0000 7f16defe0640 1 --2- 172.21.15.112:0/2460931655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d8102a70 0x7f16d8102e80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:30.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.020+0000 7f16defe0640 1 -- 172.21.15.112:0/2460931655 >> 172.21.15.112:0/2460931655 conn(0x7f16d8079710 msgr2=0x7f16d8079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:30.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.021+0000 7f16defe0640 1 -- 172.21.15.112:0/2460931655 shutdown_connections 2024-09-21T03:26:30.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.021+0000 7f16defe0640 1 -- 172.21.15.112:0/2460931655 wait complete. 2024-09-21T03:26:30.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.021+0000 7f16defe0640 1 Processor -- start 2024-09-21T03:26:30.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.021+0000 7f16defe0640 1 -- start start 2024-09-21T03:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16defe0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16d8102a70 0x7f16d819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16defe0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d81083b0 0x7f16d819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16defe0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d8108d00 0x7f16d819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16defe0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8113ae0 con 0x7f16d8108d00 2024-09-21T03:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16defe0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8113960 con 0x7f16d8102a70 2024-09-21T03:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16defe0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8113c60 con 0x7f16d81083b0 2024-09-21T03:26:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16dd57d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d8108d00 0x7f16d819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16cffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d81083b0 0x7f16d819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16dd57d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d8108d00 0x7f16d819fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56866/0 (socket says 172.21.15.112:56866) 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16cffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d81083b0 0x7f16d819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:53314/0 (socket says 172.21.15.112:53314) 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16dd57d640 1 -- 172.21.15.112:0/1794794236 learned_addr learned my addr 172.21.15.112:0/1794794236 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16dcd7c640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16d8102a70 0x7f16d819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16dd57d640 1 -- 172.21.15.112:0/1794794236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d81083b0 msgr2=0x7f16d819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16dd57d640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d81083b0 0x7f16d819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16dd57d640 1 -- 172.21.15.112:0/1794794236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16d8102a70 msgr2=0x7f16d819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16dd57d640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16d8102a70 0x7f16d819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16dd57d640 1 -- 172.21.15.112:0/1794794236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d81a0380 con 0x7f16d8108d00 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16dcd7c640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16d8102a70 0x7f16d819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16dd57d640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d8108d00 0x7f16d819fd10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f16d000cce0 tx=0x7f16d000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:30.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.022+0000 7f16cdffb640 1 -- 172.21.15.112:0/1794794236 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d000f450 con 0x7f16d8108d00 2024-09-21T03:26:30.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.023+0000 7f16defe0640 1 -- 172.21.15.112:0/1794794236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d81a0670 con 0x7f16d8108d00 2024-09-21T03:26:30.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.023+0000 7f16cdffb640 1 -- 172.21.15.112:0/1794794236 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16d000faa0 con 0x7f16d8108d00 2024-09-21T03:26:30.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.023+0000 7f16cdffb640 1 -- 172.21.15.112:0/1794794236 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0018b70 con 0x7f16d8108d00 2024-09-21T03:26:30.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.023+0000 7f16defe0640 1 -- 172.21.15.112:0/1794794236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d81a6e90 con 0x7f16d8108d00 2024-09-21T03:26:30.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.024+0000 7f16defe0640 1 -- 172.21.15.112:0/1794794236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d8103cf0 con 0x7f16d8108d00 2024-09-21T03:26:30.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.026+0000 7f16cdffb640 1 -- 172.21.15.112:0/1794794236 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f16d0018d10 con 0x7f16d8108d00 2024-09-21T03:26:30.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.027+0000 7f16cdffb640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f16c00b6590 0x7f16c00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:30.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.027+0000 7f16dcd7c640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f16c00b6590 0x7f16c00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:30.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.027+0000 7f16cdffb640 1 -- 172.21.15.112:0/1794794236 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f16d00e7490 con 0x7f16d8108d00 2024-09-21T03:26:30.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.027+0000 7f16dcd7c640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f16c00b6590 0x7f16c00b8aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f16d4031440 tx=0x7f16d4009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:30.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.030+0000 7f16cdffb640 1 -- 172.21.15.112:0/1794794236 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f16d0014030 con 0x7f16d8108d00 2024-09-21T03:26:30.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.246+0000 7f16defe0640 1 -- 172.21.15.112:0/1794794236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16d8102e80 con 0x7f16d8108d00 2024-09-21T03:26:30.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.247+0000 7f16cdffb640 1 -- 172.21.15.112:0/1794794236 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7f16d8102e80 con 0x7f16d8108d00 2024-09-21T03:26:30.249 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:30.249 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":4,"btime":"2024-09-21T03:26:29:377176+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:26:30.249 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 4 2024-09-21T03:26:30.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.249+0000 7f16defe0640 1 -- 172.21.15.112:0/1794794236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f16c00b6590 msgr2=0x7f16c00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:30.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.249+0000 7f16defe0640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f16c00b6590 0x7f16c00b8aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f16d4031440 tx=0x7f16d4009680 comp rx=0 tx=0).stop 2024-09-21T03:26:30.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.250+0000 7f16defe0640 1 -- 172.21.15.112:0/1794794236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d8108d00 msgr2=0x7f16d819fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:30.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.250+0000 7f16defe0640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d8108d00 0x7f16d819fd10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f16d000cce0 tx=0x7f16d000a620 comp rx=0 tx=0).stop 2024-09-21T03:26:30.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.250+0000 7f16defe0640 1 -- 172.21.15.112:0/1794794236 shutdown_connections 2024-09-21T03:26:30.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.250+0000 7f16defe0640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f16c00b6590 0x7f16c00b8aa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:30.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.250+0000 7f16defe0640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16d8108d00 0x7f16d819fd10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:30.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.250+0000 7f16defe0640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16d81083b0 0x7f16d819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:30.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.250+0000 7f16defe0640 1 --2- 172.21.15.112:0/1794794236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16d8102a70 0x7f16d819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:30.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.250+0000 7f16defe0640 1 -- 172.21.15.112:0/1794794236 >> 172.21.15.112:0/1794794236 conn(0x7f16d8079710 msgr2=0x7f16d8106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:30.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.250+0000 7f16defe0640 1 -- 172.21.15.112:0/1794794236 shutdown_connections 2024-09-21T03:26:30.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:30.250+0000 7f16defe0640 1 -- 172.21.15.112:0/1794794236 wait complete. 2024-09-21T03:26:30.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:30 smithi112 ceph-mon[31442]: Deploying daemon mds.d on smithi112 2024-09-21T03:26:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:30 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.112:6834/2583791645,v1:172.21.15.112:6835/2583791645] up:boot 2024-09-21T03:26:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:30 smithi112 ceph-mon[31442]: fsmap 3 up:standby 2024-09-21T03:26:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:30 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:26:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:30 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1794794236' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:30 smithi203 ceph-mon[34723]: Deploying daemon mds.d on smithi112 2024-09-21T03:26:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:30 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.112:6834/2583791645,v1:172.21.15.112:6835/2583791645] up:boot 2024-09-21T03:26:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:30 smithi203 ceph-mon[34723]: fsmap 3 up:standby 2024-09-21T03:26:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:30 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:26:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:30 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1794794236' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:30 smithi187 ceph-mon[34603]: Deploying daemon mds.d on smithi112 2024-09-21T03:26:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:30 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.112:6834/2583791645,v1:172.21.15.112:6835/2583791645] up:boot 2024-09-21T03:26:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:30 smithi187 ceph-mon[34603]: fsmap 3 up:standby 2024-09-21T03:26:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:30 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T03:26:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:30 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1794794236' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:31.344 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:26:31 smithi112 systemd[1]: Starting Ceph mds.d for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:26:31.608 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:31 smithi112 ceph-mon[31442]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:31 smithi203 ceph-mon[34723]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:31 smithi187 ceph-mon[34603]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:31.864 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:26:31 smithi112 podman[73821]: 2024-09-21 03:26:31.509794132 +0000 UTC m=+0.015969940 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:26:31.864 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:26:31 smithi112 podman[73821]: 2024-09-21 03:26:31.652292246 +0000 UTC m=+0.158468058 container create 694f730c71badc224c2ade3a3e13f7f09faf954f4e0f7ba9b0cbb9899d2e8ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-d, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T03:26:31.864 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:26:31 smithi112 podman[73821]: 2024-09-21 03:26:31.862948758 +0000 UTC m=+0.369124573 container init 694f730c71badc224c2ade3a3e13f7f09faf954f4e0f7ba9b0cbb9899d2e8ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:26:32.287 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:26:31 smithi112 podman[73821]: 2024-09-21 03:26:31.866052527 +0000 UTC m=+0.372228343 container start 694f730c71badc224c2ade3a3e13f7f09faf954f4e0f7ba9b0cbb9899d2e8ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-d, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T03:26:32.287 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:26:31 smithi112 ceph-mds[73841]: -- 172.21.15.112:0/2216114970 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562473539860 con 0x5624776e1a80 2024-09-21T03:26:32.287 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:26:31 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-d[73837]: starting mds.d at 2024-09-21T03:26:32.287 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:26:31 smithi112 bash[73821]: 694f730c71badc224c2ade3a3e13f7f09faf954f4e0f7ba9b0cbb9899d2e8ffa 2024-09-21T03:26:32.287 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:26:31 smithi112 systemd[1]: Started Ceph mds.d for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:33.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:33.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:32 smithi112 ceph-mon[31442]: Deploying daemon mds.e on smithi187 2024-09-21T03:26:33.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:32 smithi112 ceph-mon[31442]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:33.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:32 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.112:6836/3042717510,v1:172.21.15.112:6837/3042717510] up:boot 2024-09-21T03:26:33.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:32 smithi112 ceph-mon[31442]: fsmap 4 up:standby 2024-09-21T03:26:33.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:26:33.365 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:33.365 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:33.365 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:33.365 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:33.365 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:33.365 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:33.365 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:32 smithi187 ceph-mon[34603]: Deploying daemon mds.e on smithi187 2024-09-21T03:26:33.365 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:32 smithi187 ceph-mon[34603]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:33.365 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:32 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.112:6836/3042717510,v1:172.21.15.112:6837/3042717510] up:boot 2024-09-21T03:26:33.365 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:32 smithi187 ceph-mon[34603]: fsmap 4 up:standby 2024-09-21T03:26:33.365 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:26:33.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:26:33.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:33.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:33.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:33.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:33.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:33.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:33.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:32 smithi203 ceph-mon[34723]: Deploying daemon mds.e on smithi187 2024-09-21T03:26:33.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:32 smithi203 ceph-mon[34723]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:33.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:32 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.112:6836/3042717510,v1:172.21.15.112:6837/3042717510] up:boot 2024-09-21T03:26:33.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:32 smithi203 ceph-mon[34723]: fsmap 4 up:standby 2024-09-21T03:26:33.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T03:26:33.634 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:33.930 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:26:33 smithi187 systemd[1]: Starting Ceph mds.e for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:26:34.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.056+0000 7fcd6549c640 1 -- 172.21.15.112:0/1691765129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd601083a0 msgr2=0x7fcd601087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:34.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.056+0000 7fcd6549c640 1 --2- 172.21.15.112:0/1691765129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd601083a0 0x7fcd601087b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009900 tx=0x7fcd4802a410 comp rx=0 tx=0).stop 2024-09-21T03:26:34.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.056+0000 7fcd6549c640 1 -- 172.21.15.112:0/1691765129 shutdown_connections 2024-09-21T03:26:34.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.056+0000 7fcd6549c640 1 --2- 172.21.15.112:0/1691765129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd60108cf0 0x7fcd6010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:34.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.056+0000 7fcd6549c640 1 --2- 172.21.15.112:0/1691765129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd601083a0 0x7fcd601087b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:34.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.056+0000 7fcd6549c640 1 --2- 172.21.15.112:0/1691765129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd60102a60 0x7fcd60102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:34.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.056+0000 7fcd6549c640 1 -- 172.21.15.112:0/1691765129 >> 172.21.15.112:0/1691765129 conn(0x7fcd60079730 msgr2=0x7fcd60079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:34.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.056+0000 7fcd6549c640 1 -- 172.21.15.112:0/1691765129 shutdown_connections 2024-09-21T03:26:34.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.056+0000 7fcd6549c640 1 -- 172.21.15.112:0/1691765129 wait complete. 2024-09-21T03:26:34.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.057+0000 7fcd6549c640 1 Processor -- start 2024-09-21T03:26:34.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.057+0000 7fcd6549c640 1 -- start start 2024-09-21T03:26:34.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.057+0000 7fcd6549c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd60102a60 0x7fcd60199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:34.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.057+0000 7fcd6549c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd601083a0 0x7fcd601999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:34.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.057+0000 7fcd6549c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd60108cf0 0x7fcd6019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:34.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.057+0000 7fcd6549c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60112960 con 0x7fcd601083a0 2024-09-21T03:26:34.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.057+0000 7fcd6549c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601127e0 con 0x7fcd60102a60 2024-09-21T03:26:34.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.057+0000 7fcd6549c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60112ae0 con 0x7fcd60108cf0 2024-09-21T03:26:34.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd5e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd601083a0 0x7fcd601999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:34.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd5e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd601083a0 0x7fcd601999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56908/0 (socket says 172.21.15.112:56908) 2024-09-21T03:26:34.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd5e7fc640 1 -- 172.21.15.112:0/1761610332 learned_addr learned my addr 172.21.15.112:0/1761610332 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:34.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd5f7fe640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd60108cf0 0x7fcd6019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd5effd640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd60102a60 0x7fcd60199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd5e7fc640 1 -- 172.21.15.112:0/1761610332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd60108cf0 msgr2=0x7fcd6019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd5e7fc640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd60108cf0 0x7fcd6019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd5e7fc640 1 -- 172.21.15.112:0/1761610332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd60102a60 msgr2=0x7fcd60199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd5e7fc640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd60102a60 0x7fcd60199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd5e7fc640 1 -- 172.21.15.112:0/1761610332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6019f2e0 con 0x7fcd601083a0 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd5e7fc640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd601083a0 0x7fcd601999d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd48032fd0 tx=0x7fcd48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd57fff640 1 -- 172.21.15.112:0/1761610332 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48046070 con 0x7fcd601083a0 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd57fff640 1 -- 172.21.15.112:0/1761610332 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd48003aa0 con 0x7fcd601083a0 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd6549c640 1 -- 172.21.15.112:0/1761610332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6019f570 con 0x7fcd601083a0 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd57fff640 1 -- 172.21.15.112:0/1761610332 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48035050 con 0x7fcd601083a0 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.058+0000 7fcd6549c640 1 -- 172.21.15.112:0/1761610332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6019fa50 con 0x7fcd601083a0 2024-09-21T03:26:34.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.059+0000 7fcd6549c640 1 -- 172.21.15.112:0/1761610332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd60103ce0 con 0x7fcd601083a0 2024-09-21T03:26:34.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.061+0000 7fcd57fff640 1 -- 172.21.15.112:0/1761610332 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fcd48049070 con 0x7fcd601083a0 2024-09-21T03:26:34.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.062+0000 7fcd57fff640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd4c0b6590 0x7fcd4c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:34.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.062+0000 7fcd5effd640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd4c0b6590 0x7fcd4c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:34.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.062+0000 7fcd57fff640 1 -- 172.21.15.112:0/1761610332 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7fcd48108bd0 con 0x7fcd601083a0 2024-09-21T03:26:34.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.062+0000 7fcd5effd640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd4c0b6590 0x7fcd4c0b8aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005fd0 tx=0x7fcd5800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:34.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.065+0000 7fcd57fff640 1 -- 172.21.15.112:0/1761610332 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcd48050080 con 0x7fcd601083a0 2024-09-21T03:26:34.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.279+0000 7fcd6549c640 1 -- 172.21.15.112:0/1761610332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd60102e70 con 0x7fcd601083a0 2024-09-21T03:26:34.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.280+0000 7fcd57fff640 1 -- 172.21.15.112:0/1761610332 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7fcd480d3c10 con 0x7fcd601083a0 2024-09-21T03:26:34.282 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:34.282 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":5,"btime":"2024-09-21T03:26:32:391750+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T03:26:34.282 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 5 2024-09-21T03:26:34.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 -- 172.21.15.112:0/1761610332 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd4c0b6590 msgr2=0x7fcd4c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:34.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd4c0b6590 0x7fcd4c0b8aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005fd0 tx=0x7fcd5800b390 comp rx=0 tx=0).stop 2024-09-21T03:26:34.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 -- 172.21.15.112:0/1761610332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd601083a0 msgr2=0x7fcd601999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:34.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd601083a0 0x7fcd601999d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd48032fd0 tx=0x7fcd48004270 comp rx=0 tx=0).stop 2024-09-21T03:26:34.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 -- 172.21.15.112:0/1761610332 shutdown_connections 2024-09-21T03:26:34.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd4c0b6590 0x7fcd4c0b8aa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:34.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd60108cf0 0x7fcd6019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:34.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd601083a0 0x7fcd601999d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:34.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 --2- 172.21.15.112:0/1761610332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd60102a60 0x7fcd60199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:34.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 -- 172.21.15.112:0/1761610332 >> 172.21.15.112:0/1761610332 conn(0x7fcd60079730 msgr2=0x7fcd60104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:34.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 -- 172.21.15.112:0/1761610332 shutdown_connections 2024-09-21T03:26:34.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:34.282+0000 7fcd6549c640 1 -- 172.21.15.112:0/1761610332 wait complete. 2024-09-21T03:26:34.321 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:26:34 smithi187 podman[62804]: 2024-09-21 03:26:33.942977187 +0000 UTC m=+0.015884224 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:26:34.321 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:26:34 smithi187 podman[62804]: 2024-09-21 03:26:34.040605169 +0000 UTC m=+0.113512201 container create 0e278118c1b8a4110a424ea51bc7b6065789f8e3df6f19f3ebdeb14531cdb5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T03:26:34.321 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:26:34 smithi187 podman[62804]: 2024-09-21 03:26:34.284707374 +0000 UTC m=+0.357614405 container init 0e278118c1b8a4110a424ea51bc7b6065789f8e3df6f19f3ebdeb14531cdb5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:26:34.322 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:26:34 smithi187 podman[62804]: 2024-09-21 03:26:34.287413193 +0000 UTC m=+0.360320234 container start 0e278118c1b8a4110a424ea51bc7b6065789f8e3df6f19f3ebdeb14531cdb5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T03:26:34.322 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:26:34 smithi187 ceph-mds[62823]: -- 172.21.15.187:0/1596168991 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560679193860 con 0x56067d33ba80 2024-09-21T03:26:34.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:34 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1761610332' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:34.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:34 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1761610332' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:34.679 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:26:34 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e[62819]: starting mds.e at 2024-09-21T03:26:34.679 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:26:34 smithi187 bash[62804]: 0e278118c1b8a4110a424ea51bc7b6065789f8e3df6f19f3ebdeb14531cdb5b0 2024-09-21T03:26:34.679 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:26:34 smithi187 systemd[1]: Started Ceph mds.e for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:34.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:34 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1761610332' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:35.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:35 smithi203 ceph-mon[34723]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:35 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.187:6834/2552593896,v1:172.21.15.187:6835/2552593896] up:boot 2024-09-21T03:26:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:35 smithi203 ceph-mon[34723]: fsmap 5 up:standby 2024-09-21T03:26:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:26:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:35 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:35 smithi187 ceph-mon[34603]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:35 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.187:6834/2552593896,v1:172.21.15.187:6835/2552593896] up:boot 2024-09-21T03:26:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:35 smithi187 ceph-mon[34603]: fsmap 5 up:standby 2024-09-21T03:26:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:26:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:35.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:35.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:35.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:35.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:35.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:35 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:35 smithi112 ceph-mon[31442]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:35 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.187:6834/2552593896,v1:172.21.15.187:6835/2552593896] up:boot 2024-09-21T03:26:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:35 smithi112 ceph-mon[31442]: fsmap 5 up:standby 2024-09-21T03:26:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:26:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:35 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:36.376 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:26:36 smithi203 systemd[1]: Starting Ceph mds.f for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:26:36.647 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:36 smithi203 ceph-mon[34723]: Deploying daemon mds.f on smithi203 2024-09-21T03:26:36.648 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:26:36 smithi203 podman[63794]: 2024-09-21 03:26:36.374687819 +0000 UTC m=+0.111676774 container create 7ca03a4afbb15b99dc668c0d961cf2e2655609a2c9bc053868f28dc039fd0cb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T03:26:36.648 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:26:36 smithi203 podman[63794]: 2024-09-21 03:26:36.279069872 +0000 UTC m=+0.016058841 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:26:36.648 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:26:36 smithi203 podman[63794]: 2024-09-21 03:26:36.610542623 +0000 UTC m=+0.347531578 container init 7ca03a4afbb15b99dc668c0d961cf2e2655609a2c9bc053868f28dc039fd0cb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T03:26:36.648 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:26:36 smithi203 podman[63794]: 2024-09-21 03:26:36.613363139 +0000 UTC m=+0.350352105 container start 7ca03a4afbb15b99dc668c0d961cf2e2655609a2c9bc053868f28dc039fd0cb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T03:26:36.648 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:26:36 smithi203 ceph-mds[63813]: -- 172.21.15.203:0/1484140229 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f22151fc20 con 0x55f2256c7a80 2024-09-21T03:26:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:36 smithi187 ceph-mon[34603]: Deploying daemon mds.f on smithi203 2024-09-21T03:26:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:36 smithi112 ceph-mon[31442]: Deploying daemon mds.f on smithi203 2024-09-21T03:26:36.910 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:26:36 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f[63809]: starting mds.f at 2024-09-21T03:26:36.911 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:26:36 smithi203 bash[63794]: 7ca03a4afbb15b99dc668c0d961cf2e2655609a2c9bc053868f28dc039fd0cb0 2024-09-21T03:26:36.911 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:26:36 smithi203 systemd[1]: Started Ceph mds.f for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:37.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:26:37.309 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:37.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:37 smithi203 ceph-mon[34723]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:37.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:37 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:37.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:37 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:37.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:37 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:37.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:37 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:37.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:37 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:37.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:37 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:37.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:37 smithi112 ceph-mon[31442]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:37.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:37 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:37.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:37 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:37.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:37 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:37.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:37 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:37.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:37 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:37.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:37 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:37.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:37 smithi187 ceph-mon[34603]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:37.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:37 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:37.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:37 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:37.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:37 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:37.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:37 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:37.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:37 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:37.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:37 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:37.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.754+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2056235449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedb8108d70 msgr2=0x7fedb810f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:37.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.754+0000 7fedbf0f1640 1 --2- 172.21.15.112:0/2056235449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedb8108d70 0x7fedb810f5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feda0009900 tx=0x7feda002a410 comp rx=0 tx=0).stop 2024-09-21T03:26:37.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.754+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2056235449 shutdown_connections 2024-09-21T03:26:37.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.754+0000 7fedbf0f1640 1 --2- 172.21.15.112:0/2056235449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedb8108d70 0x7fedb810f5a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:37.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.754+0000 7fedbf0f1640 1 --2- 172.21.15.112:0/2056235449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedb8108420 0x7fedb8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:37.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.754+0000 7fedbf0f1640 1 --2- 172.21.15.112:0/2056235449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedb8102ae0 0x7fedb8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:37.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.754+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2056235449 >> 172.21.15.112:0/2056235449 conn(0x7fedb8079770 msgr2=0x7fedb8079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:37.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.754+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2056235449 shutdown_connections 2024-09-21T03:26:37.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.755+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2056235449 wait complete. 2024-09-21T03:26:37.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.755+0000 7fedbf0f1640 1 Processor -- start 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.755+0000 7fedbf0f1640 1 -- start start 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.755+0000 7fedbf0f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedb8102ae0 0x7fedb819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.755+0000 7fedbf0f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedb8108420 0x7fedb819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.755+0000 7fedbf0f1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedb8108d70 0x7fedb819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.755+0000 7fedbf0f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fedb8113b50 con 0x7fedb8108420 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.755+0000 7fedbf0f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fedb81139d0 con 0x7fedb8102ae0 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.755+0000 7fedbf0f1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fedb8113cd0 con 0x7fedb8108d70 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedb8108420 0x7fedb819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedb8108420 0x7fedb819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43268/0 (socket says 172.21.15.112:43268) 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb7fff640 1 -- 172.21.15.112:0/2994538371 learned_addr learned my addr 172.21.15.112:0/2994538371 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedbd68e640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedb8108d70 0x7fedb819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedbce8d640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedb8102ae0 0x7fedb819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:37.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb7fff640 1 -- 172.21.15.112:0/2994538371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedb8108d70 msgr2=0x7fedb819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:37.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb7fff640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedb8108d70 0x7fedb819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:37.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb7fff640 1 -- 172.21.15.112:0/2994538371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedb8102ae0 msgr2=0x7fedb819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:37.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb7fff640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedb8102ae0 0x7fedb819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:37.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb7fff640 1 -- 172.21.15.112:0/2994538371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb81a0350 con 0x7fedb8108420 2024-09-21T03:26:37.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb7fff640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedb8108420 0x7fedb819aa40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feda800cce0 tx=0x7feda800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:37.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb5ffb640 1 -- 172.21.15.112:0/2994538371 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda800f450 con 0x7fedb8108420 2024-09-21T03:26:37.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb5ffb640 1 -- 172.21.15.112:0/2994538371 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feda800faa0 con 0x7fedb8108420 2024-09-21T03:26:37.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2994538371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb81a0640 con 0x7fedb8108420 2024-09-21T03:26:37.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.756+0000 7fedb5ffb640 1 -- 172.21.15.112:0/2994538371 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8018c50 con 0x7fedb8108420 2024-09-21T03:26:37.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.757+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2994538371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb81a6e60 con 0x7fedb8108420 2024-09-21T03:26:37.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.758+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2994538371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb8103d60 con 0x7fedb8108420 2024-09-21T03:26:37.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.759+0000 7fedb5ffb640 1 -- 172.21.15.112:0/2994538371 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7feda800fc50 con 0x7fedb8108420 2024-09-21T03:26:37.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.759+0000 7fedb5ffb640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feda40b6590 0x7feda40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:37.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.760+0000 7fedbce8d640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feda40b6590 0x7feda40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:37.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.760+0000 7fedb5ffb640 1 -- 172.21.15.112:0/2994538371 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7feda80e7310 con 0x7fedb8108420 2024-09-21T03:26:37.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.760+0000 7fedbce8d640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feda40b6590 0x7feda40b8aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fedb0005d00 tx=0x7fedb000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:37.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.764+0000 7fedb5ffb640 1 -- 172.21.15.112:0/2994538371 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feda8014030 con 0x7fedb8108420 2024-09-21T03:26:37.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.978+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2994538371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedb819b5d0 con 0x7fedb8108420 2024-09-21T03:26:37.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.979+0000 7fedb5ffb640 1 -- 172.21.15.112:0/2994538371 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5578 (secure 0 0 0) 0x7fedb819b5d0 con 0x7fedb8108420 2024-09-21T03:26:37.982 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:37.982 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"btime":"2024-09-21T03:26:37:384615+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-21T03:26:37.983 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2024-09-21T03:26:37.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.981+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2994538371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feda40b6590 msgr2=0x7feda40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:37.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.982+0000 7fedbf0f1640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feda40b6590 0x7feda40b8aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fedb0005d00 tx=0x7fedb000b410 comp rx=0 tx=0).stop 2024-09-21T03:26:37.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.982+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2994538371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedb8108420 msgr2=0x7fedb819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:37.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.982+0000 7fedbf0f1640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedb8108420 0x7fedb819aa40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feda800cce0 tx=0x7feda800a620 comp rx=0 tx=0).stop 2024-09-21T03:26:37.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.982+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2994538371 shutdown_connections 2024-09-21T03:26:37.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.982+0000 7fedbf0f1640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feda40b6590 0x7feda40b8aa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:37.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.982+0000 7fedbf0f1640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedb8108d70 0x7fedb819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:37.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.982+0000 7fedbf0f1640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedb8108420 0x7fedb819aa40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:37.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.982+0000 7fedbf0f1640 1 --2- 172.21.15.112:0/2994538371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedb8102ae0 0x7fedb819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:37.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.982+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2994538371 >> 172.21.15.112:0/2994538371 conn(0x7fedb8079770 msgr2=0x7fedb81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:37.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.982+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2994538371 shutdown_connections 2024-09-21T03:26:37.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:37.982+0000 7fedbf0f1640 1 -- 172.21.15.112:0/2994538371 wait complete. 2024-09-21T03:26:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:38 smithi203 ceph-mon[34723]: Deploying daemon mds.c on smithi203 2024-09-21T03:26:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:38 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.203:6834/3541929468,v1:172.21.15.203:6835/3541929468] up:boot 2024-09-21T03:26:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:38 smithi203 ceph-mon[34723]: fsmap 6 up:standby 2024-09-21T03:26:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:38 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:26:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:38 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2994538371' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:38.661 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:26:38 smithi203 systemd[1]: Starting Ceph mds.c for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:26:38.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:38 smithi187 ceph-mon[34603]: Deploying daemon mds.c on smithi203 2024-09-21T03:26:38.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:38 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.203:6834/3541929468,v1:172.21.15.203:6835/3541929468] up:boot 2024-09-21T03:26:38.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:38 smithi187 ceph-mon[34603]: fsmap 6 up:standby 2024-09-21T03:26:38.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:38 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:26:38.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:38 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2994538371' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:38 smithi112 ceph-mon[31442]: Deploying daemon mds.c on smithi203 2024-09-21T03:26:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:38 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.203:6834/3541929468,v1:172.21.15.203:6835/3541929468] up:boot 2024-09-21T03:26:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:38 smithi112 ceph-mon[31442]: fsmap 6 up:standby 2024-09-21T03:26:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:38 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:26:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:38 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2994538371' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:39.019 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:26:38 smithi203 podman[64064]: 2024-09-21 03:26:38.657041918 +0000 UTC m=+0.016095787 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:26:39.019 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:26:38 smithi203 podman[64064]: 2024-09-21 03:26:38.769672199 +0000 UTC m=+0.128726095 container create ad91d241c5b11163d5c67eb9af288f49293bf8c1c3b3915e1f9cbcca769bcb46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:26:39.019 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:26:38 smithi203 podman[64064]: 2024-09-21 03:26:38.980250913 +0000 UTC m=+0.339304768 container init ad91d241c5b11163d5c67eb9af288f49293bf8c1c3b3915e1f9cbcca769bcb46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:26:39.019 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:26:38 smithi203 podman[64064]: 2024-09-21 03:26:38.983790394 +0000 UTC m=+0.342844248 container start ad91d241c5b11163d5c67eb9af288f49293bf8c1c3b3915e1f9cbcca769bcb46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:26:39.019 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:26:39 smithi203 ceph-mds[64083]: -- 172.21.15.203:0/613332397 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d14d891860 con 0x55d151a39a80 2024-09-21T03:26:39.399 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:26:39 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c[64079]: starting mds.c at 2024-09-21T03:26:39.399 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:26:39 smithi203 bash[64064]: ad91d241c5b11163d5c67eb9af288f49293bf8c1c3b3915e1f9cbcca769bcb46 2024-09-21T03:26:39.399 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:26:39 smithi203 systemd[1]: Started Ceph mds.c for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:39 smithi203 ceph-mon[34723]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:39.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:39 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:39.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:39 smithi187 ceph-mon[34603]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:39.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:39.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:39.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:39.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:39.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:39.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:39 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:39 smithi112 ceph-mon[31442]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:39.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:39.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:39 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:40 smithi203 ceph-mon[34723]: Deploying daemon mds.h on smithi187 2024-09-21T03:26:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:40 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] up:boot 2024-09-21T03:26:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:40 smithi203 ceph-mon[34723]: fsmap 7 up:standby 2024-09-21T03:26:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:40 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:26:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:40 smithi187 ceph-mon[34603]: Deploying daemon mds.h on smithi187 2024-09-21T03:26:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:40 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] up:boot 2024-09-21T03:26:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:40 smithi187 ceph-mon[34603]: fsmap 7 up:standby 2024-09-21T03:26:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:40 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:26:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:40 smithi112 ceph-mon[31442]: Deploying daemon mds.h on smithi187 2024-09-21T03:26:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:40 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] up:boot 2024-09-21T03:26:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:40 smithi112 ceph-mon[31442]: fsmap 7 up:standby 2024-09-21T03:26:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:40 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:26:40.823 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:26:41.079 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:41.411 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:26:41 smithi187 podman[63074]: 2024-09-21 03:26:41.335585607 +0000 UTC m=+0.135288020 container create 0471c51f1b1fd225e9cf93c67193b68e495f97ee97f68dc3293bcd36d744fe81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-21T03:26:41.465 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:41 smithi112 ceph-mon[31442]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:41.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.552+0000 7f372428a640 1 -- 172.21.15.112:0/1337396349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f371c1094f0 msgr2=0x7f371c109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.552+0000 7f372428a640 1 --2- 172.21.15.112:0/1337396349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f371c1094f0 0x7f371c109900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3710009900 tx=0x7f371002a410 comp rx=0 tx=0).stop 2024-09-21T03:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.552+0000 7f372428a640 1 -- 172.21.15.112:0/1337396349 shutdown_connections 2024-09-21T03:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.552+0000 7f372428a640 1 --2- 172.21.15.112:0/1337396349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f371c109e40 0x7f371c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.552+0000 7f372428a640 1 --2- 172.21.15.112:0/1337396349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f371c1094f0 0x7f371c109900 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.552+0000 7f372428a640 1 --2- 172.21.15.112:0/1337396349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f371c103bb0 0x7f371c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.552+0000 7f372428a640 1 -- 172.21.15.112:0/1337396349 >> 172.21.15.112:0/1337396349 conn(0x7f371c0ff600 msgr2=0x7f371c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.552+0000 7f372428a640 1 -- 172.21.15.112:0/1337396349 shutdown_connections 2024-09-21T03:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.552+0000 7f372428a640 1 -- 172.21.15.112:0/1337396349 wait complete. 2024-09-21T03:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.552+0000 7f372428a640 1 Processor -- start 2024-09-21T03:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.552+0000 7f372428a640 1 -- start start 2024-09-21T03:26:41.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f372428a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f371c103bb0 0x7f371c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f372428a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f371c1094f0 0x7f371c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f372428a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f371c109e40 0x7f371c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f372428a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f371c113b00 con 0x7f371c103bb0 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f372428a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f371c113980 con 0x7f371c1094f0 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f372428a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f371c113c80 con 0x7f371c109e40 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3722026640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f371c103bb0 0x7f371c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3722026640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f371c103bb0 0x7f371c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43282/0 (socket says 172.21.15.112:43282) 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3722026640 1 -- 172.21.15.112:0/3554029980 learned_addr learned my addr 172.21.15.112:0/3554029980 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3721825640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f371c1094f0 0x7f371c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3722827640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f371c109e40 0x7f371c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3722026640 1 -- 172.21.15.112:0/3554029980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f371c109e40 msgr2=0x7f371c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3722026640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f371c109e40 0x7f371c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3722026640 1 -- 172.21.15.112:0/3554029980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f371c1094f0 msgr2=0x7f371c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3722026640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f371c1094f0 0x7f371c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3722026640 1 -- 172.21.15.112:0/3554029980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371c1a0440 con 0x7f371c103bb0 2024-09-21T03:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3721825640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f371c1094f0 0x7f371c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.553+0000 7f3722026640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f371c103bb0 0x7f371c19a4d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f370c00ea80 tx=0x7f370c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.554+0000 7f371b7fe640 1 -- 172.21.15.112:0/3554029980 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c00ce40 con 0x7f371c103bb0 2024-09-21T03:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.554+0000 7f371b7fe640 1 -- 172.21.15.112:0/3554029980 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f370c005020 con 0x7f371c103bb0 2024-09-21T03:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.554+0000 7f372428a640 1 -- 172.21.15.112:0/3554029980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371c1a0730 con 0x7f371c103bb0 2024-09-21T03:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.554+0000 7f371b7fe640 1 -- 172.21.15.112:0/3554029980 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c0116d0 con 0x7f371c103bb0 2024-09-21T03:26:41.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.554+0000 7f372428a640 1 -- 172.21.15.112:0/3554029980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371c1a6e90 con 0x7f371c103bb0 2024-09-21T03:26:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.555+0000 7f372428a640 1 -- 172.21.15.112:0/3554029980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371c103fc0 con 0x7f371c103bb0 2024-09-21T03:26:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.557+0000 7f371b7fe640 1 -- 172.21.15.112:0/3554029980 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f370c0054d0 con 0x7f371c103bb0 2024-09-21T03:26:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.557+0000 7f371b7fe640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37040b65e0 0x7f37040b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.557+0000 7f3721825640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37040b65e0 0x7f37040b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.557+0000 7f371b7fe640 1 -- 172.21.15.112:0/3554029980 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f370c0e7500 con 0x7f371c103bb0 2024-09-21T03:26:41.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.558+0000 7f3721825640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37040b65e0 0x7f37040b8af0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3710009230 tx=0x7f3710031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:41.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.560+0000 7f371b7fe640 1 -- 172.21.15.112:0/3554029980 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f370c016030 con 0x7f371c103bb0 2024-09-21T03:26:41.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:41 smithi203 ceph-mon[34723]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:41.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:41 smithi187 ceph-mon[34603]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:41.679 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:26:41 smithi187 podman[63074]: 2024-09-21 03:26:41.546363541 +0000 UTC m=+0.346065909 container init 0471c51f1b1fd225e9cf93c67193b68e495f97ee97f68dc3293bcd36d744fe81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T03:26:41.679 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:26:41 smithi187 podman[63074]: 2024-09-21 03:26:41.549132719 +0000 UTC m=+0.348835087 container start 0471c51f1b1fd225e9cf93c67193b68e495f97ee97f68dc3293bcd36d744fe81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T03:26:41.679 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:26:41 smithi187 ceph-mds[63093]: -- 172.21.15.187:0/1962717069 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56040ef25860 con 0x5604130cd600 2024-09-21T03:26:41.680 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:26:41 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h[63089]: starting mds.h at 2024-09-21T03:26:41.680 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:26:41 smithi187 bash[63074]: 0471c51f1b1fd225e9cf93c67193b68e495f97ee97f68dc3293bcd36d744fe81 2024-09-21T03:26:41.680 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:26:41 smithi187 systemd[1]: Started Ceph mds.h for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:41.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.783+0000 7f372428a640 1 -- 172.21.15.112:0/3554029980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f371c19b600 con 0x7f371c103bb0 2024-09-21T03:26:41.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.784+0000 7f371b7fe640 1 -- 172.21.15.112:0/3554029980 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6406 (secure 0 0 0) 0x7f371c19b600 con 0x7f371c103bb0 2024-09-21T03:26:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:41.786 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"btime":"2024-09-21T03:26:39:396713+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T03:26:41.786 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-09-21T03:26:41.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.786+0000 7f372428a640 1 -- 172.21.15.112:0/3554029980 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37040b65e0 msgr2=0x7f37040b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.786+0000 7f372428a640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37040b65e0 0x7f37040b8af0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3710009230 tx=0x7f3710031400 comp rx=0 tx=0).stop 2024-09-21T03:26:41.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.786+0000 7f372428a640 1 -- 172.21.15.112:0/3554029980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f371c103bb0 msgr2=0x7f371c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:41.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.786+0000 7f372428a640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f371c103bb0 0x7f371c19a4d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f370c00ea80 tx=0x7f370c00ef40 comp rx=0 tx=0).stop 2024-09-21T03:26:41.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.786+0000 7f372428a640 1 -- 172.21.15.112:0/3554029980 shutdown_connections 2024-09-21T03:26:41.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.786+0000 7f372428a640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37040b65e0 0x7f37040b8af0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.786+0000 7f372428a640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f371c109e40 0x7f371c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.786+0000 7f372428a640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f371c1094f0 0x7f371c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.786+0000 7f372428a640 1 --2- 172.21.15.112:0/3554029980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f371c103bb0 0x7f371c19a4d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:41.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.786+0000 7f372428a640 1 -- 172.21.15.112:0/3554029980 >> 172.21.15.112:0/3554029980 conn(0x7f371c0ff600 msgr2=0x7f371c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:41.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.787+0000 7f372428a640 1 -- 172.21.15.112:0/3554029980 shutdown_connections 2024-09-21T03:26:41.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:41.787+0000 7f372428a640 1 -- 172.21.15.112:0/3554029980 wait complete. 2024-09-21T03:26:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: Deploying daemon mds.i on smithi203 2024-09-21T03:26:42.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3554029980' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:42.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:42.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.187:6836/2217535204,v1:172.21.15.187:6837/2217535204] up:boot 2024-09-21T03:26:42.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: fsmap 8 up:standby 2024-09-21T03:26:42.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:42 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:26:43.087 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:43.087 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:43.087 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:43.087 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:43.087 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:43.087 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:43.087 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: Deploying daemon mds.i on smithi203 2024-09-21T03:26:43.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3554029980' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:43.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:43.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.187:6836/2217535204,v1:172.21.15.187:6837/2217535204] up:boot 2024-09-21T03:26:43.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: fsmap 8 up:standby 2024-09-21T03:26:43.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:42 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:26:43.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:43.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:43.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:43.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:43.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:43.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:43.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: Deploying daemon mds.i on smithi203 2024-09-21T03:26:43.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3554029980' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:43.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:43.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.187:6836/2217535204,v1:172.21.15.187:6837/2217535204] up:boot 2024-09-21T03:26:43.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: fsmap 8 up:standby 2024-09-21T03:26:43.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:42 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:26:43.661 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:26:43 smithi203 systemd[1]: Starting Ceph mds.i for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:26:44.036 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:26:43 smithi203 podman[64332]: 2024-09-21 03:26:43.667280813 +0000 UTC m=+0.015860430 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:26:44.037 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:26:43 smithi203 podman[64332]: 2024-09-21 03:26:43.779491947 +0000 UTC m=+0.128071549 container create 2b2a8383862e4093e8f7c4ca6f69435e764c657fd09a33d44791999b4c108349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T03:26:44.037 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:26:43 smithi203 podman[64332]: 2024-09-21 03:26:43.998425922 +0000 UTC m=+0.347005525 container init 2b2a8383862e4093e8f7c4ca6f69435e764c657fd09a33d44791999b4c108349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T03:26:44.037 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:26:44 smithi203 podman[64332]: 2024-09-21 03:26:44.001391682 +0000 UTC m=+0.349971292 container start 2b2a8383862e4093e8f7c4ca6f69435e764c657fd09a33d44791999b4c108349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T03:26:44.411 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:26:44 smithi203 ceph-mds[64351]: -- 172.21.15.203:0/749998686 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df14231860 con 0x55df183d9600 2024-09-21T03:26:44.411 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:26:44 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i[64347]: starting mds.i at 2024-09-21T03:26:44.411 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:26:44 smithi203 bash[64332]: 2b2a8383862e4093e8f7c4ca6f69435e764c657fd09a33d44791999b4c108349 2024-09-21T03:26:44.411 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:26:44 smithi203 systemd[1]: Started Ceph mds.i for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:44.511 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:26:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:45 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] up:boot 2024-09-21T03:26:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:45 smithi203 ceph-mon[34723]: fsmap 9 up:standby 2024-09-21T03:26:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:45 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:26:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:45 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:45 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:45 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:45 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:45 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:45 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:45.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:45 smithi203 ceph-mon[34723]: Deploying daemon mds.g on smithi112 2024-09-21T03:26:45.412 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:45 smithi203 ceph-mon[34723]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:45.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:45 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] up:boot 2024-09-21T03:26:45.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:45 smithi187 ceph-mon[34603]: fsmap 9 up:standby 2024-09-21T03:26:45.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:45 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:26:45.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:45 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:45.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:45 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:45.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:45 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:45.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:45 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:45.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:45 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:45.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:45 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:45.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:45 smithi187 ceph-mon[34603]: Deploying daemon mds.g on smithi112 2024-09-21T03:26:45.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:45 smithi187 ceph-mon[34603]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:45.509 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:45 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] up:boot 2024-09-21T03:26:45.509 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:45 smithi112 ceph-mon[31442]: fsmap 9 up:standby 2024-09-21T03:26:45.509 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:45 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:26:45.509 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:45 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:45.509 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:45 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:45.509 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:45 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:45.509 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:45 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:45.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:45 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:45.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:45 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:45.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:45 smithi112 ceph-mon[31442]: Deploying daemon mds.g on smithi112 2024-09-21T03:26:45.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:45 smithi112 ceph-mon[31442]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:45.522 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:46.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.064+0000 7ff459f96640 1 -- 172.21.15.112:0/574698021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff454101f70 msgr2=0x7ff454102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:46.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.064+0000 7ff459f96640 1 --2- 172.21.15.112:0/574698021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff454101f70 0x7ff454102380 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff444009900 tx=0x7ff44402a410 comp rx=0 tx=0).stop 2024-09-21T03:26:46.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.064+0000 7ff459f96640 1 -- 172.21.15.112:0/574698021 shutdown_connections 2024-09-21T03:26:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.064+0000 7ff459f96640 1 --2- 172.21.15.112:0/574698021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff454109e00 0x7ff454110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.064+0000 7ff459f96640 1 --2- 172.21.15.112:0/574698021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4541094b0 0x7ff4541098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.064+0000 7ff459f96640 1 --2- 172.21.15.112:0/574698021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff454101f70 0x7ff454102380 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.064+0000 7ff459f96640 1 -- 172.21.15.112:0/574698021 >> 172.21.15.112:0/574698021 conn(0x7ff454077c60 msgr2=0x7ff4540780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.064+0000 7ff459f96640 1 -- 172.21.15.112:0/574698021 shutdown_connections 2024-09-21T03:26:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.064+0000 7ff459f96640 1 -- 172.21.15.112:0/574698021 wait complete. 2024-09-21T03:26:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.065+0000 7ff459f96640 1 Processor -- start 2024-09-21T03:26:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.065+0000 7ff459f96640 1 -- start start 2024-09-21T03:26:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.065+0000 7ff459f96640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff454101f70 0x7ff45419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.065+0000 7ff459f96640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4541094b0 0x7ff45419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:46.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.065+0000 7ff459f96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff454109e00 0x7ff45419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.065+0000 7ff459f96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff454113ac0 con 0x7ff454109e00 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.065+0000 7ff459f96640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff454113940 con 0x7ff4541094b0 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.065+0000 7ff459f96640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff454113c40 con 0x7ff454101f70 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.065+0000 7ff4537fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff454101f70 0x7ff45419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4541094b0 0x7ff45419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff452ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4541094b0 0x7ff45419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49980/0 (socket says 172.21.15.112:49980) 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff4537fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff454101f70 0x7ff45419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52602/0 (socket says 172.21.15.112:52602) 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff452ffd640 1 -- 172.21.15.112:0/2857387256 learned_addr learned my addr 172.21.15.112:0/2857387256 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff453fff640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff454109e00 0x7ff45419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff4537fe640 1 -- 172.21.15.112:0/2857387256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4541094b0 msgr2=0x7ff45419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff4537fe640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4541094b0 0x7ff45419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff4537fe640 1 -- 172.21.15.112:0/2857387256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff454109e00 msgr2=0x7ff45419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff4537fe640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff454109e00 0x7ff45419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:46.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff4537fe640 1 -- 172.21.15.112:0/2857387256 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4541a03e0 con 0x7ff454101f70 2024-09-21T03:26:46.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff4537fe640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff454101f70 0x7ff45419a470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff444006010 tx=0x7ff444004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:46.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff450ff9640 1 -- 172.21.15.112:0/2857387256 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44403f070 con 0x7ff454101f70 2024-09-21T03:26:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff450ff9640 1 -- 172.21.15.112:0/2857387256 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff444003ca0 con 0x7ff454101f70 2024-09-21T03:26:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff450ff9640 1 -- 172.21.15.112:0/2857387256 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444035070 con 0x7ff454101f70 2024-09-21T03:26:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.066+0000 7ff459f96640 1 -- 172.21.15.112:0/2857387256 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4541a0670 con 0x7ff454101f70 2024-09-21T03:26:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.067+0000 7ff459f96640 1 -- 172.21.15.112:0/2857387256 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4541a6e30 con 0x7ff454101f70 2024-09-21T03:26:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.068+0000 7ff459f96640 1 -- 172.21.15.112:0/2857387256 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45406ac50 con 0x7ff454101f70 2024-09-21T03:26:46.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.069+0000 7ff450ff9640 1 -- 172.21.15.112:0/2857387256 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7ff44403b460 con 0x7ff454101f70 2024-09-21T03:26:46.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.070+0000 7ff450ff9640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff43c0b65e0 0x7ff43c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:46.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.070+0000 7ff452ffd640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff43c0b65e0 0x7ff43c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:46.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.070+0000 7ff450ff9640 1 -- 172.21.15.112:0/2857387256 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7ff444107740 con 0x7ff454101f70 2024-09-21T03:26:46.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.070+0000 7ff452ffd640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff43c0b65e0 0x7ff43c0b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff454109140 tx=0x7ff44800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:46.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.073+0000 7ff450ff9640 1 -- 172.21.15.112:0/2857387256 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff4440d26e0 con 0x7ff454101f70 2024-09-21T03:26:46.287 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:26:46 smithi112 systemd[1]: Starting Ceph mds.g for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:26:46.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.307+0000 7ff459f96640 1 -- 172.21.15.112:0/2857387256 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff45410bc30 con 0x7ff454101f70 2024-09-21T03:26:46.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.308+0000 7ff450ff9640 1 -- 172.21.15.112:0/2857387256 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8061 (secure 0 0 0) 0x7ff4440d7620 con 0x7ff454101f70 2024-09-21T03:26:46.310 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:46.310 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":10,"btime":"2024-09-21T03:26:44:091173+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24647,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24685,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T03:26:46.310 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 10 2024-09-21T03:26:46.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 -- 172.21.15.112:0/2857387256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff43c0b65e0 msgr2=0x7ff43c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:46.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff43c0b65e0 0x7ff43c0b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff454109140 tx=0x7ff44800b410 comp rx=0 tx=0).stop 2024-09-21T03:26:46.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 -- 172.21.15.112:0/2857387256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff454101f70 msgr2=0x7ff45419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:46.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff454101f70 0x7ff45419a470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff444006010 tx=0x7ff444004060 comp rx=0 tx=0).stop 2024-09-21T03:26:46.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 -- 172.21.15.112:0/2857387256 shutdown_connections 2024-09-21T03:26:46.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff43c0b65e0 0x7ff43c0b8af0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:46.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff454109e00 0x7ff45419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:46.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4541094b0 0x7ff45419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:46.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 --2- 172.21.15.112:0/2857387256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff454101f70 0x7ff45419a470 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:46.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 -- 172.21.15.112:0/2857387256 >> 172.21.15.112:0/2857387256 conn(0x7ff454077c60 msgr2=0x7ff454107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:46.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 -- 172.21.15.112:0/2857387256 shutdown_connections 2024-09-21T03:26:46.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:46.310+0000 7ff459f96640 1 -- 172.21.15.112:0/2857387256 wait complete. 2024-09-21T03:26:46.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:46 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2857387256' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:46.664 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:46 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2857387256' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:46.675 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:46 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2857387256' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:46.676 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:26:46 smithi112 podman[74549]: 2024-09-21 03:26:46.205039636 +0000 UTC m=+0.016107256 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:26:46.676 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:26:46 smithi112 podman[74549]: 2024-09-21 03:26:46.338366208 +0000 UTC m=+0.149433818 container create 5d392b3753e0eacc4ef2483c19f17d83deee250d4e353f6fa82ea2fac6b5891d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-g, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T03:26:47.037 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:26:46 smithi112 podman[74549]: 2024-09-21 03:26:46.792572924 +0000 UTC m=+0.603640539 container init 5d392b3753e0eacc4ef2483c19f17d83deee250d4e353f6fa82ea2fac6b5891d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T03:26:47.037 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:26:46 smithi112 podman[74549]: 2024-09-21 03:26:46.795410762 +0000 UTC m=+0.606478375 container start 5d392b3753e0eacc4ef2483c19f17d83deee250d4e353f6fa82ea2fac6b5891d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-g, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:26:47.037 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:26:46 smithi112 ceph-mds[74585]: -- 172.21.15.112:0/3105527665 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556acdb65860 con 0x556ad1d0d180 2024-09-21T03:26:47.037 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:26:46 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-g[74577]: starting mds.g at 2024-09-21T03:26:47.386 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:26:47 smithi112 bash[74549]: 5d392b3753e0eacc4ef2483c19f17d83deee250d4e353f6fa82ea2fac6b5891d 2024-09-21T03:26:47.386 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:26:47 smithi112 systemd[1]: Started Ceph mds.g for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:47.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:47 smithi203 ceph-mon[34723]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:47.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:47.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:47.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:47.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:47.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:47.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:47 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:47.667 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:47 smithi112 ceph-mon[31442]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:47.667 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:47.667 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:47.667 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:47.667 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:47.667 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:47.667 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:47 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:47 smithi187 ceph-mon[34603]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:47 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:48.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:48 smithi203 ceph-mon[34723]: Deploying daemon mds.a on smithi112 2024-09-21T03:26:48.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:48 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.112:6838/1272407908,v1:172.21.15.112:6839/1272407908] up:boot 2024-09-21T03:26:48.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:48 smithi203 ceph-mon[34723]: fsmap 10 up:standby 2024-09-21T03:26:48.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:48 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:26:48.675 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:48 smithi112 ceph-mon[31442]: Deploying daemon mds.a on smithi112 2024-09-21T03:26:48.675 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:48 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.112:6838/1272407908,v1:172.21.15.112:6839/1272407908] up:boot 2024-09-21T03:26:48.675 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:48 smithi112 ceph-mon[31442]: fsmap 10 up:standby 2024-09-21T03:26:48.675 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:48 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:26:48.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:48 smithi187 ceph-mon[34603]: Deploying daemon mds.a on smithi112 2024-09-21T03:26:48.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:48 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.112:6838/1272407908,v1:172.21.15.112:6839/1272407908] up:boot 2024-09-21T03:26:48.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:48 smithi187 ceph-mon[34603]: fsmap 10 up:standby 2024-09-21T03:26:48.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:48 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:26:49.148 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:26:49.401 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:26:49 smithi112 podman[74853]: 2024-09-21 03:26:49.312228491 +0000 UTC m=+0.117009753 container create a02fbf2e7356a0ffc52c4b70efe08322e9f27eb147fb8dd6b34a4bdc243798e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-a, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T03:26:49.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:49 smithi203 ceph-mon[34723]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:49.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:49 smithi187 ceph-mon[34603]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:49.682 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:49 smithi112 ceph-mon[31442]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:49.682 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:26:49 smithi112 podman[74853]: 2024-09-21 03:26:49.58167305 +0000 UTC m=+0.386454322 container init a02fbf2e7356a0ffc52c4b70efe08322e9f27eb147fb8dd6b34a4bdc243798e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:26:49.682 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:26:49 smithi112 podman[74853]: 2024-09-21 03:26:49.584679337 +0000 UTC m=+0.389460608 container start a02fbf2e7356a0ffc52c4b70efe08322e9f27eb147fb8dd6b34a4bdc243798e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T03:26:49.683 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:26:49 smithi112 ceph-mds[74897]: -- 172.21.15.112:0/4233940382 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2bcd27860 con 0x55d2c0ecf600 2024-09-21T03:26:49.683 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:26:49 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-a[74882]: starting mds.a at 2024-09-21T03:26:49.683 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:26:49 smithi112 bash[74853]: a02fbf2e7356a0ffc52c4b70efe08322e9f27eb147fb8dd6b34a4bdc243798e3 2024-09-21T03:26:49.723 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:50.001 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:26:49 smithi112 systemd[1]: Started Ceph mds.a for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.223+0000 7f10186dd640 1 -- 172.21.15.112:0/3999874063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1010103b60 msgr2=0x7f1010103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.223+0000 7f10186dd640 1 --2- 172.21.15.112:0/3999874063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1010103b60 0x7f1010103f70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1004009900 tx=0x7f100402a410 comp rx=0 tx=0).stop 2024-09-21T03:26:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.223+0000 7f10186dd640 1 -- 172.21.15.112:0/3999874063 shutdown_connections 2024-09-21T03:26:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.223+0000 7f10186dd640 1 --2- 172.21.15.112:0/3999874063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1010109df0 0x7f1010110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.223+0000 7f10186dd640 1 --2- 172.21.15.112:0/3999874063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10101094a0 0x7f10101098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.223+0000 7f10186dd640 1 --2- 172.21.15.112:0/3999874063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1010103b60 0x7f1010103f70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.223+0000 7f10186dd640 1 -- 172.21.15.112:0/3999874063 >> 172.21.15.112:0/3999874063 conn(0x7f101007b850 msgr2=0x7f101007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.223+0000 7f10186dd640 1 -- 172.21.15.112:0/3999874063 shutdown_connections 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.223+0000 7f10186dd640 1 -- 172.21.15.112:0/3999874063 wait complete. 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.224+0000 7f10186dd640 1 Processor -- start 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.224+0000 7f10186dd640 1 -- start start 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.224+0000 7f10186dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1010103b60 0x7f101019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.224+0000 7f10186dd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10101094a0 0x7f101019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.224+0000 7f10186dd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1010109df0 0x7f101019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.224+0000 7f10186dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1010113b30 con 0x7f1010103b60 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.224+0000 7f10186dd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f10101139b0 con 0x7f10101094a0 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.224+0000 7f10186dd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1010113cb0 con 0x7f1010109df0 2024-09-21T03:26:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.224+0000 7f1016479640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1010103b60 0x7f101019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f1016479640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1010103b60 0x7f101019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58070/0 (socket says 172.21.15.112:58070) 2024-09-21T03:26:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f1016479640 1 -- 172.21.15.112:0/1743173133 learned_addr learned my addr 172.21.15.112:0/1743173133 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f1016c7a640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1010109df0 0x7f101019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f1015c78640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10101094a0 0x7f101019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f1016479640 1 -- 172.21.15.112:0/1743173133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1010109df0 msgr2=0x7f101019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f1016479640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1010109df0 0x7f101019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f1016479640 1 -- 172.21.15.112:0/1743173133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10101094a0 msgr2=0x7f101019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f1016479640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10101094a0 0x7f101019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f1016479640 1 -- 172.21.15.112:0/1743173133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10101a0490 con 0x7f1010103b60 2024-09-21T03:26:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f1016479640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1010103b60 0x7f101019a520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1004005940 tx=0x7f10040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f10037fe640 1 -- 172.21.15.112:0/1743173133 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100403f030 con 0x7f1010103b60 2024-09-21T03:26:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f10186dd640 1 -- 172.21.15.112:0/1743173133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10101a0720 con 0x7f1010103b60 2024-09-21T03:26:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f10037fe640 1 -- 172.21.15.112:0/1743173133 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1004003cb0 con 0x7f1010103b60 2024-09-21T03:26:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f10037fe640 1 -- 172.21.15.112:0/1743173133 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004035070 con 0x7f1010103b60 2024-09-21T03:26:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.225+0000 7f10186dd640 1 -- 172.21.15.112:0/1743173133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10101a6ee0 con 0x7f1010103b60 2024-09-21T03:26:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.226+0000 7f10186dd640 1 -- 172.21.15.112:0/1743173133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1010103f70 con 0x7f1010103b60 2024-09-21T03:26:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.228+0000 7f10037fe640 1 -- 172.21.15.112:0/1743173133 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f1004042030 con 0x7f1010103b60 2024-09-21T03:26:50.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.229+0000 7f10037fe640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ffc0b6560 0x7f0ffc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:50.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.229+0000 7f1015c78640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ffc0b6560 0x7f0ffc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:50.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.229+0000 7f10037fe640 1 -- 172.21.15.112:0/1743173133 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f1004107a00 con 0x7f1010103b60 2024-09-21T03:26:50.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.229+0000 7f1015c78640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ffc0b6560 0x7f0ffc0b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f10101091a0 tx=0x7f100c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.232+0000 7f10037fe640 1 -- 172.21.15.112:0/1743173133 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10040d2a40 con 0x7f1010103b60 2024-09-21T03:26:50.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.451+0000 7f10186dd640 1 -- 172.21.15.112:0/1743173133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f101019b650 con 0x7f1010103b60 2024-09-21T03:26:50.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.451+0000 7f10037fe640 1 -- 172.21.15.112:0/1743173133 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9716 (secure 0 0 0) 0x7f101019b650 con 0x7f1010103b60 2024-09-21T03:26:50.454 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:50.454 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":12,"btime":"2024-09-21T03:26:50:399519+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24647,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24685,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T03:26:50.455 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 12 2024-09-21T03:26:50.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 -- 172.21.15.112:0/1743173133 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ffc0b6560 msgr2=0x7f0ffc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ffc0b6560 0x7f0ffc0b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f10101091a0 tx=0x7f100c00b390 comp rx=0 tx=0).stop 2024-09-21T03:26:50.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 -- 172.21.15.112:0/1743173133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1010103b60 msgr2=0x7f101019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:50.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1010103b60 0x7f101019a520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1004005940 tx=0x7f10040042f0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 -- 172.21.15.112:0/1743173133 shutdown_connections 2024-09-21T03:26:50.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ffc0b6560 0x7f0ffc0b8a70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1010109df0 0x7f101019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10101094a0 0x7f101019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 --2- 172.21.15.112:0/1743173133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1010103b60 0x7f101019a520 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:50.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 -- 172.21.15.112:0/1743173133 >> 172.21.15.112:0/1743173133 conn(0x7f101007b850 msgr2=0x7f1010107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:50.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 -- 172.21.15.112:0/1743173133 shutdown_connections 2024-09-21T03:26:50.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:50.454+0000 7f10186dd640 1 -- 172.21.15.112:0/1743173133 wait complete. 2024-09-21T03:26:51.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:51.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:51.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:51.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:51.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:51.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:51.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: Deploying daemon mds.b on smithi187 2024-09-21T03:26:51.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:51.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.112:6840/395166022,v1:172.21.15.112:6841/395166022] up:boot 2024-09-21T03:26:51.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: fsmap 11 up:standby 2024-09-21T03:26:51.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:26:51.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:50 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1743173133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:51.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:51.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:51.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:51.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:51.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:51.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:51.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: Deploying daemon mds.b on smithi187 2024-09-21T03:26:51.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:51.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.112:6840/395166022,v1:172.21.15.112:6841/395166022] up:boot 2024-09-21T03:26:51.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: fsmap 11 up:standby 2024-09-21T03:26:51.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:26:51.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:50 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1743173133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:51.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:51.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:51.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:51.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T03:26:51.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T03:26:51.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:51.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: Deploying daemon mds.b on smithi187 2024-09-21T03:26:51.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:51.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.112:6840/395166022,v1:172.21.15.112:6841/395166022] up:boot 2024-09-21T03:26:51.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: fsmap 11 up:standby 2024-09-21T03:26:51.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:26:51.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:50 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1743173133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:51.929 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:26:51 smithi187 systemd[1]: Starting Ceph mds.b for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:26:52.287 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:26:52 smithi187 podman[63354]: 2024-09-21 03:26:51.912594215 +0000 UTC m=+0.015981535 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:26:52.287 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:26:52 smithi187 podman[63354]: 2024-09-21 03:26:52.038745123 +0000 UTC m=+0.142132432 container create dc4f3bac727d7fe4ba16f7983674415b6c34db1cf21bbbd51e0a8d11e5faa8a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T03:26:52.287 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:26:52 smithi187 podman[63354]: 2024-09-21 03:26:52.249480536 +0000 UTC m=+0.352867845 container init dc4f3bac727d7fe4ba16f7983674415b6c34db1cf21bbbd51e0a8d11e5faa8a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T03:26:52.287 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:26:52 smithi187 podman[63354]: 2024-09-21 03:26:52.252491053 +0000 UTC m=+0.355878371 container start dc4f3bac727d7fe4ba16f7983674415b6c34db1cf21bbbd51e0a8d11e5faa8a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T03:26:52.679 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:26:52 smithi187 ceph-mds[63373]: -- 172.21.15.187:0/3102958653 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b3fa7d860 con 0x563b43c25600 2024-09-21T03:26:52.679 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:26:52 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b[63369]: starting mds.b at 2024-09-21T03:26:52.679 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:26:52 smithi187 bash[63354]: dc4f3bac727d7fe4ba16f7983674415b6c34db1cf21bbbd51e0a8d11e5faa8a5 2024-09-21T03:26:52.679 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:26:52 smithi187 systemd[1]: Started Ceph mds.b for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:26:53.081 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:26:53.521 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:53.642 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:53 smithi112 ceph-mon[31442]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:53.642 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:53 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] up:boot 2024-09-21T03:26:53.643 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:53 smithi112 ceph-mon[31442]: fsmap 12 up:standby 2024-09-21T03:26:53.643 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:26:53.643 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.643 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.643 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.643 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.643 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.643 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:53.643 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:53 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:53 smithi203 ceph-mon[34723]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:53 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] up:boot 2024-09-21T03:26:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:53 smithi203 ceph-mon[34723]: fsmap 12 up:standby 2024-09-21T03:26:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:26:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:53 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:53 smithi187 ceph-mon[34603]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:53 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] up:boot 2024-09-21T03:26:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:53 smithi187 ceph-mon[34603]: fsmap 12 up:standby 2024-09-21T03:26:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:26:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:53.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:53 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:53.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.993+0000 7fb117cae640 1 -- 172.21.15.112:0/2815004786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1101083a0 msgr2=0x7fb1101087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:53.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.993+0000 7fb117cae640 1 --2- 172.21.15.112:0/2815004786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1101083a0 0x7fb1101087b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb100009900 tx=0x7fb10002a410 comp rx=0 tx=0).stop 2024-09-21T03:26:53.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.993+0000 7fb117cae640 1 -- 172.21.15.112:0/2815004786 shutdown_connections 2024-09-21T03:26:53.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.993+0000 7fb117cae640 1 --2- 172.21.15.112:0/2815004786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb110108cf0 0x7fb11010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.993+0000 7fb117cae640 1 --2- 172.21.15.112:0/2815004786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1101083a0 0x7fb1101087b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.993+0000 7fb117cae640 1 --2- 172.21.15.112:0/2815004786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110102a60 0x7fb110102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:53.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.993+0000 7fb117cae640 1 -- 172.21.15.112:0/2815004786 >> 172.21.15.112:0/2815004786 conn(0x7fb110079730 msgr2=0x7fb110079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:53.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.993+0000 7fb117cae640 1 -- 172.21.15.112:0/2815004786 shutdown_connections 2024-09-21T03:26:53.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.993+0000 7fb117cae640 1 -- 172.21.15.112:0/2815004786 wait complete. 2024-09-21T03:26:53.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb117cae640 1 Processor -- start 2024-09-21T03:26:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb117cae640 1 -- start start 2024-09-21T03:26:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb117cae640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110102a60 0x7fb11019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb117cae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1101083a0 0x7fb11019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb117cae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb110108cf0 0x7fb11019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb117cae640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb110112960 con 0x7fb110108cf0 2024-09-21T03:26:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb117cae640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1101127e0 con 0x7fb110102a60 2024-09-21T03:26:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb117cae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb110112ae0 con 0x7fb1101083a0 2024-09-21T03:26:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb11624b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb110108cf0 0x7fb11019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb11624b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb110108cf0 0x7fb11019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58100/0 (socket says 172.21.15.112:58100) 2024-09-21T03:26:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb11624b640 1 -- 172.21.15.112:0/1706991621 learned_addr learned my addr 172.21.15.112:0/1706991621 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.994+0000 7fb115249640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1101083a0 0x7fb11019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb115a4a640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110102a60 0x7fb11019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb11624b640 1 -- 172.21.15.112:0/1706991621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1101083a0 msgr2=0x7fb11019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb11624b640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1101083a0 0x7fb11019b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb11624b640 1 -- 172.21.15.112:0/1706991621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110102a60 msgr2=0x7fb11019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb11624b640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110102a60 0x7fb11019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb11624b640 1 -- 172.21.15.112:0/1706991621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1101a6520 con 0x7fb110108cf0 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb115a4a640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110102a60 0x7fb11019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb11624b640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb110108cf0 0x7fb11019bc90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb10800eaa0 tx=0x7fb10800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb106ffd640 1 -- 172.21.15.112:0/1706991621 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10800cdb0 con 0x7fb110108cf0 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb106ffd640 1 -- 172.21.15.112:0/1706991621 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb108005020 con 0x7fb110108cf0 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb106ffd640 1 -- 172.21.15.112:0/1706991621 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1080116d0 con 0x7fb110108cf0 2024-09-21T03:26:54.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb117cae640 1 -- 172.21.15.112:0/1706991621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1101a6810 con 0x7fb110108cf0 2024-09-21T03:26:54.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.995+0000 7fb117cae640 1 -- 172.21.15.112:0/1706991621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1101a6cd0 con 0x7fb110108cf0 2024-09-21T03:26:54.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.996+0000 7fb117cae640 1 -- 172.21.15.112:0/1706991621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb110103ce0 con 0x7fb110108cf0 2024-09-21T03:26:54.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.998+0000 7fb106ffd640 1 -- 172.21.15.112:0/1706991621 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fb1080051d0 con 0x7fb110108cf0 2024-09-21T03:26:54.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.998+0000 7fb106ffd640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb0fc0b6560 0x7fb0fc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:54.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.999+0000 7fb115a4a640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb0fc0b6560 0x7fb0fc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:54.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.999+0000 7fb106ffd640 1 -- 172.21.15.112:0/1706991621 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7fb1080e7540 con 0x7fb110108cf0 2024-09-21T03:26:54.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:53.999+0000 7fb115a4a640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb0fc0b6560 0x7fb0fc0b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb10c005fd0 tx=0x7fb10c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:54.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.003+0000 7fb106ffd640 1 -- 172.21.15.112:0/1706991621 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb108016030 con 0x7fb110108cf0 2024-09-21T03:26:54.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.251+0000 7fb117cae640 1 -- 172.21.15.112:0/1706991621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb110102e70 con 0x7fb110108cf0 2024-09-21T03:26:54.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.251+0000 7fb106ffd640 1 -- 172.21.15.112:0/1706991621 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10542 (secure 0 0 0) 0x7fb1080ae1a0 con 0x7fb110108cf0 2024-09-21T03:26:54.254 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:54.255 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":13,"btime":"2024-09-21T03:26:52:341894+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24647,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24658,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24685,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T03:26:54.255 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 13 2024-09-21T03:26:54.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 -- 172.21.15.112:0/1706991621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb0fc0b6560 msgr2=0x7fb0fc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:54.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb0fc0b6560 0x7fb0fc0b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb10c005fd0 tx=0x7fb10c005850 comp rx=0 tx=0).stop 2024-09-21T03:26:54.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 -- 172.21.15.112:0/1706991621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb110108cf0 msgr2=0x7fb11019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:54.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb110108cf0 0x7fb11019bc90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb10800eaa0 tx=0x7fb10800ef60 comp rx=0 tx=0).stop 2024-09-21T03:26:54.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 -- 172.21.15.112:0/1706991621 shutdown_connections 2024-09-21T03:26:54.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb0fc0b6560 0x7fb0fc0b8a70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:54.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb110108cf0 0x7fb11019bc90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:54.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1101083a0 0x7fb11019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:54.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 --2- 172.21.15.112:0/1706991621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110102a60 0x7fb11019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:54.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 -- 172.21.15.112:0/1706991621 >> 172.21.15.112:0/1706991621 conn(0x7fb110079730 msgr2=0x7fb110104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:54.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 -- 172.21.15.112:0/1706991621 shutdown_connections 2024-09-21T03:26:54.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:54.254+0000 7fb117cae640 1 -- 172.21.15.112:0/1706991621 wait complete. 2024-09-21T03:26:54.925 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1706991621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:54 smithi112 ceph-mon[31442]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:54.953 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T03:26:55.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1706991621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:55.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:55.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:55.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:55.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:54 smithi203 ceph-mon[34723]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:55.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1706991621' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:26:55.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:55.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:55.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:55.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:26:55.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:54 smithi187 ceph-mon[34603]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:55.815 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:56.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.243+0000 7fdc9024b640 1 -- 172.21.15.112:0/1977262298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc88102ae0 msgr2=0x7fdc88102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:56.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.243+0000 7fdc9024b640 1 --2- 172.21.15.112:0/1977262298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc88102ae0 0x7fdc88102ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009980 tx=0x7fdc8402c620 comp rx=0 tx=0).stop 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.243+0000 7fdc9024b640 1 -- 172.21.15.112:0/1977262298 shutdown_connections 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.243+0000 7fdc9024b640 1 --2- 172.21.15.112:0/1977262298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc88108d20 0x7fdc8810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.243+0000 7fdc9024b640 1 --2- 172.21.15.112:0/1977262298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc881083d0 0x7fdc881087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.243+0000 7fdc9024b640 1 --2- 172.21.15.112:0/1977262298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc88102ae0 0x7fdc88102ef0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.243+0000 7fdc9024b640 1 -- 172.21.15.112:0/1977262298 >> 172.21.15.112:0/1977262298 conn(0x7fdc88079760 msgr2=0x7fdc88079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.243+0000 7fdc9024b640 1 -- 172.21.15.112:0/1977262298 shutdown_connections 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.244+0000 7fdc9024b640 1 -- 172.21.15.112:0/1977262298 wait complete. 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.244+0000 7fdc9024b640 1 Processor -- start 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.244+0000 7fdc9024b640 1 -- start start 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc9024b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc88102ae0 0x7fdc881982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc9024b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc881083d0 0x7fdc88198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:56.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc9024b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc88108d20 0x7fdc8819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc9024b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88111890 con 0x7fdc88102ae0 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc9024b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88111710 con 0x7fdc88108d20 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc9024b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88111a10 con 0x7fdc881083d0 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8dfe7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc88102ae0 0x7fdc881982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8dfe7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc88102ae0 0x7fdc881982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44456/0 (socket says 172.21.15.112:44456) 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8dfe7640 1 -- 172.21.15.112:0/2648214911 learned_addr learned my addr 172.21.15.112:0/2648214911 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8d7e6640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc881083d0 0x7fdc88198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8e7e8640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc88108d20 0x7fdc8819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8dfe7640 1 -- 172.21.15.112:0/2648214911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc881083d0 msgr2=0x7fdc88198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8dfe7640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc881083d0 0x7fdc88198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8dfe7640 1 -- 172.21.15.112:0/2648214911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc88108d20 msgr2=0x7fdc8819db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8dfe7640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc88108d20 0x7fdc8819db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:56.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8dfe7640 1 -- 172.21.15.112:0/2648214911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8819e250 con 0x7fdc88102ae0 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8e7e8640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc88108d20 0x7fdc8819db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc8dfe7640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc88102ae0 0x7fdc881982e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc84005940 tx=0x7fdc840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.245+0000 7fdc7effd640 1 -- 172.21.15.112:0/2648214911 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84041030 con 0x7fdc88102ae0 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.246+0000 7fdc7effd640 1 -- 172.21.15.112:0/2648214911 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc84003cb0 con 0x7fdc88102ae0 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.246+0000 7fdc9024b640 1 -- 172.21.15.112:0/2648214911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8819e4e0 con 0x7fdc88102ae0 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.246+0000 7fdc7effd640 1 -- 172.21.15.112:0/2648214911 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84037070 con 0x7fdc88102ae0 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.246+0000 7fdc9024b640 1 -- 172.21.15.112:0/2648214911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc881a4ca0 con 0x7fdc88102ae0 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.247+0000 7fdc9024b640 1 -- 172.21.15.112:0/2648214911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc88102ef0 con 0x7fdc88102ae0 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.248+0000 7fdc7effd640 1 -- 172.21.15.112:0/2648214911 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fdc84044030 con 0x7fdc88102ae0 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.249+0000 7fdc7effd640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc740b6560 0x7fdc740b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.249+0000 7fdc8d7e6640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc740b6560 0x7fdc740b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.249+0000 7fdc7effd640 1 -- 172.21.15.112:0/2648214911 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7fdc84109870 con 0x7fdc88102ae0 2024-09-21T03:26:56.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.249+0000 7fdc8d7e6640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc740b6560 0x7fdc740b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdc88079d00 tx=0x7fdc7800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:56.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.258+0000 7fdc7effd640 1 -- 172.21.15.112:0/2648214911 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdc840d04d0 con 0x7fdc88102ae0 2024-09-21T03:26:56.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.488+0000 7fdc9024b640 1 -- 172.21.15.112:0/2648214911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdc8810aaf0 con 0x7fdc88102ae0 2024-09-21T03:26:56.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.489+0000 7fdc7effd640 1 -- 172.21.15.112:0/2648214911 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v69) ==== 84+0+65 (secure 0 0 0) 0x7fdc840d48b0 con 0x7fdc88102ae0 2024-09-21T03:26:56.491 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:26:56.491 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-09-21T03:26:56.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:26:56.491 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-09-21T03:26:56.491 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-09-21T03:26:56.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:26:56.491 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-09-21T03:26:56.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 -- 172.21.15.112:0/2648214911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc740b6560 msgr2=0x7fdc740b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:56.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc740b6560 0x7fdc740b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdc88079d00 tx=0x7fdc7800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:26:56.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 -- 172.21.15.112:0/2648214911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc88102ae0 msgr2=0x7fdc881982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:56.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc88102ae0 0x7fdc881982e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc84005940 tx=0x7fdc840042f0 comp rx=0 tx=0).stop 2024-09-21T03:26:56.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 -- 172.21.15.112:0/2648214911 shutdown_connections 2024-09-21T03:26:56.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdc740b6560 0x7fdc740b8a70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:56.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc88108d20 0x7fdc8819db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:56.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc881083d0 0x7fdc88198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:56.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 --2- 172.21.15.112:0/2648214911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc88102ae0 0x7fdc881982e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:56.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 -- 172.21.15.112:0/2648214911 >> 172.21.15.112:0/2648214911 conn(0x7fdc88079760 msgr2=0x7fdc88101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:56.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 -- 172.21.15.112:0/2648214911 shutdown_connections 2024-09-21T03:26:56.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:56.491+0000 7fdc9024b640 1 -- 172.21.15.112:0/2648214911 wait complete. 2024-09-21T03:26:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:56 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:56 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:56 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:56 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:56 smithi203 ceph-mon[34723]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:56 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2648214911' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:26:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:56 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:56 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:56 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:56 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:56 smithi112 ceph-mon[31442]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:57.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:56 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2648214911' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:26:57.153 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T03:26:57.154 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T03:26:57.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:56 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:57.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:56 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:57.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:56 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:57.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:56 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:57.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:56 smithi187 ceph-mon[34603]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:57.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:56 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2648214911' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T03:26:57.438 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.895+0000 7fcce2a92640 1 -- 172.21.15.112:0/2723264995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccdc108420 msgr2=0x7fccdc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.895+0000 7fcce2a92640 1 --2- 172.21.15.112:0/2723264995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccdc108420 0x7fccdc108830 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009900 tx=0x7fccd002a410 comp rx=0 tx=0).stop 2024-09-21T03:26:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.895+0000 7fcce2a92640 1 -- 172.21.15.112:0/2723264995 shutdown_connections 2024-09-21T03:26:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.895+0000 7fcce2a92640 1 --2- 172.21.15.112:0/2723264995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccdc108d70 0x7fccdc10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.895+0000 7fcce2a92640 1 --2- 172.21.15.112:0/2723264995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccdc108420 0x7fccdc108830 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.895+0000 7fcce2a92640 1 --2- 172.21.15.112:0/2723264995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccdc102ae0 0x7fccdc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.895+0000 7fcce2a92640 1 -- 172.21.15.112:0/2723264995 >> 172.21.15.112:0/2723264995 conn(0x7fccdc079770 msgr2=0x7fccdc079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:57.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.895+0000 7fcce2a92640 1 -- 172.21.15.112:0/2723264995 shutdown_connections 2024-09-21T03:26:57.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.895+0000 7fcce2a92640 1 -- 172.21.15.112:0/2723264995 wait complete. 2024-09-21T03:26:57.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.895+0000 7fcce2a92640 1 Processor -- start 2024-09-21T03:26:57.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce2a92640 1 -- start start 2024-09-21T03:26:57.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce2a92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccdc102ae0 0x7fccdc19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:57.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce2a92640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccdc108420 0x7fccdc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:57.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce2a92640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccdc108d70 0x7fccdc19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:57.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce2a92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc112a30 con 0x7fccdc102ae0 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce2a92640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc1128b0 con 0x7fccdc108d70 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce2a92640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc112bb0 con 0x7fccdc108420 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce082e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccdc102ae0 0x7fccdc19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce082e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccdc102ae0 0x7fccdc19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44484/0 (socket says 172.21.15.112:44484) 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce082e640 1 -- 172.21.15.112:0/1460703992 learned_addr learned my addr 172.21.15.112:0/1460703992 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fccdbfff640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccdc108420 0x7fccdc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce102f640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccdc108d70 0x7fccdc19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce082e640 1 -- 172.21.15.112:0/1460703992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccdc108420 msgr2=0x7fccdc19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce082e640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccdc108420 0x7fccdc19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce082e640 1 -- 172.21.15.112:0/1460703992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccdc108d70 msgr2=0x7fccdc19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce082e640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccdc108d70 0x7fccdc19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce082e640 1 -- 172.21.15.112:0/1460703992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccdc1a0480 con 0x7fccdc102ae0 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.896+0000 7fcce082e640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccdc102ae0 0x7fccdc19a510 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00d950 tx=0x7fcccc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:57.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.897+0000 7fccd9ffb640 1 -- 172.21.15.112:0/1460703992 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc017070 con 0x7fccdc102ae0 2024-09-21T03:26:57.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.897+0000 7fccd9ffb640 1 -- 172.21.15.112:0/1460703992 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcccc009c30 con 0x7fccdc102ae0 2024-09-21T03:26:57.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.897+0000 7fccd9ffb640 1 -- 172.21.15.112:0/1460703992 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc00f620 con 0x7fccdc102ae0 2024-09-21T03:26:57.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.897+0000 7fcce2a92640 1 -- 172.21.15.112:0/1460703992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccdc1a0770 con 0x7fccdc102ae0 2024-09-21T03:26:57.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.897+0000 7fcce2a92640 1 -- 172.21.15.112:0/1460703992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccdc1a6ed0 con 0x7fccdc102ae0 2024-09-21T03:26:57.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.898+0000 7fcce2a92640 1 -- 172.21.15.112:0/1460703992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccdc103d60 con 0x7fccdc102ae0 2024-09-21T03:26:57.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.899+0000 7fccd9ffb640 1 -- 172.21.15.112:0/1460703992 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fcccc005030 con 0x7fccdc102ae0 2024-09-21T03:26:57.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.900+0000 7fccd9ffb640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fccc40b6560 0x7fccc40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:57.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.901+0000 7fccdbfff640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fccc40b6560 0x7fccc40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:57.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.901+0000 7fccd9ffb640 1 -- 172.21.15.112:0/1460703992 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7fcccc0e6b90 con 0x7fccdc102ae0 2024-09-21T03:26:57.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.901+0000 7fccdbfff640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fccc40b6560 0x7fccc40b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009230 tx=0x7fccd0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:57.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:57.904+0000 7fccd9ffb640 1 -- 172.21.15.112:0/1460703992 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcccc009de0 con 0x7fccdc102ae0 2024-09-21T03:26:58.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.082+0000 7fcce2a92640 1 -- 172.21.15.112:0/1460703992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fccdc102ef0 con 0x7fccdc102ae0 2024-09-21T03:26:58.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.292+0000 7fccd9ffb640 1 -- 172.21.15.112:0/1460703992 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v70) ==== 138+0+0 (secure 0 0 0) 0x7fcccc0b1b30 con 0x7fccdc102ae0 2024-09-21T03:26:58.294 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-09-21T03:26:58.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.294+0000 7fcce2a92640 1 -- 172.21.15.112:0/1460703992 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fccc40b6560 msgr2=0x7fccc40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:58.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.294+0000 7fcce2a92640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fccc40b6560 0x7fccc40b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009230 tx=0x7fccd0031400 comp rx=0 tx=0).stop 2024-09-21T03:26:58.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.294+0000 7fcce2a92640 1 -- 172.21.15.112:0/1460703992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccdc102ae0 msgr2=0x7fccdc19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:58.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.294+0000 7fcce2a92640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccdc102ae0 0x7fccdc19a510 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00d950 tx=0x7fcccc00de10 comp rx=0 tx=0).stop 2024-09-21T03:26:58.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.295+0000 7fcce2a92640 1 -- 172.21.15.112:0/1460703992 shutdown_connections 2024-09-21T03:26:58.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.295+0000 7fcce2a92640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fccc40b6560 0x7fccc40b8a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.295+0000 7fcce2a92640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccdc108d70 0x7fccdc19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.295+0000 7fcce2a92640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccdc108420 0x7fccdc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.295+0000 7fcce2a92640 1 --2- 172.21.15.112:0/1460703992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccdc102ae0 0x7fccdc19a510 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:58.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.295+0000 7fcce2a92640 1 -- 172.21.15.112:0/1460703992 >> 172.21.15.112:0/1460703992 conn(0x7fccdc079770 msgr2=0x7fccdc101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:58.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.295+0000 7fcce2a92640 1 -- 172.21.15.112:0/1460703992 shutdown_connections 2024-09-21T03:26:58.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:58.295+0000 7fcce2a92640 1 -- 172.21.15.112:0/1460703992 wait complete. 2024-09-21T03:26:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:58 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:58 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1460703992' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:26:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:58 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:58 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1460703992' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:26:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:26:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:26:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:58 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:26:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:58 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1460703992' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T03:26:59.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T03:26:59.251 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:26:59.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:59 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1460703992' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:26:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:59 smithi203 ceph-mon[34723]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:26:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:26:59 smithi203 ceph-mon[34723]: pgmap v251: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:59 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1460703992' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:26:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:59 smithi187 ceph-mon[34603]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:26:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:26:59 smithi187 ceph-mon[34603]: pgmap v251: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:59.684 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:59 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1460703992' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T03:26:59.684 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:59 smithi112 ceph-mon[31442]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T03:26:59.684 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:26:59 smithi112 ceph-mon[31442]: pgmap v251: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:26:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.778+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/185063795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6dc103b60 msgr2=0x7fe6dc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.778+0000 7fe6e2bd1640 1 --2- 172.21.15.112:0/185063795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6dc103b60 0x7fe6dc103f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009900 tx=0x7fe6d802a410 comp rx=0 tx=0).stop 2024-09-21T03:26:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.778+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/185063795 shutdown_connections 2024-09-21T03:26:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.778+0000 7fe6e2bd1640 1 --2- 172.21.15.112:0/185063795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6dc109df0 0x7fe6dc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.779+0000 7fe6e2bd1640 1 --2- 172.21.15.112:0/185063795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6dc1094a0 0x7fe6dc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.779+0000 7fe6e2bd1640 1 --2- 172.21.15.112:0/185063795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6dc103b60 0x7fe6dc103f70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.779+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/185063795 >> 172.21.15.112:0/185063795 conn(0x7fe6dc07b850 msgr2=0x7fe6dc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.779+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/185063795 shutdown_connections 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.779+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/185063795 wait complete. 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.779+0000 7fe6e2bd1640 1 Processor -- start 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.779+0000 7fe6e2bd1640 1 -- start start 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e2bd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6dc103b60 0x7fe6dc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e2bd1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6dc1094a0 0x7fe6dc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e2bd1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6dc109df0 0x7fe6dc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e2bd1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc113ab0 con 0x7fe6dc103b60 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e2bd1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc113930 con 0x7fe6dc109df0 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e2bd1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc113c30 con 0x7fe6dc1094a0 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e096d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6dc103b60 0x7fe6dc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e096d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6dc103b60 0x7fe6dc19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44494/0 (socket says 172.21.15.112:44494) 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e096d640 1 -- 172.21.15.112:0/1341621925 learned_addr learned my addr 172.21.15.112:0/1341621925 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6d3fff640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6dc1094a0 0x7fe6dc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e116e640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6dc109df0 0x7fe6dc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e096d640 1 -- 172.21.15.112:0/1341621925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6dc1094a0 msgr2=0x7fe6dc19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e096d640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6dc1094a0 0x7fe6dc19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e096d640 1 -- 172.21.15.112:0/1341621925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6dc109df0 msgr2=0x7fe6dc19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e096d640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6dc109df0 0x7fe6dc19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e096d640 1 -- 172.21.15.112:0/1341621925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc1a0430 con 0x7fe6dc103b60 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e096d640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6dc103b60 0x7fe6dc19a4c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8005940 tx=0x7fe6d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6d1ffb640 1 -- 172.21.15.112:0/1341621925 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d803f030 con 0x7fe6dc103b60 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6d1ffb640 1 -- 172.21.15.112:0/1341621925 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d8003cb0 con 0x7fe6dc103b60 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/1341621925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc1a06c0 con 0x7fe6dc103b60 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.780+0000 7fe6d1ffb640 1 -- 172.21.15.112:0/1341621925 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8035070 con 0x7fe6dc103b60 2024-09-21T03:26:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.781+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/1341621925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc1a6e80 con 0x7fe6dc103b60 2024-09-21T03:26:59.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.782+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/1341621925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6dc104de0 con 0x7fe6dc103b60 2024-09-21T03:26:59.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.784+0000 7fe6d1ffb640 1 -- 172.21.15.112:0/1341621925 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fe6d8042030 con 0x7fe6dc103b60 2024-09-21T03:26:59.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.785+0000 7fe6d1ffb640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6c40b6560 0x7fe6c40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:26:59.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.785+0000 7fe6d3fff640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6c40b6560 0x7fe6c40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:26:59.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.785+0000 7fe6d1ffb640 1 -- 172.21.15.112:0/1341621925 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7712+0+0 (secure 0 0 0) 0x7fe6d8107ca0 con 0x7fe6dc103b60 2024-09-21T03:26:59.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.786+0000 7fe6d3fff640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6c40b6560 0x7fe6c40b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc1062a0 tx=0x7fe6cc00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:26:59.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.789+0000 7fe6d1ffb640 1 -- 172.21.15.112:0/1341621925 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6d80ce710 con 0x7fe6dc103b60 2024-09-21T03:26:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:26:59.965+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/1341621925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe6dc103f70 con 0x7fe6dc103b60 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.305+0000 7fe6d1ffb640 1 -- 172.21.15.112:0/1341621925 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) ==== 170+0+0 (secure 0 0 0) 0x7fe6d80d2af0 con 0x7fe6dc103b60 2024-09-21T03:27:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/1341621925 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6c40b6560 msgr2=0x7fe6c40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6c40b6560 0x7fe6c40b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc1062a0 tx=0x7fe6cc00b390 comp rx=0 tx=0).stop 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/1341621925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6dc103b60 msgr2=0x7fe6dc19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6dc103b60 0x7fe6dc19a4c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8005940 tx=0x7fe6d80042f0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/1341621925 shutdown_connections 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6c40b6560 0x7fe6c40b8a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6dc109df0 0x7fe6dc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6dc1094a0 0x7fe6dc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 --2- 172.21.15.112:0/1341621925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6dc103b60 0x7fe6dc19a4c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/1341621925 >> 172.21.15.112:0/1341621925 conn(0x7fe6dc07b850 msgr2=0x7fe6dc107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/1341621925 shutdown_connections 2024-09-21T03:27:00.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:00.308+0000 7fe6e2bd1640 1 -- 172.21.15.112:0/1341621925 wait complete. 2024-09-21T03:27:00.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:00 smithi203 ceph-mon[34723]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:27:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:00 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1341621925' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:27:00.670 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:00 smithi112 ceph-mon[31442]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:27:00.670 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:00 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1341621925' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:27:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:00 smithi187 ceph-mon[34603]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T03:27:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:00 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1341621925' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T03:27:00.969 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T03:27:01.214 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:01.570 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:01 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1341621925' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:27:01.570 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:01 smithi112 ceph-mon[31442]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:27:01.570 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:01 smithi112 ceph-mon[31442]: pgmap v254: 97 pgs: 3 creating+peering, 85 unknown, 9 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:01.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:01 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1341621925' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:27:01.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:01 smithi203 ceph-mon[34723]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:27:01.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:01 smithi203 ceph-mon[34723]: pgmap v254: 97 pgs: 3 creating+peering, 85 unknown, 9 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:01.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.661+0000 7ff0dc143640 1 -- 172.21.15.112:0/1755096355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d41049e0 msgr2=0x7ff0d4104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.661+0000 7ff0dc143640 1 --2- 172.21.15.112:0/1755096355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d41049e0 0x7ff0d4104df0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009ca0 tx=0x7ff0c402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.661+0000 7ff0dc143640 1 -- 172.21.15.112:0/1755096355 shutdown_connections 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.661+0000 7ff0dc143640 1 --2- 172.21.15.112:0/1755096355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0d410de40 0x7ff0d4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.661+0000 7ff0dc143640 1 --2- 172.21.15.112:0/1755096355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d4105330 0x7ff0d410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.661+0000 7ff0dc143640 1 --2- 172.21.15.112:0/1755096355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d41049e0 0x7ff0d4104df0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.661+0000 7ff0dc143640 1 -- 172.21.15.112:0/1755096355 >> 172.21.15.112:0/1755096355 conn(0x7ff0d4079730 msgr2=0x7ff0d4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.661+0000 7ff0dc143640 1 -- 172.21.15.112:0/1755096355 shutdown_connections 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.661+0000 7ff0dc143640 1 -- 172.21.15.112:0/1755096355 wait complete. 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0dc143640 1 Processor -- start 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0dc143640 1 -- start start 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0dc143640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d41049e0 0x7ff0d419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:01.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0dc143640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0d4105330 0x7ff0d419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0dc143640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d410de40 0x7ff0d419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0dc143640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d4113b20 con 0x7ff0d41049e0 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0dc143640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41139a0 con 0x7ff0d410de40 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0dc143640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d4113ca0 con 0x7ff0d4105330 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0d9edf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d41049e0 0x7ff0d419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0d9edf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d41049e0 0x7ff0d419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44518/0 (socket says 172.21.15.112:44518) 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0d9edf640 1 -- 172.21.15.112:0/1103519183 learned_addr learned my addr 172.21.15.112:0/1103519183 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0da6e0640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d410de40 0x7ff0d419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0d9edf640 1 -- 172.21.15.112:0/1103519183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0d4105330 msgr2=0x7ff0d419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0d9edf640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0d4105330 0x7ff0d419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0d9edf640 1 -- 172.21.15.112:0/1103519183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d410de40 msgr2=0x7ff0d419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.662+0000 7ff0d9edf640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d410de40 0x7ff0d419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:01.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.663+0000 7ff0d9edf640 1 -- 172.21.15.112:0/1103519183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d41a03e0 con 0x7ff0d41049e0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.663+0000 7ff0d9edf640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d41049e0 0x7ff0d419a530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff0c402ade0 tx=0x7ff0c4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.663+0000 7ff0d2ffd640 1 -- 172.21.15.112:0/1103519183 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4003d60 con 0x7ff0d41049e0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.663+0000 7ff0d2ffd640 1 -- 172.21.15.112:0/1103519183 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c40044f0 con 0x7ff0d41049e0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.663+0000 7ff0dc143640 1 -- 172.21.15.112:0/1103519183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d41a0670 con 0x7ff0d41049e0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.663+0000 7ff0d2ffd640 1 -- 172.21.15.112:0/1103519183 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c402fd10 con 0x7ff0d41049e0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.663+0000 7ff0dc143640 1 -- 172.21.15.112:0/1103519183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d41a6ef0 con 0x7ff0d41049e0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.664+0000 7ff0dc143640 1 -- 172.21.15.112:0/1103519183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d406c280 con 0x7ff0d41049e0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.666+0000 7ff0d2ffd640 1 -- 172.21.15.112:0/1103519183 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7ff0c40046a0 con 0x7ff0d41049e0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.666+0000 7ff0d2ffd640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0bc0b6590 0x7ff0bc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.667+0000 7ff0d96de640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0bc0b6590 0x7ff0bc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.667+0000 7ff0d2ffd640 1 -- 172.21.15.112:0/1103519183 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8094+0+0 (secure 0 0 0) 0x7ff0c4108a00 con 0x7ff0d41049e0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.667+0000 7ff0d96de640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0bc0b6590 0x7ff0bc0b8aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff0d410fad0 tx=0x7ff0c8005c70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:01.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.670+0000 7ff0d2ffd640 1 -- 172.21.15.112:0/1103519183 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff0c40cf370 con 0x7ff0d41049e0 2024-09-21T03:27:01.682 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:01 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1341621925' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T03:27:01.682 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:01 smithi187 ceph-mon[34603]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T03:27:01.682 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:01 smithi187 ceph-mon[34603]: pgmap v254: 97 pgs: 3 creating+peering, 85 unknown, 9 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:01.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:01.885+0000 7ff0dc143640 1 -- 172.21.15.112:0/1103519183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff0d4104df0 con 0x7ff0d41049e0 2024-09-21T03:27:02.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.313+0000 7ff0d2ffd640 1 -- 172.21.15.112:0/1103519183 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ff0c40d3750 con 0x7ff0d41049e0 2024-09-21T03:27:02.315 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T03:27:02.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.316+0000 7ff0dc143640 1 -- 172.21.15.112:0/1103519183 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0bc0b6590 msgr2=0x7ff0bc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:02.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.316+0000 7ff0dc143640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0bc0b6590 0x7ff0bc0b8aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff0d410fad0 tx=0x7ff0c8005c70 comp rx=0 tx=0).stop 2024-09-21T03:27:02.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.316+0000 7ff0dc143640 1 -- 172.21.15.112:0/1103519183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d41049e0 msgr2=0x7ff0d419a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:02.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.316+0000 7ff0dc143640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d41049e0 0x7ff0d419a530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff0c402ade0 tx=0x7ff0c4003970 comp rx=0 tx=0).stop 2024-09-21T03:27:02.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.316+0000 7ff0dc143640 1 -- 172.21.15.112:0/1103519183 shutdown_connections 2024-09-21T03:27:02.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.316+0000 7ff0dc143640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0bc0b6590 0x7ff0bc0b8aa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:02.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.316+0000 7ff0dc143640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d410de40 0x7ff0d419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:02.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.316+0000 7ff0dc143640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0d4105330 0x7ff0d419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:02.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.316+0000 7ff0dc143640 1 --2- 172.21.15.112:0/1103519183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d41049e0 0x7ff0d419a530 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:02.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.316+0000 7ff0dc143640 1 -- 172.21.15.112:0/1103519183 >> 172.21.15.112:0/1103519183 conn(0x7ff0d4079730 msgr2=0x7ff0d4107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:02.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.317+0000 7ff0dc143640 1 -- 172.21.15.112:0/1103519183 shutdown_connections 2024-09-21T03:27:02.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:02.317+0000 7ff0dc143640 1 -- 172.21.15.112:0/1103519183 wait complete. 2024-09-21T03:27:02.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:02 smithi203 ceph-mon[34723]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:27:02.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:02 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1103519183' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:27:02.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:02 smithi203 ceph-mon[34723]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-21T03:27:02.675 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:02 smithi112 ceph-mon[31442]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:27:02.675 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:02 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1103519183' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:27:02.675 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:02 smithi112 ceph-mon[31442]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-21T03:27:02.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:02 smithi187 ceph-mon[34603]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T03:27:02.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:02 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1103519183' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T03:27:02.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:02 smithi187 ceph-mon[34603]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-21T03:27:03.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:27:03.261 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:03.620 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:03 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1103519183' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:27:03.620 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:03 smithi112 ceph-mon[31442]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:27:03.620 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:03 smithi112 ceph-mon[31442]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-21T03:27:03.620 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:03 smithi112 ceph-mon[31442]: pgmap v257: 97 pgs: 31 creating+peering, 18 unknown, 48 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:03.621 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:03 smithi112 ceph-mon[31442]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-21T03:27:03.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:03 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1103519183' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:27:03.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:03 smithi203 ceph-mon[34723]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:27:03.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:03 smithi203 ceph-mon[34723]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-21T03:27:03.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:03 smithi203 ceph-mon[34723]: pgmap v257: 97 pgs: 31 creating+peering, 18 unknown, 48 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:03.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:03 smithi203 ceph-mon[34723]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-21T03:27:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:03 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1103519183' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T03:27:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:03 smithi187 ceph-mon[34603]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T03:27:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:03 smithi187 ceph-mon[34603]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-21T03:27:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:03 smithi187 ceph-mon[34603]: pgmap v257: 97 pgs: 31 creating+peering, 18 unknown, 48 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:27:03.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:03 smithi187 ceph-mon[34603]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-21T03:27:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.706+0000 7febf9adc640 1 -- 172.21.15.112:0/2384573532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf41094f0 msgr2=0x7febf4109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:03.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.706+0000 7febf9adc640 1 --2- 172.21.15.112:0/2384573532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf41094f0 0x7febf4109900 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7febe8009900 tx=0x7febe802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.707+0000 7febf9adc640 1 -- 172.21.15.112:0/2384573532 shutdown_connections 2024-09-21T03:27:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.707+0000 7febf9adc640 1 --2- 172.21.15.112:0/2384573532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febf4109e40 0x7febf4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.707+0000 7febf9adc640 1 --2- 172.21.15.112:0/2384573532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf41094f0 0x7febf4109900 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.707+0000 7febf9adc640 1 --2- 172.21.15.112:0/2384573532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf4103bb0 0x7febf4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.707+0000 7febf9adc640 1 -- 172.21.15.112:0/2384573532 >> 172.21.15.112:0/2384573532 conn(0x7febf40ff600 msgr2=0x7febf4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.707+0000 7febf9adc640 1 -- 172.21.15.112:0/2384573532 shutdown_connections 2024-09-21T03:27:03.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.707+0000 7febf9adc640 1 -- 172.21.15.112:0/2384573532 wait complete. 2024-09-21T03:27:03.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.707+0000 7febf9adc640 1 Processor -- start 2024-09-21T03:27:03.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.707+0000 7febf9adc640 1 -- start start 2024-09-21T03:27:03.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf9adc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febf4103bb0 0x7febf419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:03.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf9adc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf41094f0 0x7febf419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:03.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf9adc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf4109e40 0x7febf419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:03.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf9adc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febf4113b00 con 0x7febf4109e40 2024-09-21T03:27:03.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf9adc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7febf4113980 con 0x7febf41094f0 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf9adc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7febf4113c80 con 0x7febf4103bb0 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf4109e40 0x7febf419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf4109e40 0x7febf419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44524/0 (socket says 172.21.15.112:44524) 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf3fff640 1 -- 172.21.15.112:0/2272704157 learned_addr learned my addr 172.21.15.112:0/2272704157 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf2ffd640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf41094f0 0x7febf419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf37fe640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febf4103bb0 0x7febf419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf3fff640 1 -- 172.21.15.112:0/2272704157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febf4103bb0 msgr2=0x7febf419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf3fff640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febf4103bb0 0x7febf419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf3fff640 1 -- 172.21.15.112:0/2272704157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf41094f0 msgr2=0x7febf419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf3fff640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf41094f0 0x7febf419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf3fff640 1 -- 172.21.15.112:0/2272704157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf41a0380 con 0x7febf4109e40 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.708+0000 7febf37fe640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febf4103bb0 0x7febf419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:03.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.709+0000 7febf3fff640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf4109e40 0x7febf419fd10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7febec00cce0 tx=0x7febec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.709+0000 7febf0ff9640 1 -- 172.21.15.112:0/2272704157 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec00f450 con 0x7febf4109e40 2024-09-21T03:27:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.709+0000 7febf0ff9640 1 -- 172.21.15.112:0/2272704157 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febec00faa0 con 0x7febf4109e40 2024-09-21T03:27:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.709+0000 7febf9adc640 1 -- 172.21.15.112:0/2272704157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf41a0670 con 0x7febf4109e40 2024-09-21T03:27:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.709+0000 7febf0ff9640 1 -- 172.21.15.112:0/2272704157 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec018b70 con 0x7febf4109e40 2024-09-21T03:27:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.709+0000 7febf9adc640 1 -- 172.21.15.112:0/2272704157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf41a0b30 con 0x7febf4109e40 2024-09-21T03:27:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.710+0000 7febf9adc640 1 -- 172.21.15.112:0/2272704157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf4103fc0 con 0x7febf4109e40 2024-09-21T03:27:03.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.712+0000 7febf0ff9640 1 -- 172.21.15.112:0/2272704157 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7febec018d10 con 0x7febf4109e40 2024-09-21T03:27:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.712+0000 7febf0ff9640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7febdc0b6590 0x7febdc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.712+0000 7febf37fe640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7febdc0b6590 0x7febdc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:03.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.713+0000 7febf0ff9640 1 -- 172.21.15.112:0/2272704157 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7febec0e7820 con 0x7febf4109e40 2024-09-21T03:27:03.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.713+0000 7febf37fe640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7febdc0b6590 0x7febdc0b8aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7febe40058c0 tx=0x7febe4005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:03.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.715+0000 7febf0ff9640 1 -- 172.21.15.112:0/2272704157 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7febec014030 con 0x7febf4109e40 2024-09-21T03:27:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.895+0000 7febf9adc640 1 -- 172.21.15.112:0/2272704157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febf402e640 con 0x7febf4109e40 2024-09-21T03:27:03.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.896+0000 7febf0ff9640 1 -- 172.21.15.112:0/2272704157 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+22632 (secure 0 0 0) 0x7febec0ae160 con 0x7febf4109e40 2024-09-21T03:27:03.898 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:27:03.899 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":74,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:27:01.886286+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:26:58.087349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:26:59.970444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:27:03.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 -- 172.21.15.112:0/2272704157 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7febdc0b6590 msgr2=0x7febdc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:03.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7febdc0b6590 0x7febdc0b8aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7febe40058c0 tx=0x7febe4005850 comp rx=0 tx=0).stop 2024-09-21T03:27:03.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 -- 172.21.15.112:0/2272704157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf4109e40 msgr2=0x7febf419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:03.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf4109e40 0x7febf419fd10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7febec00cce0 tx=0x7febec00a620 comp rx=0 tx=0).stop 2024-09-21T03:27:03.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 -- 172.21.15.112:0/2272704157 shutdown_connections 2024-09-21T03:27:03.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7febdc0b6590 0x7febdc0b8aa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:03.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf4109e40 0x7febf419fd10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:03.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf41094f0 0x7febf419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:03.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 --2- 172.21.15.112:0/2272704157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febf4103bb0 0x7febf419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 -- 172.21.15.112:0/2272704157 >> 172.21.15.112:0/2272704157 conn(0x7febf40ff600 msgr2=0x7febf4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 -- 172.21.15.112:0/2272704157 shutdown_connections 2024-09-21T03:27:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:03.899+0000 7febf9adc640 1 -- 172.21.15.112:0/2272704157 wait complete. 2024-09-21T03:27:04.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:04 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] up:active 2024-09-21T03:27:04.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:04 smithi203 ceph-mon[34723]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:04.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:04 smithi203 ceph-mon[34723]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:04.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:04 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2272704157' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:04.411 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:27:04 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i[64347]: 2024-09-21T03:27:04.100+0000 7f834f9e2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T03:27:04.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:04 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] up:active 2024-09-21T03:27:04.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:04 smithi112 ceph-mon[31442]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:04.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:04 smithi112 ceph-mon[31442]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:04.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:04 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2272704157' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:04.621 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:27:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:04 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] up:active 2024-09-21T03:27:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:04 smithi187 ceph-mon[34603]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:04 smithi187 ceph-mon[34603]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:04 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2272704157' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:04.868 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:05.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.333+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1646781683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6c0110200 msgr2=0x7fe6c0112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:05.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.333+0000 7fe6c5d4e640 1 --2- 172.21.15.112:0/1646781683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6c0110200 0x7fe6c0112620 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009ca0 tx=0x7fe6b802a950 comp rx=0 tx=0).stop 2024-09-21T03:27:05.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.333+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1646781683 shutdown_connections 2024-09-21T03:27:05.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.333+0000 7fe6c5d4e640 1 --2- 172.21.15.112:0/1646781683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6c0110200 0x7fe6c0112620 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:05.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.333+0000 7fe6c5d4e640 1 --2- 172.21.15.112:0/1646781683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c0102fa0 0x7fe6c010fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.333+0000 7fe6c5d4e640 1 --2- 172.21.15.112:0/1646781683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6c0102650 0x7fe6c0102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.333+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1646781683 >> 172.21.15.112:0/1646781683 conn(0x7fe6c0079740 msgr2=0x7fe6c0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.333+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1646781683 shutdown_connections 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.333+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1646781683 wait complete. 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.334+0000 7fe6c5d4e640 1 Processor -- start 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.334+0000 7fe6c5d4e640 1 -- start start 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6c5d4e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c0102650 0x7fe6c019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6c5d4e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6c0102fa0 0x7fe6c019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6c5d4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6c0110200 0x7fe6c01a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6c5d4e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0115cc0 con 0x7fe6c0110200 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6c5d4e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0115b40 con 0x7fe6c0102650 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6c5d4e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0115e40 con 0x7fe6c0102fa0 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6c0110200 0x7fe6c01a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:05.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6c0110200 0x7fe6c01a1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38010/0 (socket says 172.21.15.112:38010) 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6bffff640 1 -- 172.21.15.112:0/1791031949 learned_addr learned my addr 172.21.15.112:0/1791031949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6bf7fe640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c0102650 0x7fe6c019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6beffd640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6c0102fa0 0x7fe6c019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6bffff640 1 -- 172.21.15.112:0/1791031949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6c0102fa0 msgr2=0x7fe6c019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6bffff640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6c0102fa0 0x7fe6c019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6bffff640 1 -- 172.21.15.112:0/1791031949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c0102650 msgr2=0x7fe6c019c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6bffff640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c0102650 0x7fe6c019c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6bffff640 1 -- 172.21.15.112:0/1791031949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c01a2520 con 0x7fe6c0110200 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.335+0000 7fe6bffff640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6c0110200 0x7fe6c01a1df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009dc0 tx=0x7fe6b8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.337+0000 7fe6bcff9640 1 -- 172.21.15.112:0/1791031949 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8034040 con 0x7fe6c0110200 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.337+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1791031949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c01a27b0 con 0x7fe6c0110200 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.337+0000 7fe6bcff9640 1 -- 172.21.15.112:0/1791031949 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6b8030850 con 0x7fe6c0110200 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.337+0000 7fe6bcff9640 1 -- 172.21.15.112:0/1791031949 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b80047f0 con 0x7fe6c0110200 2024-09-21T03:27:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.337+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1791031949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01a8f70 con 0x7fe6c0110200 2024-09-21T03:27:05.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.338+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1791031949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c006c1e0 con 0x7fe6c0110200 2024-09-21T03:27:05.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.340+0000 7fe6bcff9640 1 -- 172.21.15.112:0/1791031949 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fe6b8037020 con 0x7fe6c0110200 2024-09-21T03:27:05.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.340+0000 7fe6bcff9640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6a80b6560 0x7fe6a80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:05.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.341+0000 7fe6bf7fe640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6a80b6560 0x7fe6a80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:05.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.341+0000 7fe6bcff9640 1 -- 172.21.15.112:0/1791031949 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7fe6b8108230 con 0x7fe6c0110200 2024-09-21T03:27:05.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.341+0000 7fe6bf7fe640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6a80b6560 0x7fe6a80b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0005d00 tx=0x7fe6b000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:05.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.344+0000 7fe6bcff9640 1 -- 172.21.15.112:0/1791031949 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6b80ceae0 con 0x7fe6c0110200 2024-09-21T03:27:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.526+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1791031949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6c0102a60 con 0x7fe6c0110200 2024-09-21T03:27:05.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:05 smithi112 ceph-mon[31442]: pgmap v258: 97 pgs: 28 creating+peering, 14 unknown, 55 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 610 B/s wr, 1 op/s 2024-09-21T03:27:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.528+0000 7fe6bcff9640 1 -- 172.21.15.112:0/1791031949 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+22632 (secure 0 0 0) 0x7fe6c0102a60 con 0x7fe6c0110200 2024-09-21T03:27:05.530 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:27:05.530 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":74,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:27:01.886286+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:26:58.087349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:26:59.970444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:27:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.530+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1791031949 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6a80b6560 msgr2=0x7fe6a80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.530+0000 7fe6c5d4e640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6a80b6560 0x7fe6a80b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0005d00 tx=0x7fe6b000b410 comp rx=0 tx=0).stop 2024-09-21T03:27:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.530+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1791031949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6c0110200 msgr2=0x7fe6c01a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.530+0000 7fe6c5d4e640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6c0110200 0x7fe6c01a1df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009dc0 tx=0x7fe6b8003f10 comp rx=0 tx=0).stop 2024-09-21T03:27:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.530+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1791031949 shutdown_connections 2024-09-21T03:27:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.530+0000 7fe6c5d4e640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6a80b6560 0x7fe6a80b8a70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.530+0000 7fe6c5d4e640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6c0110200 0x7fe6c01a1df0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.530+0000 7fe6c5d4e640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6c0102fa0 0x7fe6c019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.530+0000 7fe6c5d4e640 1 --2- 172.21.15.112:0/1791031949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c0102650 0x7fe6c019c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.530+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1791031949 >> 172.21.15.112:0/1791031949 conn(0x7fe6c0079740 msgr2=0x7fe6c0111500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.531+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1791031949 shutdown_connections 2024-09-21T03:27:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:05.531+0000 7fe6c5d4e640 1 -- 172.21.15.112:0/1791031949 wait complete. 2024-09-21T03:27:05.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:05 smithi203 ceph-mon[34723]: pgmap v258: 97 pgs: 28 creating+peering, 14 unknown, 55 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 610 B/s wr, 1 op/s 2024-09-21T03:27:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:05 smithi187 ceph-mon[34603]: pgmap v258: 97 pgs: 28 creating+peering, 14 unknown, 55 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 610 B/s wr, 1 op/s 2024-09-21T03:27:06.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T03:27:06.446 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:06.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:06 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1791031949' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:06.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:06 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1791031949' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:06.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:06 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1791031949' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:06.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.917+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/1093858953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e48108420 msgr2=0x7f0e48108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:06.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.917+0000 7f0e4ff0b640 1 --2- 172.21.15.112:0/1093858953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e48108420 0x7f0e48108830 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009900 tx=0x7f0e3802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:06.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.917+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/1093858953 shutdown_connections 2024-09-21T03:27:06.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.917+0000 7f0e4ff0b640 1 --2- 172.21.15.112:0/1093858953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e48108d70 0x7f0e4810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:06.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.917+0000 7f0e4ff0b640 1 --2- 172.21.15.112:0/1093858953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e48108420 0x7f0e48108830 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:06.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.917+0000 7f0e4ff0b640 1 --2- 172.21.15.112:0/1093858953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e48102ae0 0x7f0e48102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:06.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.917+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/1093858953 >> 172.21.15.112:0/1093858953 conn(0x7f0e48079770 msgr2=0x7f0e48079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:06.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.917+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/1093858953 shutdown_connections 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.917+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/1093858953 wait complete. 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.918+0000 7f0e4ff0b640 1 Processor -- start 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.918+0000 7f0e4ff0b640 1 -- start start 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.918+0000 7f0e4ff0b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e48102ae0 0x7f0e4819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.918+0000 7f0e4ff0b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e48108420 0x7f0e4819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.918+0000 7f0e4ff0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e48108d70 0x7f0e4819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.918+0000 7f0e4ff0b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e48112a30 con 0x7f0e48108d70 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.918+0000 7f0e4ff0b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e481128b0 con 0x7f0e48102ae0 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.918+0000 7f0e4ff0b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0e48112bb0 con 0x7f0e48108420 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.918+0000 7f0e4e4a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e48108d70 0x7f0e4819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4e4a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e48108d70 0x7f0e4819fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38022/0 (socket says 172.21.15.112:38022) 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4e4a8640 1 -- 172.21.15.112:0/83778057 learned_addr learned my addr 172.21.15.112:0/83778057 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4dca7640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e48102ae0 0x7f0e4819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:06.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4d4a6640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e48108420 0x7f0e4819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4e4a8640 1 -- 172.21.15.112:0/83778057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e48108420 msgr2=0x7f0e4819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4e4a8640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e48108420 0x7f0e4819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4e4a8640 1 -- 172.21.15.112:0/83778057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e48102ae0 msgr2=0x7f0e4819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4e4a8640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e48102ae0 0x7f0e4819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4e4a8640 1 -- 172.21.15.112:0/83778057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e481a04a0 con 0x7f0e48108d70 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4e4a8640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e48108d70 0x7f0e4819fd70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009fd0 tx=0x7f0e4000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e3effd640 1 -- 172.21.15.112:0/83778057 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40017070 con 0x7f0e48108d70 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e3effd640 1 -- 172.21.15.112:0/83778057 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e40003e20 con 0x7f0e48108d70 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/83778057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e481a0790 con 0x7f0e48108d70 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e3effd640 1 -- 172.21.15.112:0/83778057 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40010460 con 0x7f0e48108d70 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.919+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/83778057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e481a6ef0 con 0x7f0e48108d70 2024-09-21T03:27:06.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.920+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/83778057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e48103d60 con 0x7f0e48108d70 2024-09-21T03:27:06.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.922+0000 7f0e3effd640 1 -- 172.21.15.112:0/83778057 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f0e40003970 con 0x7f0e48108d70 2024-09-21T03:27:06.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.922+0000 7f0e3effd640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e340b6560 0x7f0e340b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:06.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.922+0000 7f0e4dca7640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e340b6560 0x7f0e340b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:06.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.923+0000 7f0e3effd640 1 -- 172.21.15.112:0/83778057 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f0e400e71b0 con 0x7f0e48108d70 2024-09-21T03:27:06.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.923+0000 7f0e4dca7640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e340b6560 0x7f0e340b8a70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e44005fd0 tx=0x7f0e44005e20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:06.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:06.926+0000 7f0e3effd640 1 -- 172.21.15.112:0/83778057 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e400ada60 con 0x7f0e48108d70 2024-09-21T03:27:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.144+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/83778057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0e48102ef0 con 0x7f0e48108d70 2024-09-21T03:27:07.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.341+0000 7f0e3effd640 1 -- 172.21.15.112:0/83778057 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f0e400b1e40 con 0x7f0e48108d70 2024-09-21T03:27:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.343+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/83778057 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e340b6560 msgr2=0x7f0e340b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.343+0000 7f0e4ff0b640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e340b6560 0x7f0e340b8a70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e44005fd0 tx=0x7f0e44005e20 comp rx=0 tx=0).stop 2024-09-21T03:27:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.343+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/83778057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e48108d70 msgr2=0x7f0e4819fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.343+0000 7f0e4ff0b640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e48108d70 0x7f0e4819fd70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009fd0 tx=0x7f0e4000b260 comp rx=0 tx=0).stop 2024-09-21T03:27:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.344+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/83778057 shutdown_connections 2024-09-21T03:27:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.344+0000 7f0e4ff0b640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e340b6560 0x7f0e340b8a70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.344+0000 7f0e4ff0b640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e48108d70 0x7f0e4819fd70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.344+0000 7f0e4ff0b640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e48108420 0x7f0e4819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.344+0000 7f0e4ff0b640 1 --2- 172.21.15.112:0/83778057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e48102ae0 0x7f0e4819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.344+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/83778057 >> 172.21.15.112:0/83778057 conn(0x7f0e48079770 msgr2=0x7f0e48101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.344+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/83778057 shutdown_connections 2024-09-21T03:27:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:07.344+0000 7f0e4ff0b640 1 -- 172.21.15.112:0/83778057 wait complete. 2024-09-21T03:27:07.402 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:07 smithi112 ceph-mon[31442]: pgmap v259: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-21T03:27:07.402 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:07 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/83778057' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:27:07.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:07 smithi203 ceph-mon[34723]: pgmap v259: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-21T03:27:07.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:07 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/83778057' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:27:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:07 smithi187 ceph-mon[34603]: pgmap v259: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-21T03:27:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:07 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/83778057' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T03:27:07.983 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T03:27:07.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 3 2024-09-21T03:27:08.229 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:08.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:08 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/83778057' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:27:08.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:08 smithi112 ceph-mon[31442]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:08.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:08 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/83778057' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:27:08.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:08 smithi203 ceph-mon[34723]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:08.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:08 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/83778057' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T03:27:08.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:08 smithi187 ceph-mon[34603]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:08.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.692+0000 7f37f53ef640 1 -- 172.21.15.112:0/3971859350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f0105330 msgr2=0x7f37f010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:08.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.692+0000 7f37f53ef640 1 --2- 172.21.15.112:0/3971859350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f0105330 0x7f37f010d880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009900 tx=0x7f37d802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:08.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.692+0000 7f37f53ef640 1 -- 172.21.15.112:0/3971859350 shutdown_connections 2024-09-21T03:27:08.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.692+0000 7f37f53ef640 1 --2- 172.21.15.112:0/3971859350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37f010de40 0x7f37f0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:08.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.692+0000 7f37f53ef640 1 --2- 172.21.15.112:0/3971859350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f0105330 0x7f37f010d880 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:08.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.692+0000 7f37f53ef640 1 --2- 172.21.15.112:0/3971859350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37f01049e0 0x7f37f0104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:08.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.692+0000 7f37f53ef640 1 -- 172.21.15.112:0/3971859350 >> 172.21.15.112:0/3971859350 conn(0x7f37f0079730 msgr2=0x7f37f0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:08.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.692+0000 7f37f53ef640 1 -- 172.21.15.112:0/3971859350 shutdown_connections 2024-09-21T03:27:08.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.692+0000 7f37f53ef640 1 -- 172.21.15.112:0/3971859350 wait complete. 2024-09-21T03:27:08.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.693+0000 7f37f53ef640 1 Processor -- start 2024-09-21T03:27:08.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.693+0000 7f37f53ef640 1 -- start start 2024-09-21T03:27:08.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.693+0000 7f37f53ef640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37f01049e0 0x7f37f019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:08.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.693+0000 7f37f53ef640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37f0105330 0x7f37f019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:08.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.693+0000 7f37f53ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f010de40 0x7f37f019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:08.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.693+0000 7f37f53ef640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0113ae0 con 0x7f37f010de40 2024-09-21T03:27:08.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.693+0000 7f37f53ef640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0113960 con 0x7f37f01049e0 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.693+0000 7f37f53ef640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0113c60 con 0x7f37f0105330 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37ef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f010de40 0x7f37f019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37ef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f010de40 0x7f37f019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38042/0 (socket says 172.21.15.112:38042) 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37ef7fe640 1 -- 172.21.15.112:0/1724088751 learned_addr learned my addr 172.21.15.112:0/1724088751 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37ee7fc640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37f0105330 0x7f37f019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37eeffd640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37f01049e0 0x7f37f019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37ef7fe640 1 -- 172.21.15.112:0/1724088751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37f0105330 msgr2=0x7f37f019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37ef7fe640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37f0105330 0x7f37f019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37ef7fe640 1 -- 172.21.15.112:0/1724088751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37f01049e0 msgr2=0x7f37f019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37ef7fe640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37f01049e0 0x7f37f019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37ef7fe640 1 -- 172.21.15.112:0/1724088751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f01a0480 con 0x7f37f010de40 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37ef7fe640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f010de40 0x7f37f019fd50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c970 tx=0x7f37e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:08.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37e7fff640 1 -- 172.21.15.112:0/1724088751 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8007bb0 con 0x7f37f010de40 2024-09-21T03:27:08.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37e7fff640 1 -- 172.21.15.112:0/1724088751 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e8007d50 con 0x7f37f010de40 2024-09-21T03:27:08.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37f53ef640 1 -- 172.21.15.112:0/1724088751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f01a0770 con 0x7f37f010de40 2024-09-21T03:27:08.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37e7fff640 1 -- 172.21.15.112:0/1724088751 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8004e60 con 0x7f37f010de40 2024-09-21T03:27:08.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.694+0000 7f37f53ef640 1 -- 172.21.15.112:0/1724088751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f01a6ed0 con 0x7f37f010de40 2024-09-21T03:27:08.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.695+0000 7f37f53ef640 1 -- 172.21.15.112:0/1724088751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f006c280 con 0x7f37f010de40 2024-09-21T03:27:08.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.697+0000 7f37e7fff640 1 -- 172.21.15.112:0/1724088751 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f37e8005000 con 0x7f37f010de40 2024-09-21T03:27:08.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.698+0000 7f37e7fff640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37dc0b6560 0x7f37dc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:08.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.698+0000 7f37eeffd640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37dc0b6560 0x7f37dc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:08.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.698+0000 7f37e7fff640 1 -- 172.21.15.112:0/1724088751 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f37e80e7330 con 0x7f37f010de40 2024-09-21T03:27:08.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.698+0000 7f37eeffd640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37dc0b6560 0x7f37dc0b8a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37e000a400 tx=0x7f37e000d040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:08.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.701+0000 7f37e7fff640 1 -- 172.21.15.112:0/1724088751 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37e8014030 con 0x7f37f010de40 2024-09-21T03:27:08.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:08.913+0000 7f37f53ef640 1 -- 172.21.15.112:0/1724088751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f37f01a7140 con 0x7f37f010de40 2024-09-21T03:27:09.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.354+0000 7f37e7fff640 1 -- 172.21.15.112:0/1724088751 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f37e80b1fc0 con 0x7f37f010de40 2024-09-21T03:27:09.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 -- 172.21.15.112:0/1724088751 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37dc0b6560 msgr2=0x7f37dc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:09.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37dc0b6560 0x7f37dc0b8a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37e000a400 tx=0x7f37e000d040 comp rx=0 tx=0).stop 2024-09-21T03:27:09.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 -- 172.21.15.112:0/1724088751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f010de40 msgr2=0x7f37f019fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:09.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f010de40 0x7f37f019fd50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c970 tx=0x7f37e800ce30 comp rx=0 tx=0).stop 2024-09-21T03:27:09.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 -- 172.21.15.112:0/1724088751 shutdown_connections 2024-09-21T03:27:09.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37dc0b6560 0x7f37dc0b8a70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:09.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f010de40 0x7f37f019fd50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:09.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37f0105330 0x7f37f019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:09.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 --2- 172.21.15.112:0/1724088751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37f01049e0 0x7f37f019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:09.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 -- 172.21.15.112:0/1724088751 >> 172.21.15.112:0/1724088751 conn(0x7f37f0079730 msgr2=0x7f37f0107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:09.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 -- 172.21.15.112:0/1724088751 shutdown_connections 2024-09-21T03:27:09.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:09.357+0000 7f37f53ef640 1 -- 172.21.15.112:0/1724088751 wait complete. 2024-09-21T03:27:09.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:09 smithi203 ceph-mon[34723]: pgmap v260: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T03:27:09.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:09 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1724088751' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T03:27:09.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:09 smithi187 ceph-mon[34603]: pgmap v260: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T03:27:09.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:09 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1724088751' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T03:27:09.714 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:09 smithi112 ceph-mon[31442]: pgmap v260: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T03:27:09.714 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:09 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1724088751' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T03:27:10.061 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T03:27:10.306 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:10 smithi203 ceph-mon[34723]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:27:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:10 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1724088751' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T03:27:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:10 smithi203 ceph-mon[34723]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:27:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:10 smithi203 ceph-mon[34723]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:10 smithi203 ceph-mon[34723]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-21T03:27:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:10 smithi203 ceph-mon[34723]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-21T03:27:10.668 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:10 smithi112 ceph-mon[31442]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:27:10.668 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:10 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1724088751' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T03:27:10.668 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:10 smithi112 ceph-mon[31442]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:27:10.668 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:10 smithi112 ceph-mon[31442]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:10.668 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:10 smithi112 ceph-mon[31442]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-21T03:27:10.668 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:10 smithi112 ceph-mon[31442]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-21T03:27:10.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:10 smithi187 ceph-mon[34603]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T03:27:10.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:10 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1724088751' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T03:27:10.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:10 smithi187 ceph-mon[34603]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T03:27:10.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:10 smithi187 ceph-mon[34603]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-21T03:27:10.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:10 smithi187 ceph-mon[34603]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-21T03:27:10.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:10 smithi187 ceph-mon[34603]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-21T03:27:10.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.757+0000 7fb20d162640 1 -- 172.21.15.112:0/548760596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb208108da0 msgr2=0x7fb20810f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:10.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.757+0000 7fb20d162640 1 --2- 172.21.15.112:0/548760596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb208108da0 0x7fb20810f5d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb200009ca0 tx=0x7fb20002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:27:10.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.757+0000 7fb20d162640 1 -- 172.21.15.112:0/548760596 shutdown_connections 2024-09-21T03:27:10.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.757+0000 7fb20d162640 1 --2- 172.21.15.112:0/548760596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb208108da0 0x7fb20810f5d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:10.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.757+0000 7fb20d162640 1 --2- 172.21.15.112:0/548760596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb208108450 0x7fb208108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:10.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.757+0000 7fb20d162640 1 --2- 172.21.15.112:0/548760596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb208102b10 0x7fb208102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:10.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.757+0000 7fb20d162640 1 -- 172.21.15.112:0/548760596 >> 172.21.15.112:0/548760596 conn(0x7fb2080796d0 msgr2=0x7fb208079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:10.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.757+0000 7fb20d162640 1 -- 172.21.15.112:0/548760596 shutdown_connections 2024-09-21T03:27:10.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.757+0000 7fb20d162640 1 -- 172.21.15.112:0/548760596 wait complete. 2024-09-21T03:27:10.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.758+0000 7fb20d162640 1 Processor -- start 2024-09-21T03:27:10.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.758+0000 7fb20d162640 1 -- start start 2024-09-21T03:27:10.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.758+0000 7fb20d162640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb208102b10 0x7fb208199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:10.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.758+0000 7fb20d162640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb208108450 0x7fb2081996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.758+0000 7fb20d162640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb208108da0 0x7fb208199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20d162640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb208112b20 con 0x7fb208108da0 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20d162640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081129a0 con 0x7fb208102b10 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20d162640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb208112ca0 con 0x7fb208108450 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb208108da0 0x7fb208199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb208108da0 0x7fb208199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38052/0 (socket says 172.21.15.112:38052) 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20759e640 1 -- 172.21.15.112:0/269829769 learned_addr learned my addr 172.21.15.112:0/269829769 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20659c640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb208108450 0x7fb2081996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb206d9d640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb208102b10 0x7fb208199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20759e640 1 -- 172.21.15.112:0/269829769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb208108450 msgr2=0x7fb2081996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20759e640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb208108450 0x7fb2081996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20759e640 1 -- 172.21.15.112:0/269829769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb208102b10 msgr2=0x7fb208199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20759e640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb208102b10 0x7fb208199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:10.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20759e640 1 -- 172.21.15.112:0/269829769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2081a8810 con 0x7fb208108da0 2024-09-21T03:27:10.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20759e640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb208108da0 0x7fb208199be0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb200005cc0 tx=0x7fb200003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:10.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb1effff640 1 -- 172.21.15.112:0/269829769 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200004040 con 0x7fb208108da0 2024-09-21T03:27:10.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20d162640 1 -- 172.21.15.112:0/269829769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2081a8aa0 con 0x7fb208108da0 2024-09-21T03:27:10.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb1effff640 1 -- 172.21.15.112:0/269829769 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb200004690 con 0x7fb208108da0 2024-09-21T03:27:10.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb1effff640 1 -- 172.21.15.112:0/269829769 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200030d40 con 0x7fb208108da0 2024-09-21T03:27:10.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.759+0000 7fb20d162640 1 -- 172.21.15.112:0/269829769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2081a90b0 con 0x7fb208108da0 2024-09-21T03:27:10.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.761+0000 7fb20d162640 1 -- 172.21.15.112:0/269829769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb208102f20 con 0x7fb208108da0 2024-09-21T03:27:10.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.763+0000 7fb1effff640 1 -- 172.21.15.112:0/269829769 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fb2000041e0 con 0x7fb208108da0 2024-09-21T03:27:10.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.764+0000 7fb1effff640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1f00b6560 0x7fb1f00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:10.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.764+0000 7fb206d9d640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1f00b6560 0x7fb1f00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:10.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.764+0000 7fb1effff640 1 -- 172.21.15.112:0/269829769 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7fb2001087c0 con 0x7fb208108da0 2024-09-21T03:27:10.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.764+0000 7fb206d9d640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1f00b6560 0x7fb1f00b8a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb208105430 tx=0x7fb1f8009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:10.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.767+0000 7fb1effff640 1 -- 172.21.15.112:0/269829769 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb2000cf070 con 0x7fb208108da0 2024-09-21T03:27:10.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:10.982+0000 7fb20d162640 1 -- 172.21.15.112:0/269829769 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fb20806bf70 con 0x7fb208108da0 2024-09-21T03:27:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.379+0000 7fb1effff640 1 -- 172.21.15.112:0/269829769 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fb2000d3450 con 0x7fb208108da0 2024-09-21T03:27:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 -- 172.21.15.112:0/269829769 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1f00b6560 msgr2=0x7fb1f00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1f00b6560 0x7fb1f00b8a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb208105430 tx=0x7fb1f8009110 comp rx=0 tx=0).stop 2024-09-21T03:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 -- 172.21.15.112:0/269829769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb208108da0 msgr2=0x7fb208199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb208108da0 0x7fb208199be0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb200005cc0 tx=0x7fb200003ba0 comp rx=0 tx=0).stop 2024-09-21T03:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 -- 172.21.15.112:0/269829769 shutdown_connections 2024-09-21T03:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1f00b6560 0x7fb1f00b8a70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb208108da0 0x7fb208199be0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb208108450 0x7fb2081996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 --2- 172.21.15.112:0/269829769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb208102b10 0x7fb208199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 -- 172.21.15.112:0/269829769 >> 172.21.15.112:0/269829769 conn(0x7fb2080796d0 msgr2=0x7fb208106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 -- 172.21.15.112:0/269829769 shutdown_connections 2024-09-21T03:27:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:11.382+0000 7fb20d162640 1 -- 172.21.15.112:0/269829769 wait complete. 2024-09-21T03:27:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:11 smithi203 ceph-mon[34723]: pgmap v261: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-21T03:27:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:11 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] up:active 2024-09-21T03:27:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:11 smithi203 ceph-mon[34723]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:27:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:11 smithi203 ceph-mon[34723]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:27:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:11 smithi203 ceph-mon[34723]: Cluster is now healthy 2024-09-21T03:27:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:11 smithi203 ceph-mon[34723]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-21T03:27:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:11 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-21T03:27:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:11 smithi203 ceph-mon[34723]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T03:27:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:11 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/269829769' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:27:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:11 smithi187 ceph-mon[34603]: pgmap v261: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-21T03:27:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:11 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] up:active 2024-09-21T03:27:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:11 smithi187 ceph-mon[34603]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:27:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:11 smithi187 ceph-mon[34603]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:27:11.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:11 smithi187 ceph-mon[34603]: Cluster is now healthy 2024-09-21T03:27:11.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:11 smithi187 ceph-mon[34603]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-21T03:27:11.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:11 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-21T03:27:11.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:11 smithi187 ceph-mon[34603]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T03:27:11.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:11 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/269829769' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:27:11.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:11 smithi112 ceph-mon[31442]: pgmap v261: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-21T03:27:11.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:11 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] up:active 2024-09-21T03:27:11.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:11 smithi112 ceph-mon[31442]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T03:27:11.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:11 smithi112 ceph-mon[31442]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T03:27:11.689 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:11 smithi112 ceph-mon[31442]: Cluster is now healthy 2024-09-21T03:27:11.689 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:11 smithi112 ceph-mon[31442]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-21T03:27:11.689 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:11 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-21T03:27:11.689 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:11 smithi112 ceph-mon[31442]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T03:27:11.689 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:11 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/269829769' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T03:27:12.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T03:27:12.387 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:12.661 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:12 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/269829769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:27:12.661 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:12 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] up:active 2024-09-21T03:27:12.661 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:12 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-21T03:27:12.661 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:12 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:27:12.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:12 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/269829769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:27:12.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:12 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] up:active 2024-09-21T03:27:12.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:12 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-21T03:27:12.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:12 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:27:12.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:12 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/269829769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T03:27:12.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:12 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] up:active 2024-09-21T03:27:12.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:12 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-21T03:27:12.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:12 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:27:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.875+0000 7f38260b8640 1 -- 172.21.15.112:0/4007003508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3820103b60 msgr2=0x7f3820103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.876+0000 7f38260b8640 1 --2- 172.21.15.112:0/4007003508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3820103b60 0x7f3820103f70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3810009900 tx=0x7f381002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.876+0000 7f38260b8640 1 -- 172.21.15.112:0/4007003508 shutdown_connections 2024-09-21T03:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.876+0000 7f38260b8640 1 --2- 172.21.15.112:0/4007003508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3820109df0 0x7f3820110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.876+0000 7f38260b8640 1 --2- 172.21.15.112:0/4007003508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38201094a0 0x7f38201098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.876+0000 7f38260b8640 1 --2- 172.21.15.112:0/4007003508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3820103b60 0x7f3820103f70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.876+0000 7f38260b8640 1 -- 172.21.15.112:0/4007003508 >> 172.21.15.112:0/4007003508 conn(0x7f382007b850 msgr2=0x7f382007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.876+0000 7f38260b8640 1 -- 172.21.15.112:0/4007003508 shutdown_connections 2024-09-21T03:27:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.876+0000 7f38260b8640 1 -- 172.21.15.112:0/4007003508 wait complete. 2024-09-21T03:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.876+0000 7f38260b8640 1 Processor -- start 2024-09-21T03:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f38260b8640 1 -- start start 2024-09-21T03:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f38260b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3820103b60 0x7f382019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f38260b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38201094a0 0x7f382019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f38260b8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3820109df0 0x7f382019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f38260b8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3820113ab0 con 0x7f38201094a0 2024-09-21T03:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f38260b8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3820113930 con 0x7f3820103b60 2024-09-21T03:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f38260b8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3820113c30 con 0x7f3820109df0 2024-09-21T03:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f381effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38201094a0 0x7f382019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:12.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f381effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38201094a0 0x7f382019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38072/0 (socket says 172.21.15.112:38072) 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f381effd640 1 -- 172.21.15.112:0/3600357520 learned_addr learned my addr 172.21.15.112:0/3600357520 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f381f7fe640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3820103b60 0x7f382019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f381ffff640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3820109df0 0x7f382019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f381effd640 1 -- 172.21.15.112:0/3600357520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3820109df0 msgr2=0x7f382019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f381effd640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3820109df0 0x7f382019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f381effd640 1 -- 172.21.15.112:0/3600357520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3820103b60 msgr2=0x7f382019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f381effd640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3820103b60 0x7f382019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f381effd640 1 -- 172.21.15.112:0/3600357520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38201a0410 con 0x7f38201094a0 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.877+0000 7f381f7fe640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3820103b60 0x7f382019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.878+0000 7f381effd640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38201094a0 0x7f382019aa40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f381400c970 tx=0x7f381400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.878+0000 7f381cff9640 1 -- 172.21.15.112:0/3600357520 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814007bb0 con 0x7f38201094a0 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.878+0000 7f381cff9640 1 -- 172.21.15.112:0/3600357520 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3814007d50 con 0x7f38201094a0 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.878+0000 7f38260b8640 1 -- 172.21.15.112:0/3600357520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38201a0700 con 0x7f38201094a0 2024-09-21T03:27:12.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.878+0000 7f381cff9640 1 -- 172.21.15.112:0/3600357520 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381400f450 con 0x7f38201094a0 2024-09-21T03:27:12.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.878+0000 7f38260b8640 1 -- 172.21.15.112:0/3600357520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38200755f0 con 0x7f38201094a0 2024-09-21T03:27:12.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.879+0000 7f38067fc640 1 -- 172.21.15.112:0/3600357520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3800005290 con 0x7f38201094a0 2024-09-21T03:27:12.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.881+0000 7f381cff9640 1 -- 172.21.15.112:0/3600357520 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f3814004eb0 con 0x7f38201094a0 2024-09-21T03:27:12.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.881+0000 7f381cff9640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38080b6560 0x7f38080b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:12.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.881+0000 7f381f7fe640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38080b6560 0x7f38080b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:12.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.881+0000 7f381cff9640 1 -- 172.21.15.112:0/3600357520 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f38140e7540 con 0x7f38201094a0 2024-09-21T03:27:12.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.882+0000 7f381f7fe640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38080b6560 0x7f38080b8a70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3810031440 tx=0x7f3810009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:12.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:12.885+0000 7f381cff9640 1 -- 172.21.15.112:0/3600357520 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3814014030 con 0x7f38201094a0 2024-09-21T03:27:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.103+0000 7f38067fc640 1 -- 172.21.15.112:0/3600357520 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f3800005580 con 0x7f38201094a0 2024-09-21T03:27:13.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.389+0000 7f381cff9640 1 -- 172.21.15.112:0/3600357520 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f38140addf0 con 0x7f38201094a0 2024-09-21T03:27:13.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 -- 172.21.15.112:0/3600357520 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38080b6560 msgr2=0x7f38080b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:13.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38080b6560 0x7f38080b8a70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3810031440 tx=0x7f3810009680 comp rx=0 tx=0).stop 2024-09-21T03:27:13.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 -- 172.21.15.112:0/3600357520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38201094a0 msgr2=0x7f382019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38201094a0 0x7f382019aa40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f381400c970 tx=0x7f381400ce30 comp rx=0 tx=0).stop 2024-09-21T03:27:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 -- 172.21.15.112:0/3600357520 shutdown_connections 2024-09-21T03:27:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38080b6560 0x7f38080b8a70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3820109df0 0x7f382019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38201094a0 0x7f382019aa40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 --2- 172.21.15.112:0/3600357520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3820103b60 0x7f382019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 -- 172.21.15.112:0/3600357520 >> 172.21.15.112:0/3600357520 conn(0x7f382007b850 msgr2=0x7f3820107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 -- 172.21.15.112:0/3600357520 shutdown_connections 2024-09-21T03:27:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:13.392+0000 7f38067fc640 1 -- 172.21.15.112:0/3600357520 wait complete. 2024-09-21T03:27:13.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:13 smithi203 ceph-mon[34723]: pgmap v262: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-21T03:27:13.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:13 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3600357520' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:27:13.675 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:13 smithi112 ceph-mon[31442]: pgmap v262: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-21T03:27:13.675 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:13 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3600357520' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:27:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:13 smithi187 ceph-mon[34603]: pgmap v262: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-21T03:27:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:13 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3600357520' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T03:27:14.078 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-21T03:27:14.079 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T03:27:14.334 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:14.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:14 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3600357520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:27:14.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:14 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:27:14.678 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:14 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3600357520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:27:14.678 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:14 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:27:14.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:14 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3600357520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T03:27:14.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:14 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:27:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.791+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3070622360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ec1094f0 msgr2=0x7fe3ec109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.791+0000 7fe3f27eb640 1 --2- 172.21.15.112:0/3070622360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ec1094f0 0x7fe3ec109900 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009900 tx=0x7fe3e002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.791+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3070622360 shutdown_connections 2024-09-21T03:27:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.791+0000 7fe3f27eb640 1 --2- 172.21.15.112:0/3070622360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3ec109e40 0x7fe3ec110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:14.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.791+0000 7fe3f27eb640 1 --2- 172.21.15.112:0/3070622360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ec1094f0 0x7fe3ec109900 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.791+0000 7fe3f27eb640 1 --2- 172.21.15.112:0/3070622360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3ec103bb0 0x7fe3ec103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.791+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3070622360 >> 172.21.15.112:0/3070622360 conn(0x7fe3ec0ff600 msgr2=0x7fe3ec101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.791+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3070622360 shutdown_connections 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.791+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3070622360 wait complete. 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.791+0000 7fe3f27eb640 1 Processor -- start 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3f27eb640 1 -- start start 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3f27eb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3ec103bb0 0x7fe3ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3f27eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ec1094f0 0x7fe3ec075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3f27eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3ec109e40 0x7fe3ec076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3f27eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec113b00 con 0x7fe3ec1094f0 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3f27eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec113980 con 0x7fe3ec109e40 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3f27eb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec113c80 con 0x7fe3ec103bb0 2024-09-21T03:27:14.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ec1094f0 0x7fe3ec075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ec1094f0 0x7fe3ec075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38100/0 (socket says 172.21.15.112:38100) 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3eb7fe640 1 -- 172.21.15.112:0/3273310169 learned_addr learned my addr 172.21.15.112:0/3273310169 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3f0d88640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3ec109e40 0x7fe3ec076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3ebfff640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3ec103bb0 0x7fe3ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3eb7fe640 1 -- 172.21.15.112:0/3273310169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3ec103bb0 msgr2=0x7fe3ec075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3eb7fe640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3ec103bb0 0x7fe3ec075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3eb7fe640 1 -- 172.21.15.112:0/3273310169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3ec109e40 msgr2=0x7fe3ec076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3eb7fe640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3ec109e40 0x7fe3ec076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.792+0000 7fe3eb7fe640 1 -- 172.21.15.112:0/3273310169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec1a8790 con 0x7fe3ec1094f0 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.793+0000 7fe3eb7fe640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ec1094f0 0x7fe3ec075c50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe3e002a930 tx=0x7fe3e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.793+0000 7fe3e97fa640 1 -- 172.21.15.112:0/3273310169 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e003f030 con 0x7fe3ec1094f0 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.793+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3273310169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ec1a8a20 con 0x7fe3ec1094f0 2024-09-21T03:27:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.793+0000 7fe3e97fa640 1 -- 172.21.15.112:0/3273310169 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3e0003aa0 con 0x7fe3ec1094f0 2024-09-21T03:27:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.793+0000 7fe3e97fa640 1 -- 172.21.15.112:0/3273310169 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0035070 con 0x7fe3ec1094f0 2024-09-21T03:27:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.793+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3273310169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ec1a8f50 con 0x7fe3ec1094f0 2024-09-21T03:27:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.794+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3273310169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ec103fc0 con 0x7fe3ec1094f0 2024-09-21T03:27:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.795+0000 7fe3e97fa640 1 -- 172.21.15.112:0/3273310169 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fe3e0042030 con 0x7fe3ec1094f0 2024-09-21T03:27:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.796+0000 7fe3e97fa640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe3d40b6560 0x7fe3d40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.796+0000 7fe3ebfff640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe3d40b6560 0x7fe3d40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.796+0000 7fe3e97fa640 1 -- 172.21.15.112:0/3273310169 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7fe3e0107dc0 con 0x7fe3ec1094f0 2024-09-21T03:27:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.797+0000 7fe3ebfff640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe3d40b6560 0x7fe3d40b8a70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0058c0 tx=0x7fe3dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:14.800+0000 7fe3e97fa640 1 -- 172.21.15.112:0/3273310169 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe3e00d2a50 con 0x7fe3ec1094f0 2024-09-21T03:27:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.004+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3273310169 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe3ec063800 con 0x7fe3d40b6560 2024-09-21T03:27:15.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.034+0000 7fe3e97fa640 1 -- 172.21.15.112:0/3273310169 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe3ec063800 con 0x7fe3d40b6560 2024-09-21T03:27:15.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3273310169 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe3d40b6560 msgr2=0x7fe3d40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:15.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe3d40b6560 0x7fe3d40b8a70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0058c0 tx=0x7fe3dc00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:27:15.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3273310169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ec1094f0 msgr2=0x7fe3ec075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:15.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ec1094f0 0x7fe3ec075c50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe3e002a930 tx=0x7fe3e0004270 comp rx=0 tx=0).stop 2024-09-21T03:27:15.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3273310169 shutdown_connections 2024-09-21T03:27:15.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe3d40b6560 0x7fe3d40b8a70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:15.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3ec109e40 0x7fe3ec076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:15.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe3ec1094f0 0x7fe3ec075c50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:15.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 --2- 172.21.15.112:0/3273310169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3ec103bb0 0x7fe3ec075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:15.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3273310169 >> 172.21.15.112:0/3273310169 conn(0x7fe3ec0ff600 msgr2=0x7fe3ec101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:15.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3273310169 shutdown_connections 2024-09-21T03:27:15.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:15.036+0000 7fe3f27eb640 1 -- 172.21.15.112:0/3273310169 wait complete. 2024-09-21T03:27:15.287 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:27:15 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:27:15.010+0000 7f3820b4d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:27:15.287 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:27:15 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:27:15.010+0000 7f3820b4d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:27:15.287 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:27:15 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:27:15.010+0000 7f3820b4d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:27:15.287 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:27:15 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:27:15.010+0000 7f3820b4d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:27:15.287 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:27:15 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:27:15.010+0000 7f3820b4d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:27:15.642 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T03:27:15.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:15 smithi203 ceph-mon[34723]: pgmap v263: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-21T03:27:15.665 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:15 smithi112 ceph-mon[31442]: pgmap v263: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-21T03:27:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:15 smithi187 ceph-mon[34603]: pgmap v263: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-21T03:27:15.889 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:16.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.387+0000 7f4d5705d640 1 -- 172.21.15.112:0/321046289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d50103b90 msgr2=0x7f4d50103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.387+0000 7f4d5705d640 1 --2- 172.21.15.112:0/321046289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d50103b90 0x7f4d50103fa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009900 tx=0x7f4d4002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.387+0000 7f4d5705d640 1 -- 172.21.15.112:0/321046289 shutdown_connections 2024-09-21T03:27:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.387+0000 7f4d5705d640 1 --2- 172.21.15.112:0/321046289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50109e20 0x7f4d50110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.387+0000 7f4d5705d640 1 --2- 172.21.15.112:0/321046289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d501094d0 0x7f4d501098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.387+0000 7f4d5705d640 1 --2- 172.21.15.112:0/321046289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d50103b90 0x7f4d50103fa0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.387+0000 7f4d5705d640 1 -- 172.21.15.112:0/321046289 >> 172.21.15.112:0/321046289 conn(0x7f4d500ff600 msgr2=0x7f4d50101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.387+0000 7f4d5705d640 1 -- 172.21.15.112:0/321046289 shutdown_connections 2024-09-21T03:27:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.387+0000 7f4d5705d640 1 -- 172.21.15.112:0/321046289 wait complete. 2024-09-21T03:27:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.388+0000 7f4d5705d640 1 Processor -- start 2024-09-21T03:27:16.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.388+0000 7f4d5705d640 1 -- start start 2024-09-21T03:27:16.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.388+0000 7f4d5705d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50103b90 0x7f4d5019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:16.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.388+0000 7f4d5705d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d501094d0 0x7f4d5019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:16.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.388+0000 7f4d5705d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d50109e20 0x7f4d5019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:16.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.388+0000 7f4d5705d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d50113b40 con 0x7f4d501094d0 2024-09-21T03:27:16.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.388+0000 7f4d5705d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4d501139c0 con 0x7f4d50109e20 2024-09-21T03:27:16.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.388+0000 7f4d5705d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4d50113cc0 con 0x7f4d50103b90 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.388+0000 7f4d4ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d501094d0 0x7f4d5019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d4ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d501094d0 0x7f4d5019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35494/0 (socket says 172.21.15.112:35494) 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d4ffff640 1 -- 172.21.15.112:0/309852939 learned_addr learned my addr 172.21.15.112:0/309852939 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d54df9640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50103b90 0x7f4d5019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d555fa640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d50109e20 0x7f4d5019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d4ffff640 1 -- 172.21.15.112:0/309852939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50103b90 msgr2=0x7f4d5019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d4ffff640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50103b90 0x7f4d5019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d4ffff640 1 -- 172.21.15.112:0/309852939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d50109e20 msgr2=0x7f4d5019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d4ffff640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d50109e20 0x7f4d5019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d4ffff640 1 -- 172.21.15.112:0/309852939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d501a0440 con 0x7f4d501094d0 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d4ffff640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d501094d0 0x7f4d5019aa70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c970 tx=0x7f4d3800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d4dffb640 1 -- 172.21.15.112:0/309852939 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38007bb0 con 0x7f4d501094d0 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d5705d640 1 -- 172.21.15.112:0/309852939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d501a0730 con 0x7f4d501094d0 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d4dffb640 1 -- 172.21.15.112:0/309852939 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d38007d50 con 0x7f4d501094d0 2024-09-21T03:27:16.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d4dffb640 1 -- 172.21.15.112:0/309852939 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3800f450 con 0x7f4d501094d0 2024-09-21T03:27:16.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.389+0000 7f4d5705d640 1 -- 172.21.15.112:0/309852939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d501a6e90 con 0x7f4d501094d0 2024-09-21T03:27:16.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.390+0000 7f4d5705d640 1 -- 172.21.15.112:0/309852939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d50103fa0 con 0x7f4d501094d0 2024-09-21T03:27:16.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.392+0000 7f4d4dffb640 1 -- 172.21.15.112:0/309852939 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f4d38004eb0 con 0x7f4d501094d0 2024-09-21T03:27:16.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.393+0000 7f4d4dffb640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4d3c0b6560 0x7f4d3c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:16.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.393+0000 7f4d54df9640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4d3c0b6560 0x7f4d3c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:16.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.393+0000 7f4d4dffb640 1 -- 172.21.15.112:0/309852939 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f4d380e6fd0 con 0x7f4d501094d0 2024-09-21T03:27:16.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.393+0000 7f4d54df9640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4d3c0b6560 0x7f4d3c0b8a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d40031440 tx=0x7f4d40005e80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:16.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.397+0000 7f4d4dffb640 1 -- 172.21.15.112:0/309852939 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4d38014030 con 0x7f4d501094d0 2024-09-21T03:27:16.609 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:16 smithi112 ceph-mon[31442]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:16.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.607+0000 7f4d5705d640 1 -- 172.21.15.112:0/309852939 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4d50063800 con 0x7f4d3c0b6560 2024-09-21T03:27:16.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.631+0000 7f4d4dffb640 1 -- 172.21.15.112:0/309852939 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4d50063800 con 0x7f4d3c0b6560 2024-09-21T03:27:16.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 -- 172.21.15.112:0/309852939 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4d3c0b6560 msgr2=0x7f4d3c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:16.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4d3c0b6560 0x7f4d3c0b8a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d40031440 tx=0x7f4d40005e80 comp rx=0 tx=0).stop 2024-09-21T03:27:16.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 -- 172.21.15.112:0/309852939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d501094d0 msgr2=0x7f4d5019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:16.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d501094d0 0x7f4d5019aa70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c970 tx=0x7f4d3800ce30 comp rx=0 tx=0).stop 2024-09-21T03:27:16.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 -- 172.21.15.112:0/309852939 shutdown_connections 2024-09-21T03:27:16.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4d3c0b6560 0x7f4d3c0b8a70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:16.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d50109e20 0x7f4d5019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:16.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d501094d0 0x7f4d5019aa70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:16.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 --2- 172.21.15.112:0/309852939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d50103b90 0x7f4d5019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:16.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 -- 172.21.15.112:0/309852939 >> 172.21.15.112:0/309852939 conn(0x7f4d500ff600 msgr2=0x7f4d50101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:16.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 -- 172.21.15.112:0/309852939 shutdown_connections 2024-09-21T03:27:16.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:16.633+0000 7f4d5705d640 1 -- 172.21.15.112:0/309852939 wait complete. 2024-09-21T03:27:16.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:16 smithi187 ceph-mon[34603]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:16.805 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:16 smithi203 ceph-mon[34723]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:17.244 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T03:27:17.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:17 smithi112 ceph-mon[31442]: pgmap v264: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-21T03:27:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:17.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:17 smithi187 ceph-mon[34603]: pgmap v264: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-21T03:27:17.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:17 smithi203 ceph-mon[34723]: pgmap v264: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-21T03:27:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.009+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/2708105164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdf0103b80 msgr2=0x7fcdf0103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.009+0000 7fcdf7d3b640 1 --2- 172.21.15.112:0/2708105164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdf0103b80 0x7fcdf0103f90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009900 tx=0x7fcdec02a410 comp rx=0 tx=0).stop 2024-09-21T03:27:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.010+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/2708105164 shutdown_connections 2024-09-21T03:27:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.010+0000 7fcdf7d3b640 1 --2- 172.21.15.112:0/2708105164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdf0109e10 0x7fcdf0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.010+0000 7fcdf7d3b640 1 --2- 172.21.15.112:0/2708105164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf01094c0 0x7fcdf01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.010+0000 7fcdf7d3b640 1 --2- 172.21.15.112:0/2708105164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdf0103b80 0x7fcdf0103f90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.010+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/2708105164 >> 172.21.15.112:0/2708105164 conn(0x7fcdf00ff5d0 msgr2=0x7fcdf0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.010+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/2708105164 shutdown_connections 2024-09-21T03:27:18.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.010+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/2708105164 wait complete. 2024-09-21T03:27:18.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.010+0000 7fcdf7d3b640 1 Processor -- start 2024-09-21T03:27:18.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.010+0000 7fcdf7d3b640 1 -- start start 2024-09-21T03:27:18.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf7d3b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf0103b80 0x7fcdf019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:18.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf7d3b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdf01094c0 0x7fcdf019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:18.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf7d3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdf0109e10 0x7fcdf019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:18.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf7d3b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf0113b10 con 0x7fcdf0109e10 2024-09-21T03:27:18.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf7d3b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf0113990 con 0x7fcdf01094c0 2024-09-21T03:27:18.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf7d3b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf0113c90 con 0x7fcdf0103b80 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf62d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdf0109e10 0x7fcdf019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf62d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdf0109e10 0x7fcdf019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35512/0 (socket says 172.21.15.112:35512) 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf62d8640 1 -- 172.21.15.112:0/3014592005 learned_addr learned my addr 172.21.15.112:0/3014592005 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf5ad7640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf0103b80 0x7fcdf019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf52d6640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdf01094c0 0x7fcdf019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf62d8640 1 -- 172.21.15.112:0/3014592005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf0103b80 msgr2=0x7fcdf019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf62d8640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf0103b80 0x7fcdf019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf62d8640 1 -- 172.21.15.112:0/3014592005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdf01094c0 msgr2=0x7fcdf019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf62d8640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdf01094c0 0x7fcdf019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf62d8640 1 -- 172.21.15.112:0/3014592005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf01a0470 con 0x7fcdf0109e10 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.011+0000 7fcdf62d8640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdf0109e10 0x7fcdf019fd40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009fd0 tx=0x7fcde800def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.012+0000 7fcde6ffd640 1 -- 172.21.15.112:0/3014592005 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8017070 con 0x7fcdf0109e10 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.012+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/3014592005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf01a0760 con 0x7fcdf0109e10 2024-09-21T03:27:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.012+0000 7fcde6ffd640 1 -- 172.21.15.112:0/3014592005 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde8003e20 con 0x7fcdf0109e10 2024-09-21T03:27:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.012+0000 7fcde6ffd640 1 -- 172.21.15.112:0/3014592005 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8010460 con 0x7fcdf0109e10 2024-09-21T03:27:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.012+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/3014592005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf01a6ec0 con 0x7fcdf0109e10 2024-09-21T03:27:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.013+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/3014592005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf0103f90 con 0x7fcdf0109e10 2024-09-21T03:27:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.014+0000 7fcde6ffd640 1 -- 172.21.15.112:0/3014592005 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fcde8004020 con 0x7fcdf0109e10 2024-09-21T03:27:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.015+0000 7fcde6ffd640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcddc0b65f0 0x7fcddc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.015+0000 7fcdf5ad7640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcddc0b65f0 0x7fcddc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.015+0000 7fcde6ffd640 1 -- 172.21.15.112:0/3014592005 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7fcde80e7310 con 0x7fcdf0109e10 2024-09-21T03:27:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.015+0000 7fcdf5ad7640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcddc0b65f0 0x7fcddc0b8b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdec031440 tx=0x7fcdec005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.019+0000 7fcde6ffd640 1 -- 172.21.15.112:0/3014592005 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcde80ada60 con 0x7fcdf0109e10 2024-09-21T03:27:18.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.225+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/3014592005 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fcdf010bc40 con 0x7fcddc0b65f0 2024-09-21T03:27:18.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.232+0000 7fcde6ffd640 1 -- 172.21.15.112:0/3014592005 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fcdf010bc40 con 0x7fcddc0b65f0 2024-09-21T03:27:18.233 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_0/63e63772-fd77-4cff-a774-3609ad0a23e7 2024-09-21T03:27:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/3014592005 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcddc0b65f0 msgr2=0x7fcddc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcddc0b65f0 0x7fcddc0b8b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdec031440 tx=0x7fcdec005d50 comp rx=0 tx=0).stop 2024-09-21T03:27:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/3014592005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdf0109e10 msgr2=0x7fcdf019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdf0109e10 0x7fcdf019fd40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009fd0 tx=0x7fcde800def0 comp rx=0 tx=0).stop 2024-09-21T03:27:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/3014592005 shutdown_connections 2024-09-21T03:27:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcddc0b65f0 0x7fcddc0b8b00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdf0109e10 0x7fcdf019fd40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdf01094c0 0x7fcdf019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 --2- 172.21.15.112:0/3014592005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf0103b80 0x7fcdf019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/3014592005 >> 172.21.15.112:0/3014592005 conn(0x7fcdf00ff5d0 msgr2=0x7fcdf01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/3014592005 shutdown_connections 2024-09-21T03:27:18.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:18.234+0000 7fcdf7d3b640 1 -- 172.21.15.112:0/3014592005 wait complete. 2024-09-21T03:27:18.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:18 smithi112 ceph-mon[31442]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:18.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:18 smithi112 ceph-mon[31442]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T03:27:18.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:18 smithi112 ceph-mon[31442]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:18.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:18 smithi112 ceph-mon[31442]: pgmap v265: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-21T03:27:18.865 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T03:27:18.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:18 smithi203 ceph-mon[34723]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:18.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:18 smithi203 ceph-mon[34723]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T03:27:18.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:18 smithi203 ceph-mon[34723]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:18.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:18 smithi203 ceph-mon[34723]: pgmap v265: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-21T03:27:18.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:18 smithi187 ceph-mon[34603]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:18.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:18 smithi187 ceph-mon[34603]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T03:27:18.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:18 smithi187 ceph-mon[34603]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:18.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:18 smithi187 ceph-mon[34603]: pgmap v265: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-21T03:27:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.532+0000 7f0e81bca640 1 -- 172.21.15.112:0/900944319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c1062f0 msgr2=0x7f0e7c106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.532+0000 7f0e81bca640 1 --2- 172.21.15.112:0/900944319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c1062f0 0x7f0e7c106700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009900 tx=0x7f0e6402a410 comp rx=0 tx=0).stop 2024-09-21T03:27:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.532+0000 7f0e81bca640 1 -- 172.21.15.112:0/900944319 shutdown_connections 2024-09-21T03:27:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.532+0000 7f0e81bca640 1 --2- 172.21.15.112:0/900944319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7c106c40 0x7f0e7c1106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.532+0000 7f0e81bca640 1 --2- 172.21.15.112:0/900944319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c1062f0 0x7f0e7c106700 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.532+0000 7f0e81bca640 1 --2- 172.21.15.112:0/900944319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1020a0 0x7f0e7c1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.532+0000 7f0e81bca640 1 -- 172.21.15.112:0/900944319 >> 172.21.15.112:0/900944319 conn(0x7f0e7c079f90 msgr2=0x7f0e7c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.532+0000 7f0e81bca640 1 -- 172.21.15.112:0/900944319 shutdown_connections 2024-09-21T03:27:19.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.532+0000 7f0e81bca640 1 -- 172.21.15.112:0/900944319 wait complete. 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.532+0000 7f0e81bca640 1 Processor -- start 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.532+0000 7f0e81bca640 1 -- start start 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.533+0000 7f0e81bca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7c1020a0 0x7f0e7c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.533+0000 7f0e81bca640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1062f0 0x7f0e7c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.533+0000 7f0e81bca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c106c40 0x7f0e7c19fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.533+0000 7f0e81bca640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c113bd0 con 0x7f0e7c106c40 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.533+0000 7f0e81bca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c113a50 con 0x7f0e7c1020a0 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.533+0000 7f0e81bca640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c113d50 con 0x7f0e7c1062f0 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.533+0000 7f0e7bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c106c40 0x7f0e7c19fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.533+0000 7f0e7bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c106c40 0x7f0e7c19fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35534/0 (socket says 172.21.15.112:35534) 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.533+0000 7f0e7bfff640 1 -- 172.21.15.112:0/3564292831 learned_addr learned my addr 172.21.15.112:0/3564292831 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.533+0000 7f0e7b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7c1020a0 0x7f0e7c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.533+0000 7f0e72ffd640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1062f0 0x7f0e7c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:19.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.534+0000 7f0e7bfff640 1 -- 172.21.15.112:0/3564292831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1062f0 msgr2=0x7f0e7c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.534+0000 7f0e7bfff640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1062f0 0x7f0e7c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.534+0000 7f0e7bfff640 1 -- 172.21.15.112:0/3564292831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7c1020a0 msgr2=0x7f0e7c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.534+0000 7f0e7bfff640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7c1020a0 0x7f0e7c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.534+0000 7f0e7bfff640 1 -- 172.21.15.112:0/3564292831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c1a0590 con 0x7f0e7c106c40 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.534+0000 7f0e7bfff640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c106c40 0x7f0e7c19fe60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400b970 tx=0x7f0e7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.534+0000 7f0e797fa640 1 -- 172.21.15.112:0/3564292831 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74019070 con 0x7f0e7c106c40 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.534+0000 7f0e81bca640 1 -- 172.21.15.112:0/3564292831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c1a0880 con 0x7f0e7c106c40 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.534+0000 7f0e797fa640 1 -- 172.21.15.112:0/3564292831 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e74014440 con 0x7f0e7c106c40 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.534+0000 7f0e797fa640 1 -- 172.21.15.112:0/3564292831 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e740075d0 con 0x7f0e7c106c40 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.534+0000 7f0e81bca640 1 -- 172.21.15.112:0/3564292831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c1a6fe0 con 0x7f0e7c106c40 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.535+0000 7f0e81bca640 1 -- 172.21.15.112:0/3564292831 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e7c1024b0 con 0x7f0e7c106c40 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.536+0000 7f0e797fa640 1 -- 172.21.15.112:0/3564292831 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f0e7401c070 con 0x7f0e7c106c40 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.537+0000 7f0e797fa640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e680b65f0 0x7f0e680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.538+0000 7f0e7b7fe640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e680b65f0 0x7f0e680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:19.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.538+0000 7f0e797fa640 1 -- 172.21.15.112:0/3564292831 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f0e740e68e0 con 0x7f0e7c106c40 2024-09-21T03:27:19.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.538+0000 7f0e7b7fe640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e680b65f0 0x7f0e680b8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c06bb10 tx=0x7f0e6c009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:19.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.542+0000 7f0e797fa640 1 -- 172.21.15.112:0/3564292831 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e740b1410 con 0x7f0e7c106c40 2024-09-21T03:27:19.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.749+0000 7f0e81bca640 1 -- 172.21.15.112:0/3564292831 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0e7c063800 con 0x7f0e680b65f0 2024-09-21T03:27:19.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.776+0000 7f0e797fa640 1 -- 172.21.15.112:0/3564292831 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0e7c063800 con 0x7f0e680b65f0 2024-09-21T03:27:19.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.778+0000 7f0e81bca640 1 -- 172.21.15.112:0/3564292831 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e680b65f0 msgr2=0x7f0e680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:19.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.778+0000 7f0e81bca640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e680b65f0 0x7f0e680b8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c06bb10 tx=0x7f0e6c009110 comp rx=0 tx=0).stop 2024-09-21T03:27:19.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.778+0000 7f0e81bca640 1 -- 172.21.15.112:0/3564292831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c106c40 msgr2=0x7f0e7c19fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:19.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.778+0000 7f0e81bca640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c106c40 0x7f0e7c19fe60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400b970 tx=0x7f0e7400bd70 comp rx=0 tx=0).stop 2024-09-21T03:27:19.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.778+0000 7f0e81bca640 1 -- 172.21.15.112:0/3564292831 shutdown_connections 2024-09-21T03:27:19.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.778+0000 7f0e81bca640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e680b65f0 0x7f0e680b8b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:19.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.778+0000 7f0e81bca640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e7c106c40 0x7f0e7c19fe60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:19.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.778+0000 7f0e81bca640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1062f0 0x7f0e7c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:19.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.778+0000 7f0e81bca640 1 --2- 172.21.15.112:0/3564292831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7c1020a0 0x7f0e7c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:19.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.779+0000 7f0e81bca640 1 -- 172.21.15.112:0/3564292831 >> 172.21.15.112:0/3564292831 conn(0x7f0e7c079f90 msgr2=0x7f0e7c105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:19.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.779+0000 7f0e81bca640 1 -- 172.21.15.112:0/3564292831 shutdown_connections 2024-09-21T03:27:19.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:19.779+0000 7f0e81bca640 1 -- 172.21.15.112:0/3564292831 wait complete. 2024-09-21T03:27:20.406 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T03:27:20.651 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:21.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.074+0000 7f15fa8fe640 1 -- 172.21.15.112:0/2089844458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15f4105270 msgr2=0x7f15f410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:21.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.074+0000 7f15fa8fe640 1 --2- 172.21.15.112:0/2089844458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15f4105270 0x7f15f410d8b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009900 tx=0x7f15e802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.074+0000 7f15fa8fe640 1 -- 172.21.15.112:0/2089844458 shutdown_connections 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.074+0000 7f15fa8fe640 1 --2- 172.21.15.112:0/2089844458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f410ddf0 0x7f15f4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.074+0000 7f15fa8fe640 1 --2- 172.21.15.112:0/2089844458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15f4105270 0x7f15f410d8b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.074+0000 7f15fa8fe640 1 --2- 172.21.15.112:0/2089844458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15f4104840 0x7f15f4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.074+0000 7f15fa8fe640 1 -- 172.21.15.112:0/2089844458 >> 172.21.15.112:0/2089844458 conn(0x7f15f4079730 msgr2=0x7f15f4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.074+0000 7f15fa8fe640 1 -- 172.21.15.112:0/2089844458 shutdown_connections 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.074+0000 7f15fa8fe640 1 -- 172.21.15.112:0/2089844458 wait complete. 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15fa8fe640 1 Processor -- start 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15fa8fe640 1 -- start start 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15fa8fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f4104840 0x7f15f419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15fa8fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15f4105270 0x7f15f419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15fa8fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15f410ddf0 0x7f15f419fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:21.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15fa8fe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4113ad0 con 0x7f15f410ddf0 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15fa8fe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4113950 con 0x7f15f4105270 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15fa8fe640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4113c50 con 0x7f15f4104840 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15f8e9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15f410ddf0 0x7f15f419fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15f8e9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15f410ddf0 0x7f15f419fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35546/0 (socket says 172.21.15.112:35546) 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15f8e9b640 1 -- 172.21.15.112:0/3153668577 learned_addr learned my addr 172.21.15.112:0/3153668577 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15f37fe640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15f4105270 0x7f15f419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15f3fff640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f4104840 0x7f15f419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.075+0000 7f15f8e9b640 1 -- 172.21.15.112:0/3153668577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f4104840 msgr2=0x7f15f419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.076+0000 7f15f8e9b640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f4104840 0x7f15f419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.076+0000 7f15f8e9b640 1 -- 172.21.15.112:0/3153668577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15f4105270 msgr2=0x7f15f419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.076+0000 7f15f8e9b640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15f4105270 0x7f15f419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.076+0000 7f15f8e9b640 1 -- 172.21.15.112:0/3153668577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f41a05f0 con 0x7f15f410ddf0 2024-09-21T03:27:21.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.076+0000 7f15f8e9b640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15f410ddf0 0x7f15f419fec0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00cad0 tx=0x7f15ec00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.076+0000 7f15f17fa640 1 -- 172.21.15.112:0/3153668577 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00ad60 con 0x7f15f410ddf0 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.076+0000 7f15f17fa640 1 -- 172.21.15.112:0/3153668577 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ec005020 con 0x7f15f410ddf0 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.076+0000 7f15fa8fe640 1 -- 172.21.15.112:0/3153668577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f41a08e0 con 0x7f15f410ddf0 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.076+0000 7f15f17fa640 1 -- 172.21.15.112:0/3153668577 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00f6b0 con 0x7f15f410ddf0 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.076+0000 7f15fa8fe640 1 -- 172.21.15.112:0/3153668577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f41a7040 con 0x7f15f410ddf0 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.077+0000 7f15fa8fe640 1 -- 172.21.15.112:0/3153668577 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f4104c50 con 0x7f15f410ddf0 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.078+0000 7f15f17fa640 1 -- 172.21.15.112:0/3153668577 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f15ec0051d0 con 0x7f15f410ddf0 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.079+0000 7f15f17fa640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15dc0b65f0 0x7f15dc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.079+0000 7f15f3fff640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15dc0b65f0 0x7f15dc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.080+0000 7f15f17fa640 1 -- 172.21.15.112:0/3153668577 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f15ec0e7470 con 0x7f15f410ddf0 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.080+0000 7f15f3fff640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15dc0b65f0 0x7f15dc0b8b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15f406bc00 tx=0x7f15e4009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:21.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.084+0000 7f15f17fa640 1 -- 172.21.15.112:0/3153668577 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15ec014030 con 0x7f15f410ddf0 2024-09-21T03:27:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.289+0000 7f15fa8fe640 1 -- 172.21.15.112:0/3153668577 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f15f4063800 con 0x7f15dc0b65f0 2024-09-21T03:27:21.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.293+0000 7f15f17fa640 1 -- 172.21.15.112:0/3153668577 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f15f4063800 con 0x7f15dc0b65f0 2024-09-21T03:27:21.295 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_1/0439cab4-e277-4b08-8947-0a17e694fb85 2024-09-21T03:27:21.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 -- 172.21.15.112:0/3153668577 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15dc0b65f0 msgr2=0x7f15dc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:21.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15dc0b65f0 0x7f15dc0b8b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15f406bc00 tx=0x7f15e4009110 comp rx=0 tx=0).stop 2024-09-21T03:27:21.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 -- 172.21.15.112:0/3153668577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15f410ddf0 msgr2=0x7f15f419fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:21.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15f410ddf0 0x7f15f419fec0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00cad0 tx=0x7f15ec00ced0 comp rx=0 tx=0).stop 2024-09-21T03:27:21.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 -- 172.21.15.112:0/3153668577 shutdown_connections 2024-09-21T03:27:21.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15dc0b65f0 0x7f15dc0b8b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:21.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15f410ddf0 0x7f15f419fec0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:21.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15f4105270 0x7f15f419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:21.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 --2- 172.21.15.112:0/3153668577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f4104840 0x7f15f419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:21.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 -- 172.21.15.112:0/3153668577 >> 172.21.15.112:0/3153668577 conn(0x7f15f4079730 msgr2=0x7f15f40ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:21.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 -- 172.21.15.112:0/3153668577 shutdown_connections 2024-09-21T03:27:21.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:21.296+0000 7f15fa8fe640 1 -- 172.21.15.112:0/3153668577 wait complete. 2024-09-21T03:27:21.659 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:21 smithi112 ceph-mon[31442]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:21.659 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:21 smithi112 ceph-mon[31442]: pgmap v266: 97 pgs: 97 active+clean; 604 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-21T03:27:21.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:21 smithi203 ceph-mon[34723]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:21.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:21 smithi203 ceph-mon[34723]: pgmap v266: 97 pgs: 97 active+clean; 604 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-21T03:27:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:21 smithi187 ceph-mon[34603]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:21 smithi187 ceph-mon[34603]: pgmap v266: 97 pgs: 97 active+clean; 604 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-21T03:27:21.985 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:27:22.236 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:22.649 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:22 smithi112 ceph-mon[31442]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:22.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:22 smithi203 ceph-mon[34723]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:22.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:22 smithi187 ceph-mon[34603]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.736+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/2851892183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f8108cf0 msgr2=0x7ff9f810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.736+0000 7ff9fe8c6640 1 --2- 172.21.15.112:0/2851892183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f8108cf0 0x7ff9f810f520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009ca0 tx=0x7ff9f002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.736+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/2851892183 shutdown_connections 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.736+0000 7ff9fe8c6640 1 --2- 172.21.15.112:0/2851892183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f8108cf0 0x7ff9f810f520 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.736+0000 7ff9fe8c6640 1 --2- 172.21.15.112:0/2851892183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9f81083a0 0x7ff9f81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.736+0000 7ff9fe8c6640 1 --2- 172.21.15.112:0/2851892183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9f8102a60 0x7ff9f8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.736+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/2851892183 >> 172.21.15.112:0/2851892183 conn(0x7ff9f8079730 msgr2=0x7ff9f8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.736+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/2851892183 shutdown_connections 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.736+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/2851892183 wait complete. 2024-09-21T03:27:22.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9fe8c6640 1 Processor -- start 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9fe8c6640 1 -- start start 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9fe8c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9f8102a60 0x7ff9f819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9fe8c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f81083a0 0x7ff9f819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9fe8c6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9f8108cf0 0x7ff9f819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9fe8c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f8113a80 con 0x7ff9f81083a0 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9fe8c6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f8113900 con 0x7ff9f8102a60 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9fe8c6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f8113c00 con 0x7ff9f8108cf0 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9effff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f81083a0 0x7ff9f819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9effff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f81083a0 0x7ff9f819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35552/0 (socket says 172.21.15.112:35552) 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9effff640 1 -- 172.21.15.112:0/1147606027 learned_addr learned my addr 172.21.15.112:0/1147606027 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9f7fff640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9f8102a60 0x7ff9f819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9fce63640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9f8108cf0 0x7ff9f819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9effff640 1 -- 172.21.15.112:0/1147606027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9f8108cf0 msgr2=0x7ff9f819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.737+0000 7ff9effff640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9f8108cf0 0x7ff9f819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.738+0000 7ff9effff640 1 -- 172.21.15.112:0/1147606027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9f8102a60 msgr2=0x7ff9f819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.738+0000 7ff9effff640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9f8102a60 0x7ff9f819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.738+0000 7ff9effff640 1 -- 172.21.15.112:0/1147606027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f81a0380 con 0x7ff9f81083a0 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.738+0000 7ff9fce63640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9f8108cf0 0x7ff9f819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.738+0000 7ff9f7fff640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9f8102a60 0x7ff9f819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.738+0000 7ff9effff640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f81083a0 0x7ff9f819a9b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000c970 tx=0x7ff9e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.738+0000 7ff9f5ffb640 1 -- 172.21.15.112:0/1147606027 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0007bb0 con 0x7ff9f81083a0 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.738+0000 7ff9f5ffb640 1 -- 172.21.15.112:0/1147606027 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e0007d50 con 0x7ff9f81083a0 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.738+0000 7ff9f5ffb640 1 -- 172.21.15.112:0/1147606027 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000f450 con 0x7ff9f81083a0 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.738+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/1147606027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f81a0670 con 0x7ff9f81083a0 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.738+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/1147606027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f81a6dd0 con 0x7ff9f81083a0 2024-09-21T03:27:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.739+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/1147606027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f8102e70 con 0x7ff9f81083a0 2024-09-21T03:27:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.740+0000 7ff9f5ffb640 1 -- 172.21.15.112:0/1147606027 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7ff9e0004f20 con 0x7ff9f81083a0 2024-09-21T03:27:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.741+0000 7ff9f5ffb640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9e40b6670 0x7ff9e40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.741+0000 7ff9f7fff640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9e40b6670 0x7ff9e40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.741+0000 7ff9f5ffb640 1 -- 172.21.15.112:0/1147606027 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7ff9e00e75d0 con 0x7ff9f81083a0 2024-09-21T03:27:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.742+0000 7ff9f7fff640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9e40b6670 0x7ff9e40b8b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80058c0 tx=0x7ff9e8005ec0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:22.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.745+0000 7ff9f5ffb640 1 -- 172.21.15.112:0/1147606027 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff9e0014030 con 0x7ff9f81083a0 2024-09-21T03:27:22.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.957+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/1147606027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9f802e640 con 0x7ff9f81083a0 2024-09-21T03:27:22.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.958+0000 7ff9f5ffb640 1 -- 172.21.15.112:0/1147606027 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11929 (secure 0 0 0) 0x7ff9e00adcc0 con 0x7ff9f81083a0 2024-09-21T03:27:22.960 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:27:22.960 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"btime":"2024-09-21T03:27:13:388843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:27:13.104308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:27:22.960 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-09-21T03:27:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/1147606027 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9e40b6670 msgr2=0x7ff9e40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9e40b6670 0x7ff9e40b8b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80058c0 tx=0x7ff9e8005ec0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/1147606027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f81083a0 msgr2=0x7ff9f819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f81083a0 0x7ff9f819a9b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000c970 tx=0x7ff9e000ce30 comp rx=0 tx=0).stop 2024-09-21T03:27:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/1147606027 shutdown_connections 2024-09-21T03:27:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff9e40b6670 0x7ff9e40b8b80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9f8108cf0 0x7ff9f819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f81083a0 0x7ff9f819a9b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 --2- 172.21.15.112:0/1147606027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9f8102a60 0x7ff9f819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/1147606027 >> 172.21.15.112:0/1147606027 conn(0x7ff9f8079730 msgr2=0x7ff9f8104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/1147606027 shutdown_connections 2024-09-21T03:27:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:22.960+0000 7ff9fe8c6640 1 -- 172.21.15.112:0/1147606027 wait complete. 2024-09-21T03:27:23.645 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:23 smithi112 ceph-mon[31442]: pgmap v267: 97 pgs: 97 active+clean; 604 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-21T03:27:23.645 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:23 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1147606027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:27:23.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:23 smithi203 ceph-mon[34723]: pgmap v267: 97 pgs: 97 active+clean; 604 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-21T03:27:23.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:23 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1147606027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:27:23.673 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:27:23.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:23 smithi187 ceph-mon[34603]: pgmap v267: 97 pgs: 97 active+clean; 604 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-21T03:27:23.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:23 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1147606027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:27:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:24.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.402+0000 7fb8ed74a640 1 -- 172.21.15.112:0/4175704617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8103210 msgr2=0x7fb8e8103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:24.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.402+0000 7fb8ed74a640 1 --2- 172.21.15.112:0/4175704617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8103210 0x7fb8e8103620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009900 tx=0x7fb8d802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:24.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.402+0000 7fb8ed74a640 1 -- 172.21.15.112:0/4175704617 shutdown_connections 2024-09-21T03:27:24.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.402+0000 7fb8ed74a640 1 --2- 172.21.15.112:0/4175704617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8e8105b70 0x7fb8e810f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:24.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.402+0000 7fb8ed74a640 1 --2- 172.21.15.112:0/4175704617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8e8105220 0x7fb8e8105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:24.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.402+0000 7fb8ed74a640 1 --2- 172.21.15.112:0/4175704617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8103210 0x7fb8e8103620 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.402+0000 7fb8ed74a640 1 -- 172.21.15.112:0/4175704617 >> 172.21.15.112:0/4175704617 conn(0x7fb8e8079ec0 msgr2=0x7fb8e807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.402+0000 7fb8ed74a640 1 -- 172.21.15.112:0/4175704617 shutdown_connections 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.402+0000 7fb8ed74a640 1 -- 172.21.15.112:0/4175704617 wait complete. 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.403+0000 7fb8ed74a640 1 Processor -- start 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.403+0000 7fb8ed74a640 1 -- start start 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.403+0000 7fb8ed74a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8e8103210 0x7fb8e8199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.403+0000 7fb8ed74a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8e8105220 0x7fb8e8199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.403+0000 7fb8ed74a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8105b70 0x7fb8e819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.403+0000 7fb8ed74a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8112a70 con 0x7fb8e8105b70 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.403+0000 7fb8ed74a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81128f0 con 0x7fb8e8103210 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.403+0000 7fb8ed74a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8112bf0 con 0x7fb8e8105220 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8105b70 0x7fb8e819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:24.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8105b70 0x7fb8e819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35572/0 (socket says 172.21.15.112:35572) 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e77fe640 1 -- 172.21.15.112:0/398343160 learned_addr learned my addr 172.21.15.112:0/398343160 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8dffff640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8e8105220 0x7fb8e8199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e6ffd640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8e8103210 0x7fb8e8199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e77fe640 1 -- 172.21.15.112:0/398343160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8e8105220 msgr2=0x7fb8e8199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e77fe640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8e8105220 0x7fb8e8199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e77fe640 1 -- 172.21.15.112:0/398343160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8e8103210 msgr2=0x7fb8e8199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e77fe640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8e8103210 0x7fb8e8199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e77fe640 1 -- 172.21.15.112:0/398343160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e819f330 con 0x7fb8e8105b70 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e77fe640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8105b70 0x7fb8e819ecc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000cce0 tx=0x7fb8e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e4ff9640 1 -- 172.21.15.112:0/398343160 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e000f450 con 0x7fb8e8105b70 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e4ff9640 1 -- 172.21.15.112:0/398343160 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8e000faa0 con 0x7fb8e8105b70 2024-09-21T03:27:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8ed74a640 1 -- 172.21.15.112:0/398343160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e819f620 con 0x7fb8e8105b70 2024-09-21T03:27:24.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8e4ff9640 1 -- 172.21.15.112:0/398343160 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0018b70 con 0x7fb8e8105b70 2024-09-21T03:27:24.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.404+0000 7fb8ed74a640 1 -- 172.21.15.112:0/398343160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e81a5e40 con 0x7fb8e8105b70 2024-09-21T03:27:24.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.405+0000 7fb8ed74a640 1 -- 172.21.15.112:0/398343160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e8103f90 con 0x7fb8e8105b70 2024-09-21T03:27:24.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.408+0000 7fb8e4ff9640 1 -- 172.21.15.112:0/398343160 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fb8e0018d10 con 0x7fb8e8105b70 2024-09-21T03:27:24.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.408+0000 7fb8e4ff9640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8d40b6620 0x7fb8d40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:24.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.409+0000 7fb8e6ffd640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8d40b6620 0x7fb8d40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:24.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.409+0000 7fb8e4ff9640 1 -- 172.21.15.112:0/398343160 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7fb8e00e7900 con 0x7fb8e8105b70 2024-09-21T03:27:24.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.409+0000 7fb8e6ffd640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8d40b6620 0x7fb8d40b8b30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8031440 tx=0x7fb8d8009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:24.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.412+0000 7fb8e4ff9640 1 -- 172.21.15.112:0/398343160 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8e0014030 con 0x7fb8e8105b70 2024-09-21T03:27:24.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.589+0000 7fb8ed74a640 1 -- 172.21.15.112:0/398343160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8e8103620 con 0x7fb8e8105b70 2024-09-21T03:27:24.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.591+0000 7fb8e4ff9640 1 -- 172.21.15.112:0/398343160 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+22632 (secure 0 0 0) 0x7fb8e00ae0e0 con 0x7fb8e8105b70 2024-09-21T03:27:24.593 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:27:24.593 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":74,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:27:01.886286+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:26:58.087349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:26:59.970444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 -- 172.21.15.112:0/398343160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8d40b6620 msgr2=0x7fb8d40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8d40b6620 0x7fb8d40b8b30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8031440 tx=0x7fb8d8009560 comp rx=0 tx=0).stop 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 -- 172.21.15.112:0/398343160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8105b70 msgr2=0x7fb8e819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8105b70 0x7fb8e819ecc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000cce0 tx=0x7fb8e000a620 comp rx=0 tx=0).stop 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 -- 172.21.15.112:0/398343160 shutdown_connections 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8d40b6620 0x7fb8d40b8b30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e8105b70 0x7fb8e819ecc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8e8105220 0x7fb8e8199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 --2- 172.21.15.112:0/398343160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8e8103210 0x7fb8e8199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 -- 172.21.15.112:0/398343160 >> 172.21.15.112:0/398343160 conn(0x7fb8e8079ec0 msgr2=0x7fb8e8107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 -- 172.21.15.112:0/398343160 shutdown_connections 2024-09-21T03:27:24.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:24.593+0000 7fb8ed74a640 1 -- 172.21.15.112:0/398343160 wait complete. 2024-09-21T03:27:25.272 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:27:25.273 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:27:25.526 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:25.646 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:25 smithi112 ceph-mon[31442]: pgmap v268: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-21T03:27:25.646 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:25 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/398343160' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:25.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:25 smithi203 ceph-mon[34723]: pgmap v268: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-21T03:27:25.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:25 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/398343160' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:25 smithi187 ceph-mon[34603]: pgmap v268: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-21T03:27:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:25 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/398343160' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:25.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.976+0000 7f0ef562a640 1 -- 172.21.15.112:0/497134077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0102ae0 msgr2=0x7f0ef0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:25.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.976+0000 7f0ef562a640 1 --2- 172.21.15.112:0/497134077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0102ae0 0x7f0ef0102ef0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009900 tx=0x7f0ee802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:27:25.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.976+0000 7f0ef562a640 1 -- 172.21.15.112:0/497134077 shutdown_connections 2024-09-21T03:27:25.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.976+0000 7f0ef562a640 1 --2- 172.21.15.112:0/497134077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef0108d70 0x7f0ef010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:25.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.976+0000 7f0ef562a640 1 --2- 172.21.15.112:0/497134077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef0108420 0x7f0ef0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:25.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.976+0000 7f0ef562a640 1 --2- 172.21.15.112:0/497134077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0102ae0 0x7f0ef0102ef0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:25.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.976+0000 7f0ef562a640 1 -- 172.21.15.112:0/497134077 >> 172.21.15.112:0/497134077 conn(0x7f0ef0079770 msgr2=0x7f0ef0079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:25.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.976+0000 7f0ef562a640 1 -- 172.21.15.112:0/497134077 shutdown_connections 2024-09-21T03:27:25.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.976+0000 7f0ef562a640 1 -- 172.21.15.112:0/497134077 wait complete. 2024-09-21T03:27:25.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0ef562a640 1 Processor -- start 2024-09-21T03:27:25.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0ef562a640 1 -- start start 2024-09-21T03:27:25.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0ef562a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef0102ae0 0x7f0ef019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:25.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0ef562a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0108420 0x7f0ef019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0ef562a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef0108d70 0x7f0ef019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0ef562a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0112a30 con 0x7f0ef0108420 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0ef562a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01128b0 con 0x7f0ef0108d70 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0ef562a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0112bb0 con 0x7f0ef0102ae0 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0eee7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0108420 0x7f0ef019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0eee7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0108420 0x7f0ef019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49700/0 (socket says 172.21.15.112:49700) 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0eee7fc640 1 -- 172.21.15.112:0/1353205025 learned_addr learned my addr 172.21.15.112:0/1353205025 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.977+0000 7f0eef7fe640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef0108d70 0x7f0ef019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0eeeffd640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef0102ae0 0x7f0ef019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0eee7fc640 1 -- 172.21.15.112:0/1353205025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef0102ae0 msgr2=0x7f0ef019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0eee7fc640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef0102ae0 0x7f0ef019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0eee7fc640 1 -- 172.21.15.112:0/1353205025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef0108d70 msgr2=0x7f0ef019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:25.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0eee7fc640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef0108d70 0x7f0ef019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:25.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0eee7fc640 1 -- 172.21.15.112:0/1353205025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef01a0480 con 0x7f0ef0108420 2024-09-21T03:27:25.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0eeeffd640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef0102ae0 0x7f0ef019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:25.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0eee7fc640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0108420 0x7f0ef019ab70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800d750 tx=0x7f0ed800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:25.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0ee7fff640 1 -- 172.21.15.112:0/1353205025 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8017070 con 0x7f0ef0108420 2024-09-21T03:27:25.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0ef562a640 1 -- 172.21.15.112:0/1353205025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef01a0770 con 0x7f0ef0108420 2024-09-21T03:27:25.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0ee7fff640 1 -- 172.21.15.112:0/1353205025 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed80054d0 con 0x7f0ef0108420 2024-09-21T03:27:25.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0ee7fff640 1 -- 172.21.15.112:0/1353205025 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8004c30 con 0x7f0ef0108420 2024-09-21T03:27:25.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.978+0000 7f0ef562a640 1 -- 172.21.15.112:0/1353205025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef01a6f90 con 0x7f0ef0108420 2024-09-21T03:27:25.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.979+0000 7f0ef562a640 1 -- 172.21.15.112:0/1353205025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef0103d60 con 0x7f0ef0108420 2024-09-21T03:27:25.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.981+0000 7f0ee7fff640 1 -- 172.21.15.112:0/1353205025 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f0ed8004dd0 con 0x7f0ef0108420 2024-09-21T03:27:25.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.981+0000 7f0ee7fff640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0edc0b6620 0x7f0edc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:25.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.981+0000 7f0eeeffd640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0edc0b6620 0x7f0edc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:25.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.981+0000 7f0ee7fff640 1 -- 172.21.15.112:0/1353205025 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f0ed80e72a0 con 0x7f0ef0108420 2024-09-21T03:27:25.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.982+0000 7f0eeeffd640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0edc0b6620 0x7f0edc0b8b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8031440 tx=0x7f0ee8009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:25.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:25.984+0000 7f0ee7fff640 1 -- 172.21.15.112:0/1353205025 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ed80ad9f0 con 0x7f0ef0108420 2024-09-21T03:27:26.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.162+0000 7f0ef562a640 1 -- 172.21.15.112:0/1353205025 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ef019b760 con 0x7f0edc0b6620 2024-09-21T03:27:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.166+0000 7f0ee7fff640 1 -- 172.21.15.112:0/1353205025 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258679 (secure 0 0 0) 0x7f0ef019b760 con 0x7f0edc0b6620 2024-09-21T03:27:26.171 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:27:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T03:27:26.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.169+0000 7f0ef562a640 1 -- 172.21.15.112:0/1353205025 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0edc0b6620 msgr2=0x7f0edc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.169+0000 7f0ef562a640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0edc0b6620 0x7f0edc0b8b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8031440 tx=0x7f0ee8009680 comp rx=0 tx=0).stop 2024-09-21T03:27:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.169+0000 7f0ef562a640 1 -- 172.21.15.112:0/1353205025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0108420 msgr2=0x7f0ef019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.169+0000 7f0ef562a640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0108420 0x7f0ef019ab70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800d750 tx=0x7f0ed800dc10 comp rx=0 tx=0).stop 2024-09-21T03:27:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.169+0000 7f0ef562a640 1 -- 172.21.15.112:0/1353205025 shutdown_connections 2024-09-21T03:27:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.169+0000 7f0ef562a640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0edc0b6620 0x7f0edc0b8b30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.169+0000 7f0ef562a640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef0108d70 0x7f0ef019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.169+0000 7f0ef562a640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ef0108420 0x7f0ef019ab70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.169+0000 7f0ef562a640 1 --2- 172.21.15.112:0/1353205025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef0102ae0 0x7f0ef019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.169+0000 7f0ef562a640 1 -- 172.21.15.112:0/1353205025 >> 172.21.15.112:0/1353205025 conn(0x7f0ef0079770 msgr2=0x7f0ef0101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.170+0000 7f0ef562a640 1 -- 172.21.15.112:0/1353205025 shutdown_connections 2024-09-21T03:27:26.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:26.170+0000 7f0ef562a640 1 -- 172.21.15.112:0/1353205025 wait complete. 2024-09-21T03:27:27.048 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-21T03:27:24.337415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639806,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":173,"num_read_kb":170,"num_write":270,"num_write_kb":2242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364888,"kb_used_data":11480,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576936,"statfs":{"total":1151940427776,"available":1151566782464,"internally_reserved":0,"allocated":11755520,"data_stored":6660177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44306,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":91,"num_read_kb":100,"num_write":42,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003588"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325965+0000","last_change":"2024-09-21T03:27:01.339980+0000","last_active":"2024-09-21T03:27:02.325965+0000","last_peered":"2024-09-21T03:27:02.325965+0000","last_clean":"2024-09-21T03:27:02.325965+0000","last_became_active":"2024-09-21T03:27:01.339552+0000","last_became_peered":"2024-09-21T03:27:01.339552+0000","last_unstale":"2024-09-21T03:27:02.325965+0000","last_undegraded":"2024-09-21T03:27:02.325965+0000","last_fullsized":"2024-09-21T03:27:02.325965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:04.240862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328531+0000","last_change":"2024-09-21T03:27:01.334171+0000","last_active":"2024-09-21T03:27:02.328531+0000","last_peered":"2024-09-21T03:27:02.328531+0000","last_clean":"2024-09-21T03:27:02.328531+0000","last_became_active":"2024-09-21T03:27:01.333851+0000","last_became_peered":"2024-09-21T03:27:01.333851+0000","last_unstale":"2024-09-21T03:27:02.328531+0000","last_undegraded":"2024-09-21T03:27:02.328531+0000","last_fullsized":"2024-09-21T03:27:02.328531+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:49.163142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328758+0000","last_change":"2024-09-21T03:27:01.339388+0000","last_active":"2024-09-21T03:27:02.328758+0000","last_peered":"2024-09-21T03:27:02.328758+0000","last_clean":"2024-09-21T03:27:02.328758+0000","last_became_active":"2024-09-21T03:27:01.339015+0000","last_became_peered":"2024-09-21T03:27:01.339015+0000","last_unstale":"2024-09-21T03:27:02.328758+0000","last_undegraded":"2024-09-21T03:27:02.328758+0000","last_fullsized":"2024-09-21T03:27:02.328758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:06.026397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318241+0000","last_change":"2024-09-21T03:27:01.344700+0000","last_active":"2024-09-21T03:27:02.318241+0000","last_peered":"2024-09-21T03:27:02.318241+0000","last_clean":"2024-09-21T03:27:02.318241+0000","last_became_active":"2024-09-21T03:27:01.344349+0000","last_became_peered":"2024-09-21T03:27:01.344349+0000","last_unstale":"2024-09-21T03:27:02.318241+0000","last_undegraded":"2024-09-21T03:27:02.318241+0000","last_fullsized":"2024-09-21T03:27:02.318241+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:00.271820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329641+0000","last_change":"2024-09-21T03:27:01.334904+0000","last_active":"2024-09-21T03:27:02.329641+0000","last_peered":"2024-09-21T03:27:02.329641+0000","last_clean":"2024-09-21T03:27:02.329641+0000","last_became_active":"2024-09-21T03:27:01.331916+0000","last_became_peered":"2024-09-21T03:27:01.331916+0000","last_unstale":"2024-09-21T03:27:02.329641+0000","last_undegraded":"2024-09-21T03:27:02.329641+0000","last_fullsized":"2024-09-21T03:27:02.329641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:49.376481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326109+0000","last_change":"2024-09-21T03:27:01.343256+0000","last_active":"2024-09-21T03:27:02.326109+0000","last_peered":"2024-09-21T03:27:02.326109+0000","last_clean":"2024-09-21T03:27:02.326109+0000","last_became_active":"2024-09-21T03:27:01.342861+0000","last_became_peered":"2024-09-21T03:27:01.342861+0000","last_unstale":"2024-09-21T03:27:02.326109+0000","last_undegraded":"2024-09-21T03:27:02.326109+0000","last_fullsized":"2024-09-21T03:27:02.326109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:00.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333140+0000","last_change":"2024-09-21T03:27:01.330618+0000","last_active":"2024-09-21T03:27:02.333140+0000","last_peered":"2024-09-21T03:27:02.333140+0000","last_clean":"2024-09-21T03:27:02.333140+0000","last_became_active":"2024-09-21T03:27:01.330018+0000","last_became_peered":"2024-09-21T03:27:01.330018+0000","last_unstale":"2024-09-21T03:27:02.333140+0000","last_undegraded":"2024-09-21T03:27:02.333140+0000","last_fullsized":"2024-09-21T03:27:02.333140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:03.291698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325482+0000","last_change":"2024-09-21T03:27:01.332840+0000","last_active":"2024-09-21T03:27:02.325482+0000","last_peered":"2024-09-21T03:27:02.325482+0000","last_clean":"2024-09-21T03:27:02.325482+0000","last_became_active":"2024-09-21T03:27:01.332513+0000","last_became_peered":"2024-09-21T03:27:01.332513+0000","last_unstale":"2024-09-21T03:27:02.325482+0000","last_undegraded":"2024-09-21T03:27:02.325482+0000","last_fullsized":"2024-09-21T03:27:02.325482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:17.554934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330017+0000","last_change":"2024-09-21T03:27:01.338147+0000","last_active":"2024-09-21T03:27:02.330017+0000","last_peered":"2024-09-21T03:27:02.330017+0000","last_clean":"2024-09-21T03:27:02.330017+0000","last_became_active":"2024-09-21T03:27:01.337759+0000","last_became_peered":"2024-09-21T03:27:01.337759+0000","last_unstale":"2024-09-21T03:27:02.330017+0000","last_undegraded":"2024-09-21T03:27:02.330017+0000","last_fullsized":"2024-09-21T03:27:02.330017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:38.999593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326819+0000","last_change":"2024-09-21T03:27:01.343373+0000","last_active":"2024-09-21T03:27:02.326819+0000","last_peered":"2024-09-21T03:27:02.326819+0000","last_clean":"2024-09-21T03:27:02.326819+0000","last_became_active":"2024-09-21T03:27:01.342924+0000","last_became_peered":"2024-09-21T03:27:01.342924+0000","last_unstale":"2024-09-21T03:27:02.326819+0000","last_undegraded":"2024-09-21T03:27:02.326819+0000","last_fullsized":"2024-09-21T03:27:02.326819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:33.869584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333559+0000","last_change":"2024-09-21T03:27:01.340120+0000","last_active":"2024-09-21T03:27:02.333559+0000","last_peered":"2024-09-21T03:27:02.333559+0000","last_clean":"2024-09-21T03:27:02.333559+0000","last_became_active":"2024-09-21T03:27:01.339708+0000","last_became_peered":"2024-09-21T03:27:01.339708+0000","last_unstale":"2024-09-21T03:27:02.333559+0000","last_undegraded":"2024-09-21T03:27:02.333559+0000","last_fullsized":"2024-09-21T03:27:02.333559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:56.193494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318438+0000","last_change":"2024-09-21T03:27:01.340205+0000","last_active":"2024-09-21T03:27:02.318438+0000","last_peered":"2024-09-21T03:27:02.318438+0000","last_clean":"2024-09-21T03:27:02.318438+0000","last_became_active":"2024-09-21T03:27:01.339839+0000","last_became_peered":"2024-09-21T03:27:01.339839+0000","last_unstale":"2024-09-21T03:27:02.318438+0000","last_undegraded":"2024-09-21T03:27:02.318438+0000","last_fullsized":"2024-09-21T03:27:02.318438+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:05.248636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317573+0000","last_change":"2024-09-21T03:27:01.334715+0000","last_active":"2024-09-21T03:27:02.317573+0000","last_peered":"2024-09-21T03:27:02.317573+0000","last_clean":"2024-09-21T03:27:02.317573+0000","last_became_active":"2024-09-21T03:27:01.334392+0000","last_became_peered":"2024-09-21T03:27:01.334392+0000","last_unstale":"2024-09-21T03:27:02.317573+0000","last_undegraded":"2024-09-21T03:27:02.317573+0000","last_fullsized":"2024-09-21T03:27:02.317573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:18.934063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328256+0000","last_change":"2024-09-21T03:27:01.337167+0000","last_active":"2024-09-21T03:27:02.328256+0000","last_peered":"2024-09-21T03:27:02.328256+0000","last_clean":"2024-09-21T03:27:02.328256+0000","last_became_active":"2024-09-21T03:27:01.336748+0000","last_became_peered":"2024-09-21T03:27:01.336748+0000","last_unstale":"2024-09-21T03:27:02.328256+0000","last_undegraded":"2024-09-21T03:27:02.328256+0000","last_fullsized":"2024-09-21T03:27:02.328256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:12.186727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328097+0000","last_change":"2024-09-21T03:27:01.336260+0000","last_active":"2024-09-21T03:27:02.328097+0000","last_peered":"2024-09-21T03:27:02.328097+0000","last_clean":"2024-09-21T03:27:02.328097+0000","last_became_active":"2024-09-21T03:27:01.335889+0000","last_became_peered":"2024-09-21T03:27:01.335889+0000","last_unstale":"2024-09-21T03:27:02.328097+0000","last_undegraded":"2024-09-21T03:27:02.328097+0000","last_fullsized":"2024-09-21T03:27:02.328097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:02.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328818+0000","last_change":"2024-09-21T03:27:01.337181+0000","last_active":"2024-09-21T03:27:02.328818+0000","last_peered":"2024-09-21T03:27:02.328818+0000","last_clean":"2024-09-21T03:27:02.328818+0000","last_became_active":"2024-09-21T03:27:01.336813+0000","last_became_peered":"2024-09-21T03:27:01.336813+0000","last_unstale":"2024-09-21T03:27:02.328818+0000","last_undegraded":"2024-09-21T03:27:02.328818+0000","last_fullsized":"2024-09-21T03:27:02.328818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:04.052319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326009+0000","last_change":"2024-09-21T03:27:01.342079+0000","last_active":"2024-09-21T03:27:02.326009+0000","last_peered":"2024-09-21T03:27:02.326009+0000","last_clean":"2024-09-21T03:27:02.326009+0000","last_became_active":"2024-09-21T03:27:01.341430+0000","last_became_peered":"2024-09-21T03:27:01.341430+0000","last_unstale":"2024-09-21T03:27:02.326009+0000","last_undegraded":"2024-09-21T03:27:02.326009+0000","last_fullsized":"2024-09-21T03:27:02.326009+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:08.029978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.327936+0000","last_change":"2024-09-21T03:27:01.338694+0000","last_active":"2024-09-21T03:27:02.327936+0000","last_peered":"2024-09-21T03:27:02.327936+0000","last_clean":"2024-09-21T03:27:02.327936+0000","last_became_active":"2024-09-21T03:27:01.338328+0000","last_became_peered":"2024-09-21T03:27:01.338328+0000","last_unstale":"2024-09-21T03:27:02.327936+0000","last_undegraded":"2024-09-21T03:27:02.327936+0000","last_fullsized":"2024-09-21T03:27:02.327936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:50.292238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318390+0000","last_change":"2024-09-21T03:27:01.336974+0000","last_active":"2024-09-21T03:27:02.318390+0000","last_peered":"2024-09-21T03:27:02.318390+0000","last_clean":"2024-09-21T03:27:02.318390+0000","last_became_active":"2024-09-21T03:27:01.336592+0000","last_became_peered":"2024-09-21T03:27:01.336592+0000","last_unstale":"2024-09-21T03:27:02.318390+0000","last_undegraded":"2024-09-21T03:27:02.318390+0000","last_fullsized":"2024-09-21T03:27:02.318390+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:09.399348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325473+0000","last_change":"2024-09-21T03:27:01.343326+0000","last_active":"2024-09-21T03:27:02.325473+0000","last_peered":"2024-09-21T03:27:02.325473+0000","last_clean":"2024-09-21T03:27:02.325473+0000","last_became_active":"2024-09-21T03:27:01.342937+0000","last_became_peered":"2024-09-21T03:27:01.342937+0000","last_unstale":"2024-09-21T03:27:02.325473+0000","last_undegraded":"2024-09-21T03:27:02.325473+0000","last_fullsized":"2024-09-21T03:27:02.325473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:01.782324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317457+0000","last_change":"2024-09-21T03:27:01.331242+0000","last_active":"2024-09-21T03:27:02.317457+0000","last_peered":"2024-09-21T03:27:02.317457+0000","last_clean":"2024-09-21T03:27:02.317457+0000","last_became_active":"2024-09-21T03:27:01.330883+0000","last_became_peered":"2024-09-21T03:27:01.330883+0000","last_unstale":"2024-09-21T03:27:02.317457+0000","last_undegraded":"2024-09-21T03:27:02.317457+0000","last_fullsized":"2024-09-21T03:27:02.317457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:07.158448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329300+0000","last_change":"2024-09-21T03:27:01.335522+0000","last_active":"2024-09-21T03:27:02.329300+0000","last_peered":"2024-09-21T03:27:02.329300+0000","last_clean":"2024-09-21T03:27:02.329300+0000","last_became_active":"2024-09-21T03:27:01.335114+0000","last_became_peered":"2024-09-21T03:27:01.335114+0000","last_unstale":"2024-09-21T03:27:02.329300+0000","last_undegraded":"2024-09-21T03:27:02.329300+0000","last_fullsized":"2024-09-21T03:27:02.329300+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:28.383410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325441+0000","last_change":"2024-09-21T03:27:01.329428+0000","last_active":"2024-09-21T03:27:02.325441+0000","last_peered":"2024-09-21T03:27:02.325441+0000","last_clean":"2024-09-21T03:27:02.325441+0000","last_became_active":"2024-09-21T03:27:01.328882+0000","last_became_peered":"2024-09-21T03:27:01.328882+0000","last_unstale":"2024-09-21T03:27:02.325441+0000","last_undegraded":"2024-09-21T03:27:02.325441+0000","last_fullsized":"2024-09-21T03:27:02.325441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:58.634995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330137+0000","last_change":"2024-09-21T03:27:01.325536+0000","last_active":"2024-09-21T03:27:02.330137+0000","last_peered":"2024-09-21T03:27:02.330137+0000","last_clean":"2024-09-21T03:27:02.330137+0000","last_became_active":"2024-09-21T03:27:01.324988+0000","last_became_peered":"2024-09-21T03:27:01.324988+0000","last_unstale":"2024-09-21T03:27:02.330137+0000","last_undegraded":"2024-09-21T03:27:02.330137+0000","last_fullsized":"2024-09-21T03:27:02.330137+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:51.856606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325891+0000","last_change":"2024-09-21T03:27:01.330988+0000","last_active":"2024-09-21T03:27:02.325891+0000","last_peered":"2024-09-21T03:27:02.325891+0000","last_clean":"2024-09-21T03:27:02.325891+0000","last_became_active":"2024-09-21T03:27:01.330574+0000","last_became_peered":"2024-09-21T03:27:01.330574+0000","last_unstale":"2024-09-21T03:27:02.325891+0000","last_undegraded":"2024-09-21T03:27:02.325891+0000","last_fullsized":"2024-09-21T03:27:02.325891+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:45.277947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318854+0000","last_change":"2024-09-21T03:27:01.339606+0000","last_active":"2024-09-21T03:27:02.318854+0000","last_peered":"2024-09-21T03:27:02.318854+0000","last_clean":"2024-09-21T03:27:02.318854+0000","last_became_active":"2024-09-21T03:27:01.339208+0000","last_became_peered":"2024-09-21T03:27:01.339208+0000","last_unstale":"2024-09-21T03:27:02.318854+0000","last_undegraded":"2024-09-21T03:27:02.318854+0000","last_fullsized":"2024-09-21T03:27:02.318854+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:31.593363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325740+0000","last_change":"2024-09-21T03:27:01.343634+0000","last_active":"2024-09-21T03:27:02.325740+0000","last_peered":"2024-09-21T03:27:02.325740+0000","last_clean":"2024-09-21T03:27:02.325740+0000","last_became_active":"2024-09-21T03:27:01.342965+0000","last_became_peered":"2024-09-21T03:27:01.342965+0000","last_unstale":"2024-09-21T03:27:02.325740+0000","last_undegraded":"2024-09-21T03:27:02.325740+0000","last_fullsized":"2024-09-21T03:27:02.325740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:46.916198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317451+0000","last_change":"2024-09-21T03:27:01.324993+0000","last_active":"2024-09-21T03:27:02.317451+0000","last_peered":"2024-09-21T03:27:02.317451+0000","last_clean":"2024-09-21T03:27:02.317451+0000","last_became_active":"2024-09-21T03:27:01.324593+0000","last_became_peered":"2024-09-21T03:27:01.324593+0000","last_unstale":"2024-09-21T03:27:02.317451+0000","last_undegraded":"2024-09-21T03:27:02.317451+0000","last_fullsized":"2024-09-21T03:27:02.317451+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:13.211539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328598+0000","last_change":"2024-09-21T03:27:01.325232+0000","last_active":"2024-09-21T03:27:02.328598+0000","last_peered":"2024-09-21T03:27:02.328598+0000","last_clean":"2024-09-21T03:27:02.328598+0000","last_became_active":"2024-09-21T03:27:01.324570+0000","last_became_peered":"2024-09-21T03:27:01.324570+0000","last_unstale":"2024-09-21T03:27:02.328598+0000","last_undegraded":"2024-09-21T03:27:02.328598+0000","last_fullsized":"2024-09-21T03:27:02.328598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:30.843334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317814+0000","last_change":"2024-09-21T03:27:01.338789+0000","last_active":"2024-09-21T03:27:02.317814+0000","last_peered":"2024-09-21T03:27:02.317814+0000","last_clean":"2024-09-21T03:27:02.317814+0000","last_became_active":"2024-09-21T03:27:01.338282+0000","last_became_peered":"2024-09-21T03:27:01.338282+0000","last_unstale":"2024-09-21T03:27:02.317814+0000","last_undegraded":"2024-09-21T03:27:02.317814+0000","last_fullsized":"2024-09-21T03:27:02.317814+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:57.213343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318229+0000","last_change":"2024-09-21T03:27:01.339521+0000","last_active":"2024-09-21T03:27:02.318229+0000","last_peered":"2024-09-21T03:27:02.318229+0000","last_clean":"2024-09-21T03:27:02.318229+0000","last_became_active":"2024-09-21T03:27:01.339175+0000","last_became_peered":"2024-09-21T03:27:01.339175+0000","last_unstale":"2024-09-21T03:27:02.318229+0000","last_undegraded":"2024-09-21T03:27:02.318229+0000","last_fullsized":"2024-09-21T03:27:02.318229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:46.163698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325360+0000","last_change":"2024-09-21T03:27:01.338352+0000","last_active":"2024-09-21T03:27:02.325360+0000","last_peered":"2024-09-21T03:27:02.325360+0000","last_clean":"2024-09-21T03:27:02.325360+0000","last_became_active":"2024-09-21T03:27:01.337306+0000","last_became_peered":"2024-09-21T03:27:01.337306+0000","last_unstale":"2024-09-21T03:27:02.325360+0000","last_undegraded":"2024-09-21T03:27:02.325360+0000","last_fullsized":"2024-09-21T03:27:02.325360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.799039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328608+0000","last_change":"2024-09-21T03:27:01.332461+0000","last_active":"2024-09-21T03:27:02.328608+0000","last_peered":"2024-09-21T03:27:02.328608+0000","last_clean":"2024-09-21T03:27:02.328608+0000","last_became_active":"2024-09-21T03:27:01.331912+0000","last_became_peered":"2024-09-21T03:27:01.331912+0000","last_unstale":"2024-09-21T03:27:02.328608+0000","last_undegraded":"2024-09-21T03:27:02.328608+0000","last_fullsized":"2024-09-21T03:27:02.328608+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:34.771691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:18.406803+0000","last_change":"2024-09-21T03:26:59.317040+0000","last_active":"2024-09-21T03:27:18.406803+0000","last_peered":"2024-09-21T03:27:18.406803+0000","last_clean":"2024-09-21T03:27:18.406803+0000","last_became_active":"2024-09-21T03:26:59.316571+0000","last_became_peered":"2024-09-21T03:26:59.316571+0000","last_unstale":"2024-09-21T03:27:18.406803+0000","last_undegraded":"2024-09-21T03:27:18.406803+0000","last_fullsized":"2024-09-21T03:27:18.406803+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:55.788382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329094+0000","last_change":"2024-09-21T03:27:01.324364+0000","last_active":"2024-09-21T03:27:02.329094+0000","last_peered":"2024-09-21T03:27:02.329094+0000","last_clean":"2024-09-21T03:27:02.329094+0000","last_became_active":"2024-09-21T03:27:01.324018+0000","last_became_peered":"2024-09-21T03:27:01.324018+0000","last_unstale":"2024-09-21T03:27:02.329094+0000","last_undegraded":"2024-09-21T03:27:02.329094+0000","last_fullsized":"2024-09-21T03:27:02.329094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:56.404893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'3","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.390553+0000","last_change":"2024-09-21T03:26:59.313289+0000","last_active":"2024-09-21T03:27:10.390553+0000","last_peered":"2024-09-21T03:27:10.390553+0000","last_clean":"2024-09-21T03:27:10.390553+0000","last_became_active":"2024-09-21T03:26:59.312877+0000","last_became_peered":"2024-09-21T03:26:59.312877+0000","last_unstale":"2024-09-21T03:27:10.390553+0000","last_undegraded":"2024-09-21T03:27:10.390553+0000","last_fullsized":"2024-09-21T03:27:10.390553+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:53.856541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.387703+0000","last_change":"2024-09-21T03:26:59.307098+0000","last_active":"2024-09-21T03:27:10.387703+0000","last_peered":"2024-09-21T03:27:10.387703+0000","last_clean":"2024-09-21T03:27:10.387703+0000","last_became_active":"2024-09-21T03:26:59.306650+0000","last_became_peered":"2024-09-21T03:26:59.306650+0000","last_unstale":"2024-09-21T03:27:10.387703+0000","last_undegraded":"2024-09-21T03:27:10.387703+0000","last_fullsized":"2024-09-21T03:27:10.387703+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:44.033899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325372+0000","last_change":"2024-09-21T03:27:01.328443+0000","last_active":"2024-09-21T03:27:02.325372+0000","last_peered":"2024-09-21T03:27:02.325372+0000","last_clean":"2024-09-21T03:27:02.325372+0000","last_became_active":"2024-09-21T03:27:01.328107+0000","last_became_peered":"2024-09-21T03:27:01.328107+0000","last_unstale":"2024-09-21T03:27:02.325372+0000","last_undegraded":"2024-09-21T03:27:02.325372+0000","last_fullsized":"2024-09-21T03:27:02.325372+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:47.798308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'1","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330718+0000","last_change":"2024-09-21T03:26:59.317934+0000","last_active":"2024-09-21T03:27:02.330718+0000","last_peered":"2024-09-21T03:27:02.330718+0000","last_clean":"2024-09-21T03:27:02.330718+0000","last_became_active":"2024-09-21T03:26:59.317166+0000","last_became_peered":"2024-09-21T03:26:59.317166+0000","last_unstale":"2024-09-21T03:27:02.330718+0000","last_undegraded":"2024-09-21T03:27:02.330718+0000","last_fullsized":"2024-09-21T03:27:02.330718+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:52.746406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.327216+0000","last_change":"2024-09-21T03:27:01.331226+0000","last_active":"2024-09-21T03:27:02.327216+0000","last_peered":"2024-09-21T03:27:02.327216+0000","last_clean":"2024-09-21T03:27:02.327216+0000","last_became_active":"2024-09-21T03:27:01.330889+0000","last_became_peered":"2024-09-21T03:27:01.330889+0000","last_unstale":"2024-09-21T03:27:02.327216+0000","last_undegraded":"2024-09-21T03:27:02.327216+0000","last_fullsized":"2024-09-21T03:27:02.327216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:51.181554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'3","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:22.412154+0000","last_change":"2024-09-21T03:26:59.311671+0000","last_active":"2024-09-21T03:27:22.412154+0000","last_peered":"2024-09-21T03:27:22.412154+0000","last_clean":"2024-09-21T03:27:22.412154+0000","last_became_active":"2024-09-21T03:26:59.311149+0000","last_became_peered":"2024-09-21T03:26:59.311149+0000","last_unstale":"2024-09-21T03:27:22.412154+0000","last_undegraded":"2024-09-21T03:27:22.412154+0000","last_fullsized":"2024-09-21T03:27:22.412154+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:47.878274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329762+0000","last_change":"2024-09-21T03:27:01.336142+0000","last_active":"2024-09-21T03:27:02.329762+0000","last_peered":"2024-09-21T03:27:02.329762+0000","last_clean":"2024-09-21T03:27:02.329762+0000","last_became_active":"2024-09-21T03:27:01.335795+0000","last_became_peered":"2024-09-21T03:27:01.335795+0000","last_unstale":"2024-09-21T03:27:02.329762+0000","last_undegraded":"2024-09-21T03:27:02.329762+0000","last_fullsized":"2024-09-21T03:27:02.329762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:29.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.375872+0000","last_change":"2024-09-21T03:26:59.315077+0000","last_active":"2024-09-21T03:27:09.375872+0000","last_peered":"2024-09-21T03:27:09.375872+0000","last_clean":"2024-09-21T03:27:09.375872+0000","last_became_active":"2024-09-21T03:26:59.312347+0000","last_became_peered":"2024-09-21T03:26:59.312347+0000","last_unstale":"2024-09-21T03:27:09.375872+0000","last_undegraded":"2024-09-21T03:27:09.375872+0000","last_fullsized":"2024-09-21T03:27:09.375872+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:24.529066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328919+0000","last_change":"2024-09-21T03:27:01.324346+0000","last_active":"2024-09-21T03:27:02.328919+0000","last_peered":"2024-09-21T03:27:02.328919+0000","last_clean":"2024-09-21T03:27:02.328919+0000","last_became_active":"2024-09-21T03:27:01.323976+0000","last_became_peered":"2024-09-21T03:27:01.323976+0000","last_unstale":"2024-09-21T03:27:02.328919+0000","last_undegraded":"2024-09-21T03:27:02.328919+0000","last_fullsized":"2024-09-21T03:27:02.328919+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:04.591595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'4","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.376641+0000","last_change":"2024-09-21T03:26:59.326206+0000","last_active":"2024-09-21T03:27:09.376641+0000","last_peered":"2024-09-21T03:27:09.376641+0000","last_clean":"2024-09-21T03:27:09.376641+0000","last_became_active":"2024-09-21T03:26:59.325755+0000","last_became_peered":"2024-09-21T03:26:59.325755+0000","last_unstale":"2024-09-21T03:27:09.376641+0000","last_undegraded":"2024-09-21T03:27:09.376641+0000","last_fullsized":"2024-09-21T03:27:09.376641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:23.220795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329706+0000","last_change":"2024-09-21T03:27:01.341501+0000","last_active":"2024-09-21T03:27:02.329706+0000","last_peered":"2024-09-21T03:27:02.329706+0000","last_clean":"2024-09-21T03:27:02.329706+0000","last_became_active":"2024-09-21T03:27:01.341039+0000","last_became_peered":"2024-09-21T03:27:01.341039+0000","last_unstale":"2024-09-21T03:27:02.329706+0000","last_undegraded":"2024-09-21T03:27:02.329706+0000","last_fullsized":"2024-09-21T03:27:02.329706+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:15.132180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"74'21","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:23.419481+0000","last_change":"2024-09-21T03:26:59.313489+0000","last_active":"2024-09-21T03:27:23.419481+0000","last_peered":"2024-09-21T03:27:23.419481+0000","last_clean":"2024-09-21T03:27:23.419481+0000","last_became_active":"2024-09-21T03:26:59.313022+0000","last_became_peered":"2024-09-21T03:26:59.313022+0000","last_unstale":"2024-09-21T03:27:23.419481+0000","last_undegraded":"2024-09-21T03:27:23.419481+0000","last_fullsized":"2024-09-21T03:27:23.419481+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:08.541315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45203,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":59,"num_write":22,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329332+0000","last_change":"2024-09-21T03:27:01.339329+0000","last_active":"2024-09-21T03:27:02.329332+0000","last_peered":"2024-09-21T03:27:02.329332+0000","last_clean":"2024-09-21T03:27:02.329332+0000","last_became_active":"2024-09-21T03:27:01.338982+0000","last_became_peered":"2024-09-21T03:27:01.338982+0000","last_unstale":"2024-09-21T03:27:02.329332+0000","last_undegraded":"2024-09-21T03:27:02.329332+0000","last_fullsized":"2024-09-21T03:27:02.329332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:41.759450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'6","reported_seq":26,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.387802+0000","last_change":"2024-09-21T03:26:59.313270+0000","last_active":"2024-09-21T03:27:10.387802+0000","last_peered":"2024-09-21T03:27:10.387802+0000","last_clean":"2024-09-21T03:27:10.387802+0000","last_became_active":"2024-09-21T03:26:59.312867+0000","last_became_peered":"2024-09-21T03:26:59.312867+0000","last_unstale":"2024-09-21T03:27:10.387802+0000","last_undegraded":"2024-09-21T03:27:10.387802+0000","last_fullsized":"2024-09-21T03:27:10.387802+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:53.798022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317064+0000","last_change":"2024-09-21T03:27:01.331268+0000","last_active":"2024-09-21T03:27:02.317064+0000","last_peered":"2024-09-21T03:27:02.317064+0000","last_clean":"2024-09-21T03:27:02.317064+0000","last_became_active":"2024-09-21T03:27:01.330922+0000","last_became_peered":"2024-09-21T03:27:01.330922+0000","last_unstale":"2024-09-21T03:27:02.317064+0000","last_undegraded":"2024-09-21T03:27:02.317064+0000","last_fullsized":"2024-09-21T03:27:02.317064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'3","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.375592+0000","last_change":"2024-09-21T03:26:59.318388+0000","last_active":"2024-09-21T03:27:09.375592+0000","last_peered":"2024-09-21T03:27:09.375592+0000","last_clean":"2024-09-21T03:27:09.375592+0000","last_became_active":"2024-09-21T03:26:59.317938+0000","last_became_peered":"2024-09-21T03:26:59.317938+0000","last_unstale":"2024-09-21T03:27:09.375592+0000","last_undegraded":"2024-09-21T03:27:09.375592+0000","last_fullsized":"2024-09-21T03:27:09.375592+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.499699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333593+0000","last_change":"2024-09-21T03:27:01.340240+0000","last_active":"2024-09-21T03:27:02.333593+0000","last_peered":"2024-09-21T03:27:02.333593+0000","last_clean":"2024-09-21T03:27:02.333593+0000","last_became_active":"2024-09-21T03:27:01.339531+0000","last_became_peered":"2024-09-21T03:27:01.339531+0000","last_unstale":"2024-09-21T03:27:02.333593+0000","last_undegraded":"2024-09-21T03:27:02.333593+0000","last_fullsized":"2024-09-21T03:27:02.333593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:45.268210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'4","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:18.406965+0000","last_change":"2024-09-21T03:26:59.311345+0000","last_active":"2024-09-21T03:27:18.406965+0000","last_peered":"2024-09-21T03:27:18.406965+0000","last_clean":"2024-09-21T03:27:18.406965+0000","last_became_active":"2024-09-21T03:26:59.310928+0000","last_became_peered":"2024-09-21T03:26:59.310928+0000","last_unstale":"2024-09-21T03:27:18.406965+0000","last_undegraded":"2024-09-21T03:27:18.406965+0000","last_fullsized":"2024-09-21T03:27:18.406965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.614955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333101+0000","last_change":"2024-09-21T03:27:01.335785+0000","last_active":"2024-09-21T03:27:02.333101+0000","last_peered":"2024-09-21T03:27:02.333101+0000","last_clean":"2024-09-21T03:27:02.333101+0000","last_became_active":"2024-09-21T03:27:01.335396+0000","last_became_peered":"2024-09-21T03:27:01.335396+0000","last_unstale":"2024-09-21T03:27:02.333101+0000","last_undegraded":"2024-09-21T03:27:02.333101+0000","last_fullsized":"2024-09-21T03:27:02.333101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:52.785641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'15","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.391594+0000","last_change":"2024-09-21T03:26:59.325574+0000","last_active":"2024-09-21T03:27:10.391594+0000","last_peered":"2024-09-21T03:27:10.391594+0000","last_clean":"2024-09-21T03:27:10.391594+0000","last_became_active":"2024-09-21T03:26:59.325074+0000","last_became_peered":"2024-09-21T03:26:59.325074+0000","last_unstale":"2024-09-21T03:27:10.391594+0000","last_undegraded":"2024-09-21T03:27:10.391594+0000","last_fullsized":"2024-09-21T03:27:10.391594+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:19.989009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330112+0000","last_change":"2024-09-21T03:27:01.342282+0000","last_active":"2024-09-21T03:27:02.330112+0000","last_peered":"2024-09-21T03:27:02.330112+0000","last_clean":"2024-09-21T03:27:02.330112+0000","last_became_active":"2024-09-21T03:27:01.341790+0000","last_became_peered":"2024-09-21T03:27:01.341790+0000","last_unstale":"2024-09-21T03:27:02.330112+0000","last_undegraded":"2024-09-21T03:27:02.330112+0000","last_fullsized":"2024-09-21T03:27:02.330112+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:05.432720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:11.395758+0000","last_change":"2024-09-21T03:26:59.312442+0000","last_active":"2024-09-21T03:27:11.395758+0000","last_peered":"2024-09-21T03:27:11.395758+0000","last_clean":"2024-09-21T03:27:11.395758+0000","last_became_active":"2024-09-21T03:26:59.311601+0000","last_became_peered":"2024-09-21T03:26:59.311601+0000","last_unstale":"2024-09-21T03:27:11.395758+0000","last_undegraded":"2024-09-21T03:27:11.395758+0000","last_fullsized":"2024-09-21T03:27:11.395758+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:32.421228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325744+0000","last_change":"2024-09-21T03:27:01.337917+0000","last_active":"2024-09-21T03:27:02.325744+0000","last_peered":"2024-09-21T03:27:02.325744+0000","last_clean":"2024-09-21T03:27:02.325744+0000","last_became_active":"2024-09-21T03:27:01.337457+0000","last_became_peered":"2024-09-21T03:27:01.337457+0000","last_unstale":"2024-09-21T03:27:02.325744+0000","last_undegraded":"2024-09-21T03:27:02.325744+0000","last_fullsized":"2024-09-21T03:27:02.325744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:49.011795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.324662+0000","last_change":"2024-09-21T03:26:59.309265+0000","last_active":"2024-09-21T03:27:02.324662+0000","last_peered":"2024-09-21T03:27:02.324662+0000","last_clean":"2024-09-21T03:27:02.324662+0000","last_became_active":"2024-09-21T03:26:59.308904+0000","last_became_peered":"2024-09-21T03:26:59.308904+0000","last_unstale":"2024-09-21T03:27:02.324662+0000","last_undegraded":"2024-09-21T03:27:02.324662+0000","last_fullsized":"2024-09-21T03:27:02.324662+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:07.335210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333819+0000","last_change":"2024-09-21T03:27:01.338489+0000","last_active":"2024-09-21T03:27:02.333819+0000","last_peered":"2024-09-21T03:27:02.333819+0000","last_clean":"2024-09-21T03:27:02.333819+0000","last_became_active":"2024-09-21T03:27:01.338123+0000","last_became_peered":"2024-09-21T03:27:01.338123+0000","last_unstale":"2024-09-21T03:27:02.333819+0000","last_undegraded":"2024-09-21T03:27:02.333819+0000","last_fullsized":"2024-09-21T03:27:02.333819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:15.137546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.334246+0000","last_change":"2024-09-21T03:26:59.315371+0000","last_active":"2024-09-21T03:27:02.334246+0000","last_peered":"2024-09-21T03:27:02.334246+0000","last_clean":"2024-09-21T03:27:02.334246+0000","last_became_active":"2024-09-21T03:26:59.314439+0000","last_became_peered":"2024-09-21T03:26:59.314439+0000","last_unstale":"2024-09-21T03:27:02.334246+0000","last_undegraded":"2024-09-21T03:27:02.334246+0000","last_fullsized":"2024-09-21T03:27:02.334246+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:13.656324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333518+0000","last_change":"2024-09-21T03:27:01.331086+0000","last_active":"2024-09-21T03:27:02.333518+0000","last_peered":"2024-09-21T03:27:02.333518+0000","last_clean":"2024-09-21T03:27:02.333518+0000","last_became_active":"2024-09-21T03:27:01.329113+0000","last_became_peered":"2024-09-21T03:27:01.329113+0000","last_unstale":"2024-09-21T03:27:02.333518+0000","last_undegraded":"2024-09-21T03:27:02.333518+0000","last_fullsized":"2024-09-21T03:27:02.333518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:25.514730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'3","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.388642+0000","last_change":"2024-09-21T03:26:59.554087+0000","last_active":"2024-09-21T03:27:10.388642+0000","last_peered":"2024-09-21T03:27:10.388642+0000","last_clean":"2024-09-21T03:27:10.388642+0000","last_became_active":"2024-09-21T03:26:59.553610+0000","last_became_peered":"2024-09-21T03:26:59.553610+0000","last_unstale":"2024-09-21T03:27:10.388642+0000","last_undegraded":"2024-09-21T03:27:10.388642+0000","last_fullsized":"2024-09-21T03:27:10.388642+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:22.048057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333861+0000","last_change":"2024-09-21T03:27:01.339987+0000","last_active":"2024-09-21T03:27:02.333861+0000","last_peered":"2024-09-21T03:27:02.333861+0000","last_clean":"2024-09-21T03:27:02.333861+0000","last_became_active":"2024-09-21T03:27:01.339060+0000","last_became_peered":"2024-09-21T03:27:01.339060+0000","last_unstale":"2024-09-21T03:27:02.333861+0000","last_undegraded":"2024-09-21T03:27:02.333861+0000","last_fullsized":"2024-09-21T03:27:02.333861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.901337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":91,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333471+0000","last_change":"2024-09-21T03:25:50.381920+0000","last_active":"2024-09-21T03:27:02.333471+0000","last_peered":"2024-09-21T03:27:02.333471+0000","last_clean":"2024-09-21T03:27:02.333471+0000","last_became_active":"2024-09-21T03:25:50.376170+0000","last_became_peered":"2024-09-21T03:25:50.376170+0000","last_unstale":"2024-09-21T03:27:02.333471+0000","last_undegraded":"2024-09-21T03:27:02.333471+0000","last_fullsized":"2024-09-21T03:27:02.333471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_clean_scrub_stamp":"2024-09-21T03:22:31.269883+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:57.281150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'3","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:11.393293+0000","last_change":"2024-09-21T03:26:59.313612+0000","last_active":"2024-09-21T03:27:11.393293+0000","last_peered":"2024-09-21T03:27:11.393293+0000","last_clean":"2024-09-21T03:27:11.393293+0000","last_became_active":"2024-09-21T03:26:59.313060+0000","last_became_peered":"2024-09-21T03:26:59.313060+0000","last_unstale":"2024-09-21T03:27:11.393293+0000","last_undegraded":"2024-09-21T03:27:11.393293+0000","last_fullsized":"2024-09-21T03:27:11.393293+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:06.193526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326447+0000","last_change":"2024-09-21T03:27:01.337784+0000","last_active":"2024-09-21T03:27:02.326447+0000","last_peered":"2024-09-21T03:27:02.326447+0000","last_clean":"2024-09-21T03:27:02.326447+0000","last_became_active":"2024-09-21T03:27:01.337415+0000","last_became_peered":"2024-09-21T03:27:01.337415+0000","last_unstale":"2024-09-21T03:27:02.326447+0000","last_undegraded":"2024-09-21T03:27:02.326447+0000","last_fullsized":"2024-09-21T03:27:02.326447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:40.482054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'3","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.387643+0000","last_change":"2024-09-21T03:26:59.318553+0000","last_active":"2024-09-21T03:27:10.387643+0000","last_peered":"2024-09-21T03:27:10.387643+0000","last_clean":"2024-09-21T03:27:10.387643+0000","last_became_active":"2024-09-21T03:26:59.318061+0000","last_became_peered":"2024-09-21T03:26:59.318061+0000","last_unstale":"2024-09-21T03:27:10.387643+0000","last_undegraded":"2024-09-21T03:27:10.387643+0000","last_fullsized":"2024-09-21T03:27:10.387643+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:13.157253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.316926+0000","last_change":"2024-09-21T03:27:01.333769+0000","last_active":"2024-09-21T03:27:02.316926+0000","last_peered":"2024-09-21T03:27:02.316926+0000","last_clean":"2024-09-21T03:27:02.316926+0000","last_became_active":"2024-09-21T03:27:01.333436+0000","last_became_peered":"2024-09-21T03:27:01.333436+0000","last_unstale":"2024-09-21T03:27:02.316926+0000","last_undegraded":"2024-09-21T03:27:02.316926+0000","last_fullsized":"2024-09-21T03:27:02.316926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:30.395916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'4","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:19.407840+0000","last_change":"2024-09-21T03:26:59.325314+0000","last_active":"2024-09-21T03:27:19.407840+0000","last_peered":"2024-09-21T03:27:19.407840+0000","last_clean":"2024-09-21T03:27:19.407840+0000","last_became_active":"2024-09-21T03:26:59.324752+0000","last_became_peered":"2024-09-21T03:26:59.324752+0000","last_unstale":"2024-09-21T03:27:19.407840+0000","last_undegraded":"2024-09-21T03:27:19.407840+0000","last_fullsized":"2024-09-21T03:27:19.407840+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.252596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330400+0000","last_change":"2024-09-21T03:27:01.335154+0000","last_active":"2024-09-21T03:27:02.330400+0000","last_peered":"2024-09-21T03:27:02.330400+0000","last_clean":"2024-09-21T03:27:02.330400+0000","last_became_active":"2024-09-21T03:27:01.334633+0000","last_became_peered":"2024-09-21T03:27:01.334633+0000","last_unstale":"2024-09-21T03:27:02.330400+0000","last_undegraded":"2024-09-21T03:27:02.330400+0000","last_fullsized":"2024-09-21T03:27:02.330400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:27.992858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'1","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333802+0000","last_change":"2024-09-21T03:26:59.314019+0000","last_active":"2024-09-21T03:27:02.333802+0000","last_peered":"2024-09-21T03:27:02.333802+0000","last_clean":"2024-09-21T03:27:02.333802+0000","last_became_active":"2024-09-21T03:26:59.311915+0000","last_became_peered":"2024-09-21T03:26:59.311915+0000","last_unstale":"2024-09-21T03:27:02.333802+0000","last_undegraded":"2024-09-21T03:27:02.333802+0000","last_fullsized":"2024-09-21T03:27:02.333802+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:51.412303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326370+0000","last_change":"2024-09-21T03:27:01.344921+0000","last_active":"2024-09-21T03:27:02.326370+0000","last_peered":"2024-09-21T03:27:02.326370+0000","last_clean":"2024-09-21T03:27:02.326370+0000","last_became_active":"2024-09-21T03:27:01.344517+0000","last_became_peered":"2024-09-21T03:27:01.344517+0000","last_unstale":"2024-09-21T03:27:02.326370+0000","last_undegraded":"2024-09-21T03:27:02.326370+0000","last_fullsized":"2024-09-21T03:27:02.326370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:38.549938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.327627+0000","last_change":"2024-09-21T03:26:59.325943+0000","last_active":"2024-09-21T03:27:02.327627+0000","last_peered":"2024-09-21T03:27:02.327627+0000","last_clean":"2024-09-21T03:27:02.327627+0000","last_became_active":"2024-09-21T03:26:59.325524+0000","last_became_peered":"2024-09-21T03:26:59.325524+0000","last_unstale":"2024-09-21T03:27:02.327627+0000","last_undegraded":"2024-09-21T03:27:02.327627+0000","last_fullsized":"2024-09-21T03:27:02.327627+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:45.111480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:03.425334+0000","last_change":"2024-09-21T03:27:01.335369+0000","last_active":"2024-09-21T03:27:03.425334+0000","last_peered":"2024-09-21T03:27:03.425334+0000","last_clean":"2024-09-21T03:27:03.425334+0000","last_became_active":"2024-09-21T03:27:01.334997+0000","last_became_peered":"2024-09-21T03:27:01.334997+0000","last_unstale":"2024-09-21T03:27:03.425334+0000","last_undegraded":"2024-09-21T03:27:03.425334+0000","last_fullsized":"2024-09-21T03:27:03.425334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:10.257451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'1","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.375338+0000","last_change":"2024-09-21T03:26:59.326204+0000","last_active":"2024-09-21T03:27:09.375338+0000","last_peered":"2024-09-21T03:27:09.375338+0000","last_clean":"2024-09-21T03:27:09.375338+0000","last_became_active":"2024-09-21T03:26:59.325431+0000","last_became_peered":"2024-09-21T03:26:59.325431+0000","last_unstale":"2024-09-21T03:27:09.375338+0000","last_undegraded":"2024-09-21T03:27:09.375338+0000","last_fullsized":"2024-09-21T03:27:09.375338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:27.863217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317522+0000","last_change":"2024-09-21T03:27:01.333861+0000","last_active":"2024-09-21T03:27:02.317522+0000","last_peered":"2024-09-21T03:27:02.317522+0000","last_clean":"2024-09-21T03:27:02.317522+0000","last_became_active":"2024-09-21T03:27:01.333485+0000","last_became_peered":"2024-09-21T03:27:01.333485+0000","last_unstale":"2024-09-21T03:27:02.317522+0000","last_undegraded":"2024-09-21T03:27:02.317522+0000","last_fullsized":"2024-09-21T03:27:02.317522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:04.475811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"74'6","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:17.324020+0000","last_change":"2024-09-21T03:26:59.309767+0000","last_active":"2024-09-21T03:27:17.324020+0000","last_peered":"2024-09-21T03:27:17.324020+0000","last_clean":"2024-09-21T03:27:17.324020+0000","last_became_active":"2024-09-21T03:26:59.309354+0000","last_became_peered":"2024-09-21T03:26:59.309354+0000","last_unstale":"2024-09-21T03:27:17.324020+0000","last_undegraded":"2024-09-21T03:27:17.324020+0000","last_fullsized":"2024-09-21T03:27:17.324020+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:26.728719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326117+0000","last_change":"2024-09-21T03:27:01.333097+0000","last_active":"2024-09-21T03:27:02.326117+0000","last_peered":"2024-09-21T03:27:02.326117+0000","last_clean":"2024-09-21T03:27:02.326117+0000","last_became_active":"2024-09-21T03:27:01.332743+0000","last_became_peered":"2024-09-21T03:27:01.332743+0000","last_unstale":"2024-09-21T03:27:02.326117+0000","last_undegraded":"2024-09-21T03:27:02.326117+0000","last_fullsized":"2024-09-21T03:27:02.326117+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:41.004213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'4","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.376149+0000","last_change":"2024-09-21T03:26:59.553974+0000","last_active":"2024-09-21T03:27:09.376149+0000","last_peered":"2024-09-21T03:27:09.376149+0000","last_clean":"2024-09-21T03:27:09.376149+0000","last_became_active":"2024-09-21T03:26:59.553647+0000","last_became_peered":"2024-09-21T03:26:59.553647+0000","last_unstale":"2024-09-21T03:27:09.376149+0000","last_undegraded":"2024-09-21T03:27:09.376149+0000","last_fullsized":"2024-09-21T03:27:09.376149+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:00.490951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328791+0000","last_change":"2024-09-21T03:27:01.325308+0000","last_active":"2024-09-21T03:27:02.328791+0000","last_peered":"2024-09-21T03:27:02.328791+0000","last_clean":"2024-09-21T03:27:02.328791+0000","last_became_active":"2024-09-21T03:27:01.324977+0000","last_became_peered":"2024-09-21T03:27:01.324977+0000","last_unstale":"2024-09-21T03:27:02.328791+0000","last_undegraded":"2024-09-21T03:27:02.328791+0000","last_fullsized":"2024-09-21T03:27:02.328791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:10.093856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.375403+0000","last_change":"2024-09-21T03:26:59.326017+0000","last_active":"2024-09-21T03:27:09.375403+0000","last_peered":"2024-09-21T03:27:09.375403+0000","last_clean":"2024-09-21T03:27:09.375403+0000","last_became_active":"2024-09-21T03:26:59.325588+0000","last_became_peered":"2024-09-21T03:26:59.325588+0000","last_unstale":"2024-09-21T03:27:09.375403+0000","last_undegraded":"2024-09-21T03:27:09.375403+0000","last_fullsized":"2024-09-21T03:27:09.375403+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:39.740449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328577+0000","last_change":"2024-09-21T03:27:01.336871+0000","last_active":"2024-09-21T03:27:02.328577+0000","last_peered":"2024-09-21T03:27:02.328577+0000","last_clean":"2024-09-21T03:27:02.328577+0000","last_became_active":"2024-09-21T03:27:01.336507+0000","last_became_peered":"2024-09-21T03:27:01.336507+0000","last_unstale":"2024-09-21T03:27:02.328577+0000","last_undegraded":"2024-09-21T03:27:02.328577+0000","last_fullsized":"2024-09-21T03:27:02.328577+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:58.674375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'3","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.375555+0000","last_change":"2024-09-21T03:26:59.308997+0000","last_active":"2024-09-21T03:27:09.375555+0000","last_peered":"2024-09-21T03:27:09.375555+0000","last_clean":"2024-09-21T03:27:09.375555+0000","last_became_active":"2024-09-21T03:26:59.308338+0000","last_became_peered":"2024-09-21T03:26:59.308338+0000","last_unstale":"2024-09-21T03:27:09.375555+0000","last_undegraded":"2024-09-21T03:27:09.375555+0000","last_fullsized":"2024-09-21T03:27:09.375555+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:15.328468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325382+0000","last_change":"2024-09-21T03:27:01.338808+0000","last_active":"2024-09-21T03:27:02.325382+0000","last_peered":"2024-09-21T03:27:02.325382+0000","last_clean":"2024-09-21T03:27:02.325382+0000","last_became_active":"2024-09-21T03:27:01.337987+0000","last_became_peered":"2024-09-21T03:27:01.337987+0000","last_unstale":"2024-09-21T03:27:02.325382+0000","last_undegraded":"2024-09-21T03:27:02.325382+0000","last_fullsized":"2024-09-21T03:27:02.325382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:56.265014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'1","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318214+0000","last_change":"2024-09-21T03:26:59.312684+0000","last_active":"2024-09-21T03:27:02.318214+0000","last_peered":"2024-09-21T03:27:02.318214+0000","last_clean":"2024-09-21T03:27:02.318214+0000","last_became_active":"2024-09-21T03:26:59.312038+0000","last_became_peered":"2024-09-21T03:26:59.312038+0000","last_unstale":"2024-09-21T03:27:02.318214+0000","last_undegraded":"2024-09-21T03:27:02.318214+0000","last_fullsized":"2024-09-21T03:27:02.318214+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:02.672255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325588+0000","last_change":"2024-09-21T03:27:01.333080+0000","last_active":"2024-09-21T03:27:02.325588+0000","last_peered":"2024-09-21T03:27:02.325588+0000","last_clean":"2024-09-21T03:27:02.325588+0000","last_became_active":"2024-09-21T03:27:01.332663+0000","last_became_peered":"2024-09-21T03:27:01.332663+0000","last_unstale":"2024-09-21T03:27:02.325588+0000","last_undegraded":"2024-09-21T03:27:02.325588+0000","last_fullsized":"2024-09-21T03:27:02.325588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:34.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.376057+0000","last_change":"2024-09-21T03:26:59.315490+0000","last_active":"2024-09-21T03:27:09.376057+0000","last_peered":"2024-09-21T03:27:09.376057+0000","last_clean":"2024-09-21T03:27:09.376057+0000","last_became_active":"2024-09-21T03:26:59.313789+0000","last_became_peered":"2024-09-21T03:26:59.313789+0000","last_unstale":"2024-09-21T03:27:09.376057+0000","last_undegraded":"2024-09-21T03:27:09.376057+0000","last_fullsized":"2024-09-21T03:27:09.376057+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:10.140711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333412+0000","last_change":"2024-09-21T03:27:01.341019+0000","last_active":"2024-09-21T03:27:02.333412+0000","last_peered":"2024-09-21T03:27:02.333412+0000","last_clean":"2024-09-21T03:27:02.333412+0000","last_became_active":"2024-09-21T03:27:01.340660+0000","last_became_peered":"2024-09-21T03:27:01.340660+0000","last_unstale":"2024-09-21T03:27:02.333412+0000","last_undegraded":"2024-09-21T03:27:02.333412+0000","last_fullsized":"2024-09-21T03:27:02.333412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:16.736993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330044+0000","last_change":"2024-09-21T03:27:01.341165+0000","last_active":"2024-09-21T03:27:02.330044+0000","last_peered":"2024-09-21T03:27:02.330044+0000","last_clean":"2024-09-21T03:27:02.330044+0000","last_became_active":"2024-09-21T03:27:01.340765+0000","last_became_peered":"2024-09-21T03:27:01.340765+0000","last_unstale":"2024-09-21T03:27:02.330044+0000","last_undegraded":"2024-09-21T03:27:02.330044+0000","last_fullsized":"2024-09-21T03:27:02.330044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:15.455944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.335947+0000","last_change":"2024-09-21T03:26:59.313410+0000","last_active":"2024-09-21T03:27:02.335947+0000","last_peered":"2024-09-21T03:27:02.335947+0000","last_clean":"2024-09-21T03:27:02.335947+0000","last_became_active":"2024-09-21T03:26:59.312233+0000","last_became_peered":"2024-09-21T03:26:59.312233+0000","last_unstale":"2024-09-21T03:27:02.335947+0000","last_undegraded":"2024-09-21T03:27:02.335947+0000","last_fullsized":"2024-09-21T03:27:02.335947+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:26.089611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329376+0000","last_change":"2024-09-21T03:27:01.335332+0000","last_active":"2024-09-21T03:27:02.329376+0000","last_peered":"2024-09-21T03:27:02.329376+0000","last_clean":"2024-09-21T03:27:02.329376+0000","last_became_active":"2024-09-21T03:27:01.334872+0000","last_became_peered":"2024-09-21T03:27:01.334872+0000","last_unstale":"2024-09-21T03:27:02.329376+0000","last_undegraded":"2024-09-21T03:27:02.329376+0000","last_fullsized":"2024-09-21T03:27:02.329376+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:39.631246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'4","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.389895+0000","last_change":"2024-09-21T03:26:59.314250+0000","last_active":"2024-09-21T03:27:10.389895+0000","last_peered":"2024-09-21T03:27:10.389895+0000","last_clean":"2024-09-21T03:27:10.389895+0000","last_became_active":"2024-09-21T03:26:59.313707+0000","last_became_peered":"2024-09-21T03:26:59.313707+0000","last_unstale":"2024-09-21T03:27:10.389895+0000","last_undegraded":"2024-09-21T03:27:10.389895+0000","last_fullsized":"2024-09-21T03:27:10.389895+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:22.935292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328895+0000","last_change":"2024-09-21T03:27:01.339230+0000","last_active":"2024-09-21T03:27:02.328895+0000","last_peered":"2024-09-21T03:27:02.328895+0000","last_clean":"2024-09-21T03:27:02.328895+0000","last_became_active":"2024-09-21T03:27:01.338881+0000","last_became_peered":"2024-09-21T03:27:01.338881+0000","last_unstale":"2024-09-21T03:27:02.328895+0000","last_undegraded":"2024-09-21T03:27:02.328895+0000","last_fullsized":"2024-09-21T03:27:02.328895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:49.189180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328445+0000","last_change":"2024-09-21T03:26:59.326549+0000","last_active":"2024-09-21T03:27:02.328445+0000","last_peered":"2024-09-21T03:27:02.328445+0000","last_clean":"2024-09-21T03:27:02.328445+0000","last_became_active":"2024-09-21T03:26:59.326091+0000","last_became_peered":"2024-09-21T03:26:59.326091+0000","last_unstale":"2024-09-21T03:27:02.328445+0000","last_undegraded":"2024-09-21T03:27:02.328445+0000","last_fullsized":"2024-09-21T03:27:02.328445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:13.507716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325716+0000","last_change":"2024-09-21T03:27:01.331367+0000","last_active":"2024-09-21T03:27:02.325716+0000","last_peered":"2024-09-21T03:27:02.325716+0000","last_clean":"2024-09-21T03:27:02.325716+0000","last_became_active":"2024-09-21T03:27:01.330604+0000","last_became_peered":"2024-09-21T03:27:01.330604+0000","last_unstale":"2024-09-21T03:27:02.325716+0000","last_undegraded":"2024-09-21T03:27:02.325716+0000","last_fullsized":"2024-09-21T03:27:02.325716+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:32.207561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'9","reported_seq":29,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.389361+0000","last_change":"2024-09-21T03:26:59.311912+0000","last_active":"2024-09-21T03:27:10.389361+0000","last_peered":"2024-09-21T03:27:10.389361+0000","last_clean":"2024-09-21T03:27:10.389361+0000","last_became_active":"2024-09-21T03:26:59.311571+0000","last_became_peered":"2024-09-21T03:26:59.311571+0000","last_unstale":"2024-09-21T03:27:10.389361+0000","last_undegraded":"2024-09-21T03:27:10.389361+0000","last_fullsized":"2024-09-21T03:27:10.389361+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:55.946494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49204,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":92,"num_read_kb":101,"num_write":145,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":147612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":802816,"data_stored":395655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939669,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1482752,"data_stored":1032730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103194,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":395891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299423,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":395811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32564,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712588,"statfs":{"total":95995035648,"available":95961690112,"internally_reserved":0,"allocated":1413120,"data_stored":987741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659176,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":806912,"data_stored":395280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855406,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28412,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716740,"statfs":{"total":95995035648,"available":95965941760,"internally_reserved":0,"allocated":1421312,"data_stored":986917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018932,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":811008,"data_stored":396108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":823296,"data_stored":395591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":441626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542211,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":851968,"data_stored":440276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705735,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":827392,"data_stored":396551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":45203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":46553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":47172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:27:27.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:27:27.306 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:27.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:27 smithi203 ceph-mon[34723]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:27:27.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:27 smithi203 ceph-mon[34723]: pgmap v269: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T03:27:27.664 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:27 smithi112 ceph-mon[31442]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:27:27.664 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:27 smithi112 ceph-mon[31442]: pgmap v269: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T03:27:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:27 smithi187 ceph-mon[34603]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:27:27.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:27 smithi187 ceph-mon[34603]: pgmap v269: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T03:27:27.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.752+0000 7f44bdc25640 1 -- 172.21.15.112:0/1199095627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b81094a0 msgr2=0x7f44b81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.752+0000 7f44bdc25640 1 --2- 172.21.15.112:0/1199095627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b81094a0 0x7f44b81098b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009900 tx=0x7f44ac02a410 comp rx=0 tx=0).stop 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.752+0000 7f44bdc25640 1 -- 172.21.15.112:0/1199095627 shutdown_connections 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.752+0000 7f44bdc25640 1 --2- 172.21.15.112:0/1199095627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b8109df0 0x7f44b8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.752+0000 7f44bdc25640 1 --2- 172.21.15.112:0/1199095627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b81094a0 0x7f44b81098b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.752+0000 7f44bdc25640 1 --2- 172.21.15.112:0/1199095627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b8103b60 0x7f44b8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.752+0000 7f44bdc25640 1 -- 172.21.15.112:0/1199095627 >> 172.21.15.112:0/1199095627 conn(0x7f44b807b850 msgr2=0x7f44b807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.752+0000 7f44bdc25640 1 -- 172.21.15.112:0/1199095627 shutdown_connections 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.752+0000 7f44bdc25640 1 -- 172.21.15.112:0/1199095627 wait complete. 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44bdc25640 1 Processor -- start 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44bdc25640 1 -- start start 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44bdc25640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b8103b60 0x7f44b819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44bdc25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b81094a0 0x7f44b819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44bdc25640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b8109df0 0x7f44b819ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44bdc25640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8113b50 con 0x7f44b81094a0 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44bdc25640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81139d0 con 0x7f44b8103b60 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44bdc25640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8113cd0 con 0x7f44b8109df0 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44b6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b81094a0 0x7f44b819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44b6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b81094a0 0x7f44b819ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49726/0 (socket says 172.21.15.112:49726) 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44b6ffd640 1 -- 172.21.15.112:0/2203183293 learned_addr learned my addr 172.21.15.112:0/2203183293 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44b77fe640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b8103b60 0x7f44b819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44b7fff640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b8109df0 0x7f44b819ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44b6ffd640 1 -- 172.21.15.112:0/2203183293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b8109df0 msgr2=0x7f44b819ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44b6ffd640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b8109df0 0x7f44b819ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44b6ffd640 1 -- 172.21.15.112:0/2203183293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b8103b60 msgr2=0x7f44b819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44b6ffd640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b8103b60 0x7f44b819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.753+0000 7f44b6ffd640 1 -- 172.21.15.112:0/2203183293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b81a0630 con 0x7f44b81094a0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.754+0000 7f44b7fff640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b8109df0 0x7f44b819ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.754+0000 7f44b6ffd640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b81094a0 0x7f44b819ac60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f44ac02a930 tx=0x7f44ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.754+0000 7f44b4ff9640 1 -- 172.21.15.112:0/2203183293 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac03f030 con 0x7f44b81094a0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.754+0000 7f44b4ff9640 1 -- 172.21.15.112:0/2203183293 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44ac003aa0 con 0x7f44b81094a0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.754+0000 7f44bdc25640 1 -- 172.21.15.112:0/2203183293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b81a08c0 con 0x7f44b81094a0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.754+0000 7f44b4ff9640 1 -- 172.21.15.112:0/2203183293 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac035070 con 0x7f44b81094a0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.754+0000 7f44bdc25640 1 -- 172.21.15.112:0/2203183293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b81a7080 con 0x7f44b81094a0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.755+0000 7f44bdc25640 1 -- 172.21.15.112:0/2203183293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b8104de0 con 0x7f44b81094a0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.756+0000 7f44b4ff9640 1 -- 172.21.15.112:0/2203183293 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f44ac042030 con 0x7f44b81094a0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.757+0000 7f44b4ff9640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a00b65f0 0x7f44a00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.757+0000 7f44b77fe640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a00b65f0 0x7f44a00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.757+0000 7f44b4ff9640 1 -- 172.21.15.112:0/2203183293 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f44ac1080a0 con 0x7f44b81094a0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.757+0000 7f44b77fe640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a00b65f0 0x7f44a00b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009990 tx=0x7f44a8009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:27.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.760+0000 7f44b4ff9640 1 -- 172.21.15.112:0/2203183293 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f44ac0ce7f0 con 0x7f44b81094a0 2024-09-21T03:27:27.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.933+0000 7f44bdc25640 1 -- 172.21.15.112:0/2203183293 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44b8063800 con 0x7f44a00b65f0 2024-09-21T03:27:27.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.938+0000 7f44b4ff9640 1 -- 172.21.15.112:0/2203183293 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258682 (secure 0 0 0) 0x7f44b8063800 con 0x7f44a00b65f0 2024-09-21T03:27:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:27:27.944 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T03:27:27.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 -- 172.21.15.112:0/2203183293 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a00b65f0 msgr2=0x7f44a00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:27.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a00b65f0 0x7f44a00b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009990 tx=0x7f44a8009110 comp rx=0 tx=0).stop 2024-09-21T03:27:27.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 -- 172.21.15.112:0/2203183293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b81094a0 msgr2=0x7f44b819ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:27.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b81094a0 0x7f44b819ac60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f44ac02a930 tx=0x7f44ac004270 comp rx=0 tx=0).stop 2024-09-21T03:27:27.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 -- 172.21.15.112:0/2203183293 shutdown_connections 2024-09-21T03:27:27.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a00b65f0 0x7f44a00b8b00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b8109df0 0x7f44b819ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b81094a0 0x7f44b819ac60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 --2- 172.21.15.112:0/2203183293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b8103b60 0x7f44b819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:27.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 -- 172.21.15.112:0/2203183293 >> 172.21.15.112:0/2203183293 conn(0x7f44b807b850 msgr2=0x7f44b81076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:27.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 -- 172.21.15.112:0/2203183293 shutdown_connections 2024-09-21T03:27:27.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:27.941+0000 7f44bdc25640 1 -- 172.21.15.112:0/2203183293 wait complete. 2024-09-21T03:27:28.669 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-21T03:27:26.337849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639806,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":188,"num_read_kb":175,"num_write":277,"num_write_kb":2244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364892,"kb_used_data":11484,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576932,"statfs":{"total":1151940427776,"available":1151566778368,"internally_reserved":0,"allocated":11759616,"data_stored":6660294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44216,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":95,"num_read_kb":97,"num_write":39,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003340"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325965+0000","last_change":"2024-09-21T03:27:01.339980+0000","last_active":"2024-09-21T03:27:02.325965+0000","last_peered":"2024-09-21T03:27:02.325965+0000","last_clean":"2024-09-21T03:27:02.325965+0000","last_became_active":"2024-09-21T03:27:01.339552+0000","last_became_peered":"2024-09-21T03:27:01.339552+0000","last_unstale":"2024-09-21T03:27:02.325965+0000","last_undegraded":"2024-09-21T03:27:02.325965+0000","last_fullsized":"2024-09-21T03:27:02.325965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:04.240862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328531+0000","last_change":"2024-09-21T03:27:01.334171+0000","last_active":"2024-09-21T03:27:02.328531+0000","last_peered":"2024-09-21T03:27:02.328531+0000","last_clean":"2024-09-21T03:27:02.328531+0000","last_became_active":"2024-09-21T03:27:01.333851+0000","last_became_peered":"2024-09-21T03:27:01.333851+0000","last_unstale":"2024-09-21T03:27:02.328531+0000","last_undegraded":"2024-09-21T03:27:02.328531+0000","last_fullsized":"2024-09-21T03:27:02.328531+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:49.163142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328758+0000","last_change":"2024-09-21T03:27:01.339388+0000","last_active":"2024-09-21T03:27:02.328758+0000","last_peered":"2024-09-21T03:27:02.328758+0000","last_clean":"2024-09-21T03:27:02.328758+0000","last_became_active":"2024-09-21T03:27:01.339015+0000","last_became_peered":"2024-09-21T03:27:01.339015+0000","last_unstale":"2024-09-21T03:27:02.328758+0000","last_undegraded":"2024-09-21T03:27:02.328758+0000","last_fullsized":"2024-09-21T03:27:02.328758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:06.026397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318241+0000","last_change":"2024-09-21T03:27:01.344700+0000","last_active":"2024-09-21T03:27:02.318241+0000","last_peered":"2024-09-21T03:27:02.318241+0000","last_clean":"2024-09-21T03:27:02.318241+0000","last_became_active":"2024-09-21T03:27:01.344349+0000","last_became_peered":"2024-09-21T03:27:01.344349+0000","last_unstale":"2024-09-21T03:27:02.318241+0000","last_undegraded":"2024-09-21T03:27:02.318241+0000","last_fullsized":"2024-09-21T03:27:02.318241+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:00.271820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329641+0000","last_change":"2024-09-21T03:27:01.334904+0000","last_active":"2024-09-21T03:27:02.329641+0000","last_peered":"2024-09-21T03:27:02.329641+0000","last_clean":"2024-09-21T03:27:02.329641+0000","last_became_active":"2024-09-21T03:27:01.331916+0000","last_became_peered":"2024-09-21T03:27:01.331916+0000","last_unstale":"2024-09-21T03:27:02.329641+0000","last_undegraded":"2024-09-21T03:27:02.329641+0000","last_fullsized":"2024-09-21T03:27:02.329641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:49.376481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326109+0000","last_change":"2024-09-21T03:27:01.343256+0000","last_active":"2024-09-21T03:27:02.326109+0000","last_peered":"2024-09-21T03:27:02.326109+0000","last_clean":"2024-09-21T03:27:02.326109+0000","last_became_active":"2024-09-21T03:27:01.342861+0000","last_became_peered":"2024-09-21T03:27:01.342861+0000","last_unstale":"2024-09-21T03:27:02.326109+0000","last_undegraded":"2024-09-21T03:27:02.326109+0000","last_fullsized":"2024-09-21T03:27:02.326109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:00.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333140+0000","last_change":"2024-09-21T03:27:01.330618+0000","last_active":"2024-09-21T03:27:02.333140+0000","last_peered":"2024-09-21T03:27:02.333140+0000","last_clean":"2024-09-21T03:27:02.333140+0000","last_became_active":"2024-09-21T03:27:01.330018+0000","last_became_peered":"2024-09-21T03:27:01.330018+0000","last_unstale":"2024-09-21T03:27:02.333140+0000","last_undegraded":"2024-09-21T03:27:02.333140+0000","last_fullsized":"2024-09-21T03:27:02.333140+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:03.291698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325482+0000","last_change":"2024-09-21T03:27:01.332840+0000","last_active":"2024-09-21T03:27:02.325482+0000","last_peered":"2024-09-21T03:27:02.325482+0000","last_clean":"2024-09-21T03:27:02.325482+0000","last_became_active":"2024-09-21T03:27:01.332513+0000","last_became_peered":"2024-09-21T03:27:01.332513+0000","last_unstale":"2024-09-21T03:27:02.325482+0000","last_undegraded":"2024-09-21T03:27:02.325482+0000","last_fullsized":"2024-09-21T03:27:02.325482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:17.554934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330017+0000","last_change":"2024-09-21T03:27:01.338147+0000","last_active":"2024-09-21T03:27:02.330017+0000","last_peered":"2024-09-21T03:27:02.330017+0000","last_clean":"2024-09-21T03:27:02.330017+0000","last_became_active":"2024-09-21T03:27:01.337759+0000","last_became_peered":"2024-09-21T03:27:01.337759+0000","last_unstale":"2024-09-21T03:27:02.330017+0000","last_undegraded":"2024-09-21T03:27:02.330017+0000","last_fullsized":"2024-09-21T03:27:02.330017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:38.999593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326819+0000","last_change":"2024-09-21T03:27:01.343373+0000","last_active":"2024-09-21T03:27:02.326819+0000","last_peered":"2024-09-21T03:27:02.326819+0000","last_clean":"2024-09-21T03:27:02.326819+0000","last_became_active":"2024-09-21T03:27:01.342924+0000","last_became_peered":"2024-09-21T03:27:01.342924+0000","last_unstale":"2024-09-21T03:27:02.326819+0000","last_undegraded":"2024-09-21T03:27:02.326819+0000","last_fullsized":"2024-09-21T03:27:02.326819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:33.869584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333559+0000","last_change":"2024-09-21T03:27:01.340120+0000","last_active":"2024-09-21T03:27:02.333559+0000","last_peered":"2024-09-21T03:27:02.333559+0000","last_clean":"2024-09-21T03:27:02.333559+0000","last_became_active":"2024-09-21T03:27:01.339708+0000","last_became_peered":"2024-09-21T03:27:01.339708+0000","last_unstale":"2024-09-21T03:27:02.333559+0000","last_undegraded":"2024-09-21T03:27:02.333559+0000","last_fullsized":"2024-09-21T03:27:02.333559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:56.193494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318438+0000","last_change":"2024-09-21T03:27:01.340205+0000","last_active":"2024-09-21T03:27:02.318438+0000","last_peered":"2024-09-21T03:27:02.318438+0000","last_clean":"2024-09-21T03:27:02.318438+0000","last_became_active":"2024-09-21T03:27:01.339839+0000","last_became_peered":"2024-09-21T03:27:01.339839+0000","last_unstale":"2024-09-21T03:27:02.318438+0000","last_undegraded":"2024-09-21T03:27:02.318438+0000","last_fullsized":"2024-09-21T03:27:02.318438+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:05.248636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317573+0000","last_change":"2024-09-21T03:27:01.334715+0000","last_active":"2024-09-21T03:27:02.317573+0000","last_peered":"2024-09-21T03:27:02.317573+0000","last_clean":"2024-09-21T03:27:02.317573+0000","last_became_active":"2024-09-21T03:27:01.334392+0000","last_became_peered":"2024-09-21T03:27:01.334392+0000","last_unstale":"2024-09-21T03:27:02.317573+0000","last_undegraded":"2024-09-21T03:27:02.317573+0000","last_fullsized":"2024-09-21T03:27:02.317573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:18.934063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328256+0000","last_change":"2024-09-21T03:27:01.337167+0000","last_active":"2024-09-21T03:27:02.328256+0000","last_peered":"2024-09-21T03:27:02.328256+0000","last_clean":"2024-09-21T03:27:02.328256+0000","last_became_active":"2024-09-21T03:27:01.336748+0000","last_became_peered":"2024-09-21T03:27:01.336748+0000","last_unstale":"2024-09-21T03:27:02.328256+0000","last_undegraded":"2024-09-21T03:27:02.328256+0000","last_fullsized":"2024-09-21T03:27:02.328256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:12.186727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328097+0000","last_change":"2024-09-21T03:27:01.336260+0000","last_active":"2024-09-21T03:27:02.328097+0000","last_peered":"2024-09-21T03:27:02.328097+0000","last_clean":"2024-09-21T03:27:02.328097+0000","last_became_active":"2024-09-21T03:27:01.335889+0000","last_became_peered":"2024-09-21T03:27:01.335889+0000","last_unstale":"2024-09-21T03:27:02.328097+0000","last_undegraded":"2024-09-21T03:27:02.328097+0000","last_fullsized":"2024-09-21T03:27:02.328097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:02.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328818+0000","last_change":"2024-09-21T03:27:01.337181+0000","last_active":"2024-09-21T03:27:02.328818+0000","last_peered":"2024-09-21T03:27:02.328818+0000","last_clean":"2024-09-21T03:27:02.328818+0000","last_became_active":"2024-09-21T03:27:01.336813+0000","last_became_peered":"2024-09-21T03:27:01.336813+0000","last_unstale":"2024-09-21T03:27:02.328818+0000","last_undegraded":"2024-09-21T03:27:02.328818+0000","last_fullsized":"2024-09-21T03:27:02.328818+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:04.052319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326009+0000","last_change":"2024-09-21T03:27:01.342079+0000","last_active":"2024-09-21T03:27:02.326009+0000","last_peered":"2024-09-21T03:27:02.326009+0000","last_clean":"2024-09-21T03:27:02.326009+0000","last_became_active":"2024-09-21T03:27:01.341430+0000","last_became_peered":"2024-09-21T03:27:01.341430+0000","last_unstale":"2024-09-21T03:27:02.326009+0000","last_undegraded":"2024-09-21T03:27:02.326009+0000","last_fullsized":"2024-09-21T03:27:02.326009+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:08.029978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.327936+0000","last_change":"2024-09-21T03:27:01.338694+0000","last_active":"2024-09-21T03:27:02.327936+0000","last_peered":"2024-09-21T03:27:02.327936+0000","last_clean":"2024-09-21T03:27:02.327936+0000","last_became_active":"2024-09-21T03:27:01.338328+0000","last_became_peered":"2024-09-21T03:27:01.338328+0000","last_unstale":"2024-09-21T03:27:02.327936+0000","last_undegraded":"2024-09-21T03:27:02.327936+0000","last_fullsized":"2024-09-21T03:27:02.327936+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:50.292238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318390+0000","last_change":"2024-09-21T03:27:01.336974+0000","last_active":"2024-09-21T03:27:02.318390+0000","last_peered":"2024-09-21T03:27:02.318390+0000","last_clean":"2024-09-21T03:27:02.318390+0000","last_became_active":"2024-09-21T03:27:01.336592+0000","last_became_peered":"2024-09-21T03:27:01.336592+0000","last_unstale":"2024-09-21T03:27:02.318390+0000","last_undegraded":"2024-09-21T03:27:02.318390+0000","last_fullsized":"2024-09-21T03:27:02.318390+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:09.399348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325473+0000","last_change":"2024-09-21T03:27:01.343326+0000","last_active":"2024-09-21T03:27:02.325473+0000","last_peered":"2024-09-21T03:27:02.325473+0000","last_clean":"2024-09-21T03:27:02.325473+0000","last_became_active":"2024-09-21T03:27:01.342937+0000","last_became_peered":"2024-09-21T03:27:01.342937+0000","last_unstale":"2024-09-21T03:27:02.325473+0000","last_undegraded":"2024-09-21T03:27:02.325473+0000","last_fullsized":"2024-09-21T03:27:02.325473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:01.782324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317457+0000","last_change":"2024-09-21T03:27:01.331242+0000","last_active":"2024-09-21T03:27:02.317457+0000","last_peered":"2024-09-21T03:27:02.317457+0000","last_clean":"2024-09-21T03:27:02.317457+0000","last_became_active":"2024-09-21T03:27:01.330883+0000","last_became_peered":"2024-09-21T03:27:01.330883+0000","last_unstale":"2024-09-21T03:27:02.317457+0000","last_undegraded":"2024-09-21T03:27:02.317457+0000","last_fullsized":"2024-09-21T03:27:02.317457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:07.158448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329300+0000","last_change":"2024-09-21T03:27:01.335522+0000","last_active":"2024-09-21T03:27:02.329300+0000","last_peered":"2024-09-21T03:27:02.329300+0000","last_clean":"2024-09-21T03:27:02.329300+0000","last_became_active":"2024-09-21T03:27:01.335114+0000","last_became_peered":"2024-09-21T03:27:01.335114+0000","last_unstale":"2024-09-21T03:27:02.329300+0000","last_undegraded":"2024-09-21T03:27:02.329300+0000","last_fullsized":"2024-09-21T03:27:02.329300+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:28.383410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325441+0000","last_change":"2024-09-21T03:27:01.329428+0000","last_active":"2024-09-21T03:27:02.325441+0000","last_peered":"2024-09-21T03:27:02.325441+0000","last_clean":"2024-09-21T03:27:02.325441+0000","last_became_active":"2024-09-21T03:27:01.328882+0000","last_became_peered":"2024-09-21T03:27:01.328882+0000","last_unstale":"2024-09-21T03:27:02.325441+0000","last_undegraded":"2024-09-21T03:27:02.325441+0000","last_fullsized":"2024-09-21T03:27:02.325441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:58.634995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330137+0000","last_change":"2024-09-21T03:27:01.325536+0000","last_active":"2024-09-21T03:27:02.330137+0000","last_peered":"2024-09-21T03:27:02.330137+0000","last_clean":"2024-09-21T03:27:02.330137+0000","last_became_active":"2024-09-21T03:27:01.324988+0000","last_became_peered":"2024-09-21T03:27:01.324988+0000","last_unstale":"2024-09-21T03:27:02.330137+0000","last_undegraded":"2024-09-21T03:27:02.330137+0000","last_fullsized":"2024-09-21T03:27:02.330137+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:51.856606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325891+0000","last_change":"2024-09-21T03:27:01.330988+0000","last_active":"2024-09-21T03:27:02.325891+0000","last_peered":"2024-09-21T03:27:02.325891+0000","last_clean":"2024-09-21T03:27:02.325891+0000","last_became_active":"2024-09-21T03:27:01.330574+0000","last_became_peered":"2024-09-21T03:27:01.330574+0000","last_unstale":"2024-09-21T03:27:02.325891+0000","last_undegraded":"2024-09-21T03:27:02.325891+0000","last_fullsized":"2024-09-21T03:27:02.325891+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:45.277947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318854+0000","last_change":"2024-09-21T03:27:01.339606+0000","last_active":"2024-09-21T03:27:02.318854+0000","last_peered":"2024-09-21T03:27:02.318854+0000","last_clean":"2024-09-21T03:27:02.318854+0000","last_became_active":"2024-09-21T03:27:01.339208+0000","last_became_peered":"2024-09-21T03:27:01.339208+0000","last_unstale":"2024-09-21T03:27:02.318854+0000","last_undegraded":"2024-09-21T03:27:02.318854+0000","last_fullsized":"2024-09-21T03:27:02.318854+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:31.593363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325740+0000","last_change":"2024-09-21T03:27:01.343634+0000","last_active":"2024-09-21T03:27:02.325740+0000","last_peered":"2024-09-21T03:27:02.325740+0000","last_clean":"2024-09-21T03:27:02.325740+0000","last_became_active":"2024-09-21T03:27:01.342965+0000","last_became_peered":"2024-09-21T03:27:01.342965+0000","last_unstale":"2024-09-21T03:27:02.325740+0000","last_undegraded":"2024-09-21T03:27:02.325740+0000","last_fullsized":"2024-09-21T03:27:02.325740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:46.916198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317451+0000","last_change":"2024-09-21T03:27:01.324993+0000","last_active":"2024-09-21T03:27:02.317451+0000","last_peered":"2024-09-21T03:27:02.317451+0000","last_clean":"2024-09-21T03:27:02.317451+0000","last_became_active":"2024-09-21T03:27:01.324593+0000","last_became_peered":"2024-09-21T03:27:01.324593+0000","last_unstale":"2024-09-21T03:27:02.317451+0000","last_undegraded":"2024-09-21T03:27:02.317451+0000","last_fullsized":"2024-09-21T03:27:02.317451+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:13.211539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328598+0000","last_change":"2024-09-21T03:27:01.325232+0000","last_active":"2024-09-21T03:27:02.328598+0000","last_peered":"2024-09-21T03:27:02.328598+0000","last_clean":"2024-09-21T03:27:02.328598+0000","last_became_active":"2024-09-21T03:27:01.324570+0000","last_became_peered":"2024-09-21T03:27:01.324570+0000","last_unstale":"2024-09-21T03:27:02.328598+0000","last_undegraded":"2024-09-21T03:27:02.328598+0000","last_fullsized":"2024-09-21T03:27:02.328598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:30.843334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317814+0000","last_change":"2024-09-21T03:27:01.338789+0000","last_active":"2024-09-21T03:27:02.317814+0000","last_peered":"2024-09-21T03:27:02.317814+0000","last_clean":"2024-09-21T03:27:02.317814+0000","last_became_active":"2024-09-21T03:27:01.338282+0000","last_became_peered":"2024-09-21T03:27:01.338282+0000","last_unstale":"2024-09-21T03:27:02.317814+0000","last_undegraded":"2024-09-21T03:27:02.317814+0000","last_fullsized":"2024-09-21T03:27:02.317814+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:57.213343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318229+0000","last_change":"2024-09-21T03:27:01.339521+0000","last_active":"2024-09-21T03:27:02.318229+0000","last_peered":"2024-09-21T03:27:02.318229+0000","last_clean":"2024-09-21T03:27:02.318229+0000","last_became_active":"2024-09-21T03:27:01.339175+0000","last_became_peered":"2024-09-21T03:27:01.339175+0000","last_unstale":"2024-09-21T03:27:02.318229+0000","last_undegraded":"2024-09-21T03:27:02.318229+0000","last_fullsized":"2024-09-21T03:27:02.318229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:46.163698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325360+0000","last_change":"2024-09-21T03:27:01.338352+0000","last_active":"2024-09-21T03:27:02.325360+0000","last_peered":"2024-09-21T03:27:02.325360+0000","last_clean":"2024-09-21T03:27:02.325360+0000","last_became_active":"2024-09-21T03:27:01.337306+0000","last_became_peered":"2024-09-21T03:27:01.337306+0000","last_unstale":"2024-09-21T03:27:02.325360+0000","last_undegraded":"2024-09-21T03:27:02.325360+0000","last_fullsized":"2024-09-21T03:27:02.325360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.799039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328608+0000","last_change":"2024-09-21T03:27:01.332461+0000","last_active":"2024-09-21T03:27:02.328608+0000","last_peered":"2024-09-21T03:27:02.328608+0000","last_clean":"2024-09-21T03:27:02.328608+0000","last_became_active":"2024-09-21T03:27:01.331912+0000","last_became_peered":"2024-09-21T03:27:01.331912+0000","last_unstale":"2024-09-21T03:27:02.328608+0000","last_undegraded":"2024-09-21T03:27:02.328608+0000","last_fullsized":"2024-09-21T03:27:02.328608+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:34.771691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":40,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:23.414480+0000","last_change":"2024-09-21T03:26:59.317040+0000","last_active":"2024-09-21T03:27:23.414480+0000","last_peered":"2024-09-21T03:27:23.414480+0000","last_clean":"2024-09-21T03:27:23.414480+0000","last_became_active":"2024-09-21T03:26:59.316571+0000","last_became_peered":"2024-09-21T03:26:59.316571+0000","last_unstale":"2024-09-21T03:27:23.414480+0000","last_undegraded":"2024-09-21T03:27:23.414480+0000","last_fullsized":"2024-09-21T03:27:23.414480+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:55.788382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329094+0000","last_change":"2024-09-21T03:27:01.324364+0000","last_active":"2024-09-21T03:27:02.329094+0000","last_peered":"2024-09-21T03:27:02.329094+0000","last_clean":"2024-09-21T03:27:02.329094+0000","last_became_active":"2024-09-21T03:27:01.324018+0000","last_became_peered":"2024-09-21T03:27:01.324018+0000","last_unstale":"2024-09-21T03:27:02.329094+0000","last_undegraded":"2024-09-21T03:27:02.329094+0000","last_fullsized":"2024-09-21T03:27:02.329094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:56.404893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'3","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.390553+0000","last_change":"2024-09-21T03:26:59.313289+0000","last_active":"2024-09-21T03:27:10.390553+0000","last_peered":"2024-09-21T03:27:10.390553+0000","last_clean":"2024-09-21T03:27:10.390553+0000","last_became_active":"2024-09-21T03:26:59.312877+0000","last_became_peered":"2024-09-21T03:26:59.312877+0000","last_unstale":"2024-09-21T03:27:10.390553+0000","last_undegraded":"2024-09-21T03:27:10.390553+0000","last_fullsized":"2024-09-21T03:27:10.390553+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:53.856541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.387703+0000","last_change":"2024-09-21T03:26:59.307098+0000","last_active":"2024-09-21T03:27:10.387703+0000","last_peered":"2024-09-21T03:27:10.387703+0000","last_clean":"2024-09-21T03:27:10.387703+0000","last_became_active":"2024-09-21T03:26:59.306650+0000","last_became_peered":"2024-09-21T03:26:59.306650+0000","last_unstale":"2024-09-21T03:27:10.387703+0000","last_undegraded":"2024-09-21T03:27:10.387703+0000","last_fullsized":"2024-09-21T03:27:10.387703+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:44.033899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325372+0000","last_change":"2024-09-21T03:27:01.328443+0000","last_active":"2024-09-21T03:27:02.325372+0000","last_peered":"2024-09-21T03:27:02.325372+0000","last_clean":"2024-09-21T03:27:02.325372+0000","last_became_active":"2024-09-21T03:27:01.328107+0000","last_became_peered":"2024-09-21T03:27:01.328107+0000","last_unstale":"2024-09-21T03:27:02.325372+0000","last_undegraded":"2024-09-21T03:27:02.325372+0000","last_fullsized":"2024-09-21T03:27:02.325372+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:47.798308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'1","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330718+0000","last_change":"2024-09-21T03:26:59.317934+0000","last_active":"2024-09-21T03:27:02.330718+0000","last_peered":"2024-09-21T03:27:02.330718+0000","last_clean":"2024-09-21T03:27:02.330718+0000","last_became_active":"2024-09-21T03:26:59.317166+0000","last_became_peered":"2024-09-21T03:26:59.317166+0000","last_unstale":"2024-09-21T03:27:02.330718+0000","last_undegraded":"2024-09-21T03:27:02.330718+0000","last_fullsized":"2024-09-21T03:27:02.330718+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:52.746406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.327216+0000","last_change":"2024-09-21T03:27:01.331226+0000","last_active":"2024-09-21T03:27:02.327216+0000","last_peered":"2024-09-21T03:27:02.327216+0000","last_clean":"2024-09-21T03:27:02.327216+0000","last_became_active":"2024-09-21T03:27:01.330889+0000","last_became_peered":"2024-09-21T03:27:01.330889+0000","last_unstale":"2024-09-21T03:27:02.327216+0000","last_undegraded":"2024-09-21T03:27:02.327216+0000","last_fullsized":"2024-09-21T03:27:02.327216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:51.181554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'3","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:22.412154+0000","last_change":"2024-09-21T03:26:59.311671+0000","last_active":"2024-09-21T03:27:22.412154+0000","last_peered":"2024-09-21T03:27:22.412154+0000","last_clean":"2024-09-21T03:27:22.412154+0000","last_became_active":"2024-09-21T03:26:59.311149+0000","last_became_peered":"2024-09-21T03:26:59.311149+0000","last_unstale":"2024-09-21T03:27:22.412154+0000","last_undegraded":"2024-09-21T03:27:22.412154+0000","last_fullsized":"2024-09-21T03:27:22.412154+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:47.878274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329762+0000","last_change":"2024-09-21T03:27:01.336142+0000","last_active":"2024-09-21T03:27:02.329762+0000","last_peered":"2024-09-21T03:27:02.329762+0000","last_clean":"2024-09-21T03:27:02.329762+0000","last_became_active":"2024-09-21T03:27:01.335795+0000","last_became_peered":"2024-09-21T03:27:01.335795+0000","last_unstale":"2024-09-21T03:27:02.329762+0000","last_undegraded":"2024-09-21T03:27:02.329762+0000","last_fullsized":"2024-09-21T03:27:02.329762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:29.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.375872+0000","last_change":"2024-09-21T03:26:59.315077+0000","last_active":"2024-09-21T03:27:09.375872+0000","last_peered":"2024-09-21T03:27:09.375872+0000","last_clean":"2024-09-21T03:27:09.375872+0000","last_became_active":"2024-09-21T03:26:59.312347+0000","last_became_peered":"2024-09-21T03:26:59.312347+0000","last_unstale":"2024-09-21T03:27:09.375872+0000","last_undegraded":"2024-09-21T03:27:09.375872+0000","last_fullsized":"2024-09-21T03:27:09.375872+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:24.529066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328919+0000","last_change":"2024-09-21T03:27:01.324346+0000","last_active":"2024-09-21T03:27:02.328919+0000","last_peered":"2024-09-21T03:27:02.328919+0000","last_clean":"2024-09-21T03:27:02.328919+0000","last_became_active":"2024-09-21T03:27:01.323976+0000","last_became_peered":"2024-09-21T03:27:01.323976+0000","last_unstale":"2024-09-21T03:27:02.328919+0000","last_undegraded":"2024-09-21T03:27:02.328919+0000","last_fullsized":"2024-09-21T03:27:02.328919+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:04.591595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'4","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.376641+0000","last_change":"2024-09-21T03:26:59.326206+0000","last_active":"2024-09-21T03:27:09.376641+0000","last_peered":"2024-09-21T03:27:09.376641+0000","last_clean":"2024-09-21T03:27:09.376641+0000","last_became_active":"2024-09-21T03:26:59.325755+0000","last_became_peered":"2024-09-21T03:26:59.325755+0000","last_unstale":"2024-09-21T03:27:09.376641+0000","last_undegraded":"2024-09-21T03:27:09.376641+0000","last_fullsized":"2024-09-21T03:27:09.376641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:23.220795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329706+0000","last_change":"2024-09-21T03:27:01.341501+0000","last_active":"2024-09-21T03:27:02.329706+0000","last_peered":"2024-09-21T03:27:02.329706+0000","last_clean":"2024-09-21T03:27:02.329706+0000","last_became_active":"2024-09-21T03:27:01.341039+0000","last_became_peered":"2024-09-21T03:27:01.341039+0000","last_unstale":"2024-09-21T03:27:02.329706+0000","last_undegraded":"2024-09-21T03:27:02.329706+0000","last_fullsized":"2024-09-21T03:27:02.329706+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:15.132180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"74'21","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:23.419481+0000","last_change":"2024-09-21T03:26:59.313489+0000","last_active":"2024-09-21T03:27:23.419481+0000","last_peered":"2024-09-21T03:27:23.419481+0000","last_clean":"2024-09-21T03:27:23.419481+0000","last_became_active":"2024-09-21T03:26:59.313022+0000","last_became_peered":"2024-09-21T03:26:59.313022+0000","last_unstale":"2024-09-21T03:27:23.419481+0000","last_undegraded":"2024-09-21T03:27:23.419481+0000","last_fullsized":"2024-09-21T03:27:23.419481+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:08.541315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45203,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":59,"num_write":22,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329332+0000","last_change":"2024-09-21T03:27:01.339329+0000","last_active":"2024-09-21T03:27:02.329332+0000","last_peered":"2024-09-21T03:27:02.329332+0000","last_clean":"2024-09-21T03:27:02.329332+0000","last_became_active":"2024-09-21T03:27:01.338982+0000","last_became_peered":"2024-09-21T03:27:01.338982+0000","last_unstale":"2024-09-21T03:27:02.329332+0000","last_undegraded":"2024-09-21T03:27:02.329332+0000","last_fullsized":"2024-09-21T03:27:02.329332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:41.759450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'6","reported_seq":26,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.387802+0000","last_change":"2024-09-21T03:26:59.313270+0000","last_active":"2024-09-21T03:27:10.387802+0000","last_peered":"2024-09-21T03:27:10.387802+0000","last_clean":"2024-09-21T03:27:10.387802+0000","last_became_active":"2024-09-21T03:26:59.312867+0000","last_became_peered":"2024-09-21T03:26:59.312867+0000","last_unstale":"2024-09-21T03:27:10.387802+0000","last_undegraded":"2024-09-21T03:27:10.387802+0000","last_fullsized":"2024-09-21T03:27:10.387802+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:53.798022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317064+0000","last_change":"2024-09-21T03:27:01.331268+0000","last_active":"2024-09-21T03:27:02.317064+0000","last_peered":"2024-09-21T03:27:02.317064+0000","last_clean":"2024-09-21T03:27:02.317064+0000","last_became_active":"2024-09-21T03:27:01.330922+0000","last_became_peered":"2024-09-21T03:27:01.330922+0000","last_unstale":"2024-09-21T03:27:02.317064+0000","last_undegraded":"2024-09-21T03:27:02.317064+0000","last_fullsized":"2024-09-21T03:27:02.317064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'3","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.375592+0000","last_change":"2024-09-21T03:26:59.318388+0000","last_active":"2024-09-21T03:27:09.375592+0000","last_peered":"2024-09-21T03:27:09.375592+0000","last_clean":"2024-09-21T03:27:09.375592+0000","last_became_active":"2024-09-21T03:26:59.317938+0000","last_became_peered":"2024-09-21T03:26:59.317938+0000","last_unstale":"2024-09-21T03:27:09.375592+0000","last_undegraded":"2024-09-21T03:27:09.375592+0000","last_fullsized":"2024-09-21T03:27:09.375592+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.499699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333593+0000","last_change":"2024-09-21T03:27:01.340240+0000","last_active":"2024-09-21T03:27:02.333593+0000","last_peered":"2024-09-21T03:27:02.333593+0000","last_clean":"2024-09-21T03:27:02.333593+0000","last_became_active":"2024-09-21T03:27:01.339531+0000","last_became_peered":"2024-09-21T03:27:01.339531+0000","last_unstale":"2024-09-21T03:27:02.333593+0000","last_undegraded":"2024-09-21T03:27:02.333593+0000","last_fullsized":"2024-09-21T03:27:02.333593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:45.268210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'4","reported_seq":40,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:23.414227+0000","last_change":"2024-09-21T03:26:59.311345+0000","last_active":"2024-09-21T03:27:23.414227+0000","last_peered":"2024-09-21T03:27:23.414227+0000","last_clean":"2024-09-21T03:27:23.414227+0000","last_became_active":"2024-09-21T03:26:59.310928+0000","last_became_peered":"2024-09-21T03:26:59.310928+0000","last_unstale":"2024-09-21T03:27:23.414227+0000","last_undegraded":"2024-09-21T03:27:23.414227+0000","last_fullsized":"2024-09-21T03:27:23.414227+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.614955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333101+0000","last_change":"2024-09-21T03:27:01.335785+0000","last_active":"2024-09-21T03:27:02.333101+0000","last_peered":"2024-09-21T03:27:02.333101+0000","last_clean":"2024-09-21T03:27:02.333101+0000","last_became_active":"2024-09-21T03:27:01.335396+0000","last_became_peered":"2024-09-21T03:27:01.335396+0000","last_unstale":"2024-09-21T03:27:02.333101+0000","last_undegraded":"2024-09-21T03:27:02.333101+0000","last_fullsized":"2024-09-21T03:27:02.333101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:52.785641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'15","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.391594+0000","last_change":"2024-09-21T03:26:59.325574+0000","last_active":"2024-09-21T03:27:10.391594+0000","last_peered":"2024-09-21T03:27:10.391594+0000","last_clean":"2024-09-21T03:27:10.391594+0000","last_became_active":"2024-09-21T03:26:59.325074+0000","last_became_peered":"2024-09-21T03:26:59.325074+0000","last_unstale":"2024-09-21T03:27:10.391594+0000","last_undegraded":"2024-09-21T03:27:10.391594+0000","last_fullsized":"2024-09-21T03:27:10.391594+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:19.989009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330112+0000","last_change":"2024-09-21T03:27:01.342282+0000","last_active":"2024-09-21T03:27:02.330112+0000","last_peered":"2024-09-21T03:27:02.330112+0000","last_clean":"2024-09-21T03:27:02.330112+0000","last_became_active":"2024-09-21T03:27:01.341790+0000","last_became_peered":"2024-09-21T03:27:01.341790+0000","last_unstale":"2024-09-21T03:27:02.330112+0000","last_undegraded":"2024-09-21T03:27:02.330112+0000","last_fullsized":"2024-09-21T03:27:02.330112+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:05.432720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:11.395758+0000","last_change":"2024-09-21T03:26:59.312442+0000","last_active":"2024-09-21T03:27:11.395758+0000","last_peered":"2024-09-21T03:27:11.395758+0000","last_clean":"2024-09-21T03:27:11.395758+0000","last_became_active":"2024-09-21T03:26:59.311601+0000","last_became_peered":"2024-09-21T03:26:59.311601+0000","last_unstale":"2024-09-21T03:27:11.395758+0000","last_undegraded":"2024-09-21T03:27:11.395758+0000","last_fullsized":"2024-09-21T03:27:11.395758+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:32.421228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325744+0000","last_change":"2024-09-21T03:27:01.337917+0000","last_active":"2024-09-21T03:27:02.325744+0000","last_peered":"2024-09-21T03:27:02.325744+0000","last_clean":"2024-09-21T03:27:02.325744+0000","last_became_active":"2024-09-21T03:27:01.337457+0000","last_became_peered":"2024-09-21T03:27:01.337457+0000","last_unstale":"2024-09-21T03:27:02.325744+0000","last_undegraded":"2024-09-21T03:27:02.325744+0000","last_fullsized":"2024-09-21T03:27:02.325744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:49.011795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.324662+0000","last_change":"2024-09-21T03:26:59.309265+0000","last_active":"2024-09-21T03:27:02.324662+0000","last_peered":"2024-09-21T03:27:02.324662+0000","last_clean":"2024-09-21T03:27:02.324662+0000","last_became_active":"2024-09-21T03:26:59.308904+0000","last_became_peered":"2024-09-21T03:26:59.308904+0000","last_unstale":"2024-09-21T03:27:02.324662+0000","last_undegraded":"2024-09-21T03:27:02.324662+0000","last_fullsized":"2024-09-21T03:27:02.324662+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:07.335210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333819+0000","last_change":"2024-09-21T03:27:01.338489+0000","last_active":"2024-09-21T03:27:02.333819+0000","last_peered":"2024-09-21T03:27:02.333819+0000","last_clean":"2024-09-21T03:27:02.333819+0000","last_became_active":"2024-09-21T03:27:01.338123+0000","last_became_peered":"2024-09-21T03:27:01.338123+0000","last_unstale":"2024-09-21T03:27:02.333819+0000","last_undegraded":"2024-09-21T03:27:02.333819+0000","last_fullsized":"2024-09-21T03:27:02.333819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:15.137546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.334246+0000","last_change":"2024-09-21T03:26:59.315371+0000","last_active":"2024-09-21T03:27:02.334246+0000","last_peered":"2024-09-21T03:27:02.334246+0000","last_clean":"2024-09-21T03:27:02.334246+0000","last_became_active":"2024-09-21T03:26:59.314439+0000","last_became_peered":"2024-09-21T03:26:59.314439+0000","last_unstale":"2024-09-21T03:27:02.334246+0000","last_undegraded":"2024-09-21T03:27:02.334246+0000","last_fullsized":"2024-09-21T03:27:02.334246+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:13.656324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333518+0000","last_change":"2024-09-21T03:27:01.331086+0000","last_active":"2024-09-21T03:27:02.333518+0000","last_peered":"2024-09-21T03:27:02.333518+0000","last_clean":"2024-09-21T03:27:02.333518+0000","last_became_active":"2024-09-21T03:27:01.329113+0000","last_became_peered":"2024-09-21T03:27:01.329113+0000","last_unstale":"2024-09-21T03:27:02.333518+0000","last_undegraded":"2024-09-21T03:27:02.333518+0000","last_fullsized":"2024-09-21T03:27:02.333518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:25.514730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'3","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.388642+0000","last_change":"2024-09-21T03:26:59.554087+0000","last_active":"2024-09-21T03:27:10.388642+0000","last_peered":"2024-09-21T03:27:10.388642+0000","last_clean":"2024-09-21T03:27:10.388642+0000","last_became_active":"2024-09-21T03:26:59.553610+0000","last_became_peered":"2024-09-21T03:26:59.553610+0000","last_unstale":"2024-09-21T03:27:10.388642+0000","last_undegraded":"2024-09-21T03:27:10.388642+0000","last_fullsized":"2024-09-21T03:27:10.388642+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:22.048057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333861+0000","last_change":"2024-09-21T03:27:01.339987+0000","last_active":"2024-09-21T03:27:02.333861+0000","last_peered":"2024-09-21T03:27:02.333861+0000","last_clean":"2024-09-21T03:27:02.333861+0000","last_became_active":"2024-09-21T03:27:01.339060+0000","last_became_peered":"2024-09-21T03:27:01.339060+0000","last_unstale":"2024-09-21T03:27:02.333861+0000","last_undegraded":"2024-09-21T03:27:02.333861+0000","last_fullsized":"2024-09-21T03:27:02.333861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.901337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":91,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333471+0000","last_change":"2024-09-21T03:25:50.381920+0000","last_active":"2024-09-21T03:27:02.333471+0000","last_peered":"2024-09-21T03:27:02.333471+0000","last_clean":"2024-09-21T03:27:02.333471+0000","last_became_active":"2024-09-21T03:25:50.376170+0000","last_became_peered":"2024-09-21T03:25:50.376170+0000","last_unstale":"2024-09-21T03:27:02.333471+0000","last_undegraded":"2024-09-21T03:27:02.333471+0000","last_fullsized":"2024-09-21T03:27:02.333471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_clean_scrub_stamp":"2024-09-21T03:22:31.269883+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:57.281150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'3","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:11.393293+0000","last_change":"2024-09-21T03:26:59.313612+0000","last_active":"2024-09-21T03:27:11.393293+0000","last_peered":"2024-09-21T03:27:11.393293+0000","last_clean":"2024-09-21T03:27:11.393293+0000","last_became_active":"2024-09-21T03:26:59.313060+0000","last_became_peered":"2024-09-21T03:26:59.313060+0000","last_unstale":"2024-09-21T03:27:11.393293+0000","last_undegraded":"2024-09-21T03:27:11.393293+0000","last_fullsized":"2024-09-21T03:27:11.393293+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:06.193526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326447+0000","last_change":"2024-09-21T03:27:01.337784+0000","last_active":"2024-09-21T03:27:02.326447+0000","last_peered":"2024-09-21T03:27:02.326447+0000","last_clean":"2024-09-21T03:27:02.326447+0000","last_became_active":"2024-09-21T03:27:01.337415+0000","last_became_peered":"2024-09-21T03:27:01.337415+0000","last_unstale":"2024-09-21T03:27:02.326447+0000","last_undegraded":"2024-09-21T03:27:02.326447+0000","last_fullsized":"2024-09-21T03:27:02.326447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:40.482054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'3","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.387643+0000","last_change":"2024-09-21T03:26:59.318553+0000","last_active":"2024-09-21T03:27:10.387643+0000","last_peered":"2024-09-21T03:27:10.387643+0000","last_clean":"2024-09-21T03:27:10.387643+0000","last_became_active":"2024-09-21T03:26:59.318061+0000","last_became_peered":"2024-09-21T03:26:59.318061+0000","last_unstale":"2024-09-21T03:27:10.387643+0000","last_undegraded":"2024-09-21T03:27:10.387643+0000","last_fullsized":"2024-09-21T03:27:10.387643+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:13.157253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.316926+0000","last_change":"2024-09-21T03:27:01.333769+0000","last_active":"2024-09-21T03:27:02.316926+0000","last_peered":"2024-09-21T03:27:02.316926+0000","last_clean":"2024-09-21T03:27:02.316926+0000","last_became_active":"2024-09-21T03:27:01.333436+0000","last_became_peered":"2024-09-21T03:27:01.333436+0000","last_unstale":"2024-09-21T03:27:02.316926+0000","last_undegraded":"2024-09-21T03:27:02.316926+0000","last_fullsized":"2024-09-21T03:27:02.316926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:30.395916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'4","reported_seq":38,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:24.415041+0000","last_change":"2024-09-21T03:26:59.325314+0000","last_active":"2024-09-21T03:27:24.415041+0000","last_peered":"2024-09-21T03:27:24.415041+0000","last_clean":"2024-09-21T03:27:24.415041+0000","last_became_active":"2024-09-21T03:26:59.324752+0000","last_became_peered":"2024-09-21T03:26:59.324752+0000","last_unstale":"2024-09-21T03:27:24.415041+0000","last_undegraded":"2024-09-21T03:27:24.415041+0000","last_fullsized":"2024-09-21T03:27:24.415041+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.252596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330400+0000","last_change":"2024-09-21T03:27:01.335154+0000","last_active":"2024-09-21T03:27:02.330400+0000","last_peered":"2024-09-21T03:27:02.330400+0000","last_clean":"2024-09-21T03:27:02.330400+0000","last_became_active":"2024-09-21T03:27:01.334633+0000","last_became_peered":"2024-09-21T03:27:01.334633+0000","last_unstale":"2024-09-21T03:27:02.330400+0000","last_undegraded":"2024-09-21T03:27:02.330400+0000","last_fullsized":"2024-09-21T03:27:02.330400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:27.992858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'1","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333802+0000","last_change":"2024-09-21T03:26:59.314019+0000","last_active":"2024-09-21T03:27:02.333802+0000","last_peered":"2024-09-21T03:27:02.333802+0000","last_clean":"2024-09-21T03:27:02.333802+0000","last_became_active":"2024-09-21T03:26:59.311915+0000","last_became_peered":"2024-09-21T03:26:59.311915+0000","last_unstale":"2024-09-21T03:27:02.333802+0000","last_undegraded":"2024-09-21T03:27:02.333802+0000","last_fullsized":"2024-09-21T03:27:02.333802+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:51.412303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326370+0000","last_change":"2024-09-21T03:27:01.344921+0000","last_active":"2024-09-21T03:27:02.326370+0000","last_peered":"2024-09-21T03:27:02.326370+0000","last_clean":"2024-09-21T03:27:02.326370+0000","last_became_active":"2024-09-21T03:27:01.344517+0000","last_became_peered":"2024-09-21T03:27:01.344517+0000","last_unstale":"2024-09-21T03:27:02.326370+0000","last_undegraded":"2024-09-21T03:27:02.326370+0000","last_fullsized":"2024-09-21T03:27:02.326370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:38.549938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.327627+0000","last_change":"2024-09-21T03:26:59.325943+0000","last_active":"2024-09-21T03:27:02.327627+0000","last_peered":"2024-09-21T03:27:02.327627+0000","last_clean":"2024-09-21T03:27:02.327627+0000","last_became_active":"2024-09-21T03:26:59.325524+0000","last_became_peered":"2024-09-21T03:26:59.325524+0000","last_unstale":"2024-09-21T03:27:02.327627+0000","last_undegraded":"2024-09-21T03:27:02.327627+0000","last_fullsized":"2024-09-21T03:27:02.327627+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:45.111480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:03.425334+0000","last_change":"2024-09-21T03:27:01.335369+0000","last_active":"2024-09-21T03:27:03.425334+0000","last_peered":"2024-09-21T03:27:03.425334+0000","last_clean":"2024-09-21T03:27:03.425334+0000","last_became_active":"2024-09-21T03:27:01.334997+0000","last_became_peered":"2024-09-21T03:27:01.334997+0000","last_unstale":"2024-09-21T03:27:03.425334+0000","last_undegraded":"2024-09-21T03:27:03.425334+0000","last_fullsized":"2024-09-21T03:27:03.425334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:10.257451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'1","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.375338+0000","last_change":"2024-09-21T03:26:59.326204+0000","last_active":"2024-09-21T03:27:09.375338+0000","last_peered":"2024-09-21T03:27:09.375338+0000","last_clean":"2024-09-21T03:27:09.375338+0000","last_became_active":"2024-09-21T03:26:59.325431+0000","last_became_peered":"2024-09-21T03:26:59.325431+0000","last_unstale":"2024-09-21T03:27:09.375338+0000","last_undegraded":"2024-09-21T03:27:09.375338+0000","last_fullsized":"2024-09-21T03:27:09.375338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:27.863217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.317522+0000","last_change":"2024-09-21T03:27:01.333861+0000","last_active":"2024-09-21T03:27:02.317522+0000","last_peered":"2024-09-21T03:27:02.317522+0000","last_clean":"2024-09-21T03:27:02.317522+0000","last_became_active":"2024-09-21T03:27:01.333485+0000","last_became_peered":"2024-09-21T03:27:01.333485+0000","last_unstale":"2024-09-21T03:27:02.317522+0000","last_undegraded":"2024-09-21T03:27:02.317522+0000","last_fullsized":"2024-09-21T03:27:02.317522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:04.475811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"74'9","reported_seq":28,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:22.325833+0000","last_change":"2024-09-21T03:26:59.309767+0000","last_active":"2024-09-21T03:27:22.325833+0000","last_peered":"2024-09-21T03:27:22.325833+0000","last_clean":"2024-09-21T03:27:22.325833+0000","last_became_active":"2024-09-21T03:26:59.309354+0000","last_became_peered":"2024-09-21T03:26:59.309354+0000","last_unstale":"2024-09-21T03:27:22.325833+0000","last_undegraded":"2024-09-21T03:27:22.325833+0000","last_fullsized":"2024-09-21T03:27:22.325833+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:26.728719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.326117+0000","last_change":"2024-09-21T03:27:01.333097+0000","last_active":"2024-09-21T03:27:02.326117+0000","last_peered":"2024-09-21T03:27:02.326117+0000","last_clean":"2024-09-21T03:27:02.326117+0000","last_became_active":"2024-09-21T03:27:01.332743+0000","last_became_peered":"2024-09-21T03:27:01.332743+0000","last_unstale":"2024-09-21T03:27:02.326117+0000","last_undegraded":"2024-09-21T03:27:02.326117+0000","last_fullsized":"2024-09-21T03:27:02.326117+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:41.004213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'4","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.376149+0000","last_change":"2024-09-21T03:26:59.553974+0000","last_active":"2024-09-21T03:27:09.376149+0000","last_peered":"2024-09-21T03:27:09.376149+0000","last_clean":"2024-09-21T03:27:09.376149+0000","last_became_active":"2024-09-21T03:26:59.553647+0000","last_became_peered":"2024-09-21T03:26:59.553647+0000","last_unstale":"2024-09-21T03:27:09.376149+0000","last_undegraded":"2024-09-21T03:27:09.376149+0000","last_fullsized":"2024-09-21T03:27:09.376149+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:00.490951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328791+0000","last_change":"2024-09-21T03:27:01.325308+0000","last_active":"2024-09-21T03:27:02.328791+0000","last_peered":"2024-09-21T03:27:02.328791+0000","last_clean":"2024-09-21T03:27:02.328791+0000","last_became_active":"2024-09-21T03:27:01.324977+0000","last_became_peered":"2024-09-21T03:27:01.324977+0000","last_unstale":"2024-09-21T03:27:02.328791+0000","last_undegraded":"2024-09-21T03:27:02.328791+0000","last_fullsized":"2024-09-21T03:27:02.328791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:10.093856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.375403+0000","last_change":"2024-09-21T03:26:59.326017+0000","last_active":"2024-09-21T03:27:09.375403+0000","last_peered":"2024-09-21T03:27:09.375403+0000","last_clean":"2024-09-21T03:27:09.375403+0000","last_became_active":"2024-09-21T03:26:59.325588+0000","last_became_peered":"2024-09-21T03:26:59.325588+0000","last_unstale":"2024-09-21T03:27:09.375403+0000","last_undegraded":"2024-09-21T03:27:09.375403+0000","last_fullsized":"2024-09-21T03:27:09.375403+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:39.740449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328577+0000","last_change":"2024-09-21T03:27:01.336871+0000","last_active":"2024-09-21T03:27:02.328577+0000","last_peered":"2024-09-21T03:27:02.328577+0000","last_clean":"2024-09-21T03:27:02.328577+0000","last_became_active":"2024-09-21T03:27:01.336507+0000","last_became_peered":"2024-09-21T03:27:01.336507+0000","last_unstale":"2024-09-21T03:27:02.328577+0000","last_undegraded":"2024-09-21T03:27:02.328577+0000","last_fullsized":"2024-09-21T03:27:02.328577+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:58.674375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'3","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.375555+0000","last_change":"2024-09-21T03:26:59.308997+0000","last_active":"2024-09-21T03:27:09.375555+0000","last_peered":"2024-09-21T03:27:09.375555+0000","last_clean":"2024-09-21T03:27:09.375555+0000","last_became_active":"2024-09-21T03:26:59.308338+0000","last_became_peered":"2024-09-21T03:26:59.308338+0000","last_unstale":"2024-09-21T03:27:09.375555+0000","last_undegraded":"2024-09-21T03:27:09.375555+0000","last_fullsized":"2024-09-21T03:27:09.375555+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:15.328468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325382+0000","last_change":"2024-09-21T03:27:01.338808+0000","last_active":"2024-09-21T03:27:02.325382+0000","last_peered":"2024-09-21T03:27:02.325382+0000","last_clean":"2024-09-21T03:27:02.325382+0000","last_became_active":"2024-09-21T03:27:01.337987+0000","last_became_peered":"2024-09-21T03:27:01.337987+0000","last_unstale":"2024-09-21T03:27:02.325382+0000","last_undegraded":"2024-09-21T03:27:02.325382+0000","last_fullsized":"2024-09-21T03:27:02.325382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:56.265014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'1","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.318214+0000","last_change":"2024-09-21T03:26:59.312684+0000","last_active":"2024-09-21T03:27:02.318214+0000","last_peered":"2024-09-21T03:27:02.318214+0000","last_clean":"2024-09-21T03:27:02.318214+0000","last_became_active":"2024-09-21T03:26:59.312038+0000","last_became_peered":"2024-09-21T03:26:59.312038+0000","last_unstale":"2024-09-21T03:27:02.318214+0000","last_undegraded":"2024-09-21T03:27:02.318214+0000","last_fullsized":"2024-09-21T03:27:02.318214+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:02.672255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325588+0000","last_change":"2024-09-21T03:27:01.333080+0000","last_active":"2024-09-21T03:27:02.325588+0000","last_peered":"2024-09-21T03:27:02.325588+0000","last_clean":"2024-09-21T03:27:02.325588+0000","last_became_active":"2024-09-21T03:27:01.332663+0000","last_became_peered":"2024-09-21T03:27:01.332663+0000","last_unstale":"2024-09-21T03:27:02.325588+0000","last_undegraded":"2024-09-21T03:27:02.325588+0000","last_fullsized":"2024-09-21T03:27:02.325588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:34.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":24,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:09.376057+0000","last_change":"2024-09-21T03:26:59.315490+0000","last_active":"2024-09-21T03:27:09.376057+0000","last_peered":"2024-09-21T03:27:09.376057+0000","last_clean":"2024-09-21T03:27:09.376057+0000","last_became_active":"2024-09-21T03:26:59.313789+0000","last_became_peered":"2024-09-21T03:26:59.313789+0000","last_unstale":"2024-09-21T03:27:09.376057+0000","last_undegraded":"2024-09-21T03:27:09.376057+0000","last_fullsized":"2024-09-21T03:27:09.376057+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:10.140711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.333412+0000","last_change":"2024-09-21T03:27:01.341019+0000","last_active":"2024-09-21T03:27:02.333412+0000","last_peered":"2024-09-21T03:27:02.333412+0000","last_clean":"2024-09-21T03:27:02.333412+0000","last_became_active":"2024-09-21T03:27:01.340660+0000","last_became_peered":"2024-09-21T03:27:01.340660+0000","last_unstale":"2024-09-21T03:27:02.333412+0000","last_undegraded":"2024-09-21T03:27:02.333412+0000","last_fullsized":"2024-09-21T03:27:02.333412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:16.736993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.330044+0000","last_change":"2024-09-21T03:27:01.341165+0000","last_active":"2024-09-21T03:27:02.330044+0000","last_peered":"2024-09-21T03:27:02.330044+0000","last_clean":"2024-09-21T03:27:02.330044+0000","last_became_active":"2024-09-21T03:27:01.340765+0000","last_became_peered":"2024-09-21T03:27:01.340765+0000","last_unstale":"2024-09-21T03:27:02.330044+0000","last_undegraded":"2024-09-21T03:27:02.330044+0000","last_fullsized":"2024-09-21T03:27:02.330044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:15.455944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.335947+0000","last_change":"2024-09-21T03:26:59.313410+0000","last_active":"2024-09-21T03:27:02.335947+0000","last_peered":"2024-09-21T03:27:02.335947+0000","last_clean":"2024-09-21T03:27:02.335947+0000","last_became_active":"2024-09-21T03:26:59.312233+0000","last_became_peered":"2024-09-21T03:26:59.312233+0000","last_unstale":"2024-09-21T03:27:02.335947+0000","last_undegraded":"2024-09-21T03:27:02.335947+0000","last_fullsized":"2024-09-21T03:27:02.335947+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:26.089611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.329376+0000","last_change":"2024-09-21T03:27:01.335332+0000","last_active":"2024-09-21T03:27:02.329376+0000","last_peered":"2024-09-21T03:27:02.329376+0000","last_clean":"2024-09-21T03:27:02.329376+0000","last_became_active":"2024-09-21T03:27:01.334872+0000","last_became_peered":"2024-09-21T03:27:01.334872+0000","last_unstale":"2024-09-21T03:27:02.329376+0000","last_undegraded":"2024-09-21T03:27:02.329376+0000","last_fullsized":"2024-09-21T03:27:02.329376+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:39.631246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'4","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.389895+0000","last_change":"2024-09-21T03:26:59.314250+0000","last_active":"2024-09-21T03:27:10.389895+0000","last_peered":"2024-09-21T03:27:10.389895+0000","last_clean":"2024-09-21T03:27:10.389895+0000","last_became_active":"2024-09-21T03:26:59.313707+0000","last_became_peered":"2024-09-21T03:26:59.313707+0000","last_unstale":"2024-09-21T03:27:10.389895+0000","last_undegraded":"2024-09-21T03:27:10.389895+0000","last_fullsized":"2024-09-21T03:27:10.389895+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:22.935292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328895+0000","last_change":"2024-09-21T03:27:01.339230+0000","last_active":"2024-09-21T03:27:02.328895+0000","last_peered":"2024-09-21T03:27:02.328895+0000","last_clean":"2024-09-21T03:27:02.328895+0000","last_became_active":"2024-09-21T03:27:01.338881+0000","last_became_peered":"2024-09-21T03:27:01.338881+0000","last_unstale":"2024-09-21T03:27:02.328895+0000","last_undegraded":"2024-09-21T03:27:02.328895+0000","last_fullsized":"2024-09-21T03:27:02.328895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:49.189180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.328445+0000","last_change":"2024-09-21T03:26:59.326549+0000","last_active":"2024-09-21T03:27:02.328445+0000","last_peered":"2024-09-21T03:27:02.328445+0000","last_clean":"2024-09-21T03:27:02.328445+0000","last_became_active":"2024-09-21T03:26:59.326091+0000","last_became_peered":"2024-09-21T03:26:59.326091+0000","last_unstale":"2024-09-21T03:27:02.328445+0000","last_undegraded":"2024-09-21T03:27:02.328445+0000","last_fullsized":"2024-09-21T03:27:02.328445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:13.507716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:02.325716+0000","last_change":"2024-09-21T03:27:01.331367+0000","last_active":"2024-09-21T03:27:02.325716+0000","last_peered":"2024-09-21T03:27:02.325716+0000","last_clean":"2024-09-21T03:27:02.325716+0000","last_became_active":"2024-09-21T03:27:01.330604+0000","last_became_peered":"2024-09-21T03:27:01.330604+0000","last_unstale":"2024-09-21T03:27:02.325716+0000","last_undegraded":"2024-09-21T03:27:02.325716+0000","last_fullsized":"2024-09-21T03:27:02.325716+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:32.207561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'9","reported_seq":29,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-21T03:27:10.389361+0000","last_change":"2024-09-21T03:26:59.311912+0000","last_active":"2024-09-21T03:27:10.389361+0000","last_peered":"2024-09-21T03:27:10.389361+0000","last_clean":"2024-09-21T03:27:10.389361+0000","last_became_active":"2024-09-21T03:26:59.311571+0000","last_became_peered":"2024-09-21T03:26:59.311571+0000","last_unstale":"2024-09-21T03:27:10.389361+0000","last_undegraded":"2024-09-21T03:27:10.389361+0000","last_fullsized":"2024-09-21T03:27:10.389361+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:55.946494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49204,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":107,"num_read_kb":106,"num_write":152,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":147612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":802816,"data_stored":395655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939669,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1482752,"data_stored":1032730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103195,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":395891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299424,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":395811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32564,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712588,"statfs":{"total":95995035648,"available":95961690112,"internally_reserved":0,"allocated":1413120,"data_stored":987741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":806912,"data_stored":395280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855407,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1425408,"data_stored":987034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":811008,"data_stored":396108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":823296,"data_stored":395591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":441626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542211,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":851968,"data_stored":440276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705736,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":827392,"data_stored":396551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":45203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":46553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":47172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:27:28.673 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:27:28.673 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T03:27:28.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T03:27:28.925 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.446+0000 7f72ceca0640 1 -- 172.21.15.112:0/1755076892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72c8108d00 msgr2=0x7f72c810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.446+0000 7f72ceca0640 1 --2- 172.21.15.112:0/1755076892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72c8108d00 0x7f72c810f530 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009ca0 tx=0x7f72c002a8e0 comp rx=0 tx=0).stop 2024-09-21T03:27:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.446+0000 7f72ceca0640 1 -- 172.21.15.112:0/1755076892 shutdown_connections 2024-09-21T03:27:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.446+0000 7f72ceca0640 1 --2- 172.21.15.112:0/1755076892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72c8108d00 0x7f72c810f530 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.446+0000 7f72ceca0640 1 --2- 172.21.15.112:0/1755076892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72c81083b0 0x7f72c81087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.446+0000 7f72ceca0640 1 --2- 172.21.15.112:0/1755076892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c8102a70 0x7f72c8102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.446+0000 7f72ceca0640 1 -- 172.21.15.112:0/1755076892 >> 172.21.15.112:0/1755076892 conn(0x7f72c8079870 msgr2=0x7f72c8079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.446+0000 7f72ceca0640 1 -- 172.21.15.112:0/1755076892 shutdown_connections 2024-09-21T03:27:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.446+0000 7f72ceca0640 1 -- 172.21.15.112:0/1755076892 wait complete. 2024-09-21T03:27:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72ceca0640 1 Processor -- start 2024-09-21T03:27:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72ceca0640 1 -- start start 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72ceca0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c8102a70 0x7f72c8110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72ceca0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72c81083b0 0x7f72c8110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72ceca0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72c8108d00 0x7f72c8110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72ceca0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8113b80 con 0x7f72c81083b0 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72ceca0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8113a00 con 0x7f72c8102a70 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72ceca0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8113d00 con 0x7f72c8108d00 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72c7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72c81083b0 0x7f72c8110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72c7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72c81083b0 0x7f72c8110550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49744/0 (socket says 172.21.15.112:49744) 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72c7fff640 1 -- 172.21.15.112:0/4177944239 learned_addr learned my addr 172.21.15.112:0/4177944239 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72cca3c640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c8102a70 0x7f72c8110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.447+0000 7f72cd23d640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72c8108d00 0x7f72c8110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72c7fff640 1 -- 172.21.15.112:0/4177944239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72c8108d00 msgr2=0x7f72c8110a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72c7fff640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72c8108d00 0x7f72c8110a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72c7fff640 1 -- 172.21.15.112:0/4177944239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c8102a70 msgr2=0x7f72c8110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72c7fff640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c8102a70 0x7f72c8110010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72c7fff640 1 -- 172.21.15.112:0/4177944239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c81a87f0 con 0x7f72c81083b0 2024-09-21T03:27:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72cca3c640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c8102a70 0x7f72c8110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72c7fff640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72c81083b0 0x7f72c8110550 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f72b4006fd0 tx=0x7f72b400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72c5ffb640 1 -- 172.21.15.112:0/4177944239 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4015020 con 0x7f72c81083b0 2024-09-21T03:27:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72c5ffb640 1 -- 172.21.15.112:0/4177944239 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b4011ce0 con 0x7f72c81083b0 2024-09-21T03:27:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72ceca0640 1 -- 172.21.15.112:0/4177944239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c81a8ae0 con 0x7f72c81083b0 2024-09-21T03:27:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72c5ffb640 1 -- 172.21.15.112:0/4177944239 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4010d90 con 0x7f72c81083b0 2024-09-21T03:27:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.448+0000 7f72ceca0640 1 -- 172.21.15.112:0/4177944239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c81a9020 con 0x7f72c81083b0 2024-09-21T03:27:29.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.449+0000 7f72ceca0640 1 -- 172.21.15.112:0/4177944239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c8102f00 con 0x7f72c81083b0 2024-09-21T03:27:29.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.450+0000 7f72c5ffb640 1 -- 172.21.15.112:0/4177944239 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f72b4018020 con 0x7f72c81083b0 2024-09-21T03:27:29.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.451+0000 7f72c5ffb640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72b80b6670 0x7f72b80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:29.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.451+0000 7f72cca3c640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72b80b6670 0x7f72b80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:29.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.451+0000 7f72c5ffb640 1 -- 172.21.15.112:0/4177944239 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f72b40e6b90 con 0x7f72c81083b0 2024-09-21T03:27:29.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.452+0000 7f72cca3c640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72b80b6670 0x7f72b80b8b80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f72c8111a60 tx=0x7f72bc009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:29.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.456+0000 7f72c5ffb640 1 -- 172.21.15.112:0/4177944239 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f72b401f020 con 0x7f72c81083b0 2024-09-21T03:27:29.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:29 smithi203 ceph-mon[34723]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:27:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:29 smithi203 ceph-mon[34723]: pgmap v270: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-09-21T03:27:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:29 smithi187 ceph-mon[34603]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:27:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:29 smithi187 ceph-mon[34603]: pgmap v270: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-09-21T03:27:29.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.702+0000 7f72ceca0640 1 -- 172.21.15.112:0/4177944239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f72c802e640 con 0x7f72c81083b0 2024-09-21T03:27:29.705 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:29 smithi112 ceph-mon[31442]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:27:29.705 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:29 smithi112 ceph-mon[31442]: pgmap v270: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-09-21T03:27:29.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.705+0000 7f72c5ffb640 1 -- 172.21.15.112:0/4177944239 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f72b40ad2e0 con 0x7f72c81083b0 2024-09-21T03:27:29.708 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-09-21T03:27:29.708 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQAhPe5muoMGKhAAcbaGXb4X9CzEzOU2j42b6g== 2024-09-21T03:27:29.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 -- 172.21.15.112:0/4177944239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72b80b6670 msgr2=0x7f72b80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:29.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72b80b6670 0x7f72b80b8b80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f72c8111a60 tx=0x7f72bc009110 comp rx=0 tx=0).stop 2024-09-21T03:27:29.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 -- 172.21.15.112:0/4177944239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72c81083b0 msgr2=0x7f72c8110550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:29.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72c81083b0 0x7f72c8110550 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f72b4006fd0 tx=0x7f72b400b820 comp rx=0 tx=0).stop 2024-09-21T03:27:29.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 -- 172.21.15.112:0/4177944239 shutdown_connections 2024-09-21T03:27:29.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72b80b6670 0x7f72b80b8b80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:29.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72c8108d00 0x7f72c8110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:29.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72c81083b0 0x7f72c8110550 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:29.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 --2- 172.21.15.112:0/4177944239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c8102a70 0x7f72c8110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:29.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 -- 172.21.15.112:0/4177944239 >> 172.21.15.112:0/4177944239 conn(0x7f72c8079870 msgr2=0x7f72c8106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:29.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 -- 172.21.15.112:0/4177944239 shutdown_connections 2024-09-21T03:27:29.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:29.708+0000 7f72ceca0640 1 -- 172.21.15.112:0/4177944239 wait complete. 2024-09-21T03:27:30.507 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:27:30.508 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T03:27:30.508 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T03:27:30.545 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T03:27:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:30 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/4177944239' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:27:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:30 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/4177944239' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:27:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:30 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/4177944239' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:27:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:30 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/4177944239' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:27:30.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:30 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/4177944239' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T03:27:30.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:30 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/4177944239' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T03:27:30.829 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.310+0000 7f620e3dc640 1 -- 172.21.15.112:0/2523641198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6208108cf0 msgr2=0x7f620810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.311+0000 7f620e3dc640 1 --2- 172.21.15.112:0/2523641198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6208108cf0 0x7f620810f520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6200009ca0 tx=0x7f620002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.311+0000 7f620e3dc640 1 -- 172.21.15.112:0/2523641198 shutdown_connections 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.311+0000 7f620e3dc640 1 --2- 172.21.15.112:0/2523641198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6208108cf0 0x7f620810f520 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.311+0000 7f620e3dc640 1 --2- 172.21.15.112:0/2523641198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62081083a0 0x7f62081087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.311+0000 7f620e3dc640 1 --2- 172.21.15.112:0/2523641198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6208102a60 0x7f6208102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.311+0000 7f620e3dc640 1 -- 172.21.15.112:0/2523641198 >> 172.21.15.112:0/2523641198 conn(0x7f6208079730 msgr2=0x7f6208079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.311+0000 7f620e3dc640 1 -- 172.21.15.112:0/2523641198 shutdown_connections 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.311+0000 7f620e3dc640 1 -- 172.21.15.112:0/2523641198 wait complete. 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.311+0000 7f620e3dc640 1 Processor -- start 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.311+0000 7f620e3dc640 1 -- start start 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f620e3dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6208102a60 0x7f620819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f620e3dc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62081083a0 0x7f620819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f620e3dc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6208108cf0 0x7f620819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f620e3dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6208112960 con 0x7f6208102a60 2024-09-21T03:27:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f620e3dc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62081127e0 con 0x7f62081083a0 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f620e3dc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6208112ae0 con 0x7f6208108cf0 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f6207fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6208102a60 0x7f620819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f6207fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6208102a60 0x7f620819b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49770/0 (socket says 172.21.15.112:49770) 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f6207fff640 1 -- 172.21.15.112:0/2545562298 learned_addr learned my addr 172.21.15.112:0/2545562298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f620c979640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6208108cf0 0x7f620819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f62077fe640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62081083a0 0x7f620819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f6207fff640 1 -- 172.21.15.112:0/2545562298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6208108cf0 msgr2=0x7f620819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f6207fff640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6208108cf0 0x7f620819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f6207fff640 1 -- 172.21.15.112:0/2545562298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62081083a0 msgr2=0x7f620819b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f6207fff640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62081083a0 0x7f620819b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f6207fff640 1 -- 172.21.15.112:0/2545562298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62081a6520 con 0x7f6208102a60 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.312+0000 7f62077fe640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62081083a0 0x7f620819b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.313+0000 7f6207fff640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6208102a60 0x7f620819b210 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f61f800c890 tx=0x7f61f800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:31.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.313+0000 7f62057fa640 1 -- 172.21.15.112:0/2545562298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8004220 con 0x7f6208102a60 2024-09-21T03:27:31.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.313+0000 7f62057fa640 1 -- 172.21.15.112:0/2545562298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61f80043c0 con 0x7f6208102a60 2024-09-21T03:27:31.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.313+0000 7f620e3dc640 1 -- 172.21.15.112:0/2545562298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62081a6810 con 0x7f6208102a60 2024-09-21T03:27:31.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.313+0000 7f62057fa640 1 -- 172.21.15.112:0/2545562298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8004a00 con 0x7f6208102a60 2024-09-21T03:27:31.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.313+0000 7f620e3dc640 1 -- 172.21.15.112:0/2545562298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62081a6d50 con 0x7f6208102a60 2024-09-21T03:27:31.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.314+0000 7f620e3dc640 1 -- 172.21.15.112:0/2545562298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6208102e70 con 0x7f6208102a60 2024-09-21T03:27:31.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.315+0000 7f62057fa640 1 -- 172.21.15.112:0/2545562298 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f61f80079b0 con 0x7f6208102a60 2024-09-21T03:27:31.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.316+0000 7f62057fa640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61f00b65f0 0x7f61f00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:31.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.316+0000 7f62077fe640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61f00b65f0 0x7f61f00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:31.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.316+0000 7f62057fa640 1 -- 172.21.15.112:0/2545562298 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f61f80e7710 con 0x7f6208102a60 2024-09-21T03:27:31.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.316+0000 7f62077fe640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61f00b65f0 0x7f61f00b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f61fc005fd0 tx=0x7f61fc00b5d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:31.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.319+0000 7f62057fa640 1 -- 172.21.15.112:0/2545562298 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61f8014030 con 0x7f6208102a60 2024-09-21T03:27:31.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.498+0000 7f620e3dc640 1 -- 172.21.15.112:0/2545562298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f620806b2d0 con 0x7f6208102a60 2024-09-21T03:27:31.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:31 smithi112 ceph-mon[31442]: pgmap v271: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 9 op/s 2024-09-21T03:27:31.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.499+0000 7f62057fa640 1 -- 172.21.15.112:0/2545562298 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v56) ==== 86+0+133 (secure 0 0 0) 0x7f620806b2d0 con 0x7f6208102a60 2024-09-21T03:27:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:27:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 -- 172.21.15.112:0/2545562298 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61f00b65f0 msgr2=0x7f61f00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61f00b65f0 0x7f61f00b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f61fc005fd0 tx=0x7f61fc00b5d0 comp rx=0 tx=0).stop 2024-09-21T03:27:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 -- 172.21.15.112:0/2545562298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6208102a60 msgr2=0x7f620819b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6208102a60 0x7f620819b210 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f61f800c890 tx=0x7f61f800cd50 comp rx=0 tx=0).stop 2024-09-21T03:27:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 -- 172.21.15.112:0/2545562298 shutdown_connections 2024-09-21T03:27:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61f00b65f0 0x7f61f00b8b00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:31.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6208108cf0 0x7f620819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:31.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62081083a0 0x7f620819b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:31.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 --2- 172.21.15.112:0/2545562298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6208102a60 0x7f620819b210 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:31.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 -- 172.21.15.112:0/2545562298 >> 172.21.15.112:0/2545562298 conn(0x7f6208079730 msgr2=0x7f6208104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:31.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 -- 172.21.15.112:0/2545562298 shutdown_connections 2024-09-21T03:27:31.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:31.501+0000 7f620e3dc640 1 -- 172.21.15.112:0/2545562298 wait complete. 2024-09-21T03:27:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:31 smithi203 ceph-mon[34723]: pgmap v271: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 9 op/s 2024-09-21T03:27:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:31 smithi187 ceph-mon[34603]: pgmap v271: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 9 op/s 2024-09-21T03:27:32.232 INFO:teuthology.orchestra.run.smithi112.stdout:[{"version":56,"timestamp":"2024-09-21T03:26:52.399651+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T03:27:32.232 INFO:tasks.ceph_manager:config epoch is 56 2024-09-21T03:27:32.232 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T03:27:32.232 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T03:27:32.232 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T03:27:32.488 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:32 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2545562298' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:27:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]} : dispatch 2024-09-21T03:27:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T03:27:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-21T03:27:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-21T03:27:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T03:27:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T03:27:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]} : dispatch 2024-09-21T03:27:32.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:32 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-09-21T03:27:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:32 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2545562298' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:27:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]} : dispatch 2024-09-21T03:27:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T03:27:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-21T03:27:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-21T03:27:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T03:27:32.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T03:27:32.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]} : dispatch 2024-09-21T03:27:32.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:32 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-09-21T03:27:32.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:32 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2545562298' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T03:27:32.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]} : dispatch 2024-09-21T03:27:32.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T03:27:32.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-21T03:27:32.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-21T03:27:32.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T03:27:32.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T03:27:32.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]} : dispatch 2024-09-21T03:27:32.914 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:32 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-09-21T03:27:32.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.959+0000 7f0ed80dd640 1 -- 172.21.15.112:0/2557489242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed0102a70 msgr2=0x7f0ed0102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.959+0000 7f0ed80dd640 1 --2- 172.21.15.112:0/2557489242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed0102a70 0x7f0ed0102e80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009900 tx=0x7f0ecc02a410 comp rx=0 tx=0).stop 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.959+0000 7f0ed80dd640 1 -- 172.21.15.112:0/2557489242 shutdown_connections 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.959+0000 7f0ed80dd640 1 --2- 172.21.15.112:0/2557489242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed0108d00 0x7f0ed010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.959+0000 7f0ed80dd640 1 --2- 172.21.15.112:0/2557489242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ed01083b0 0x7f0ed01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.959+0000 7f0ed80dd640 1 --2- 172.21.15.112:0/2557489242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed0102a70 0x7f0ed0102e80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.959+0000 7f0ed80dd640 1 -- 172.21.15.112:0/2557489242 >> 172.21.15.112:0/2557489242 conn(0x7f0ed0079710 msgr2=0x7f0ed0079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.959+0000 7f0ed80dd640 1 -- 172.21.15.112:0/2557489242 shutdown_connections 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.959+0000 7f0ed80dd640 1 -- 172.21.15.112:0/2557489242 wait complete. 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.960+0000 7f0ed80dd640 1 Processor -- start 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.960+0000 7f0ed80dd640 1 -- start start 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed80dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed0102a70 0x7f0ed01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed80dd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ed01083b0 0x7f0ed0199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed80dd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed0108d00 0x7f0ed019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed80dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed01129c0 con 0x7f0ed0102a70 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed80dd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed0112840 con 0x7f0ed01083b0 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed80dd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed0112b40 con 0x7f0ed0108d00 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed5e79640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed0102a70 0x7f0ed01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed5e79640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed0102a70 0x7f0ed01993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49786/0 (socket says 172.21.15.112:49786) 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed5e79640 1 -- 172.21.15.112:0/811021835 learned_addr learned my addr 172.21.15.112:0/811021835 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed667a640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed0108d00 0x7f0ed019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed5678640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ed01083b0 0x7f0ed0199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed5e79640 1 -- 172.21.15.112:0/811021835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed0108d00 msgr2=0x7f0ed019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed5e79640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed0108d00 0x7f0ed019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed5e79640 1 -- 172.21.15.112:0/811021835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ed01083b0 msgr2=0x7f0ed0199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed5e79640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ed01083b0 0x7f0ed0199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:32.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed5e79640 1 -- 172.21.15.112:0/811021835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed019f320 con 0x7f0ed0102a70 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ed5e79640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed0102a70 0x7f0ed01993b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc005940 tx=0x7f0ecc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.961+0000 7f0ec6ffd640 1 -- 172.21.15.112:0/811021835 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc03f030 con 0x7f0ed0102a70 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.962+0000 7f0ec6ffd640 1 -- 172.21.15.112:0/811021835 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ecc003cb0 con 0x7f0ed0102a70 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.962+0000 7f0ed80dd640 1 -- 172.21.15.112:0/811021835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed019f5b0 con 0x7f0ed0102a70 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.962+0000 7f0ec6ffd640 1 -- 172.21.15.112:0/811021835 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc035070 con 0x7f0ed0102a70 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.962+0000 7f0ed80dd640 1 -- 172.21.15.112:0/811021835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed01a5d70 con 0x7f0ed0102a70 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.963+0000 7f0ed80dd640 1 -- 172.21.15.112:0/811021835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed0103cf0 con 0x7f0ed0102a70 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.964+0000 7f0ec6ffd640 1 -- 172.21.15.112:0/811021835 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f0ecc042030 con 0x7f0ed0102a70 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.965+0000 7f0ec6ffd640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ebc0b65f0 0x7f0ebc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.965+0000 7f0ed5678640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ebc0b65f0 0x7f0ebc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.965+0000 7f0ec6ffd640 1 -- 172.21.15.112:0/811021835 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8426+0+0 (secure 0 0 0) 0x7f0ecc1080d0 con 0x7f0ed0102a70 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.965+0000 7f0ed5678640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ebc0b65f0 0x7f0ebc0b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0005fd0 tx=0x7f0ec0005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:32.968+0000 7f0ec6ffd640 1 -- 172.21.15.112:0/811021835 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ecc0ce740 con 0x7f0ed0102a70 2024-09-21T03:27:33.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.191+0000 7f0ed80dd640 1 -- 172.21.15.112:0/811021835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0ed010aad0 con 0x7f0ed0102a70 2024-09-21T03:27:33.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.198+0000 7f0ec6ffd640 1 -- 172.21.15.112:0/811021835 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+296461 (secure 0 0 0) 0x7f0ecc0d2b20 con 0x7f0ed0102a70 2024-09-21T03:27:33.200 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:27:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 -- 172.21.15.112:0/811021835 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ebc0b65f0 msgr2=0x7f0ebc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ebc0b65f0 0x7f0ebc0b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0005fd0 tx=0x7f0ec0005850 comp rx=0 tx=0).stop 2024-09-21T03:27:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 -- 172.21.15.112:0/811021835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed0102a70 msgr2=0x7f0ed01993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed0102a70 0x7f0ed01993b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc005940 tx=0x7f0ecc0042f0 comp rx=0 tx=0).stop 2024-09-21T03:27:33.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 -- 172.21.15.112:0/811021835 shutdown_connections 2024-09-21T03:27:33.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0ebc0b65f0 0x7f0ebc0b8b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:33.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ed0108d00 0x7f0ed019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:33.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ed01083b0 0x7f0ed0199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:33.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 --2- 172.21.15.112:0/811021835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ed0102a70 0x7f0ed01993b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:33.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 -- 172.21.15.112:0/811021835 >> 172.21.15.112:0/811021835 conn(0x7f0ed0079710 msgr2=0x7f0ed0106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:33.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 -- 172.21.15.112:0/811021835 shutdown_connections 2024-09-21T03:27:33.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:33.201+0000 7f0ed80dd640 1 -- 172.21.15.112:0/811021835 wait complete. 2024-09-21T03:27:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:33 smithi112 ceph-mon[31442]: pgmap v272: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-21T03:27:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-21T03:27:33.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T03:27:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-21T03:27:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-21T03:27:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T03:27:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T03:27:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-21T03:27:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:33 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-21T03:27:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:33 smithi112 ceph-mon[31442]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:27:33.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:33 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/811021835' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:27:33.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:33 smithi187 ceph-mon[34603]: pgmap v272: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-21T03:27:33.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-21T03:27:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T03:27:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-21T03:27:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-21T03:27:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T03:27:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T03:27:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-21T03:27:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:33 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-21T03:27:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:33 smithi187 ceph-mon[34603]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:27:33.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:33 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/811021835' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:27:33.879 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"flags":0,"active_gid":14226,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":4024464582},{"type":"v1","addr":"172.21.15.112:6801","nonce":4024464582}]},"active_addr":"172.21.15.112:6801/4024464582","active_change":"2024-09-21T03:20:32.227422+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24145,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24155,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.112:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3312220801}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3017432490}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":79700488}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2215771593}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2913928311}]}]} 2024-09-21T03:27:33.886 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T03:27:33.887 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T03:27:33.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:27:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:33 smithi203 ceph-mon[34723]: pgmap v272: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-09-21T03:27:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-21T03:27:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T03:27:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-21T03:27:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-21T03:27:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T03:27:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T03:27:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-09-21T03:27:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:33 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-21T03:27:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:33 smithi203 ceph-mon[34723]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T03:27:33.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:33 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/811021835' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T03:27:34.165 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:34.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:34 smithi112 ceph-mon[31442]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:27:34.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:34 smithi112 ceph-mon[31442]: pgmap v275: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.611+0000 7fef0984f640 1 -- 172.21.15.112:0/3398911106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04105360 msgr2=0x7fef0410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.611+0000 7fef0984f640 1 --2- 172.21.15.112:0/3398911106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04105360 0x7fef0410d8b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feeec009900 tx=0x7feeec02a410 comp rx=0 tx=0).stop 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.611+0000 7fef0984f640 1 -- 172.21.15.112:0/3398911106 shutdown_connections 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.611+0000 7fef0984f640 1 --2- 172.21.15.112:0/3398911106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef0410de40 0x7fef04110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.611+0000 7fef0984f640 1 --2- 172.21.15.112:0/3398911106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04105360 0x7fef0410d8b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.611+0000 7fef0984f640 1 --2- 172.21.15.112:0/3398911106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04104a10 0x7fef04104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.611+0000 7fef0984f640 1 -- 172.21.15.112:0/3398911106 >> 172.21.15.112:0/3398911106 conn(0x7fef04079720 msgr2=0x7fef04079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.611+0000 7fef0984f640 1 -- 172.21.15.112:0/3398911106 shutdown_connections 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.611+0000 7fef0984f640 1 -- 172.21.15.112:0/3398911106 wait complete. 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.611+0000 7fef0984f640 1 Processor -- start 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.611+0000 7fef0984f640 1 -- start start 2024-09-21T03:27:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7fef0984f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04104a10 0x7fef0419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7fef0984f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04105360 0x7fef0419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7fef0984f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef0410de40 0x7fef0419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7fef0984f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef04113a80 con 0x7fef04105360 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7fef0984f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fef04113900 con 0x7fef0410de40 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7fef0984f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef04113c00 con 0x7fef04104a10 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7feefa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04105360 0x7fef0419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7feefa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04105360 0x7fef0419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49794/0 (socket says 172.21.15.112:49794) 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7feefa7fc640 1 -- 172.21.15.112:0/3779472048 learned_addr learned my addr 172.21.15.112:0/3779472048 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7fef037fe640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef0410de40 0x7fef0419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7fef02ffd640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04104a10 0x7fef0419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7feefa7fc640 1 -- 172.21.15.112:0/3779472048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04104a10 msgr2=0x7fef0419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7feefa7fc640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04104a10 0x7fef0419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7feefa7fc640 1 -- 172.21.15.112:0/3779472048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef0410de40 msgr2=0x7fef0419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7feefa7fc640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef0410de40 0x7fef0419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7feefa7fc640 1 -- 172.21.15.112:0/3779472048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef041a04c0 con 0x7fef04105360 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7feefa7fc640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04105360 0x7fef0419aaf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feeec02a930 tx=0x7feeec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.612+0000 7fef0884d640 1 -- 172.21.15.112:0/3779472048 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec03f030 con 0x7fef04105360 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.613+0000 7fef0884d640 1 -- 172.21.15.112:0/3779472048 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeec003aa0 con 0x7fef04105360 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.613+0000 7fef0884d640 1 -- 172.21.15.112:0/3779472048 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec035070 con 0x7fef04105360 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.613+0000 7fef0984f640 1 -- 172.21.15.112:0/3779472048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef041a0750 con 0x7fef04105360 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.613+0000 7fef0984f640 1 -- 172.21.15.112:0/3779472048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041a6f10 con 0x7fef04105360 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.614+0000 7fef0984f640 1 -- 172.21.15.112:0/3779472048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0406bed0 con 0x7fef04105360 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.615+0000 7fef0884d640 1 -- 172.21.15.112:0/3779472048 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7feeec042030 con 0x7fef04105360 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.616+0000 7fef0884d640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feef00b65f0 0x7feef00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.616+0000 7fef02ffd640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feef00b65f0 0x7feef00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.616+0000 7fef0884d640 1 -- 172.21.15.112:0/3779472048 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7feeec107d00 con 0x7fef04105360 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.616+0000 7fef02ffd640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feef00b65f0 0x7feef00b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feef4005fd0 tx=0x7feef4005e20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:34.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.620+0000 7fef0884d640 1 -- 172.21.15.112:0/3779472048 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feeec0ce370 con 0x7fef04105360 2024-09-21T03:27:34.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.800+0000 7fef0984f640 1 -- 172.21.15.112:0/3779472048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef04108f20 con 0x7fef04105360 2024-09-21T03:27:34.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.802+0000 7fef0884d640 1 -- 172.21.15.112:0/3779472048 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23065 (secure 0 0 0) 0x7feeec0d2750 con 0x7fef04105360 2024-09-21T03:27:34.804 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:27:34.804 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":76,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:27:33.414969+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:26:58.087349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:26:59.970444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 -- 172.21.15.112:0/3779472048 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feef00b65f0 msgr2=0x7feef00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feef00b65f0 0x7feef00b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feef4005fd0 tx=0x7feef4005e20 comp rx=0 tx=0).stop 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 -- 172.21.15.112:0/3779472048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04105360 msgr2=0x7fef0419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04105360 0x7fef0419aaf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feeec02a930 tx=0x7feeec004270 comp rx=0 tx=0).stop 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 -- 172.21.15.112:0/3779472048 shutdown_connections 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7feef00b65f0 0x7feef00b8b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef0410de40 0x7fef0419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04105360 0x7fef0419aaf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 --2- 172.21.15.112:0/3779472048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04104a10 0x7fef0419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 -- 172.21.15.112:0/3779472048 >> 172.21.15.112:0/3779472048 conn(0x7fef04079720 msgr2=0x7fef04100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 -- 172.21.15.112:0/3779472048 shutdown_connections 2024-09-21T03:27:34.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:34.804+0000 7fef0984f640 1 -- 172.21.15.112:0/3779472048 wait complete. 2024-09-21T03:27:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:34 smithi203 ceph-mon[34723]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:27:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:34 smithi203 ceph-mon[34723]: pgmap v275: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-09-21T03:27:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:34 smithi187 ceph-mon[34603]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T03:27:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:34 smithi187 ceph-mon[34603]: pgmap v275: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-09-21T03:27:35.456 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T03:27:35.456 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:27:35.701 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:35 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3779472048' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:35.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:35 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3779472048' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:35.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:35 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3779472048' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:36.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.168+0000 7fb3920c3640 1 -- 172.21.15.112:0/3023959393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb38c102a80 msgr2=0x7fb38c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.168+0000 7fb3920c3640 1 --2- 172.21.15.112:0/3023959393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb38c102a80 0x7fb38c102e90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009900 tx=0x7fb37c02a410 comp rx=0 tx=0).stop 2024-09-21T03:27:36.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.168+0000 7fb3920c3640 1 -- 172.21.15.112:0/3023959393 shutdown_connections 2024-09-21T03:27:36.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.168+0000 7fb3920c3640 1 --2- 172.21.15.112:0/3023959393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb38c108d10 0x7fb38c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.168+0000 7fb3920c3640 1 --2- 172.21.15.112:0/3023959393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38c1083c0 0x7fb38c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.168+0000 7fb3920c3640 1 --2- 172.21.15.112:0/3023959393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb38c102a80 0x7fb38c102e90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.168+0000 7fb3920c3640 1 -- 172.21.15.112:0/3023959393 >> 172.21.15.112:0/3023959393 conn(0x7fb38c079740 msgr2=0x7fb38c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:36.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.168+0000 7fb3920c3640 1 -- 172.21.15.112:0/3023959393 shutdown_connections 2024-09-21T03:27:36.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.168+0000 7fb3920c3640 1 -- 172.21.15.112:0/3023959393 wait complete. 2024-09-21T03:27:36.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.168+0000 7fb3920c3640 1 Processor -- start 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb3920c3640 1 -- start start 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb3920c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb38c102a80 0x7fb38c10ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb3920c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38c1083c0 0x7fb38c110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb3920c3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb38c108d10 0x7fb38c110a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb3920c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c113aa0 con 0x7fb38c102a80 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb3920c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c113920 con 0x7fb38c1083c0 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb3920c3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c113c20 con 0x7fb38c108d10 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb38b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb38c102a80 0x7fb38c10ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb38b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb38c102a80 0x7fb38c10ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42840/0 (socket says 172.21.15.112:42840) 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb38b7fe640 1 -- 172.21.15.112:0/350522697 learned_addr learned my addr 172.21.15.112:0/350522697 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb38bfff640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb38c108d10 0x7fb38c110a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb38affd640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38c1083c0 0x7fb38c110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb38b7fe640 1 -- 172.21.15.112:0/350522697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb38c108d10 msgr2=0x7fb38c110a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb38b7fe640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb38c108d10 0x7fb38c110a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb38b7fe640 1 -- 172.21.15.112:0/350522697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38c1083c0 msgr2=0x7fb38c110510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb38b7fe640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38c1083c0 0x7fb38c110510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.169+0000 7fb38b7fe640 1 -- 172.21.15.112:0/350522697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38c1a8660 con 0x7fb38c102a80 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.170+0000 7fb38b7fe640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb38c102a80 0x7fb38c10ffd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb37c005940 tx=0x7fb37c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.170+0000 7fb388ff9640 1 -- 172.21.15.112:0/350522697 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c03f030 con 0x7fb38c102a80 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.170+0000 7fb3920c3640 1 -- 172.21.15.112:0/350522697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38c1a88f0 con 0x7fb38c102a80 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.170+0000 7fb388ff9640 1 -- 172.21.15.112:0/350522697 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb37c003cb0 con 0x7fb38c102a80 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.170+0000 7fb388ff9640 1 -- 172.21.15.112:0/350522697 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c035070 con 0x7fb38c102a80 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.170+0000 7fb3920c3640 1 -- 172.21.15.112:0/350522697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38c1a8dd0 con 0x7fb38c102a80 2024-09-21T03:27:36.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.171+0000 7fb3920c3640 1 -- 172.21.15.112:0/350522697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38c103d00 con 0x7fb38c102a80 2024-09-21T03:27:36.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.172+0000 7fb388ff9640 1 -- 172.21.15.112:0/350522697 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fb37c042030 con 0x7fb38c102a80 2024-09-21T03:27:36.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.173+0000 7fb388ff9640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3740b65f0 0x7fb3740b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:36.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.173+0000 7fb38affd640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3740b65f0 0x7fb3740b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:36.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.173+0000 7fb388ff9640 1 -- 172.21.15.112:0/350522697 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fb37c107e00 con 0x7fb38c102a80 2024-09-21T03:27:36.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.174+0000 7fb38affd640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3740b65f0 0x7fb3740b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb38c100c00 tx=0x7fb38000a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:36.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.178+0000 7fb388ff9640 1 -- 172.21.15.112:0/350522697 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb37c0ce470 con 0x7fb38c102a80 2024-09-21T03:27:36.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.358+0000 7fb3920c3640 1 -- 172.21.15.112:0/350522697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb38c1112d0 con 0x7fb38c102a80 2024-09-21T03:27:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.360+0000 7fb388ff9640 1 -- 172.21.15.112:0/350522697 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23065 (secure 0 0 0) 0x7fb37c0d2850 con 0x7fb38c102a80 2024-09-21T03:27:36.362 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:27:36.362 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":76,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:27:33.414969+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:26:58.087349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:26:59.970444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:27:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 -- 172.21.15.112:0/350522697 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3740b65f0 msgr2=0x7fb3740b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3740b65f0 0x7fb3740b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb38c100c00 tx=0x7fb38000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 -- 172.21.15.112:0/350522697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb38c102a80 msgr2=0x7fb38c10ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb38c102a80 0x7fb38c10ffd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb37c005940 tx=0x7fb37c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 -- 172.21.15.112:0/350522697 shutdown_connections 2024-09-21T03:27:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3740b65f0 0x7fb3740b8b00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb38c108d10 0x7fb38c110a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb38c1083c0 0x7fb38c110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 --2- 172.21.15.112:0/350522697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb38c102a80 0x7fb38c10ffd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:36.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 -- 172.21.15.112:0/350522697 >> 172.21.15.112:0/350522697 conn(0x7fb38c079740 msgr2=0x7fb38c1013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:36.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 -- 172.21.15.112:0/350522697 shutdown_connections 2024-09-21T03:27:36.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:36.362+0000 7fb3920c3640 1 -- 172.21.15.112:0/350522697 wait complete. 2024-09-21T03:27:36.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:36 smithi112 ceph-mon[31442]: pgmap v276: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.5 KiB/s wr, 8 op/s; 91 B/s, 1 objects/s recovering 2024-09-21T03:27:36.734 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:36 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/350522697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:36.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:36 smithi203 ceph-mon[34723]: pgmap v276: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.5 KiB/s wr, 8 op/s; 91 B/s, 1 objects/s recovering 2024-09-21T03:27:36.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:36 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/350522697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:36.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:36 smithi187 ceph-mon[34603]: pgmap v276: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.5 KiB/s wr, 8 op/s; 91 B/s, 1 objects/s recovering 2024-09-21T03:27:36.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:36 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/350522697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:27:37.156 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T03:27:37.156 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T03:27:37.157 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T03:27:37.157 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T03:27:37.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T03:27:37.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T03:27:37.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T03:27:37.159 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T03:27:37.159 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T03:27:37.159 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T03:27:37.160 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T03:27:37.160 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T03:27:37.953 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:37.956 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:37.959 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:37.962 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:37.962 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:37.969 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:37.969 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:37.973 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:37.977 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:37.981 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:37.983 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:37.991 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.757+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2319743546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd4104840 msgr2=0x7f7dd4104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.757+0000 7f7dd9fc7640 1 --2- 172.21.15.112:0/2319743546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd4104840 0x7f7dd4104c50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009900 tx=0x7f7dc402a410 comp rx=0 tx=0).stop 2024-09-21T03:27:39.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.757+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2319743546 shutdown_connections 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.757+0000 7f7dd9fc7640 1 --2- 172.21.15.112:0/2319743546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dd410ddf0 0x7f7dd4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.757+0000 7f7dd9fc7640 1 --2- 172.21.15.112:0/2319743546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd4105270 0x7f7dd410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.757+0000 7f7dd9fc7640 1 --2- 172.21.15.112:0/2319743546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd4104840 0x7f7dd4104c50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.757+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2319743546 >> 172.21.15.112:0/2319743546 conn(0x7f7dd4079730 msgr2=0x7f7dd4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.757+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2319743546 shutdown_connections 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.757+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2319743546 wait complete. 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.758+0000 7f7dd9fc7640 1 Processor -- start 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.758+0000 7f7dd9fc7640 1 -- start start 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.758+0000 7f7dd9fc7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd4104840 0x7f7dd419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.758+0000 7f7dd9fc7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd4105270 0x7f7dd419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.758+0000 7f7dd9fc7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dd410ddf0 0x7f7dd419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd9fc7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd4113a70 con 0x7f7dd4105270 2024-09-21T03:27:39.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd9fc7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd41138f0 con 0x7f7dd4104840 2024-09-21T03:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd9fc7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd4113bf0 con 0x7f7dd410ddf0 2024-09-21T03:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd4105270 0x7f7dd419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd4105270 0x7f7dd419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42850/0 (socket says 172.21.15.112:42850) 2024-09-21T03:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd2ffd640 1 -- 172.21.15.112:0/2814332879 learned_addr learned my addr 172.21.15.112:0/2814332879 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd3fff640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dd410ddf0 0x7f7dd419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd37fe640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd4104840 0x7f7dd419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd2ffd640 1 -- 172.21.15.112:0/2814332879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dd410ddf0 msgr2=0x7f7dd419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd2ffd640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dd410ddf0 0x7f7dd419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd2ffd640 1 -- 172.21.15.112:0/2814332879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd4104840 msgr2=0x7f7dd419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd2ffd640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd4104840 0x7f7dd419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd2ffd640 1 -- 172.21.15.112:0/2814332879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd41a04d0 con 0x7f7dd4105270 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd37fe640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dd4104840 0x7f7dd419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd2ffd640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd4105270 0x7f7dd419ab00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800d740 tx=0x7f7dc800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd0ff9640 1 -- 172.21.15.112:0/2814332879 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8017070 con 0x7f7dd4105270 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd0ff9640 1 -- 172.21.15.112:0/2814332879 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dc800de50 con 0x7f7dd4105270 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2814332879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd41a07c0 con 0x7f7dd4105270 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd0ff9640 1 -- 172.21.15.112:0/2814332879 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc800f620 con 0x7f7dd4105270 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.759+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2814332879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd41a6f20 con 0x7f7dd4105270 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.760+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2814332879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7dd41072d0 con 0x7f7dd4105270 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.762+0000 7f7dd0ff9640 1 -- 172.21.15.112:0/2814332879 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f7dc801a070 con 0x7f7dd4105270 2024-09-21T03:27:39.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.762+0000 7f7dd0ff9640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7dbc0b65f0 0x7f7dbc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.763+0000 7f7dd37fe640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7dbc0b65f0 0x7f7dbc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.763+0000 7f7dd0ff9640 1 -- 172.21.15.112:0/2814332879 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f7dc80e7490 con 0x7f7dd4105270 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.763+0000 7f7dd0ff9640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739] conn(0x7f7dbc0c0360 0x7f7dbc0c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.763+0000 7f7dd0ff9640 1 -- 172.21.15.112:0/2814332879 --> [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7dbc0c2f20 con 0x7f7dbc0c0360 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.763+0000 7f7dd0ff9640 1 -- 172.21.15.112:0/2814332879 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7dc80e78e0 con 0x7f7dd4105270 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.763+0000 7f7dd3fff640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739] conn(0x7f7dbc0c0360 0x7f7dbc0c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.763+0000 7f7dd37fe640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7dbc0b65f0 0x7f7dbc0b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40096c0 tx=0x7f7dc4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.763+0000 7f7dd3fff640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739] conn(0x7f7dbc0c0360 0x7f7dbc0c2850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.764+0000 7f7dd0ff9640 1 -- 172.21.15.112:0/2814332879 <== osd.3 v2:172.21.15.112:6810/2827467739 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7dbc0c2f20 con 0x7f7dbc0c0360 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.777+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2814332879 --> [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7dd406b500 con 0x7f7dbc0c0360 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd0ff9640 1 -- 172.21.15.112:0/2814332879 <== osd.3 v2:172.21.15.112:6810/2827467739 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7dd406b500 con 0x7f7dbc0c0360 2024-09-21T03:27:39.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739] conn(0x7f7dbc0c0360 msgr2=0x7f7dbc0c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd9fc7640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739] conn(0x7f7dbc0c0360 0x7f7dbc0c2850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7dbc0b65f0 msgr2=0x7f7dbc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd9fc7640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7dbc0b65f0 0x7f7dbc0b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40096c0 tx=0x7f7dc4031400 comp rx=0 tx=0).stop 2024-09-21T03:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd4105270 msgr2=0x7f7dd419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd9fc7640 1 --2- 172.21.15.112:0/2814332879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7dd4105270 0x7f7dd419ab00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800d740 tx=0x7f7dc800dc00 comp rx=0 tx=0).stop 2024-09-21T03:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd3fff640 1 -- 172.21.15.112:0/2814332879 reap_dead start 2024-09-21T03:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2814332879 shutdown_connections 2024-09-21T03:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2814332879 >> 172.21.15.112:0/2814332879 conn(0x7f7dd4079730 msgr2=0x7f7dd41068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2814332879 shutdown_connections 2024-09-21T03:27:39.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:38.778+0000 7f7dd9fc7640 1 -- 172.21.15.112:0/2814332879 wait complete. 2024-09-21T03:27:39.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:39 smithi203 ceph-mon[34723]: pgmap v277: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 639 B/s wr, 4 op/s; 91 B/s, 1 objects/s recovering 2024-09-21T03:27:39.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:39 smithi187 ceph-mon[34603]: pgmap v277: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 639 B/s wr, 4 op/s; 91 B/s, 1 objects/s recovering 2024-09-21T03:27:39.711 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:39 smithi112 ceph-mon[31442]: pgmap v277: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 639 B/s wr, 4 op/s; 91 B/s, 1 objects/s recovering 2024-09-21T03:27:40.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.271+0000 7f67200f4640 1 -- 172.21.15.112:0/1021754164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67181020e0 msgr2=0x7f67181024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.271+0000 7f67200f4640 1 --2- 172.21.15.112:0/1021754164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67181020e0 0x7f67181024f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6714009900 tx=0x7f671402a410 comp rx=0 tx=0).stop 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.271+0000 7f67200f4640 1 -- 172.21.15.112:0/1021754164 shutdown_connections 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.271+0000 7f67200f4640 1 --2- 172.21.15.112:0/1021754164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6718106c80 0x7f6718110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.271+0000 7f67200f4640 1 --2- 172.21.15.112:0/1021754164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718106330 0x7f6718106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.271+0000 7f67200f4640 1 --2- 172.21.15.112:0/1021754164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67181020e0 0x7f67181024f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.271+0000 7f67200f4640 1 -- 172.21.15.112:0/1021754164 >> 172.21.15.112:0/1021754164 conn(0x7f6718079ec0 msgr2=0x7f671807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.271+0000 7f67200f4640 1 -- 172.21.15.112:0/1021754164 shutdown_connections 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.271+0000 7f67200f4640 1 -- 172.21.15.112:0/1021754164 wait complete. 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f67200f4640 1 Processor -- start 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f67200f4640 1 -- start start 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f67200f4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67181020e0 0x7f671819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f67200f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718106330 0x7f671819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:40.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f67200f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6718106c80 0x7f671819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f67200f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6718113bb0 con 0x7f6718106c80 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f67200f4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6718113a30 con 0x7f6718106330 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f67200f4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6718113d30 con 0x7f67181020e0 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f671e691640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6718106c80 0x7f671819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f671e691640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6718106c80 0x7f671819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42878/0 (socket says 172.21.15.112:42878) 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f671e691640 1 -- 172.21.15.112:0/3778937483 learned_addr learned my addr 172.21.15.112:0/3778937483 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f671de90640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67181020e0 0x7f671819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.272+0000 7f671d68f640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718106330 0x7f671819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.273+0000 7f671e691640 1 -- 172.21.15.112:0/3778937483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67181020e0 msgr2=0x7f671819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.273+0000 7f671e691640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67181020e0 0x7f671819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.273+0000 7f671e691640 1 -- 172.21.15.112:0/3778937483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718106330 msgr2=0x7f671819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.273+0000 7f671e691640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718106330 0x7f671819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.273+0000 7f671e691640 1 -- 172.21.15.112:0/3778937483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67181a0510 con 0x7f6718106c80 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.273+0000 7f671e691640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6718106c80 0x7f671819fde0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6710009fd0 tx=0x7f671000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.273+0000 7f670effd640 1 -- 172.21.15.112:0/3778937483 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710017070 con 0x7f6718106c80 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.273+0000 7f670effd640 1 -- 172.21.15.112:0/3778937483 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6710003e20 con 0x7f6718106c80 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.273+0000 7f67200f4640 1 -- 172.21.15.112:0/3778937483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67181a0800 con 0x7f6718106c80 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.273+0000 7f670effd640 1 -- 172.21.15.112:0/3778937483 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710010460 con 0x7f6718106c80 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.273+0000 7f67200f4640 1 -- 172.21.15.112:0/3778937483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67181a6f60 con 0x7f6718106c80 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.274+0000 7f67200f4640 1 -- 172.21.15.112:0/3778937483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6718063ad0 con 0x7f6718106c80 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.275+0000 7f670effd640 1 -- 172.21.15.112:0/3778937483 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f6710003970 con 0x7f6718106c80 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.276+0000 7f670effd640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f67040b65f0 0x7f67040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.276+0000 7f671de90640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f67040b65f0 0x7f67040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.276+0000 7f670effd640 1 -- 172.21.15.112:0/3778937483 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f67100e77a0 con 0x7f6718106c80 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.277+0000 7f670effd640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149] conn(0x7f67040c0360 0x7f67040c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.277+0000 7f670effd640 1 -- 172.21.15.112:0/3778937483 --> [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67040c2f20 con 0x7f67040c0360 2024-09-21T03:27:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.277+0000 7f670effd640 1 -- 172.21.15.112:0/3778937483 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f67100e7bf0 con 0x7f6718106c80 2024-09-21T03:27:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.277+0000 7f671de90640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f67040b65f0 0x7f67040b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6714004770 tx=0x7f67140058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.277+0000 7f671d68f640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149] conn(0x7f67040c0360 0x7f67040c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.277+0000 7f671d68f640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149] conn(0x7f67040c0360 0x7f67040c2850 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.278+0000 7f670effd640 1 -- 172.21.15.112:0/3778937483 <== osd.10 v2:172.21.15.187:6824/499555149 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67040c2f20 con 0x7f67040c0360 2024-09-21T03:27:40.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.292+0000 7f67200f4640 1 -- 172.21.15.112:0/3778937483 --> [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6718063f50 con 0x7f67040c0360 2024-09-21T03:27:40.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.292+0000 7f670effd640 1 -- 172.21.15.112:0/3778937483 <== osd.10 v2:172.21.15.187:6824/499555149 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6718063f50 con 0x7f67040c0360 2024-09-21T03:27:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.293+0000 7f67200f4640 1 -- 172.21.15.112:0/3778937483 >> [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149] conn(0x7f67040c0360 msgr2=0x7f67040c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.293+0000 7f67200f4640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149] conn(0x7f67040c0360 0x7f67040c2850 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.293+0000 7f67200f4640 1 -- 172.21.15.112:0/3778937483 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f67040b65f0 msgr2=0x7f67040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.293+0000 7f67200f4640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f67040b65f0 0x7f67040b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6714004770 tx=0x7f67140058d0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.293+0000 7f67200f4640 1 -- 172.21.15.112:0/3778937483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6718106c80 msgr2=0x7f671819fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.293+0000 7f67200f4640 1 --2- 172.21.15.112:0/3778937483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6718106c80 0x7f671819fde0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6710009fd0 tx=0x7f671000b260 comp rx=0 tx=0).stop 2024-09-21T03:27:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.293+0000 7f671e691640 1 -- 172.21.15.112:0/3778937483 reap_dead start 2024-09-21T03:27:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.293+0000 7f67200f4640 1 -- 172.21.15.112:0/3778937483 shutdown_connections 2024-09-21T03:27:40.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.293+0000 7f67200f4640 1 -- 172.21.15.112:0/3778937483 >> 172.21.15.112:0/3778937483 conn(0x7f6718079ec0 msgr2=0x7f6718107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:40.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.293+0000 7f67200f4640 1 -- 172.21.15.112:0/3778937483 shutdown_connections 2024-09-21T03:27:40.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.293+0000 7f67200f4640 1 -- 172.21.15.112:0/3778937483 wait complete. 2024-09-21T03:27:40.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.474+0000 7f1e76299640 1 -- 172.21.15.112:0/3148761952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e70103b60 msgr2=0x7f1e70103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.474+0000 7f1e76299640 1 --2- 172.21.15.112:0/3148761952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e70103b60 0x7f1e70103f70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009900 tx=0x7f1e6002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:40.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.474+0000 7f1e76299640 1 -- 172.21.15.112:0/3148761952 shutdown_connections 2024-09-21T03:27:40.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.474+0000 7f1e76299640 1 --2- 172.21.15.112:0/3148761952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e70109df0 0x7f1e70110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.474+0000 7f1e76299640 1 --2- 172.21.15.112:0/3148761952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e701094a0 0x7f1e701098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.474+0000 7f1e76299640 1 --2- 172.21.15.112:0/3148761952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e70103b60 0x7f1e70103f70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.474+0000 7f1e76299640 1 -- 172.21.15.112:0/3148761952 >> 172.21.15.112:0/3148761952 conn(0x7f1e7007b850 msgr2=0x7f1e7007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:40.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.474+0000 7f1e76299640 1 -- 172.21.15.112:0/3148761952 shutdown_connections 2024-09-21T03:27:40.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.474+0000 7f1e76299640 1 -- 172.21.15.112:0/3148761952 wait complete. 2024-09-21T03:27:40.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.474+0000 7f1e76299640 1 Processor -- start 2024-09-21T03:27:40.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.474+0000 7f1e76299640 1 -- start start 2024-09-21T03:27:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e76299640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e70103b60 0x7f1e7019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e76299640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e701094a0 0x7f1e7019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e76299640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e70109df0 0x7f1e7019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e76299640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e70113b70 con 0x7f1e70109df0 2024-09-21T03:27:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e76299640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e701139f0 con 0x7f1e70103b60 2024-09-21T03:27:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e76299640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1e70113cf0 con 0x7f1e701094a0 2024-09-21T03:27:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e74836640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e70109df0 0x7f1e7019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e74836640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e70109df0 0x7f1e7019fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42898/0 (socket says 172.21.15.112:42898) 2024-09-21T03:27:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e74836640 1 -- 172.21.15.112:0/3929685331 learned_addr learned my addr 172.21.15.112:0/3929685331 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e6f7fe640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e701094a0 0x7f1e7019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:40.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e6ffff640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e70103b60 0x7f1e7019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:40.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e74836640 1 -- 172.21.15.112:0/3929685331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e701094a0 msgr2=0x7f1e7019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e74836640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e701094a0 0x7f1e7019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e74836640 1 -- 172.21.15.112:0/3929685331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e70103b60 msgr2=0x7f1e7019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e74836640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e70103b60 0x7f1e7019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e74836640 1 -- 172.21.15.112:0/3929685331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e701a05b0 con 0x7f1e70109df0 2024-09-21T03:27:40.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.475+0000 7f1e74836640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e70109df0 0x7f1e7019fe80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b970 tx=0x7f1e6800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:40.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.476+0000 7f1e6d7fa640 1 -- 172.21.15.112:0/3929685331 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68019070 con 0x7f1e70109df0 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.476+0000 7f1e6d7fa640 1 -- 172.21.15.112:0/3929685331 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e68014440 con 0x7f1e70109df0 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.476+0000 7f1e76299640 1 -- 172.21.15.112:0/3929685331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e701a08a0 con 0x7f1e70109df0 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.476+0000 7f1e6d7fa640 1 -- 172.21.15.112:0/3929685331 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e680075d0 con 0x7f1e70109df0 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.476+0000 7f1e76299640 1 -- 172.21.15.112:0/3929685331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e701a7000 con 0x7f1e70109df0 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.476+0000 7f1e76299640 1 -- 172.21.15.112:0/3929685331 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e70063b50 con 0x7f1e70109df0 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.478+0000 7f1e6d7fa640 1 -- 172.21.15.112:0/3929685331 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f1e6801c070 con 0x7f1e70109df0 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.479+0000 7f1e6d7fa640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1e580b6670 0x7f1e580b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.479+0000 7f1e6ffff640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1e580b6670 0x7f1e580b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.479+0000 7f1e6d7fa640 1 -- 172.21.15.112:0/3929685331 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f1e680e71c0 con 0x7f1e70109df0 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.479+0000 7f1e6d7fa640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172] conn(0x7f1e580c03e0 0x7f1e580c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.479+0000 7f1e6d7fa640 1 -- 172.21.15.112:0/3929685331 --> [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e580c2fa0 con 0x7f1e580c03e0 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.479+0000 7f1e6d7fa640 1 -- 172.21.15.112:0/3929685331 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1e680e7610 con 0x7f1e70109df0 2024-09-21T03:27:40.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.479+0000 7f1e6ffff640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1e580b6670 0x7f1e580b8b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e60005f70 tx=0x7f1e60000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:40.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.479+0000 7f1e6f7fe640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172] conn(0x7f1e580c03e0 0x7f1e580c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:40.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.480+0000 7f1e6f7fe640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172] conn(0x7f1e580c03e0 0x7f1e580c28d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:40.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.481+0000 7f1e6d7fa640 1 -- 172.21.15.112:0/3929685331 <== osd.7 v2:172.21.15.187:6816/4003842172 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e580c2fa0 con 0x7f1e580c03e0 2024-09-21T03:27:40.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.494+0000 7f1e76299640 1 -- 172.21.15.112:0/3929685331 --> [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e70063fd0 con 0x7f1e580c03e0 2024-09-21T03:27:40.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.494+0000 7f1e6d7fa640 1 -- 172.21.15.112:0/3929685331 <== osd.7 v2:172.21.15.187:6816/4003842172 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1e70063fd0 con 0x7f1e580c03e0 2024-09-21T03:27:40.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.495+0000 7f1e76299640 1 -- 172.21.15.112:0/3929685331 >> [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172] conn(0x7f1e580c03e0 msgr2=0x7f1e580c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.495+0000 7f1e76299640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172] conn(0x7f1e580c03e0 0x7f1e580c28d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.495+0000 7f1e76299640 1 -- 172.21.15.112:0/3929685331 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1e580b6670 msgr2=0x7f1e580b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.495+0000 7f1e76299640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1e580b6670 0x7f1e580b8b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e60005f70 tx=0x7f1e60000bc0 comp rx=0 tx=0).stop 2024-09-21T03:27:40.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.495+0000 7f1e76299640 1 -- 172.21.15.112:0/3929685331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e70109df0 msgr2=0x7f1e7019fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:40.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.495+0000 7f1e76299640 1 --2- 172.21.15.112:0/3929685331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e70109df0 0x7f1e7019fe80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b970 tx=0x7f1e6800bd70 comp rx=0 tx=0).stop 2024-09-21T03:27:40.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.495+0000 7f1e74836640 1 -- 172.21.15.112:0/3929685331 reap_dead start 2024-09-21T03:27:40.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.495+0000 7f1e76299640 1 -- 172.21.15.112:0/3929685331 shutdown_connections 2024-09-21T03:27:40.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.495+0000 7f1e76299640 1 -- 172.21.15.112:0/3929685331 >> 172.21.15.112:0/3929685331 conn(0x7f1e7007b850 msgr2=0x7f1e701076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:40.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.495+0000 7f1e76299640 1 -- 172.21.15.112:0/3929685331 shutdown_connections 2024-09-21T03:27:40.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:40.495+0000 7f1e76299640 1 -- 172.21.15.112:0/3929685331 wait complete. 2024-09-21T03:27:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.254+0000 7fcad1497640 1 -- 172.21.15.112:0/380467664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcacc1083a0 msgr2=0x7fcacc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.254+0000 7fcad1497640 1 --2- 172.21.15.112:0/380467664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcacc1083a0 0x7fcacc1087b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009900 tx=0x7fcac002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.254+0000 7fcad1497640 1 -- 172.21.15.112:0/380467664 shutdown_connections 2024-09-21T03:27:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.254+0000 7fcad1497640 1 --2- 172.21.15.112:0/380467664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcacc108cf0 0x7fcacc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.254+0000 7fcad1497640 1 --2- 172.21.15.112:0/380467664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcacc1083a0 0x7fcacc1087b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.254+0000 7fcad1497640 1 --2- 172.21.15.112:0/380467664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcacc102a60 0x7fcacc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.254+0000 7fcad1497640 1 -- 172.21.15.112:0/380467664 >> 172.21.15.112:0/380467664 conn(0x7fcacc079730 msgr2=0x7fcacc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.254+0000 7fcad1497640 1 -- 172.21.15.112:0/380467664 shutdown_connections 2024-09-21T03:27:41.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.254+0000 7fcad1497640 1 -- 172.21.15.112:0/380467664 wait complete. 2024-09-21T03:27:41.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.255+0000 7fcad1497640 1 Processor -- start 2024-09-21T03:27:41.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.255+0000 7fcad1497640 1 -- start start 2024-09-21T03:27:41.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.255+0000 7fcad1497640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcacc102a60 0x7fcacc19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.255+0000 7fcad1497640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcacc1083a0 0x7fcacc19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.255+0000 7fcad1497640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcacc108cf0 0x7fcacc19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.255+0000 7fcad1497640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc112960 con 0x7fcacc102a60 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.255+0000 7fcad1497640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc1127e0 con 0x7fcacc108cf0 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.255+0000 7fcad1497640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc112ae0 con 0x7fcacc1083a0 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.255+0000 7fcacaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcacc102a60 0x7fcacc19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcacaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcacc102a60 0x7fcacc19b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42918/0 (socket says 172.21.15.112:42918) 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcacaffd640 1 -- 172.21.15.112:0/136333085 learned_addr learned my addr 172.21.15.112:0/136333085 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcaca7fc640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcacc1083a0 0x7fcacc19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcacb7fe640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcacc108cf0 0x7fcacc19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcacaffd640 1 -- 172.21.15.112:0/136333085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcacc1083a0 msgr2=0x7fcacc19b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcacaffd640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcacc1083a0 0x7fcacc19b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcacaffd640 1 -- 172.21.15.112:0/136333085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcacc108cf0 msgr2=0x7fcacc19bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcacaffd640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcacc108cf0 0x7fcacc19bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcacaffd640 1 -- 172.21.15.112:0/136333085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc1a6550 con 0x7fcacc102a60 2024-09-21T03:27:41.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcacb7fe640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcacc108cf0 0x7fcacc19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:41.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcacaffd640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcacc102a60 0x7fcacc19b290 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00d770 tx=0x7fcabc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcab3fff640 1 -- 172.21.15.112:0/136333085 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc017070 con 0x7fcacc102a60 2024-09-21T03:27:41.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcad1497640 1 -- 172.21.15.112:0/136333085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcacc1a6840 con 0x7fcacc102a60 2024-09-21T03:27:41.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcab3fff640 1 -- 172.21.15.112:0/136333085 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcabc0054d0 con 0x7fcacc102a60 2024-09-21T03:27:41.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcab3fff640 1 -- 172.21.15.112:0/136333085 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc004c30 con 0x7fcacc102a60 2024-09-21T03:27:41.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.256+0000 7fcad1497640 1 -- 172.21.15.112:0/136333085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcacc1a6d50 con 0x7fcacc102a60 2024-09-21T03:27:41.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.257+0000 7fcad1497640 1 -- 172.21.15.112:0/136333085 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcacc063b10 con 0x7fcacc102a60 2024-09-21T03:27:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.259+0000 7fcab3fff640 1 -- 172.21.15.112:0/136333085 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fcabc004dd0 con 0x7fcacc102a60 2024-09-21T03:27:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.260+0000 7fcab3fff640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcab40b65f0 0x7fcab40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.260+0000 7fcaca7fc640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcab40b65f0 0x7fcab40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.260+0000 7fcab3fff640 1 -- 172.21.15.112:0/136333085 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fcabc0e74d0 con 0x7fcacc102a60 2024-09-21T03:27:41.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.260+0000 7fcab3fff640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741] conn(0x7fcab40c0360 0x7fcab40c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.260+0000 7fcab3fff640 1 -- 172.21.15.112:0/136333085 --> [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcabc021e50 con 0x7fcab40c0360 2024-09-21T03:27:41.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.260+0000 7fcab3fff640 1 -- 172.21.15.112:0/136333085 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fcabc0e7920 con 0x7fcacc102a60 2024-09-21T03:27:41.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.260+0000 7fcaca7fc640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcab40b65f0 0x7fcab40b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcac0000c00 tx=0x7fcac0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.260+0000 7fcacb7fe640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741] conn(0x7fcab40c0360 0x7fcab40c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.261+0000 7fcacb7fe640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741] conn(0x7fcab40c0360 0x7fcab40c2850 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.262+0000 7fcab3fff640 1 -- 172.21.15.112:0/136333085 <== osd.5 v2:172.21.15.203:6808/3996799741 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcabc021e50 con 0x7fcab40c0360 2024-09-21T03:27:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.275+0000 7fcad1497640 1 -- 172.21.15.112:0/136333085 --> [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcacc063f90 con 0x7fcab40c0360 2024-09-21T03:27:41.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcab3fff640 1 -- 172.21.15.112:0/136333085 <== osd.5 v2:172.21.15.203:6808/3996799741 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcacc063f90 con 0x7fcab40c0360 2024-09-21T03:27:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcad1497640 1 -- 172.21.15.112:0/136333085 >> [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741] conn(0x7fcab40c0360 msgr2=0x7fcab40c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcad1497640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741] conn(0x7fcab40c0360 0x7fcab40c2850 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcad1497640 1 -- 172.21.15.112:0/136333085 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcab40b65f0 msgr2=0x7fcab40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcad1497640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcab40b65f0 0x7fcab40b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcac0000c00 tx=0x7fcac0031400 comp rx=0 tx=0).stop 2024-09-21T03:27:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcad1497640 1 -- 172.21.15.112:0/136333085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcacc102a60 msgr2=0x7fcacc19b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcad1497640 1 --2- 172.21.15.112:0/136333085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcacc102a60 0x7fcacc19b290 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00d770 tx=0x7fcabc00dc30 comp rx=0 tx=0).stop 2024-09-21T03:27:41.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcacb7fe640 1 -- 172.21.15.112:0/136333085 reap_dead start 2024-09-21T03:27:41.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcad1497640 1 -- 172.21.15.112:0/136333085 shutdown_connections 2024-09-21T03:27:41.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcad1497640 1 -- 172.21.15.112:0/136333085 >> 172.21.15.112:0/136333085 conn(0x7fcacc079730 msgr2=0x7fcacc104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcad1497640 1 -- 172.21.15.112:0/136333085 shutdown_connections 2024-09-21T03:27:41.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.276+0000 7fcad1497640 1 -- 172.21.15.112:0/136333085 wait complete. 2024-09-21T03:27:41.467 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:41 smithi112 ceph-mon[31442]: pgmap v278: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 6 op/s; 91 B/s, 1 objects/s recovering 2024-09-21T03:27:41.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.556+0000 7f9c18846640 1 -- 172.21.15.112:0/3112799871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c141052f0 msgr2=0x7f9c1410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.556+0000 7f9c18846640 1 --2- 172.21.15.112:0/3112799871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c141052f0 0x7f9c1410d930 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009900 tx=0x7f9bfc02a410 comp rx=0 tx=0).stop 2024-09-21T03:27:41.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.556+0000 7f9c18846640 1 -- 172.21.15.112:0/3112799871 shutdown_connections 2024-09-21T03:27:41.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.556+0000 7f9c18846640 1 --2- 172.21.15.112:0/3112799871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c1410de70 0x7f9c14110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.556+0000 7f9c18846640 1 --2- 172.21.15.112:0/3112799871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c141052f0 0x7f9c1410d930 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.556+0000 7f9c18846640 1 --2- 172.21.15.112:0/3112799871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c141048c0 0x7f9c14104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.556+0000 7f9c18846640 1 -- 172.21.15.112:0/3112799871 >> 172.21.15.112:0/3112799871 conn(0x7f9c14079740 msgr2=0x7f9c14079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.556+0000 7f9c18846640 1 -- 172.21.15.112:0/3112799871 shutdown_connections 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.556+0000 7f9c18846640 1 -- 172.21.15.112:0/3112799871 wait complete. 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c18846640 1 Processor -- start 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c18846640 1 -- start start 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c18846640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c141048c0 0x7f9c1419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c18846640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c141052f0 0x7f9c1419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c18846640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1410de70 0x7f9c1419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c18846640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c14113a80 con 0x7f9c1410de70 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c18846640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9c14113900 con 0x7f9c141048c0 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c18846640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9c14113c00 con 0x7f9c141052f0 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c12d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1410de70 0x7f9c1419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c12d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1410de70 0x7f9c1419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42938/0 (socket says 172.21.15.112:42938) 2024-09-21T03:27:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c12d9d640 1 -- 172.21.15.112:0/405438269 learned_addr learned my addr 172.21.15.112:0/405438269 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.557+0000 7f9c1259c640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c141048c0 0x7f9c1419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c12d9d640 1 -- 172.21.15.112:0/405438269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c141052f0 msgr2=0x7f9c1419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c0bfff640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c141052f0 0x7f9c1419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c12d9d640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c141052f0 0x7f9c1419ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c12d9d640 1 -- 172.21.15.112:0/405438269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c141048c0 msgr2=0x7f9c1419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c12d9d640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c141048c0 0x7f9c1419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c12d9d640 1 -- 172.21.15.112:0/405438269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c141a0420 con 0x7f9c1410de70 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c1259c640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c141048c0 0x7f9c1419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c12d9d640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1410de70 0x7f9c1419fdb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00cce0 tx=0x7f9c0c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c0b7fe640 1 -- 172.21.15.112:0/405438269 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00f450 con 0x7f9c1410de70 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c18846640 1 -- 172.21.15.112:0/405438269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c141a0710 con 0x7f9c1410de70 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c0b7fe640 1 -- 172.21.15.112:0/405438269 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0c00faa0 con 0x7f9c1410de70 2024-09-21T03:27:41.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c0b7fe640 1 -- 172.21.15.112:0/405438269 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c018b70 con 0x7f9c1410de70 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.558+0000 7f9c18846640 1 -- 172.21.15.112:0/405438269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c141a6f30 con 0x7f9c1410de70 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.559+0000 7f9c18846640 1 -- 172.21.15.112:0/405438269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c14063b50 con 0x7f9c1410de70 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.560+0000 7f9c0b7fe640 1 -- 172.21.15.112:0/405438269 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f9c0c018d10 con 0x7f9c1410de70 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.561+0000 7f9c0b7fe640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9c000b6620 0x7f9c000b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.561+0000 7f9c1259c640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9c000b6620 0x7f9c000b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.561+0000 7f9c0b7fe640 1 -- 172.21.15.112:0/405438269 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f9c0c0e7cc0 con 0x7f9c1410de70 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.561+0000 7f9c0b7fe640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217] conn(0x7f9c000c0390 0x7f9c000c2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.561+0000 7f9c0b7fe640 1 -- 172.21.15.112:0/405438269 --> [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c0c012ec0 con 0x7f9c000c0390 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.561+0000 7f9c0b7fe640 1 -- 172.21.15.112:0/405438269 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f9c0c0e8110 con 0x7f9c1410de70 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.562+0000 7f9c1259c640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9c000b6620 0x7f9c000b8b30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9c040058c0 tx=0x7f9c04005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.562+0000 7f9c0bfff640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217] conn(0x7f9c000c0390 0x7f9c000c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.562+0000 7f9c0bfff640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217] conn(0x7f9c000c0390 0x7f9c000c2880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.563+0000 7f9c0b7fe640 1 -- 172.21.15.112:0/405438269 <== osd.2 v2:172.21.15.203:6800/388961217 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c0c012ec0 con 0x7f9c000c0390 2024-09-21T03:27:41.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.576+0000 7f9c18846640 1 -- 172.21.15.112:0/405438269 --> [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c14063f50 con 0x7f9c000c0390 2024-09-21T03:27:41.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.577+0000 7f9c0b7fe640 1 -- 172.21.15.112:0/405438269 <== osd.2 v2:172.21.15.203:6800/388961217 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9c14063f50 con 0x7f9c000c0390 2024-09-21T03:27:41.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.577+0000 7f9c18846640 1 -- 172.21.15.112:0/405438269 >> [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217] conn(0x7f9c000c0390 msgr2=0x7f9c000c2880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.577+0000 7f9c18846640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217] conn(0x7f9c000c0390 0x7f9c000c2880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.577+0000 7f9c18846640 1 -- 172.21.15.112:0/405438269 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9c000b6620 msgr2=0x7f9c000b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.577+0000 7f9c18846640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9c000b6620 0x7f9c000b8b30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9c040058c0 tx=0x7f9c04005d50 comp rx=0 tx=0).stop 2024-09-21T03:27:41.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.577+0000 7f9c18846640 1 -- 172.21.15.112:0/405438269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1410de70 msgr2=0x7f9c1419fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.577+0000 7f9c18846640 1 --2- 172.21.15.112:0/405438269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1410de70 0x7f9c1419fdb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00cce0 tx=0x7f9c0c00a620 comp rx=0 tx=0).stop 2024-09-21T03:27:41.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.577+0000 7f9c12d9d640 1 -- 172.21.15.112:0/405438269 reap_dead start 2024-09-21T03:27:41.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.577+0000 7f9c18846640 1 -- 172.21.15.112:0/405438269 shutdown_connections 2024-09-21T03:27:41.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.577+0000 7f9c18846640 1 -- 172.21.15.112:0/405438269 >> 172.21.15.112:0/405438269 conn(0x7f9c14079740 msgr2=0x7f9c14100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.578+0000 7f9c18846640 1 -- 172.21.15.112:0/405438269 shutdown_connections 2024-09-21T03:27:41.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.578+0000 7f9c18846640 1 -- 172.21.15.112:0/405438269 wait complete. 2024-09-21T03:27:41.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:41 smithi203 ceph-mon[34723]: pgmap v278: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 6 op/s; 91 B/s, 1 objects/s recovering 2024-09-21T03:27:41.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:41 smithi187 ceph-mon[34603]: pgmap v278: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 6 op/s; 91 B/s, 1 objects/s recovering 2024-09-21T03:27:41.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.776+0000 7f433de9a640 1 -- 172.21.15.112:0/3377327991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4338108cf0 msgr2=0x7f433810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.776+0000 7f433de9a640 1 --2- 172.21.15.112:0/3377327991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4338108cf0 0x7f433810f520 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4330009ca0 tx=0x7f433002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.776+0000 7f433de9a640 1 -- 172.21.15.112:0/3377327991 shutdown_connections 2024-09-21T03:27:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.776+0000 7f433de9a640 1 --2- 172.21.15.112:0/3377327991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4338108cf0 0x7f433810f520 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.777+0000 7f433de9a640 1 --2- 172.21.15.112:0/3377327991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43381083a0 0x7f43381087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.777+0000 7f433de9a640 1 --2- 172.21.15.112:0/3377327991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4338102a60 0x7f4338102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.777+0000 7f433de9a640 1 -- 172.21.15.112:0/3377327991 >> 172.21.15.112:0/3377327991 conn(0x7f4338079730 msgr2=0x7f4338079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.777+0000 7f433de9a640 1 -- 172.21.15.112:0/3377327991 shutdown_connections 2024-09-21T03:27:41.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.777+0000 7f433de9a640 1 -- 172.21.15.112:0/3377327991 wait complete. 2024-09-21T03:27:41.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.777+0000 7f433de9a640 1 Processor -- start 2024-09-21T03:27:41.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.777+0000 7f433de9a640 1 -- start start 2024-09-21T03:27:41.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f433de9a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4338102a60 0x7f433819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f433de9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43381083a0 0x7f433819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f433de9a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4338108cf0 0x7f433819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f433de9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4338113a80 con 0x7f43381083a0 2024-09-21T03:27:41.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f433de9a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4338113900 con 0x7f4338108cf0 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f433de9a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4338113c00 con 0x7f4338102a60 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f4336ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43381083a0 0x7f433819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f4336ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43381083a0 0x7f433819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42944/0 (socket says 172.21.15.112:42944) 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f4336ffd640 1 -- 172.21.15.112:0/615573261 learned_addr learned my addr 172.21.15.112:0/615573261 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f43377fe640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4338102a60 0x7f433819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f4337fff640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4338108cf0 0x7f433819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f4336ffd640 1 -- 172.21.15.112:0/615573261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4338102a60 msgr2=0x7f433819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f4336ffd640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4338102a60 0x7f433819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f4336ffd640 1 -- 172.21.15.112:0/615573261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4338108cf0 msgr2=0x7f433819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f4336ffd640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4338108cf0 0x7f433819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f4336ffd640 1 -- 172.21.15.112:0/615573261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43381a03c0 con 0x7f43381083a0 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.778+0000 7f4336ffd640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43381083a0 0x7f433819a9f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f432000c970 tx=0x7f432000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.779+0000 7f4334ff9640 1 -- 172.21.15.112:0/615573261 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320007bb0 con 0x7f43381083a0 2024-09-21T03:27:41.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.779+0000 7f4334ff9640 1 -- 172.21.15.112:0/615573261 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4320007d50 con 0x7f43381083a0 2024-09-21T03:27:41.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.779+0000 7f433de9a640 1 -- 172.21.15.112:0/615573261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43381a06b0 con 0x7f43381083a0 2024-09-21T03:27:41.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.779+0000 7f4334ff9640 1 -- 172.21.15.112:0/615573261 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432000f450 con 0x7f43381083a0 2024-09-21T03:27:41.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.779+0000 7f433de9a640 1 -- 172.21.15.112:0/615573261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433810fe90 con 0x7f43381083a0 2024-09-21T03:27:41.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.780+0000 7f433de9a640 1 -- 172.21.15.112:0/615573261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4338063b10 con 0x7f43381083a0 2024-09-21T03:27:41.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.782+0000 7f4334ff9640 1 -- 172.21.15.112:0/615573261 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f4320004f20 con 0x7f43381083a0 2024-09-21T03:27:41.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.782+0000 7f4334ff9640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f43240b65f0 0x7f43240b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.782+0000 7f43377fe640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f43240b65f0 0x7f43240b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.782+0000 7f4334ff9640 1 -- 172.21.15.112:0/615573261 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f43200e7780 con 0x7f43381083a0 2024-09-21T03:27:41.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.782+0000 7f4334ff9640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548] conn(0x7f43240c0360 0x7f43240c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.783+0000 7f4334ff9640 1 -- 172.21.15.112:0/615573261 --> [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4320012ec0 con 0x7f43240c0360 2024-09-21T03:27:41.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.783+0000 7f4334ff9640 1 -- 172.21.15.112:0/615573261 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f43200e7bd0 con 0x7f43381083a0 2024-09-21T03:27:41.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.783+0000 7f43377fe640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f43240b65f0 0x7f43240b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f43280058c0 tx=0x7f432800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.783+0000 7f4337fff640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548] conn(0x7f43240c0360 0x7f43240c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.783+0000 7f4337fff640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548] conn(0x7f43240c0360 0x7f43240c2850 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.784+0000 7f4334ff9640 1 -- 172.21.15.112:0/615573261 <== osd.1 v2:172.21.15.187:6800/279076548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4320012ec0 con 0x7f43240c0360 2024-09-21T03:27:41.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.798+0000 7f433de9a640 1 -- 172.21.15.112:0/615573261 --> [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4338063f10 con 0x7f43240c0360 2024-09-21T03:27:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.798+0000 7f4334ff9640 1 -- 172.21.15.112:0/615573261 <== osd.1 v2:172.21.15.187:6800/279076548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4338063f10 con 0x7f43240c0360 2024-09-21T03:27:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.799+0000 7f433de9a640 1 -- 172.21.15.112:0/615573261 >> [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548] conn(0x7f43240c0360 msgr2=0x7f43240c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.799+0000 7f433de9a640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548] conn(0x7f43240c0360 0x7f43240c2850 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.799+0000 7f433de9a640 1 -- 172.21.15.112:0/615573261 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f43240b65f0 msgr2=0x7f43240b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.799+0000 7f433de9a640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f43240b65f0 0x7f43240b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f43280058c0 tx=0x7f432800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.799+0000 7f433de9a640 1 -- 172.21.15.112:0/615573261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43381083a0 msgr2=0x7f433819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.799+0000 7f433de9a640 1 --2- 172.21.15.112:0/615573261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43381083a0 0x7f433819a9f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f432000c970 tx=0x7f432000ce30 comp rx=0 tx=0).stop 2024-09-21T03:27:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.799+0000 7f4337fff640 1 -- 172.21.15.112:0/615573261 reap_dead start 2024-09-21T03:27:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.799+0000 7f433de9a640 1 -- 172.21.15.112:0/615573261 shutdown_connections 2024-09-21T03:27:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.799+0000 7f433de9a640 1 -- 172.21.15.112:0/615573261 >> 172.21.15.112:0/615573261 conn(0x7f4338079730 msgr2=0x7f4338104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.799+0000 7f433de9a640 1 -- 172.21.15.112:0/615573261 shutdown_connections 2024-09-21T03:27:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.799+0000 7f433de9a640 1 -- 172.21.15.112:0/615573261 wait complete. 2024-09-21T03:27:41.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.983+0000 7fc02db8b640 1 -- 172.21.15.112:0/2089821141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc028101f70 msgr2=0x7fc028102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.983+0000 7fc02db8b640 1 --2- 172.21.15.112:0/2089821141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc028101f70 0x7fc028102380 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc018009900 tx=0x7fc01802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:41.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.983+0000 7fc02db8b640 1 -- 172.21.15.112:0/2089821141 shutdown_connections 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.983+0000 7fc02db8b640 1 --2- 172.21.15.112:0/2089821141 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc028109e00 0x7fc028110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.983+0000 7fc02db8b640 1 --2- 172.21.15.112:0/2089821141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0281094b0 0x7fc0281098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.983+0000 7fc02db8b640 1 --2- 172.21.15.112:0/2089821141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc028101f70 0x7fc028102380 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.983+0000 7fc02db8b640 1 -- 172.21.15.112:0/2089821141 >> 172.21.15.112:0/2089821141 conn(0x7fc028077c60 msgr2=0x7fc0280780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.983+0000 7fc02db8b640 1 -- 172.21.15.112:0/2089821141 shutdown_connections 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.983+0000 7fc02db8b640 1 -- 172.21.15.112:0/2089821141 wait complete. 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc02db8b640 1 Processor -- start 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc02db8b640 1 -- start start 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc02db8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc028101f70 0x7fc02819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc02db8b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0281094b0 0x7fc02819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc02db8b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc028109e00 0x7fc0281a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc02db8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc028113ac0 con 0x7fc028101f70 2024-09-21T03:27:41.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc02db8b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc028113940 con 0x7fc0281094b0 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc02db8b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc028113c40 con 0x7fc028109e00 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc0277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc028101f70 0x7fc02819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc0277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc028101f70 0x7fc02819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42964/0 (socket says 172.21.15.112:42964) 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc0277fe640 1 -- 172.21.15.112:0/41325629 learned_addr learned my addr 172.21.15.112:0/41325629 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc027fff640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc028109e00 0x7fc0281a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc01ffff640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0281094b0 0x7fc02819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc0277fe640 1 -- 172.21.15.112:0/41325629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc028109e00 msgr2=0x7fc0281a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc0277fe640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc028109e00 0x7fc0281a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc0277fe640 1 -- 172.21.15.112:0/41325629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0281094b0 msgr2=0x7fc02819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc0277fe640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0281094b0 0x7fc02819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc0277fe640 1 -- 172.21.15.112:0/41325629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0281a2600 con 0x7fc028101f70 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc027fff640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc028109e00 0x7fc0281a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:41.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.984+0000 7fc01ffff640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0281094b0 0x7fc02819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.985+0000 7fc0277fe640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc028101f70 0x7fc02819c690 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc018005940 tx=0x7fc0180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.985+0000 7fc0257fa640 1 -- 172.21.15.112:0/41325629 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01803f030 con 0x7fc028101f70 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.985+0000 7fc0257fa640 1 -- 172.21.15.112:0/41325629 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc018003cb0 con 0x7fc028101f70 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.985+0000 7fc02db8b640 1 -- 172.21.15.112:0/41325629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0281a2890 con 0x7fc028101f70 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.985+0000 7fc0257fa640 1 -- 172.21.15.112:0/41325629 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018035070 con 0x7fc028101f70 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.985+0000 7fc02db8b640 1 -- 172.21.15.112:0/41325629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0281a9050 con 0x7fc028101f70 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.986+0000 7fc02db8b640 1 -- 172.21.15.112:0/41325629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc028107d20 con 0x7fc028101f70 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.987+0000 7fc0257fa640 1 -- 172.21.15.112:0/41325629 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fc018042030 con 0x7fc028101f70 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.988+0000 7fc0257fa640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc0140b65f0 0x7fc0140b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.988+0000 7fc01ffff640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc0140b65f0 0x7fc0140b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.988+0000 7fc0257fa640 1 -- 172.21.15.112:0/41325629 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc018108270 con 0x7fc028101f70 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.988+0000 7fc0257fa640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939] conn(0x7fc0140c0360 0x7fc0140c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:41.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.988+0000 7fc0257fa640 1 -- 172.21.15.112:0/41325629 --> [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc018030e80 con 0x7fc0140c0360 2024-09-21T03:27:41.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.988+0000 7fc0257fa640 1 -- 172.21.15.112:0/41325629 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc0181086c0 con 0x7fc028101f70 2024-09-21T03:27:41.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.989+0000 7fc01ffff640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc0140b65f0 0x7fc0140b8b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc028108d40 tx=0x7fc010005e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.989+0000 7fc027fff640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939] conn(0x7fc0140c0360 0x7fc0140c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:41.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.989+0000 7fc027fff640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939] conn(0x7fc0140c0360 0x7fc0140c2850 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:41.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:41.990+0000 7fc0257fa640 1 -- 172.21.15.112:0/41325629 <== osd.4 v2:172.21.15.187:6808/3494239939 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc018030e80 con 0x7fc0140c0360 2024-09-21T03:27:42.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.003+0000 7fc02db8b640 1 -- 172.21.15.112:0/41325629 --> [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc02806ac50 con 0x7fc0140c0360 2024-09-21T03:27:42.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.004+0000 7fc0257fa640 1 -- 172.21.15.112:0/41325629 <== osd.4 v2:172.21.15.187:6808/3494239939 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc02806ac50 con 0x7fc0140c0360 2024-09-21T03:27:42.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.004+0000 7fc02db8b640 1 -- 172.21.15.112:0/41325629 >> [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939] conn(0x7fc0140c0360 msgr2=0x7fc0140c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:42.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.004+0000 7fc02db8b640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939] conn(0x7fc0140c0360 0x7fc0140c2850 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:42.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.005+0000 7fc02db8b640 1 -- 172.21.15.112:0/41325629 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc0140b65f0 msgr2=0x7fc0140b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:42.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.005+0000 7fc02db8b640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc0140b65f0 0x7fc0140b8b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc028108d40 tx=0x7fc010005e50 comp rx=0 tx=0).stop 2024-09-21T03:27:42.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.005+0000 7fc02db8b640 1 -- 172.21.15.112:0/41325629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc028101f70 msgr2=0x7fc02819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:42.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.005+0000 7fc02db8b640 1 --2- 172.21.15.112:0/41325629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc028101f70 0x7fc02819c690 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc018005940 tx=0x7fc0180042f0 comp rx=0 tx=0).stop 2024-09-21T03:27:42.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.005+0000 7fc027fff640 1 -- 172.21.15.112:0/41325629 reap_dead start 2024-09-21T03:27:42.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.005+0000 7fc02db8b640 1 -- 172.21.15.112:0/41325629 shutdown_connections 2024-09-21T03:27:42.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.005+0000 7fc02db8b640 1 -- 172.21.15.112:0/41325629 >> 172.21.15.112:0/41325629 conn(0x7fc028077c60 msgr2=0x7fc0281072c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:42.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.005+0000 7fc02db8b640 1 -- 172.21.15.112:0/41325629 shutdown_connections 2024-09-21T03:27:42.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:42.005+0000 7fc02db8b640 1 -- 172.21.15.112:0/41325629 wait complete. 2024-09-21T03:27:43.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:43 smithi112 ceph-mon[31442]: pgmap v279: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 825 B/s wr, 6 op/s; 73 B/s, 1 objects/s recovering 2024-09-21T03:27:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:43 smithi203 ceph-mon[34723]: pgmap v279: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 825 B/s wr, 6 op/s; 73 B/s, 1 objects/s recovering 2024-09-21T03:27:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:43 smithi187 ceph-mon[34603]: pgmap v279: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 825 B/s wr, 6 op/s; 73 B/s, 1 objects/s recovering 2024-09-21T03:27:43.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.903+0000 7fef581b9640 1 -- 172.21.15.112:0/3631426872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef5010de30 msgr2=0x7fef50110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.903+0000 7fef581b9640 1 --2- 172.21.15.112:0/3631426872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef5010de30 0x7fef50110320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fef48009ca0 tx=0x7fef4802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.903+0000 7fef581b9640 1 -- 172.21.15.112:0/3631426872 shutdown_connections 2024-09-21T03:27:43.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.903+0000 7fef581b9640 1 --2- 172.21.15.112:0/3631426872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef5010de30 0x7fef50110320 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.903+0000 7fef581b9640 1 --2- 172.21.15.112:0/3631426872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef50105320 0x7fef5010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.903+0000 7fef581b9640 1 --2- 172.21.15.112:0/3631426872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef501049d0 0x7fef50104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.903+0000 7fef581b9640 1 -- 172.21.15.112:0/3631426872 >> 172.21.15.112:0/3631426872 conn(0x7fef50079740 msgr2=0x7fef50079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:43.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.903+0000 7fef581b9640 1 -- 172.21.15.112:0/3631426872 shutdown_connections 2024-09-21T03:27:43.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.903+0000 7fef581b9640 1 -- 172.21.15.112:0/3631426872 wait complete. 2024-09-21T03:27:43.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.903+0000 7fef581b9640 1 Processor -- start 2024-09-21T03:27:43.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.903+0000 7fef581b9640 1 -- start start 2024-09-21T03:27:43.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef581b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501049d0 0x7fef5019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef581b9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef50105320 0x7fef5019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef581b9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef5010de30 0x7fef5019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef581b9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef50113ab0 con 0x7fef501049d0 2024-09-21T03:27:43.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef581b9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fef50113930 con 0x7fef5010de30 2024-09-21T03:27:43.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef581b9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef50113c30 con 0x7fef50105320 2024-09-21T03:27:43.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef55f55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501049d0 0x7fef5019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef55f55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501049d0 0x7fef5019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42982/0 (socket says 172.21.15.112:42982) 2024-09-21T03:27:43.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef55f55640 1 -- 172.21.15.112:0/716407301 learned_addr learned my addr 172.21.15.112:0/716407301 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:43.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef56756640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef5010de30 0x7fef5019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef55754640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef50105320 0x7fef5019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef55f55640 1 -- 172.21.15.112:0/716407301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef50105320 msgr2=0x7fef5019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef55f55640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef50105320 0x7fef5019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef55f55640 1 -- 172.21.15.112:0/716407301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef5010de30 msgr2=0x7fef5019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef55f55640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef5010de30 0x7fef5019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef55f55640 1 -- 172.21.15.112:0/716407301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef501a0410 con 0x7fef501049d0 2024-09-21T03:27:43.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef55754640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef50105320 0x7fef5019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:43.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef55f55640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501049d0 0x7fef5019a4a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00c970 tx=0x7fef4c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:43.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.904+0000 7fef46ffd640 1 -- 172.21.15.112:0/716407301 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c004220 con 0x7fef501049d0 2024-09-21T03:27:43.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.905+0000 7fef581b9640 1 -- 172.21.15.112:0/716407301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef501a0700 con 0x7fef501049d0 2024-09-21T03:27:43.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.905+0000 7fef46ffd640 1 -- 172.21.15.112:0/716407301 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef4c0043c0 con 0x7fef501049d0 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.905+0000 7fef46ffd640 1 -- 172.21.15.112:0/716407301 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c004a00 con 0x7fef501049d0 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.905+0000 7fef581b9640 1 -- 172.21.15.112:0/716407301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef501a6e60 con 0x7fef501049d0 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.905+0000 7fef581b9640 1 -- 172.21.15.112:0/716407301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef50100fe0 con 0x7fef501049d0 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.907+0000 7fef46ffd640 1 -- 172.21.15.112:0/716407301 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fef4c0079b0 con 0x7fef501049d0 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.908+0000 7fef46ffd640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fef3c0b65f0 0x7fef3c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.908+0000 7fef55754640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fef3c0b65f0 0x7fef3c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.908+0000 7fef46ffd640 1 -- 172.21.15.112:0/716407301 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fef4c0e7770 con 0x7fef501049d0 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.908+0000 7fef46ffd640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356] conn(0x7fef3c0c0360 0x7fef3c0c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.908+0000 7fef46ffd640 1 -- 172.21.15.112:0/716407301 --> [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef3c0c2f20 con 0x7fef3c0c0360 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.908+0000 7fef46ffd640 1 -- 172.21.15.112:0/716407301 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fef4c0e7bc0 con 0x7fef501049d0 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.908+0000 7fef56756640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356] conn(0x7fef3c0c0360 0x7fef3c0c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.908+0000 7fef55754640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fef3c0b65f0 0x7fef3c0b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fef400058c0 tx=0x7fef40005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.909+0000 7fef56756640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356] conn(0x7fef3c0c0360 0x7fef3c0c2850 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:43.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.909+0000 7fef46ffd640 1 -- 172.21.15.112:0/716407301 <== osd.6 v2:172.21.15.112:6818/92569356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef3c0c2f20 con 0x7fef3c0c0360 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef581b9640 1 -- 172.21.15.112:0/716407301 --> [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef5006bf20 con 0x7fef3c0c0360 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef46ffd640 1 -- 172.21.15.112:0/716407301 <== osd.6 v2:172.21.15.112:6818/92569356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fef5006bf20 con 0x7fef3c0c0360 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef581b9640 1 -- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356] conn(0x7fef3c0c0360 msgr2=0x7fef3c0c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef581b9640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356] conn(0x7fef3c0c0360 0x7fef3c0c2850 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef581b9640 1 -- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fef3c0b65f0 msgr2=0x7fef3c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef581b9640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fef3c0b65f0 0x7fef3c0b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fef400058c0 tx=0x7fef40005850 comp rx=0 tx=0).stop 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef581b9640 1 -- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501049d0 msgr2=0x7fef5019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef581b9640 1 --2- 172.21.15.112:0/716407301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef501049d0 0x7fef5019a4a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00c970 tx=0x7fef4c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef56756640 1 -- 172.21.15.112:0/716407301 reap_dead start 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef581b9640 1 -- 172.21.15.112:0/716407301 shutdown_connections 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef581b9640 1 -- 172.21.15.112:0/716407301 >> 172.21.15.112:0/716407301 conn(0x7fef50079740 msgr2=0x7fef500ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef581b9640 1 -- 172.21.15.112:0/716407301 shutdown_connections 2024-09-21T03:27:43.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.923+0000 7fef581b9640 1 -- 172.21.15.112:0/716407301 wait complete. 2024-09-21T03:27:44.965 INFO:teuthology.orchestra.run.smithi112.stdout:176093659181 2024-09-21T03:27:44.966 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T03:27:45.270 INFO:teuthology.orchestra.run.smithi112.stdout:270582939673 2024-09-21T03:27:45.271 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T03:27:45.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.816+0000 7f12108f9640 1 -- 172.21.15.112:0/2594522716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120c107b30 msgr2=0x7f120c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.816+0000 7f12108f9640 1 --2- 172.21.15.112:0/2594522716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120c107b30 0x7f120c107f40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1200009900 tx=0x7f120002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:45.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.816+0000 7f12108f9640 1 -- 172.21.15.112:0/2594522716 shutdown_connections 2024-09-21T03:27:45.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.817+0000 7f12108f9640 1 --2- 172.21.15.112:0/2594522716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f120c108480 0x7f120c10ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.817+0000 7f12108f9640 1 --2- 172.21.15.112:0/2594522716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120c107b30 0x7f120c107f40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.817+0000 7f12108f9640 1 --2- 172.21.15.112:0/2594522716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f120c1021f0 0x7f120c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.817+0000 7f12108f9640 1 -- 172.21.15.112:0/2594522716 >> 172.21.15.112:0/2594522716 conn(0x7f120c077c60 msgr2=0x7f120c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.817+0000 7f12108f9640 1 -- 172.21.15.112:0/2594522716 shutdown_connections 2024-09-21T03:27:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.817+0000 7f12108f9640 1 -- 172.21.15.112:0/2594522716 wait complete. 2024-09-21T03:27:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f12108f9640 1 Processor -- start 2024-09-21T03:27:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f12108f9640 1 -- start start 2024-09-21T03:27:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f12108f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f120c1021f0 0x7f120c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f12108f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120c107b30 0x7f120c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f12108f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f120c108480 0x7f120c19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f12108f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f120c111900 con 0x7f120c107b30 2024-09-21T03:27:45.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f12108f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f120c111780 con 0x7f120c108480 2024-09-21T03:27:45.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f12108f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f120c111a80 con 0x7f120c1021f0 2024-09-21T03:27:45.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f1209d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120c107b30 0x7f120c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:45.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f1209d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120c107b30 0x7f120c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42974/0 (socket says 172.21.15.112:42974) 2024-09-21T03:27:45.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f1209d9b640 1 -- 172.21.15.112:0/974644368 learned_addr learned my addr 172.21.15.112:0/974644368 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:45.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f120a59c640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f120c1021f0 0x7f120c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:45.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.818+0000 7f120ad9d640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f120c108480 0x7f120c19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:45.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.819+0000 7f1209d9b640 1 -- 172.21.15.112:0/974644368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f120c1021f0 msgr2=0x7f120c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.819+0000 7f1209d9b640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f120c1021f0 0x7f120c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.819+0000 7f1209d9b640 1 -- 172.21.15.112:0/974644368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f120c108480 msgr2=0x7f120c19db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.819+0000 7f1209d9b640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f120c108480 0x7f120c19db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.819+0000 7f1209d9b640 1 -- 172.21.15.112:0/974644368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c19e260 con 0x7f120c107b30 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.819+0000 7f1209d9b640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120c107b30 0x7f120c198890 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f120002a930 tx=0x7f1200004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.819+0000 7f11fb7fe640 1 -- 172.21.15.112:0/974644368 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120003f030 con 0x7f120c107b30 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.819+0000 7f12108f9640 1 -- 172.21.15.112:0/974644368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c19e4f0 con 0x7f120c107b30 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.819+0000 7f11fb7fe640 1 -- 172.21.15.112:0/974644368 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1200003aa0 con 0x7f120c107b30 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.819+0000 7f11fb7fe640 1 -- 172.21.15.112:0/974644368 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200035070 con 0x7f120c107b30 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.819+0000 7f12108f9640 1 -- 172.21.15.112:0/974644368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c1a4cb0 con 0x7f120c107b30 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.820+0000 7f12108f9640 1 -- 172.21.15.112:0/974644368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f120c063b10 con 0x7f120c107b30 2024-09-21T03:27:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.822+0000 7f11fb7fe640 1 -- 172.21.15.112:0/974644368 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f1200042030 con 0x7f120c107b30 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.822+0000 7f11fb7fe640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f11f40b65f0 0x7f11f40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.823+0000 7f120a59c640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f11f40b65f0 0x7f11f40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.823+0000 7f11fb7fe640 1 -- 172.21.15.112:0/974644368 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f1200108040 con 0x7f120c107b30 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.823+0000 7f11fb7fe640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460] conn(0x7f11f40c0360 0x7f11f40c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.823+0000 7f11fb7fe640 1 -- 172.21.15.112:0/974644368 --> [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11f40c2f20 con 0x7f11f40c0360 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.823+0000 7f11fb7fe640 1 -- 172.21.15.112:0/974644368 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1200108490 con 0x7f120c107b30 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.823+0000 7f120a59c640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f11f40b65f0 0x7f11f40b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009990 tx=0x7f11fc009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.823+0000 7f120ad9d640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460] conn(0x7f11f40c0360 0x7f11f40c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.824+0000 7f120ad9d640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460] conn(0x7f11f40c0360 0x7f11f40c2850 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.824+0000 7f11fb7fe640 1 -- 172.21.15.112:0/974644368 <== osd.8 v2:172.21.15.203:6816/3883207460 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11f40c2f20 con 0x7f11f40c0360 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.838+0000 7f12108f9640 1 -- 172.21.15.112:0/974644368 --> [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f120c063f10 con 0x7f11f40c0360 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.838+0000 7f11fb7fe640 1 -- 172.21.15.112:0/974644368 <== osd.8 v2:172.21.15.203:6816/3883207460 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f120c063f10 con 0x7f11f40c0360 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.839+0000 7f12108f9640 1 -- 172.21.15.112:0/974644368 >> [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460] conn(0x7f11f40c0360 msgr2=0x7f11f40c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.839+0000 7f12108f9640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460] conn(0x7f11f40c0360 0x7f11f40c2850 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.839+0000 7f12108f9640 1 -- 172.21.15.112:0/974644368 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f11f40b65f0 msgr2=0x7f11f40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.839+0000 7f12108f9640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f11f40b65f0 0x7f11f40b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009990 tx=0x7f11fc009110 comp rx=0 tx=0).stop 2024-09-21T03:27:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.839+0000 7f12108f9640 1 -- 172.21.15.112:0/974644368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120c107b30 msgr2=0x7f120c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.839+0000 7f12108f9640 1 --2- 172.21.15.112:0/974644368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f120c107b30 0x7f120c198890 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f120002a930 tx=0x7f1200004270 comp rx=0 tx=0).stop 2024-09-21T03:27:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.839+0000 7f120ad9d640 1 -- 172.21.15.112:0/974644368 reap_dead start 2024-09-21T03:27:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.839+0000 7f12108f9640 1 -- 172.21.15.112:0/974644368 shutdown_connections 2024-09-21T03:27:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.839+0000 7f12108f9640 1 -- 172.21.15.112:0/974644368 >> 172.21.15.112:0/974644368 conn(0x7f120c077c60 msgr2=0x7f120c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.839+0000 7f12108f9640 1 -- 172.21.15.112:0/974644368 shutdown_connections 2024-09-21T03:27:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:43.839+0000 7f12108f9640 1 -- 172.21.15.112:0/974644368 wait complete. 2024-09-21T03:27:45.465 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:45 smithi112 ceph-mon[31442]: pgmap v280: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 750 B/s wr, 7 op/s; 67 B/s, 1 objects/s recovering 2024-09-21T03:27:45.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:45 smithi203 ceph-mon[34723]: pgmap v280: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 750 B/s wr, 7 op/s; 67 B/s, 1 objects/s recovering 2024-09-21T03:27:45.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:45 smithi187 ceph-mon[34603]: pgmap v280: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 750 B/s wr, 7 op/s; 67 B/s, 1 objects/s recovering 2024-09-21T03:27:45.834 INFO:teuthology.orchestra.run.smithi112.stdout:201863462953 2024-09-21T03:27:45.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T03:27:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.663+0000 7f5b8672d640 1 -- 172.21.15.112:0/3904743270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b801049b0 msgr2=0x7f5b80104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.663+0000 7f5b8672d640 1 --2- 172.21.15.112:0/3904743270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b801049b0 0x7f5b80104dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009900 tx=0x7f5b7002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.663+0000 7f5b8672d640 1 -- 172.21.15.112:0/3904743270 shutdown_connections 2024-09-21T03:27:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.663+0000 7f5b8672d640 1 --2- 172.21.15.112:0/3904743270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8010de10 0x7f5b80110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.663+0000 7f5b8672d640 1 --2- 172.21.15.112:0/3904743270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b80105300 0x7f5b8010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.663+0000 7f5b8672d640 1 --2- 172.21.15.112:0/3904743270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b801049b0 0x7f5b80104dc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.663+0000 7f5b8672d640 1 -- 172.21.15.112:0/3904743270 >> 172.21.15.112:0/3904743270 conn(0x7f5b80079730 msgr2=0x7f5b80079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.663+0000 7f5b8672d640 1 -- 172.21.15.112:0/3904743270 shutdown_connections 2024-09-21T03:27:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.663+0000 7f5b8672d640 1 -- 172.21.15.112:0/3904743270 wait complete. 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.664+0000 7f5b8672d640 1 Processor -- start 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.664+0000 7f5b8672d640 1 -- start start 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b8672d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b801049b0 0x7f5b8019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b8672d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b80105300 0x7f5b8019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b8672d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b8010de10 0x7f5b8019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b8672d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80113a70 con 0x7f5b801049b0 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b8672d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b801138f0 con 0x7f5b80105300 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b8672d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80113bf0 con 0x7f5b8010de10 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b7ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b801049b0 0x7f5b8019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b7ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b801049b0 0x7f5b8019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42992/0 (socket says 172.21.15.112:42992) 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b7ffff640 1 -- 172.21.15.112:0/1354183928 learned_addr learned my addr 172.21.15.112:0/1354183928 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:45.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b84cca640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b8010de10 0x7f5b8019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:45.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b7ffff640 1 -- 172.21.15.112:0/1354183928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b8010de10 msgr2=0x7f5b8019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b7ffff640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b8010de10 0x7f5b8019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b7ffff640 1 -- 172.21.15.112:0/1354183928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b80105300 msgr2=0x7f5b8019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b7ffff640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b80105300 0x7f5b8019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b7ffff640 1 -- 172.21.15.112:0/1354183928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b801a0350 con 0x7f5b801049b0 2024-09-21T03:27:45.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b7ffff640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b801049b0 0x7f5b8019a4a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b70031a90 tx=0x7f5b70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:45.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b7d7fa640 1 -- 172.21.15.112:0/1354183928 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7003f030 con 0x7f5b801049b0 2024-09-21T03:27:45.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.665+0000 7f5b8672d640 1 -- 172.21.15.112:0/1354183928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b801a05e0 con 0x7f5b801049b0 2024-09-21T03:27:45.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.666+0000 7f5b7d7fa640 1 -- 172.21.15.112:0/1354183928 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b70003cb0 con 0x7f5b801049b0 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.666+0000 7f5b8672d640 1 -- 172.21.15.112:0/1354183928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b801a0b10 con 0x7f5b801049b0 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.666+0000 7f5b7d7fa640 1 -- 172.21.15.112:0/1354183928 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70035070 con 0x7f5b801049b0 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.666+0000 7f5b8672d640 1 -- 172.21.15.112:0/1354183928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b80063b10 con 0x7f5b801049b0 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.669+0000 7f5b7d7fa640 1 -- 172.21.15.112:0/1354183928 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f5b70042030 con 0x7f5b801049b0 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.670+0000 7f5b7d7fa640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5b680b66a0 0x7f5b680b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.670+0000 7f5b7f7fe640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5b680b66a0 0x7f5b680b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.670+0000 7f5b7d7fa640 1 -- 172.21.15.112:0/1354183928 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f5b70108010 con 0x7f5b801049b0 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.670+0000 7f5b7d7fa640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655] conn(0x7f5b680c0410 0x7f5b680c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.670+0000 7f5b7d7fa640 1 -- 172.21.15.112:0/1354183928 --> [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b680c2fd0 con 0x7f5b680c0410 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.670+0000 7f5b7d7fa640 1 -- 172.21.15.112:0/1354183928 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f5b70108460 con 0x7f5b801049b0 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.670+0000 7f5b7f7fe640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5b680b66a0 0x7f5b680b8bb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5b80079fb0 tx=0x7f5b74005c70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.670+0000 7f5b84cca640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655] conn(0x7f5b680c0410 0x7f5b680c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.671+0000 7f5b84cca640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655] conn(0x7f5b680c0410 0x7f5b680c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.672+0000 7f5b7d7fa640 1 -- 172.21.15.112:0/1354183928 <== osd.11 v2:172.21.15.203:6824/520562655 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b680c2fd0 con 0x7f5b680c0410 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.685+0000 7f5b8672d640 1 -- 172.21.15.112:0/1354183928 --> [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b80063f10 con 0x7f5b680c0410 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.685+0000 7f5b7d7fa640 1 -- 172.21.15.112:0/1354183928 <== osd.11 v2:172.21.15.203:6824/520562655 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5b80063f10 con 0x7f5b680c0410 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.686+0000 7f5b8672d640 1 -- 172.21.15.112:0/1354183928 >> [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655] conn(0x7f5b680c0410 msgr2=0x7f5b680c2900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.686+0000 7f5b8672d640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655] conn(0x7f5b680c0410 0x7f5b680c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.686+0000 7f5b8672d640 1 -- 172.21.15.112:0/1354183928 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5b680b66a0 msgr2=0x7f5b680b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.686+0000 7f5b8672d640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5b680b66a0 0x7f5b680b8bb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5b80079fb0 tx=0x7f5b74005c70 comp rx=0 tx=0).stop 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.686+0000 7f5b8672d640 1 -- 172.21.15.112:0/1354183928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b801049b0 msgr2=0x7f5b8019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.686+0000 7f5b8672d640 1 --2- 172.21.15.112:0/1354183928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b801049b0 0x7f5b8019a4a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b70031a90 tx=0x7f5b70004270 comp rx=0 tx=0).stop 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.686+0000 7f5b84cca640 1 -- 172.21.15.112:0/1354183928 reap_dead start 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.686+0000 7f5b8672d640 1 -- 172.21.15.112:0/1354183928 shutdown_connections 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.686+0000 7f5b8672d640 1 -- 172.21.15.112:0/1354183928 >> 172.21.15.112:0/1354183928 conn(0x7f5b80079730 msgr2=0x7f5b801007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.686+0000 7f5b8672d640 1 -- 172.21.15.112:0/1354183928 shutdown_connections 2024-09-21T03:27:45.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:44.686+0000 7f5b8672d640 1 -- 172.21.15.112:0/1354183928 wait complete. 2024-09-21T03:27:46.625 INFO:teuthology.orchestra.run.smithi112.stdout:292057776150 2024-09-21T03:27:46.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T03:27:46.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.947+0000 7fa125f0e640 1 -- 172.21.15.112:0/3809870302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa120102a60 msgr2=0x7fa120102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:46.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.947+0000 7fa125f0e640 1 --2- 172.21.15.112:0/3809870302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa120102a60 0x7fa120102e70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa110009900 tx=0x7fa11002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:46.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.947+0000 7fa125f0e640 1 -- 172.21.15.112:0/3809870302 shutdown_connections 2024-09-21T03:27:46.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.947+0000 7fa125f0e640 1 --2- 172.21.15.112:0/3809870302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa120108cf0 0x7fa12010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:46.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.947+0000 7fa125f0e640 1 --2- 172.21.15.112:0/3809870302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1201083a0 0x7fa1201087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:46.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.947+0000 7fa125f0e640 1 --2- 172.21.15.112:0/3809870302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa120102a60 0x7fa120102e70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:46.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.947+0000 7fa125f0e640 1 -- 172.21.15.112:0/3809870302 >> 172.21.15.112:0/3809870302 conn(0x7fa120079730 msgr2=0x7fa120079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:46.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.947+0000 7fa125f0e640 1 -- 172.21.15.112:0/3809870302 shutdown_connections 2024-09-21T03:27:46.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.947+0000 7fa125f0e640 1 -- 172.21.15.112:0/3809870302 wait complete. 2024-09-21T03:27:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa125f0e640 1 Processor -- start 2024-09-21T03:27:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa125f0e640 1 -- start start 2024-09-21T03:27:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa125f0e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa120102a60 0x7fa12010ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa125f0e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1201083a0 0x7fa1201104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa125f0e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa120108cf0 0x7fa120110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:46.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa125f0e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa120113ac0 con 0x7fa120108cf0 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa125f0e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa120113940 con 0x7fa120102a60 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa125f0e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa120113c40 con 0x7fa1201083a0 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa11ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa120108cf0 0x7fa120110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa11ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa120108cf0 0x7fa120110a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40756/0 (socket says 172.21.15.112:40756) 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa11ffff640 1 -- 172.21.15.112:0/2140471428 learned_addr learned my addr 172.21.15.112:0/2140471428 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa11f7fe640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa120102a60 0x7fa12010ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.948+0000 7fa11effd640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1201083a0 0x7fa1201104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.949+0000 7fa11ffff640 1 -- 172.21.15.112:0/2140471428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1201083a0 msgr2=0x7fa1201104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.949+0000 7fa11ffff640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1201083a0 0x7fa1201104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.949+0000 7fa11ffff640 1 -- 172.21.15.112:0/2140471428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa120102a60 msgr2=0x7fa12010ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.949+0000 7fa11ffff640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa120102a60 0x7fa12010ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.949+0000 7fa11ffff640 1 -- 172.21.15.112:0/2140471428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1201a8770 con 0x7fa120108cf0 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.949+0000 7fa11ffff640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa120108cf0 0x7fa120110a30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa11800eaa0 tx=0x7fa11800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:46.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.949+0000 7fa11cff9640 1 -- 172.21.15.112:0/2140471428 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11800cdb0 con 0x7fa120108cf0 2024-09-21T03:27:46.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.949+0000 7fa11cff9640 1 -- 172.21.15.112:0/2140471428 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa118005020 con 0x7fa120108cf0 2024-09-21T03:27:46.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.949+0000 7fa125f0e640 1 -- 172.21.15.112:0/2140471428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1201a8a60 con 0x7fa120108cf0 2024-09-21T03:27:46.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.949+0000 7fa11cff9640 1 -- 172.21.15.112:0/2140471428 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180116d0 con 0x7fa120108cf0 2024-09-21T03:27:46.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.949+0000 7fa125f0e640 1 -- 172.21.15.112:0/2140471428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1201a8fd0 con 0x7fa120108cf0 2024-09-21T03:27:46.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.950+0000 7fa125f0e640 1 -- 172.21.15.112:0/2140471428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa120063b10 con 0x7fa120108cf0 2024-09-21T03:27:46.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.951+0000 7fa11cff9640 1 -- 172.21.15.112:0/2140471428 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fa1180051d0 con 0x7fa120108cf0 2024-09-21T03:27:46.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.952+0000 7fa11cff9640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa10c0b65f0 0x7fa10c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:46.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.952+0000 7fa11f7fe640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa10c0b65f0 0x7fa10c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:46.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.952+0000 7fa11cff9640 1 -- 172.21.15.112:0/2140471428 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa1180e7bd0 con 0x7fa120108cf0 2024-09-21T03:27:46.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.952+0000 7fa11cff9640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143] conn(0x7fa10c0c0360 0x7fa10c0c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:46.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.953+0000 7fa11cff9640 1 -- 172.21.15.112:0/2140471428 --> [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa11801be50 con 0x7fa10c0c0360 2024-09-21T03:27:46.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.953+0000 7fa11cff9640 1 -- 172.21.15.112:0/2140471428 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa1180e8020 con 0x7fa120108cf0 2024-09-21T03:27:46.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.953+0000 7fa11effd640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143] conn(0x7fa10c0c0360 0x7fa10c0c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:46.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.953+0000 7fa11f7fe640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa10c0b65f0 0x7fa10c0b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa110000c00 tx=0x7fa110031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:46.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.953+0000 7fa11effd640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143] conn(0x7fa10c0c0360 0x7fa10c0c2850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:46.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.954+0000 7fa11cff9640 1 -- 172.21.15.112:0/2140471428 <== osd.9 v2:172.21.15.112:6826/2553277143 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa11801be50 con 0x7fa10c0c0360 2024-09-21T03:27:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.969+0000 7fa125f0e640 1 -- 172.21.15.112:0/2140471428 --> [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa120063f10 con 0x7fa10c0c0360 2024-09-21T03:27:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.970+0000 7fa11cff9640 1 -- 172.21.15.112:0/2140471428 <== osd.9 v2:172.21.15.112:6826/2553277143 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa120063f10 con 0x7fa10c0c0360 2024-09-21T03:27:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.970+0000 7fa125f0e640 1 -- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143] conn(0x7fa10c0c0360 msgr2=0x7fa10c0c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.970+0000 7fa125f0e640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143] conn(0x7fa10c0c0360 0x7fa10c0c2850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.970+0000 7fa125f0e640 1 -- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa10c0b65f0 msgr2=0x7fa10c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.970+0000 7fa125f0e640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa10c0b65f0 0x7fa10c0b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa110000c00 tx=0x7fa110031400 comp rx=0 tx=0).stop 2024-09-21T03:27:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.970+0000 7fa125f0e640 1 -- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa120108cf0 msgr2=0x7fa120110a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.970+0000 7fa125f0e640 1 --2- 172.21.15.112:0/2140471428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa120108cf0 0x7fa120110a30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa11800eaa0 tx=0x7fa11800ef60 comp rx=0 tx=0).stop 2024-09-21T03:27:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.970+0000 7fa11ffff640 1 -- 172.21.15.112:0/2140471428 reap_dead start 2024-09-21T03:27:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.970+0000 7fa125f0e640 1 -- 172.21.15.112:0/2140471428 shutdown_connections 2024-09-21T03:27:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.970+0000 7fa125f0e640 1 -- 172.21.15.112:0/2140471428 >> 172.21.15.112:0/2140471428 conn(0x7fa120079730 msgr2=0x7fa1201014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.971+0000 7fa125f0e640 1 -- 172.21.15.112:0/2140471428 shutdown_connections 2024-09-21T03:27:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:46.971+0000 7fa125f0e640 1 -- 172.21.15.112:0/2140471428 wait complete. 2024-09-21T03:27:47.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:47 smithi112 ceph-mon[31442]: pgmap v281: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s; 61 B/s, 0 objects/s recovering 2024-09-21T03:27:47.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:47 smithi203 ceph-mon[34723]: pgmap v281: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s; 61 B/s, 0 objects/s recovering 2024-09-21T03:27:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:47 smithi187 ceph-mon[34603]: pgmap v281: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s; 61 B/s, 0 objects/s recovering 2024-09-21T03:27:47.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.782+0000 7f842ea8b640 1 -- 172.21.15.112:0/1477181513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8428103140 msgr2=0x7f8428103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:47.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.782+0000 7f842ea8b640 1 --2- 172.21.15.112:0/1477181513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8428103140 0x7f8428103550 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8420009900 tx=0x7f842002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:47.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.782+0000 7f842ea8b640 1 -- 172.21.15.112:0/1477181513 shutdown_connections 2024-09-21T03:27:47.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.782+0000 7f842ea8b640 1 --2- 172.21.15.112:0/1477181513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8428108d00 0x7f842810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.782+0000 7f842ea8b640 1 --2- 172.21.15.112:0/1477181513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84281083b0 0x7f84281087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.782+0000 7f842ea8b640 1 --2- 172.21.15.112:0/1477181513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8428103140 0x7f8428103550 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.782+0000 7f842ea8b640 1 -- 172.21.15.112:0/1477181513 >> 172.21.15.112:0/1477181513 conn(0x7f8428077c60 msgr2=0x7f84280780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:47.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.782+0000 7f842ea8b640 1 -- 172.21.15.112:0/1477181513 shutdown_connections 2024-09-21T03:27:47.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.782+0000 7f842ea8b640 1 -- 172.21.15.112:0/1477181513 wait complete. 2024-09-21T03:27:47.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.783+0000 7f842ea8b640 1 Processor -- start 2024-09-21T03:27:47.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.783+0000 7f842ea8b640 1 -- start start 2024-09-21T03:27:47.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.783+0000 7f842ea8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8428103140 0x7f842819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:47.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.783+0000 7f842ea8b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84281083b0 0x7f842819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:47.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.783+0000 7f842ea8b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8428108d00 0x7f842819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:47.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.783+0000 7f842ea8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84281129c0 con 0x7f8428103140 2024-09-21T03:27:47.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.783+0000 7f842ea8b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8428112840 con 0x7f8428108d00 2024-09-21T03:27:47.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.783+0000 7f842ea8b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8428112b40 con 0x7f84281083b0 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.783+0000 7f842c827640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8428103140 0x7f842819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.783+0000 7f842c827640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8428103140 0x7f842819b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40780/0 (socket says 172.21.15.112:40780) 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.783+0000 7f842c827640 1 -- 172.21.15.112:0/2960514541 learned_addr learned my addr 172.21.15.112:0/2960514541 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f842d028640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8428108d00 0x7f842819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f8427fff640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84281083b0 0x7f842819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f842c827640 1 -- 172.21.15.112:0/2960514541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84281083b0 msgr2=0x7f842819b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f842c827640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84281083b0 0x7f842819b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f842c827640 1 -- 172.21.15.112:0/2960514541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8428108d00 msgr2=0x7f842819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f842c827640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8428108d00 0x7f842819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f842c827640 1 -- 172.21.15.112:0/2960514541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84281a6550 con 0x7f8428103140 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f8427fff640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84281083b0 0x7f842819b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f842c827640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8428103140 0x7f842819b2d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8420005940 tx=0x7f84200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f8425ffb640 1 -- 172.21.15.112:0/2960514541 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842003f030 con 0x7f8428103140 2024-09-21T03:27:47.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f8425ffb640 1 -- 172.21.15.112:0/2960514541 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8420003cb0 con 0x7f8428103140 2024-09-21T03:27:47.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f842ea8b640 1 -- 172.21.15.112:0/2960514541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84281a67e0 con 0x7f8428103140 2024-09-21T03:27:47.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f8425ffb640 1 -- 172.21.15.112:0/2960514541 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420035070 con 0x7f8428103140 2024-09-21T03:27:47.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.784+0000 7f842ea8b640 1 -- 172.21.15.112:0/2960514541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84281a6cc0 con 0x7f8428103140 2024-09-21T03:27:47.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.785+0000 7f842ea8b640 1 -- 172.21.15.112:0/2960514541 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8428063b50 con 0x7f8428103140 2024-09-21T03:27:47.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.787+0000 7f8425ffb640 1 -- 172.21.15.112:0/2960514541 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f8420042030 con 0x7f8428103140 2024-09-21T03:27:47.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.787+0000 7f8425ffb640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f84140b65f0 0x7f84140b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:47.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.788+0000 7f8427fff640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f84140b65f0 0x7f84140b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:47.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.788+0000 7f8425ffb640 1 -- 172.21.15.112:0/2960514541 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f84201081f0 con 0x7f8428103140 2024-09-21T03:27:47.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.788+0000 7f8425ffb640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796] conn(0x7f84140c0360 0x7f84140c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:47.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.788+0000 7f8425ffb640 1 -- 172.21.15.112:0/2960514541 --> [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84140c2f20 con 0x7f84140c0360 2024-09-21T03:27:47.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.788+0000 7f8425ffb640 1 -- 172.21.15.112:0/2960514541 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8420108640 con 0x7f8428103140 2024-09-21T03:27:47.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.788+0000 7f842d028640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796] conn(0x7f84140c0360 0x7f84140c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:47.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.788+0000 7f8427fff640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f84140b65f0 0x7f84140b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8428108120 tx=0x7f84180093e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:47.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.788+0000 7f842d028640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796] conn(0x7f84140c0360 0x7f84140c2850 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:47.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.789+0000 7f8425ffb640 1 -- 172.21.15.112:0/2960514541 <== osd.0 v2:172.21.15.112:6802/3601862796 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84140c2f20 con 0x7f84140c0360 2024-09-21T03:27:47.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.802+0000 7f842ea8b640 1 -- 172.21.15.112:0/2960514541 --> [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8428063f50 con 0x7f84140c0360 2024-09-21T03:27:47.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f8425ffb640 1 -- 172.21.15.112:0/2960514541 <== osd.0 v2:172.21.15.112:6802/3601862796 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8428063f50 con 0x7f84140c0360 2024-09-21T03:27:47.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f842ea8b640 1 -- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796] conn(0x7f84140c0360 msgr2=0x7f84140c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:47.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f842ea8b640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796] conn(0x7f84140c0360 0x7f84140c2850 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f842ea8b640 1 -- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f84140b65f0 msgr2=0x7f84140b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:47.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f842ea8b640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f84140b65f0 0x7f84140b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8428108120 tx=0x7f84180093e0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f842ea8b640 1 -- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8428103140 msgr2=0x7f842819b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:47.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f842ea8b640 1 --2- 172.21.15.112:0/2960514541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8428103140 0x7f842819b2d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8420005940 tx=0x7f84200042f0 comp rx=0 tx=0).stop 2024-09-21T03:27:47.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f842d028640 1 -- 172.21.15.112:0/2960514541 reap_dead start 2024-09-21T03:27:47.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f842ea8b640 1 -- 172.21.15.112:0/2960514541 shutdown_connections 2024-09-21T03:27:47.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f842ea8b640 1 -- 172.21.15.112:0/2960514541 >> 172.21.15.112:0/2960514541 conn(0x7f8428077c60 msgr2=0x7f84281064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:47.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f842ea8b640 1 -- 172.21.15.112:0/2960514541 shutdown_connections 2024-09-21T03:27:47.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:47.803+0000 7f842ea8b640 1 -- 172.21.15.112:0/2960514541 wait complete. 2024-09-21T03:27:48.210 INFO:teuthology.orchestra.run.smithi112.stdout:77309411395 2024-09-21T03:27:48.210 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T03:27:48.804 INFO:teuthology.orchestra.run.smithi112.stdout:150323855411 2024-09-21T03:27:48.805 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T03:27:49.352 INFO:teuthology.orchestra.run.smithi112.stdout:107374182460 2024-09-21T03:27:49.352 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T03:27:49.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:49 smithi203 ceph-mon[34723]: pgmap v282: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 4 op/s 2024-09-21T03:27:49.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:49 smithi187 ceph-mon[34603]: pgmap v282: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 4 op/s 2024-09-21T03:27:49.731 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:49 smithi112 ceph-mon[31442]: pgmap v282: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 4 op/s 2024-09-21T03:27:49.759 INFO:teuthology.orchestra.run.smithi112.stdout:60129542216 2024-09-21T03:27:49.759 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T03:27:50.344 INFO:teuthology.orchestra.run.smithi112.stdout:38654705741 2024-09-21T03:27:50.345 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T03:27:50.805 INFO:teuthology.orchestra.run.smithi112.stdout:249108103200 2024-09-21T03:27:50.806 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T03:27:51.195 INFO:teuthology.orchestra.run.smithi112.stdout:128849018937 2024-09-21T03:27:51.196 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T03:27:51.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:51 smithi112 ceph-mon[31442]: pgmap v283: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T03:27:51.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:51 smithi187 ceph-mon[34603]: pgmap v283: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T03:27:51.697 INFO:teuthology.orchestra.run.smithi112.stdout:223338299428 2024-09-21T03:27:51.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T03:27:51.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:51 smithi203 ceph-mon[34723]: pgmap v283: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T03:27:51.974 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:51.974 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:51.975 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:51.975 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:51.978 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:51.978 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:51.983 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:51.985 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:51.985 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:51.990 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:51.990 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:27:53.650 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:53 smithi112 ceph-mon[31442]: pgmap v284: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:27:53.650 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:53 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1620414693' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:27:53.650 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:53 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1374111258' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:27:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:53 smithi187 ceph-mon[34603]: pgmap v284: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:27:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:53 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1620414693' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:27:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:53 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1374111258' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:27:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.778+0000 7f31727e2640 1 -- 172.21.15.112:0/3610106452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f316c109df0 msgr2=0x7f316c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.778+0000 7f31727e2640 1 --2- 172.21.15.112:0/3610106452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f316c109df0 0x7f316c110620 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3164009ca0 tx=0x7f316402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.778+0000 7f31727e2640 1 -- 172.21.15.112:0/3610106452 shutdown_connections 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.778+0000 7f31727e2640 1 --2- 172.21.15.112:0/3610106452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f316c109df0 0x7f316c110620 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.778+0000 7f31727e2640 1 --2- 172.21.15.112:0/3610106452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f316c1094a0 0x7f316c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.778+0000 7f31727e2640 1 --2- 172.21.15.112:0/3610106452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f316c103b60 0x7f316c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.778+0000 7f31727e2640 1 -- 172.21.15.112:0/3610106452 >> 172.21.15.112:0/3610106452 conn(0x7f316c07b850 msgr2=0x7f316c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.778+0000 7f31727e2640 1 -- 172.21.15.112:0/3610106452 shutdown_connections 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.778+0000 7f31727e2640 1 -- 172.21.15.112:0/3610106452 wait complete. 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.779+0000 7f31727e2640 1 Processor -- start 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.779+0000 7f31727e2640 1 -- start start 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.779+0000 7f31727e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f316c103b60 0x7f316c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.779+0000 7f31727e2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f316c1094a0 0x7f316c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.779+0000 7f31727e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f316c109df0 0x7f316c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.779+0000 7f31727e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f316c113ab0 con 0x7f316c103b60 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.779+0000 7f31727e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f316c113930 con 0x7f316c109df0 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.779+0000 7f31727e2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f316c113c30 con 0x7f316c1094a0 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.780+0000 7f3170d7f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f316c109df0 0x7f316c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.780+0000 7f3170d7f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f316c109df0 0x7f316c19fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42816/0 (socket says 172.21.15.112:42816) 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.780+0000 7f3170d7f640 1 -- 172.21.15.112:0/1620414693 learned_addr learned my addr 172.21.15.112:0/1620414693 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.780+0000 7f316b7fe640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f316c1094a0 0x7f316c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.780+0000 7f3170d7f640 1 -- 172.21.15.112:0/1620414693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f316c1094a0 msgr2=0x7f316c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.780+0000 7f3170d7f640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f316c1094a0 0x7f316c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.780+0000 7f3170d7f640 1 -- 172.21.15.112:0/1620414693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f316c103b60 msgr2=0x7f316c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.780+0000 7f3170d7f640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f316c103b60 0x7f316c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.780+0000 7f3170d7f640 1 -- 172.21.15.112:0/1620414693 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c1a0390 con 0x7f316c109df0 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.780+0000 7f316b7fe640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f316c1094a0 0x7f316c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:27:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.781+0000 7f3170d7f640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f316c109df0 0x7f316c19fc60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3164005cc0 tx=0x7f316402f380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.781+0000 7f31697fa640 1 -- 172.21.15.112:0/1620414693 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164003bf0 con 0x7f316c109df0 2024-09-21T03:27:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.781+0000 7f31697fa640 1 -- 172.21.15.112:0/1620414693 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3164003d90 con 0x7f316c109df0 2024-09-21T03:27:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.781+0000 7f31727e2640 1 -- 172.21.15.112:0/1620414693 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c1a0620 con 0x7f316c109df0 2024-09-21T03:27:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.781+0000 7f31697fa640 1 -- 172.21.15.112:0/1620414693 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316402fbc0 con 0x7f316c109df0 2024-09-21T03:27:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.781+0000 7f31727e2640 1 -- 172.21.15.112:0/1620414693 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c1a6de0 con 0x7f316c109df0 2024-09-21T03:27:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.782+0000 7f31727e2640 1 -- 172.21.15.112:0/1620414693 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f316c103f70 con 0x7f316c109df0 2024-09-21T03:27:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.784+0000 7f31697fa640 1 -- 172.21.15.112:0/1620414693 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f316402fd60 con 0x7f316c109df0 2024-09-21T03:27:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.784+0000 7f31697fa640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31540b65f0 0x7f31540b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:53.858 INFO:teuthology.orchestra.run.smithi112.stdout:77309411397 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.785+0000 7f316bfff640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31540b65f0 0x7f31540b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.785+0000 7f31697fa640 1 -- 172.21.15.112:0/1620414693 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3164108cb0 con 0x7f316c109df0 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.785+0000 7f316bfff640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31540b65f0 0x7f31540b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f315c005fd0 tx=0x7f315c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.788+0000 7f31697fa640 1 -- 172.21.15.112:0/1620414693 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f31640cf2c0 con 0x7f316c109df0 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.974+0000 7f31727e2640 1 -- 172.21.15.112:0/1620414693 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f316c19b550 con 0x7f316c109df0 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.974+0000 7f31697fa640 1 -- 172.21.15.112:0/1620414693 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f316c19b550 con 0x7f316c109df0 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 -- 172.21.15.112:0/1620414693 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31540b65f0 msgr2=0x7f31540b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31540b65f0 0x7f31540b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f315c005fd0 tx=0x7f315c005850 comp rx=0 tx=0).stop 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 -- 172.21.15.112:0/1620414693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f316c109df0 msgr2=0x7f316c19fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f316c109df0 0x7f316c19fc60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3164005cc0 tx=0x7f316402f380 comp rx=0 tx=0).stop 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 -- 172.21.15.112:0/1620414693 shutdown_connections 2024-09-21T03:27:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31540b65f0 0x7f31540b8b00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f316c109df0 0x7f316c19fc60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f316c1094a0 0x7f316c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 --2- 172.21.15.112:0/1620414693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f316c103b60 0x7f316c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 -- 172.21.15.112:0/1620414693 >> 172.21.15.112:0/1620414693 conn(0x7f316c07b850 msgr2=0x7f316c105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 -- 172.21.15.112:0/1620414693 shutdown_connections 2024-09-21T03:27:53.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.977+0000 7f31727e2640 1 -- 172.21.15.112:0/1620414693 wait complete. 2024-09-21T03:27:53.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:53 smithi203 ceph-mon[34723]: pgmap v284: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-21T03:27:53.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:53 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1620414693' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T03:27:53.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:53 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1374111258' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T03:27:55.654 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:55 smithi112 ceph-mon[31442]: pgmap v285: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:27:55.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:55 smithi187 ceph-mon[34603]: pgmap v285: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:27:55.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:55 smithi203 ceph-mon[34723]: pgmap v285: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:27:55.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.960+0000 7ff0657f8640 1 -- 172.21.15.112:0/3237804528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff060108cf0 msgr2=0x7ff06010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:55.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.960+0000 7ff0657f8640 1 --2- 172.21.15.112:0/3237804528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff060108cf0 0x7ff06010f520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff058009900 tx=0x7ff05802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.960+0000 7ff0657f8640 1 -- 172.21.15.112:0/3237804528 shutdown_connections 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.960+0000 7ff0657f8640 1 --2- 172.21.15.112:0/3237804528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff060108cf0 0x7ff06010f520 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.960+0000 7ff0657f8640 1 --2- 172.21.15.112:0/3237804528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0601083a0 0x7ff0601087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.960+0000 7ff0657f8640 1 --2- 172.21.15.112:0/3237804528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff060102a60 0x7ff060102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.960+0000 7ff0657f8640 1 -- 172.21.15.112:0/3237804528 >> 172.21.15.112:0/3237804528 conn(0x7ff060079720 msgr2=0x7ff060079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.960+0000 7ff0657f8640 1 -- 172.21.15.112:0/3237804528 shutdown_connections 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.961+0000 7ff0657f8640 1 -- 172.21.15.112:0/3237804528 wait complete. 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.961+0000 7ff0657f8640 1 Processor -- start 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.961+0000 7ff0657f8640 1 -- start start 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff0657f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff060102a60 0x7ff06019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff0657f8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0601083a0 0x7ff06019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:55.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff0657f8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff060108cf0 0x7ff06019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff0657f8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff060112960 con 0x7ff060102a60 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff0657f8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0601127e0 con 0x7ff060108cf0 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff0657f8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff060112ae0 con 0x7ff0601083a0 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff060102a60 0x7ff06019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff060102a60 0x7ff06019b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52298/0 (socket says 172.21.15.112:52298) 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05effd640 1 -- 172.21.15.112:0/861215259 learned_addr learned my addr 172.21.15.112:0/861215259 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05e7fc640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0601083a0 0x7ff06019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05f7fe640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff060108cf0 0x7ff06019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05effd640 1 -- 172.21.15.112:0/861215259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0601083a0 msgr2=0x7ff06019b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05effd640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0601083a0 0x7ff06019b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05effd640 1 -- 172.21.15.112:0/861215259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff060108cf0 msgr2=0x7ff06019bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05effd640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff060108cf0 0x7ff06019bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05effd640 1 -- 172.21.15.112:0/861215259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0601a63b0 con 0x7ff060102a60 2024-09-21T03:27:55.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05f7fe640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff060108cf0 0x7ff06019bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff05effd640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff060102a60 0x7ff06019b1b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff05400eab0 tx=0x7ff05400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff047fff640 1 -- 172.21.15.112:0/861215259 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05400ce40 con 0x7ff060102a60 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff0657f8640 1 -- 172.21.15.112:0/861215259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0601a66a0 con 0x7ff060102a60 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.962+0000 7ff047fff640 1 -- 172.21.15.112:0/861215259 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff054011920 con 0x7ff060102a60 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.963+0000 7ff047fff640 1 -- 172.21.15.112:0/861215259 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054010a30 con 0x7ff060102a60 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.963+0000 7ff0657f8640 1 -- 172.21.15.112:0/861215259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0601a6c30 con 0x7ff060102a60 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.964+0000 7ff0657f8640 1 -- 172.21.15.112:0/861215259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff060103ce0 con 0x7ff060102a60 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.966+0000 7ff047fff640 1 -- 172.21.15.112:0/861215259 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7ff054010bd0 con 0x7ff060102a60 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.967+0000 7ff047fff640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0480b6620 0x7ff0480b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.967+0000 7ff05e7fc640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0480b6620 0x7ff0480b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.967+0000 7ff047fff640 1 -- 172.21.15.112:0/861215259 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff0540e7c20 con 0x7ff060102a60 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.967+0000 7ff05e7fc640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0480b6620 0x7ff0480b8b30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff050005e10 tx=0x7ff050005da0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:55.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.969+0000 7ff047fff640 1 -- 172.21.15.112:0/861215259 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff054016030 con 0x7ff060102a60 2024-09-21T03:27:56.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.153+0000 7ff0657f8640 1 -- 172.21.15.112:0/861215259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff060102e70 con 0x7ff060102a60 2024-09-21T03:27:56.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.154+0000 7ff047fff640 1 -- 172.21.15.112:0/861215259 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff0540ae290 con 0x7ff060102a60 2024-09-21T03:27:56.155 INFO:teuthology.orchestra.run.smithi112.stdout:38654705742 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 -- 172.21.15.112:0/861215259 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0480b6620 msgr2=0x7ff0480b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0480b6620 0x7ff0480b8b30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff050005e10 tx=0x7ff050005da0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 -- 172.21.15.112:0/861215259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff060102a60 msgr2=0x7ff06019b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff060102a60 0x7ff06019b1b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff05400eab0 tx=0x7ff05400ef70 comp rx=0 tx=0).stop 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 -- 172.21.15.112:0/861215259 shutdown_connections 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0480b6620 0x7ff0480b8b30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff060108cf0 0x7ff06019bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0601083a0 0x7ff06019b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 --2- 172.21.15.112:0/861215259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff060102a60 0x7ff06019b1b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 -- 172.21.15.112:0/861215259 >> 172.21.15.112:0/861215259 conn(0x7ff060079720 msgr2=0x7ff0601043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 -- 172.21.15.112:0/861215259 shutdown_connections 2024-09-21T03:27:56.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.156+0000 7ff0657f8640 1 -- 172.21.15.112:0/861215259 wait complete. 2024-09-21T03:27:56.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.518+0000 7f9132790640 1 -- 172.21.15.112:0/356039858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912c103b60 msgr2=0x7f912c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.518+0000 7f9132790640 1 --2- 172.21.15.112:0/356039858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912c103b60 0x7f912c103f70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f911c009900 tx=0x7f911c02a410 comp rx=0 tx=0).stop 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.518+0000 7f9132790640 1 -- 172.21.15.112:0/356039858 shutdown_connections 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.518+0000 7f9132790640 1 --2- 172.21.15.112:0/356039858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f912c109df0 0x7f912c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.518+0000 7f9132790640 1 --2- 172.21.15.112:0/356039858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c1094a0 0x7f912c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.518+0000 7f9132790640 1 --2- 172.21.15.112:0/356039858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912c103b60 0x7f912c103f70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.518+0000 7f9132790640 1 -- 172.21.15.112:0/356039858 >> 172.21.15.112:0/356039858 conn(0x7f912c07b850 msgr2=0x7f912c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.518+0000 7f9132790640 1 -- 172.21.15.112:0/356039858 shutdown_connections 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.518+0000 7f9132790640 1 -- 172.21.15.112:0/356039858 wait complete. 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.519+0000 7f9132790640 1 Processor -- start 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.519+0000 7f9132790640 1 -- start start 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.519+0000 7f9132790640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912c103b60 0x7f912c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f9132790640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f912c1094a0 0x7f912c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f9132790640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c109df0 0x7f912c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f9132790640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f912c113ab0 con 0x7f912c103b60 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f9132790640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f912c113930 con 0x7f912c1094a0 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f9132790640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f912c113c30 con 0x7f912c109df0 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f912bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912c103b60 0x7f912c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f912bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912c103b60 0x7f912c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52322/0 (socket says 172.21.15.112:52322) 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f912bfff640 1 -- 172.21.15.112:0/133811122 learned_addr learned my addr 172.21.15.112:0/133811122 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f912b7fe640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f912c1094a0 0x7f912c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f9130d2d640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c109df0 0x7f912c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f912bfff640 1 -- 172.21.15.112:0/133811122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c109df0 msgr2=0x7f912c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f912bfff640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c109df0 0x7f912c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f912bfff640 1 -- 172.21.15.112:0/133811122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f912c1094a0 msgr2=0x7f912c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f912bfff640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f912c1094a0 0x7f912c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f912bfff640 1 -- 172.21.15.112:0/133811122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c1a8730 con 0x7f912c103b60 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f9130d2d640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c109df0 0x7f912c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f912bfff640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912c103b60 0x7f912c075710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f911c005940 tx=0x7f911c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f91297fa640 1 -- 172.21.15.112:0/133811122 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c03f030 con 0x7f912c103b60 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f9132790640 1 -- 172.21.15.112:0/133811122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912c1a89c0 con 0x7f912c103b60 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f91297fa640 1 -- 172.21.15.112:0/133811122 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f911c003aa0 con 0x7f912c103b60 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f91297fa640 1 -- 172.21.15.112:0/133811122 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c035070 con 0x7f912c103b60 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.520+0000 7f9132790640 1 -- 172.21.15.112:0/133811122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912c1a8ef0 con 0x7f912c103b60 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.521+0000 7f9132790640 1 -- 172.21.15.112:0/133811122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912c103f70 con 0x7f912c103b60 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.523+0000 7f91297fa640 1 -- 172.21.15.112:0/133811122 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f911c042030 con 0x7f912c103b60 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.523+0000 7f91297fa640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91180b65f0 0x7f91180b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.524+0000 7f912b7fe640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91180b65f0 0x7f91180b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.524+0000 7f91297fa640 1 -- 172.21.15.112:0/133811122 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f911c107eb0 con 0x7f912c103b60 2024-09-21T03:27:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.524+0000 7f912b7fe640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91180b65f0 0x7f91180b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f912c1064c0 tx=0x7f9114005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:56.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.527+0000 7f91297fa640 1 -- 172.21.15.112:0/133811122 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f911c0ce520 con 0x7f912c103b60 2024-09-21T03:27:56.655 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:56 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1727168720' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:27:56.655 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:56 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/861215259' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:27:56.655 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:56 smithi112 ceph-mon[31442]: pgmap v286: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-21T03:27:56.655 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:56 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1609940324' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:27:56.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.711+0000 7f9132790640 1 -- 172.21.15.112:0/133811122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f912c02e640 con 0x7f912c103b60 2024-09-21T03:27:56.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.711+0000 7f91297fa640 1 -- 172.21.15.112:0/133811122 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f911c0d2900 con 0x7f912c103b60 2024-09-21T03:27:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:270582939676 2024-09-21T03:27:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.713+0000 7f9132790640 1 -- 172.21.15.112:0/133811122 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91180b65f0 msgr2=0x7f91180b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.713+0000 7f9132790640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91180b65f0 0x7f91180b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f912c1064c0 tx=0x7f9114005f90 comp rx=0 tx=0).stop 2024-09-21T03:27:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.713+0000 7f9132790640 1 -- 172.21.15.112:0/133811122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912c103b60 msgr2=0x7f912c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.713+0000 7f9132790640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912c103b60 0x7f912c075710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f911c005940 tx=0x7f911c004270 comp rx=0 tx=0).stop 2024-09-21T03:27:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.713+0000 7f9132790640 1 -- 172.21.15.112:0/133811122 shutdown_connections 2024-09-21T03:27:56.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.713+0000 7f9132790640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91180b65f0 0x7f91180b8b00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.713+0000 7f9132790640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f912c109df0 0x7f912c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.713+0000 7f9132790640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f912c1094a0 0x7f912c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.713+0000 7f9132790640 1 --2- 172.21.15.112:0/133811122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f912c103b60 0x7f912c075710 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.713+0000 7f9132790640 1 -- 172.21.15.112:0/133811122 >> 172.21.15.112:0/133811122 conn(0x7f912c07b850 msgr2=0x7f912c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:56.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.714+0000 7f9132790640 1 -- 172.21.15.112:0/133811122 shutdown_connections 2024-09-21T03:27:56.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.714+0000 7f9132790640 1 -- 172.21.15.112:0/133811122 wait complete. 2024-09-21T03:27:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.746+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/185310495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa0102ae0 msgr2=0x7fcaa0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.746+0000 7fcaa5d2a640 1 --2- 172.21.15.112:0/185310495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa0102ae0 0x7fcaa0102ef0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fca90009900 tx=0x7fca9002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.746+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/185310495 shutdown_connections 2024-09-21T03:27:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.746+0000 7fcaa5d2a640 1 --2- 172.21.15.112:0/185310495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaa0108d20 0x7fcaa010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.746+0000 7fcaa5d2a640 1 --2- 172.21.15.112:0/185310495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa01083d0 0x7fcaa01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.746+0000 7fcaa5d2a640 1 --2- 172.21.15.112:0/185310495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa0102ae0 0x7fcaa0102ef0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.746+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/185310495 >> 172.21.15.112:0/185310495 conn(0x7fcaa00797b0 msgr2=0x7fcaa0079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:56.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.746+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/185310495 shutdown_connections 2024-09-21T03:27:56.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.746+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/185310495 wait complete. 2024-09-21T03:27:56.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.747+0000 7fcaa5d2a640 1 Processor -- start 2024-09-21T03:27:56.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.747+0000 7fcaa5d2a640 1 -- start start 2024-09-21T03:27:56.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.747+0000 7fcaa5d2a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa0102ae0 0x7fcaa019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fcaa5d2a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa01083d0 0x7fcaa019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fcaa5d2a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaa0108d20 0x7fcaa019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fcaa5d2a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01129e0 con 0x7fcaa0102ae0 2024-09-21T03:27:56.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fcaa5d2a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0112860 con 0x7fcaa01083d0 2024-09-21T03:27:56.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fcaa5d2a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0112b60 con 0x7fcaa0108d20 2024-09-21T03:27:56.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa0102ae0 0x7fcaa019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa0102ae0 0x7fcaa019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52340/0 (socket says 172.21.15.112:52340) 2024-09-21T03:27:56.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9f7fe640 1 -- 172.21.15.112:0/1031110405 learned_addr learned my addr 172.21.15.112:0/1031110405 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:56.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9effd640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa01083d0 0x7fcaa019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9ffff640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaa0108d20 0x7fcaa019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9f7fe640 1 -- 172.21.15.112:0/1031110405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaa0108d20 msgr2=0x7fcaa019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9f7fe640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaa0108d20 0x7fcaa019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9f7fe640 1 -- 172.21.15.112:0/1031110405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa01083d0 msgr2=0x7fcaa019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9f7fe640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa01083d0 0x7fcaa019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9f7fe640 1 -- 172.21.15.112:0/1031110405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa01a0490 con 0x7fcaa0102ae0 2024-09-21T03:27:56.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9ffff640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaa0108d20 0x7fcaa019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:56.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9f7fe640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa0102ae0 0x7fcaa019a520 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fca90005940 tx=0x7fca90004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:56.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9cff9640 1 -- 172.21.15.112:0/1031110405 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9003f030 con 0x7fcaa0102ae0 2024-09-21T03:27:56.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fca9cff9640 1 -- 172.21.15.112:0/1031110405 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca90003aa0 con 0x7fcaa0102ae0 2024-09-21T03:27:56.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.748+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/1031110405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa01a0720 con 0x7fcaa0102ae0 2024-09-21T03:27:56.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.749+0000 7fca9cff9640 1 -- 172.21.15.112:0/1031110405 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90035070 con 0x7fcaa0102ae0 2024-09-21T03:27:56.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.749+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/1031110405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa01a6ee0 con 0x7fcaa0102ae0 2024-09-21T03:27:56.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.750+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/1031110405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa0102ef0 con 0x7fcaa0102ae0 2024-09-21T03:27:56.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.751+0000 7fca9cff9640 1 -- 172.21.15.112:0/1031110405 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fca90042030 con 0x7fcaa0102ae0 2024-09-21T03:27:56.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.752+0000 7fca9cff9640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca880b66f0 0x7fca880b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:56.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.752+0000 7fca9cff9640 1 -- 172.21.15.112:0/1031110405 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fca90107ec0 con 0x7fcaa0102ae0 2024-09-21T03:27:56.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.752+0000 7fca9effd640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca880b66f0 0x7fca880b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:56.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.753+0000 7fca9effd640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca880b66f0 0x7fca880b8c00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0104a40 tx=0x7fca94005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:56.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.755+0000 7fca9cff9640 1 -- 172.21.15.112:0/1031110405 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fca900ce4d0 con 0x7fcaa0102ae0 2024-09-21T03:27:56.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:56 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1727168720' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:27:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:56 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/861215259' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:27:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:56 smithi203 ceph-mon[34723]: pgmap v286: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-21T03:27:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:56 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1609940324' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:27:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:56 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1727168720' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T03:27:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:56 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/861215259' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T03:27:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:56 smithi187 ceph-mon[34603]: pgmap v286: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-21T03:27:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:56 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1609940324' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T03:27:56.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.937+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/1031110405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcaa010bb60 con 0x7fcaa0102ae0 2024-09-21T03:27:56.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.937+0000 7fca9cff9640 1 -- 172.21.15.112:0/1031110405 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fca900d28b0 con 0x7fcaa0102ae0 2024-09-21T03:27:56.939 INFO:teuthology.orchestra.run.smithi112.stdout:150323855414 2024-09-21T03:27:56.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/1031110405 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca880b66f0 msgr2=0x7fca880b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca880b66f0 0x7fca880b8c00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0104a40 tx=0x7fca94005f90 comp rx=0 tx=0).stop 2024-09-21T03:27:56.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/1031110405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa0102ae0 msgr2=0x7fcaa019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:56.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa0102ae0 0x7fcaa019a520 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fca90005940 tx=0x7fca90004270 comp rx=0 tx=0).stop 2024-09-21T03:27:56.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/1031110405 shutdown_connections 2024-09-21T03:27:56.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca880b66f0 0x7fca880b8c00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaa0108d20 0x7fcaa019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaa01083d0 0x7fcaa019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 --2- 172.21.15.112:0/1031110405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcaa0102ae0 0x7fcaa019a520 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:56.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/1031110405 >> 172.21.15.112:0/1031110405 conn(0x7fcaa00797b0 msgr2=0x7fcaa0101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:56.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/1031110405 shutdown_connections 2024-09-21T03:27:56.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.940+0000 7fcaa5d2a640 1 -- 172.21.15.112:0/1031110405 wait complete. 2024-09-21T03:27:57.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.145+0000 7f65ff32d640 1 -- 172.21.15.112:0/2179551366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f8109500 msgr2=0x7f65f8109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:57.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.145+0000 7f65ff32d640 1 --2- 172.21.15.112:0/2179551366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f8109500 0x7f65f8109910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009900 tx=0x7f65e802a410 comp rx=0 tx=0).stop 2024-09-21T03:27:57.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.146+0000 7f65ff32d640 1 -- 172.21.15.112:0/2179551366 shutdown_connections 2024-09-21T03:27:57.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.146+0000 7f65ff32d640 1 --2- 172.21.15.112:0/2179551366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f8109e50 0x7f65f8110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.146+0000 7f65ff32d640 1 --2- 172.21.15.112:0/2179551366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f8109500 0x7f65f8109910 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.146+0000 7f65ff32d640 1 --2- 172.21.15.112:0/2179551366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65f8101fc0 0x7f65f81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.146+0000 7f65ff32d640 1 -- 172.21.15.112:0/2179551366 >> 172.21.15.112:0/2179551366 conn(0x7f65f8079f90 msgr2=0x7f65f807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:57.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.146+0000 7f65ff32d640 1 -- 172.21.15.112:0/2179551366 shutdown_connections 2024-09-21T03:27:57.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.146+0000 7f65ff32d640 1 -- 172.21.15.112:0/2179551366 wait complete. 2024-09-21T03:27:57.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.146+0000 7f65ff32d640 1 Processor -- start 2024-09-21T03:27:57.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.146+0000 7f65ff32d640 1 -- start start 2024-09-21T03:27:57.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65ff32d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f8101fc0 0x7f65f819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:57.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65ff32d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65f8109500 0x7f65f819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:57.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65ff32d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f8109e50 0x7f65f819fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:57.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65ff32d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65f8113bb0 con 0x7f65f8101fc0 2024-09-21T03:27:57.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65ff32d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65f8113a30 con 0x7f65f8109500 2024-09-21T03:27:57.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65ff32d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f65f8113d30 con 0x7f65f8109e50 2024-09-21T03:27:57.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fd0c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f8101fc0 0x7f65f819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:57.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fd0c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f8101fc0 0x7f65f819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52356/0 (socket says 172.21.15.112:52356) 2024-09-21T03:27:57.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fd0c9640 1 -- 172.21.15.112:0/3231970624 learned_addr learned my addr 172.21.15.112:0/3231970624 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fc8c8640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65f8109500 0x7f65f819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fd8ca640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f8109e50 0x7f65f819fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fd0c9640 1 -- 172.21.15.112:0/3231970624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f8109e50 msgr2=0x7f65f819fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fd0c9640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f8109e50 0x7f65f819fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fd0c9640 1 -- 172.21.15.112:0/3231970624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65f8109500 msgr2=0x7f65f819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fd0c9640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65f8109500 0x7f65f819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fd0c9640 1 -- 172.21.15.112:0/3231970624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f81a05d0 con 0x7f65f8101fc0 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fd8ca640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f8109e50 0x7f65f819fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fc8c8640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65f8109500 0x7f65f819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65fd0c9640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f8101fc0 0x7f65f819a660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f65f400b4a0 tx=0x7f65f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65ee7fc640 1 -- 172.21.15.112:0/3231970624 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4014070 con 0x7f65f8101fc0 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.147+0000 7f65ee7fc640 1 -- 172.21.15.112:0/3231970624 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f4010ac0 con 0x7f65f8101fc0 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.148+0000 7f65ee7fc640 1 -- 172.21.15.112:0/3231970624 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4017c30 con 0x7f65f8101fc0 2024-09-21T03:27:57.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.148+0000 7f65ff32d640 1 -- 172.21.15.112:0/3231970624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f81a08c0 con 0x7f65f8101fc0 2024-09-21T03:27:57.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.148+0000 7f65ff32d640 1 -- 172.21.15.112:0/3231970624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f81a7020 con 0x7f65f8101fc0 2024-09-21T03:27:57.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.149+0000 7f65ff32d640 1 -- 172.21.15.112:0/3231970624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f8102450 con 0x7f65f8101fc0 2024-09-21T03:27:57.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.150+0000 7f65ee7fc640 1 -- 172.21.15.112:0/3231970624 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f65f4017070 con 0x7f65f8101fc0 2024-09-21T03:27:57.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.151+0000 7f65ee7fc640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f65e40b65f0 0x7f65e40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:57.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.151+0000 7f65fc8c8640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f65e40b65f0 0x7f65e40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:57.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.152+0000 7f65ee7fc640 1 -- 172.21.15.112:0/3231970624 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f65f40e6c80 con 0x7f65f8101fc0 2024-09-21T03:27:57.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.152+0000 7f65fc8c8640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f65e40b65f0 0x7f65e40b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009230 tx=0x7f65e8000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:57.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.155+0000 7f65ee7fc640 1 -- 172.21.15.112:0/3231970624 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65f40ad290 con 0x7f65f8101fc0 2024-09-21T03:27:57.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.183+0000 7f32f2e9e640 1 -- 172.21.15.112:0/3462082985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ec108cf0 msgr2=0x7f32ec10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:57.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.183+0000 7f32f2e9e640 1 --2- 172.21.15.112:0/3462082985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ec108cf0 0x7f32ec10f520 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009ca0 tx=0x7f32e402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.183+0000 7f32f2e9e640 1 -- 172.21.15.112:0/3462082985 shutdown_connections 2024-09-21T03:27:57.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.183+0000 7f32f2e9e640 1 --2- 172.21.15.112:0/3462082985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ec108cf0 0x7f32ec10f520 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.183+0000 7f32f2e9e640 1 --2- 172.21.15.112:0/3462082985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ec1083a0 0x7f32ec1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.183+0000 7f32f2e9e640 1 --2- 172.21.15.112:0/3462082985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ec102a60 0x7f32ec102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.183+0000 7f32f2e9e640 1 -- 172.21.15.112:0/3462082985 >> 172.21.15.112:0/3462082985 conn(0x7f32ec079730 msgr2=0x7f32ec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.183+0000 7f32f2e9e640 1 -- 172.21.15.112:0/3462082985 shutdown_connections 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.183+0000 7f32f2e9e640 1 -- 172.21.15.112:0/3462082985 wait complete. 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f2e9e640 1 Processor -- start 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f2e9e640 1 -- start start 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f2e9e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ec102a60 0x7f32ec19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f2e9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ec1083a0 0x7f32ec19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f2e9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ec108cf0 0x7f32ec19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f2e9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec112960 con 0x7f32ec108cf0 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f2e9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec1127e0 con 0x7f32ec1083a0 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f2e9e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec112ae0 con 0x7f32ec102a60 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f143b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ec108cf0 0x7f32ec19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:57.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f143b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ec108cf0 0x7f32ec19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52310/0 (socket says 172.21.15.112:52310) 2024-09-21T03:27:57.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f143b640 1 -- 172.21.15.112:0/1609940324 learned_addr learned my addr 172.21.15.112:0/1609940324 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:57.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.184+0000 7f32f0c3a640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ec102a60 0x7f32ec19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:57.245 INFO:teuthology.orchestra.run.smithi112.stdout:60129542218 2024-09-21T03:27:57.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32ebfff640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ec1083a0 0x7f32ec19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:57.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32f143b640 1 -- 172.21.15.112:0/1609940324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ec102a60 msgr2=0x7f32ec19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:57.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32f143b640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ec102a60 0x7f32ec19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32f143b640 1 -- 172.21.15.112:0/1609940324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ec1083a0 msgr2=0x7f32ec19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32f143b640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ec1083a0 0x7f32ec19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32f143b640 1 -- 172.21.15.112:0/1609940324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ec1a03d0 con 0x7f32ec108cf0 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32ebfff640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ec1083a0 0x7f32ec19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32f143b640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ec108cf0 0x7f32ec19fca0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005cc0 tx=0x7f32e402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32e9ffb640 1 -- 172.21.15.112:0/1609940324 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4003bf0 con 0x7f32ec108cf0 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32f2e9e640 1 -- 172.21.15.112:0/1609940324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ec1a0660 con 0x7f32ec108cf0 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32e9ffb640 1 -- 172.21.15.112:0/1609940324 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e4003d90 con 0x7f32ec108cf0 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32e9ffb640 1 -- 172.21.15.112:0/1609940324 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e402fbc0 con 0x7f32ec108cf0 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.185+0000 7f32f2e9e640 1 -- 172.21.15.112:0/1609940324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ec1a6e20 con 0x7f32ec108cf0 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.187+0000 7f32f2e9e640 1 -- 172.21.15.112:0/1609940324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ec102e70 con 0x7f32ec108cf0 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.188+0000 7f32e9ffb640 1 -- 172.21.15.112:0/1609940324 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f32e402fd60 con 0x7f32ec108cf0 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.189+0000 7f32e9ffb640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32dc0b65f0 0x7f32dc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:57.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.189+0000 7f32f0c3a640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32dc0b65f0 0x7f32dc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.189+0000 7f32e9ffb640 1 -- 172.21.15.112:0/1609940324 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f32e4108c60 con 0x7f32ec108cf0 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.189+0000 7f32f0c3a640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32dc0b65f0 0x7f32dc0b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f32e0005fd0 tx=0x7f32e000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.192+0000 7f32e9ffb640 1 -- 172.21.15.112:0/1609940324 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f32e40cf2d0 con 0x7f32ec108cf0 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.380+0000 7f32f2e9e640 1 -- 172.21.15.112:0/1609940324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f32ec10bad0 con 0x7f32ec108cf0 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.380+0000 7f32e9ffb640 1 -- 172.21.15.112:0/1609940324 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f32e40d36b0 con 0x7f32ec108cf0 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 -- 172.21.15.112:0/1609940324 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32dc0b65f0 msgr2=0x7f32dc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32dc0b65f0 0x7f32dc0b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f32e0005fd0 tx=0x7f32e000b390 comp rx=0 tx=0).stop 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 -- 172.21.15.112:0/1609940324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ec108cf0 msgr2=0x7f32ec19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ec108cf0 0x7f32ec19fca0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005cc0 tx=0x7f32e402f380 comp rx=0 tx=0).stop 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 -- 172.21.15.112:0/1609940324 shutdown_connections 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32dc0b65f0 0x7f32dc0b8b00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ec108cf0 0x7f32ec19fca0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ec1083a0 0x7f32ec19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 --2- 172.21.15.112:0/1609940324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ec102a60 0x7f32ec19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 -- 172.21.15.112:0/1609940324 >> 172.21.15.112:0/1609940324 conn(0x7f32ec079730 msgr2=0x7f32ec06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:57.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 -- 172.21.15.112:0/1609940324 shutdown_connections 2024-09-21T03:27:57.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:56.383+0000 7f32f2e9e640 1 -- 172.21.15.112:0/1609940324 wait complete. 2024-09-21T03:27:57.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.339+0000 7f65ff32d640 1 -- 172.21.15.112:0/3231970624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f65f819b790 con 0x7f65f8101fc0 2024-09-21T03:27:57.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.339+0000 7f65ee7fc640 1 -- 172.21.15.112:0/3231970624 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f65f819b790 con 0x7f65f8101fc0 2024-09-21T03:27:57.341 INFO:teuthology.orchestra.run.smithi112.stdout:176093659184 2024-09-21T03:27:57.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 -- 172.21.15.112:0/3231970624 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f65e40b65f0 msgr2=0x7f65e40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f65e40b65f0 0x7f65e40b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009230 tx=0x7f65e8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 -- 172.21.15.112:0/3231970624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f8101fc0 msgr2=0x7f65f819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f8101fc0 0x7f65f819a660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f65f400b4a0 tx=0x7f65f400b960 comp rx=0 tx=0).stop 2024-09-21T03:27:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 -- 172.21.15.112:0/3231970624 shutdown_connections 2024-09-21T03:27:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f65e40b65f0 0x7f65e40b8b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f8109e50 0x7f65f819fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65f8109500 0x7f65f819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 --2- 172.21.15.112:0/3231970624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65f8101fc0 0x7f65f819a660 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 -- 172.21.15.112:0/3231970624 >> 172.21.15.112:0/3231970624 conn(0x7f65f8079f90 msgr2=0x7f65f81075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 -- 172.21.15.112:0/3231970624 shutdown_connections 2024-09-21T03:27:57.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:57.342+0000 7f65ff32d640 1 -- 172.21.15.112:0/3231970624 wait complete. 2024-09-21T03:27:57.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:57 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/133811122' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:27:57.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:57 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1031110405' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:27:57.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:57 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3231970624' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:27:57.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:57.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:57 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/133811122' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:27:57.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:57 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1031110405' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:27:57.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:57 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3231970624' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:27:57.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:57.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:57 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/133811122' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T03:27:57.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:57 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1031110405' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T03:27:57.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:57 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3231970624' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T03:27:57.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:27:58.285 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542216 got 60129542218 for osd.1 2024-09-21T03:27:58.285 DEBUG:teuthology.parallel:result is None 2024-09-21T03:27:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:58 smithi112 ceph-mon[31442]: pgmap v287: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:27:58.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:58 smithi203 ceph-mon[34723]: pgmap v287: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:27:58.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:58 smithi187 ceph-mon[34603]: pgmap v287: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:27:59.090 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705741 got 38654705742 for osd.0 2024-09-21T03:27:59.091 DEBUG:teuthology.parallel:result is None 2024-09-21T03:27:59.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.280+0000 7f69b3ad6640 1 -- 172.21.15.112:0/660195519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69ac108cf0 msgr2=0x7f69ac10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.280+0000 7f69b3ad6640 1 --2- 172.21.15.112:0/660195519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69ac108cf0 0x7f69ac10f520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009ca0 tx=0x7f69a402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.280+0000 7f69b3ad6640 1 -- 172.21.15.112:0/660195519 shutdown_connections 2024-09-21T03:27:59.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.280+0000 7f69b3ad6640 1 --2- 172.21.15.112:0/660195519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69ac108cf0 0x7f69ac10f520 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.280+0000 7f69b3ad6640 1 --2- 172.21.15.112:0/660195519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69ac1083a0 0x7f69ac1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.280+0000 7f69b3ad6640 1 --2- 172.21.15.112:0/660195519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ac102a60 0x7f69ac102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.280+0000 7f69b3ad6640 1 -- 172.21.15.112:0/660195519 >> 172.21.15.112:0/660195519 conn(0x7f69ac079730 msgr2=0x7f69ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:59.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.280+0000 7f69b3ad6640 1 -- 172.21.15.112:0/660195519 shutdown_connections 2024-09-21T03:27:59.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.280+0000 7f69b3ad6640 1 -- 172.21.15.112:0/660195519 wait complete. 2024-09-21T03:27:59.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.280+0000 7f69b3ad6640 1 Processor -- start 2024-09-21T03:27:59.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b3ad6640 1 -- start start 2024-09-21T03:27:59.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b3ad6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ac102a60 0x7f69ac19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b3ad6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69ac1083a0 0x7f69ac19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b3ad6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69ac108cf0 0x7f69ac19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b3ad6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac112960 con 0x7f69ac1083a0 2024-09-21T03:27:59.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b3ad6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac1127e0 con 0x7f69ac108cf0 2024-09-21T03:27:59.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b3ad6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac112ae0 con 0x7f69ac102a60 2024-09-21T03:27:59.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b1071640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69ac1083a0 0x7f69ac19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b1071640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69ac1083a0 0x7f69ac19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52372/0 (socket says 172.21.15.112:52372) 2024-09-21T03:27:59.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b1071640 1 -- 172.21.15.112:0/2769521536 learned_addr learned my addr 172.21.15.112:0/2769521536 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:59.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b1872640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ac102a60 0x7f69ac19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b2073640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69ac108cf0 0x7f69ac19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b1071640 1 -- 172.21.15.112:0/2769521536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ac102a60 msgr2=0x7f69ac19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b1071640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ac102a60 0x7f69ac19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b1071640 1 -- 172.21.15.112:0/2769521536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69ac108cf0 msgr2=0x7f69ac19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b1071640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69ac108cf0 0x7f69ac19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b1071640 1 -- 172.21.15.112:0/2769521536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ac1a0350 con 0x7f69ac1083a0 2024-09-21T03:27:59.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b2073640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69ac108cf0 0x7f69ac19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:27:59.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.281+0000 7f69b1071640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69ac1083a0 0x7f69ac19a980 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f699c00c970 tx=0x7f699c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:59.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.282+0000 7f69a2ffd640 1 -- 172.21.15.112:0/2769521536 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c007bb0 con 0x7f69ac1083a0 2024-09-21T03:27:59.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.282+0000 7f69a2ffd640 1 -- 172.21.15.112:0/2769521536 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f699c007d50 con 0x7f69ac1083a0 2024-09-21T03:27:59.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.282+0000 7f69b3ad6640 1 -- 172.21.15.112:0/2769521536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ac1a0640 con 0x7f69ac1083a0 2024-09-21T03:27:59.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.282+0000 7f69a2ffd640 1 -- 172.21.15.112:0/2769521536 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c00f450 con 0x7f69ac1083a0 2024-09-21T03:27:59.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.282+0000 7f69b3ad6640 1 -- 172.21.15.112:0/2769521536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ac1a6da0 con 0x7f69ac1083a0 2024-09-21T03:27:59.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.283+0000 7f69b3ad6640 1 -- 172.21.15.112:0/2769521536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ac102e70 con 0x7f69ac1083a0 2024-09-21T03:27:59.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.285+0000 7f69a2ffd640 1 -- 172.21.15.112:0/2769521536 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f699c004f20 con 0x7f69ac1083a0 2024-09-21T03:27:59.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.286+0000 7f69a2ffd640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69980b6670 0x7f69980b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.286+0000 7f69b1872640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69980b6670 0x7f69980b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.286+0000 7f69a2ffd640 1 -- 172.21.15.112:0/2769521536 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f699c0e7560 con 0x7f69ac1083a0 2024-09-21T03:27:59.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.287+0000 7f69b1872640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69980b6670 0x7f69980b8b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005fd0 tx=0x7f69a800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:59.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.289+0000 7f69a2ffd640 1 -- 172.21.15.112:0/2769521536 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f699c014030 con 0x7f69ac1083a0 2024-09-21T03:27:59.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.480+0000 7f69b3ad6640 1 -- 172.21.15.112:0/2769521536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f69ac19b510 con 0x7f69ac1083a0 2024-09-21T03:27:59.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.480+0000 7f69a2ffd640 1 -- 172.21.15.112:0/2769521536 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f69ac19b510 con 0x7f69ac1083a0 2024-09-21T03:27:59.483 INFO:teuthology.orchestra.run.smithi112.stdout:201863462957 2024-09-21T03:27:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 -- 172.21.15.112:0/2769521536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69980b6670 msgr2=0x7f69980b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69980b6670 0x7f69980b8b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005fd0 tx=0x7f69a800b390 comp rx=0 tx=0).stop 2024-09-21T03:27:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 -- 172.21.15.112:0/2769521536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69ac1083a0 msgr2=0x7f69ac19a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69ac1083a0 0x7f69ac19a980 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f699c00c970 tx=0x7f699c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:27:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 -- 172.21.15.112:0/2769521536 shutdown_connections 2024-09-21T03:27:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69980b6670 0x7f69980b8b80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69ac108cf0 0x7f69ac19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69ac1083a0 0x7f69ac19a980 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 --2- 172.21.15.112:0/2769521536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69ac102a60 0x7f69ac19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 -- 172.21.15.112:0/2769521536 >> 172.21.15.112:0/2769521536 conn(0x7f69ac079730 msgr2=0x7f69ac06b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:59.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 -- 172.21.15.112:0/2769521536 shutdown_connections 2024-09-21T03:27:59.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.483+0000 7f69b3ad6640 1 -- 172.21.15.112:0/2769521536 wait complete. 2024-09-21T03:27:59.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.490+0000 7f87f57d4640 1 -- 172.21.15.112:0/3185877299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f0102ae0 msgr2=0x7f87f0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.490+0000 7f87f57d4640 1 --2- 172.21.15.112:0/3185877299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f0102ae0 0x7f87f0102ef0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009900 tx=0x7f87e002a410 comp rx=0 tx=0).stop 2024-09-21T03:27:59.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.490+0000 7f87f57d4640 1 -- 172.21.15.112:0/3185877299 shutdown_connections 2024-09-21T03:27:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.490+0000 7f87f57d4640 1 --2- 172.21.15.112:0/3185877299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87f0108d20 0x7f87f010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.490+0000 7f87f57d4640 1 --2- 172.21.15.112:0/3185877299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87f01083d0 0x7f87f01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.490+0000 7f87f57d4640 1 --2- 172.21.15.112:0/3185877299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f0102ae0 0x7f87f0102ef0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.490+0000 7f87f57d4640 1 -- 172.21.15.112:0/3185877299 >> 172.21.15.112:0/3185877299 conn(0x7f87f0079790 msgr2=0x7f87f0079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.490+0000 7f87f57d4640 1 -- 172.21.15.112:0/3185877299 shutdown_connections 2024-09-21T03:27:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.490+0000 7f87f57d4640 1 -- 172.21.15.112:0/3185877299 wait complete. 2024-09-21T03:27:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87f57d4640 1 Processor -- start 2024-09-21T03:27:59.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87f57d4640 1 -- start start 2024-09-21T03:27:59.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87f57d4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87f0102ae0 0x7f87f019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87f57d4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87f01083d0 0x7f87f019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87f57d4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f0108d20 0x7f87f019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87f57d4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01129e0 con 0x7f87f0108d20 2024-09-21T03:27:59.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87f57d4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f87f0112860 con 0x7f87f0102ae0 2024-09-21T03:27:59.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87f57d4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f87f0112b60 con 0x7f87f01083d0 2024-09-21T03:27:59.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f0108d20 0x7f87f019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f0108d20 0x7f87f019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52388/0 (socket says 172.21.15.112:52388) 2024-09-21T03:27:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87ef7fe640 1 -- 172.21.15.112:0/1541724139 learned_addr learned my addr 172.21.15.112:0/1541724139 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87eeffd640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87f0102ae0 0x7f87f019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.491+0000 7f87ee7fc640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87f01083d0 0x7f87f019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.492+0000 7f87ef7fe640 1 -- 172.21.15.112:0/1541724139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87f01083d0 msgr2=0x7f87f019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.492+0000 7f87ef7fe640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87f01083d0 0x7f87f019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.492+0000 7f87ef7fe640 1 -- 172.21.15.112:0/1541724139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87f0102ae0 msgr2=0x7f87f019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.492+0000 7f87ef7fe640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87f0102ae0 0x7f87f019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.492+0000 7f87ef7fe640 1 -- 172.21.15.112:0/1541724139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f01a6600 con 0x7f87f0108d20 2024-09-21T03:27:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.492+0000 7f87ef7fe640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f0108d20 0x7f87f019bdf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f87e800eb10 tx=0x7f87e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.492+0000 7f87d7fff640 1 -- 172.21.15.112:0/1541724139 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8004b20 con 0x7f87f0108d20 2024-09-21T03:27:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.492+0000 7f87f57d4640 1 -- 172.21.15.112:0/1541724139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f01a68f0 con 0x7f87f0108d20 2024-09-21T03:27:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.492+0000 7f87d7fff640 1 -- 172.21.15.112:0/1541724139 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e8004cc0 con 0x7f87f0108d20 2024-09-21T03:27:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.492+0000 7f87d7fff640 1 -- 172.21.15.112:0/1541724139 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8010690 con 0x7f87f0108d20 2024-09-21T03:27:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.492+0000 7f87f57d4640 1 -- 172.21.15.112:0/1541724139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f01a6e30 con 0x7f87f0108d20 2024-09-21T03:27:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.493+0000 7f87f57d4640 1 -- 172.21.15.112:0/1541724139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f0102ef0 con 0x7f87f0108d20 2024-09-21T03:27:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.495+0000 7f87d7fff640 1 -- 172.21.15.112:0/1541724139 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f87e8005020 con 0x7f87f0108d20 2024-09-21T03:27:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.496+0000 7f87d7fff640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87d80b6620 0x7f87d80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.496+0000 7f87eeffd640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87d80b6620 0x7f87d80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.496+0000 7f87d7fff640 1 -- 172.21.15.112:0/1541724139 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f87e80e7c60 con 0x7f87f0108d20 2024-09-21T03:27:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.496+0000 7f87eeffd640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87d80b6620 0x7f87d80b8b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87e0000c00 tx=0x7f87e0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.499+0000 7f87d7fff640 1 -- 172.21.15.112:0/1541724139 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87e8056710 con 0x7f87f0108d20 2024-09-21T03:27:59.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.682+0000 7f87f57d4640 1 -- 172.21.15.112:0/1541724139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f87f006bc30 con 0x7f87f0108d20 2024-09-21T03:27:59.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.682+0000 7f87d7fff640 1 -- 172.21.15.112:0/1541724139 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f87f006bc30 con 0x7f87f0108d20 2024-09-21T03:27:59.684 INFO:teuthology.orchestra.run.smithi112.stdout:292057776153 2024-09-21T03:27:59.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 -- 172.21.15.112:0/1541724139 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87d80b6620 msgr2=0x7f87d80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87d80b6620 0x7f87d80b8b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87e0000c00 tx=0x7f87e0031400 comp rx=0 tx=0).stop 2024-09-21T03:27:59.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 -- 172.21.15.112:0/1541724139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f0108d20 msgr2=0x7f87f019bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f0108d20 0x7f87f019bdf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f87e800eb10 tx=0x7f87e800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 -- 172.21.15.112:0/1541724139 shutdown_connections 2024-09-21T03:27:59.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87d80b6620 0x7f87d80b8b30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f0108d20 0x7f87f019bdf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87f01083d0 0x7f87f019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 --2- 172.21.15.112:0/1541724139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87f0102ae0 0x7f87f019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 -- 172.21.15.112:0/1541724139 >> 172.21.15.112:0/1541724139 conn(0x7f87f0079790 msgr2=0x7f87f0106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:59.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 -- 172.21.15.112:0/1541724139 shutdown_connections 2024-09-21T03:27:59.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.685+0000 7f87f57d4640 1 -- 172.21.15.112:0/1541724139 wait complete. 2024-09-21T03:27:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.931+0000 7f04c801b640 1 -- 172.21.15.112:0/4021721284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c010ddf0 msgr2=0x7f04c0110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.931+0000 7f04c801b640 1 --2- 172.21.15.112:0/4021721284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c010ddf0 0x7f04c0110300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009ca0 tx=0x7f04b802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.931+0000 7f04c801b640 1 -- 172.21.15.112:0/4021721284 shutdown_connections 2024-09-21T03:27:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.931+0000 7f04c801b640 1 --2- 172.21.15.112:0/4021721284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c010ddf0 0x7f04c0110300 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.931+0000 7f04c801b640 1 --2- 172.21.15.112:0/4021721284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c0105270 0x7f04c010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.931+0000 7f04c801b640 1 --2- 172.21.15.112:0/4021721284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c0104840 0x7f04c0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.931+0000 7f04c801b640 1 -- 172.21.15.112:0/4021721284 >> 172.21.15.112:0/4021721284 conn(0x7f04c0079730 msgr2=0x7f04c0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:27:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.932+0000 7f04c801b640 1 -- 172.21.15.112:0/4021721284 shutdown_connections 2024-09-21T03:27:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.932+0000 7f04c801b640 1 -- 172.21.15.112:0/4021721284 wait complete. 2024-09-21T03:27:59.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.932+0000 7f04c801b640 1 Processor -- start 2024-09-21T03:27:59.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.932+0000 7f04c801b640 1 -- start start 2024-09-21T03:27:59.935 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:59 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:27:59.935 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:59 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:27:59.935 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:59 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:27:59.935 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:59 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:27:59.935 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:59 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2769521536' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:27:59.935 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:27:59 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1541724139' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:27:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c801b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c0104840 0x7f04c0102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c801b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c0105270 0x7f04c0102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c801b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c010ddf0 0x7f04c01031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c801b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0113a90 con 0x7f04c010ddf0 2024-09-21T03:27:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c801b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0113910 con 0x7f04c0105270 2024-09-21T03:27:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c801b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0113c10 con 0x7f04c0104840 2024-09-21T03:27:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c65b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c010ddf0 0x7f04c01031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c65b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c010ddf0 0x7f04c01031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52398/0 (socket says 172.21.15.112:52398) 2024-09-21T03:27:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c65b8640 1 -- 172.21.15.112:0/272501662 learned_addr learned my addr 172.21.15.112:0/272501662 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:27:59.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c5db7640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c0104840 0x7f04c0102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c55b6640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c0105270 0x7f04c0102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c65b8640 1 -- 172.21.15.112:0/272501662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c0104840 msgr2=0x7f04c0102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c65b8640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c0104840 0x7f04c0102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c65b8640 1 -- 172.21.15.112:0/272501662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c0105270 msgr2=0x7f04c0102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c65b8640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c0105270 0x7f04c0102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c65b8640 1 -- 172.21.15.112:0/272501662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c01a8720 con 0x7f04c010ddf0 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.933+0000 7f04c65b8640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c010ddf0 0x7f04c01031d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04b8005cc0 tx=0x7f04b802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.934+0000 7f04b6ffd640 1 -- 172.21.15.112:0/272501662 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8003cf0 con 0x7f04c010ddf0 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.934+0000 7f04b6ffd640 1 -- 172.21.15.112:0/272501662 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b80044f0 con 0x7f04c010ddf0 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.934+0000 7f04c801b640 1 -- 172.21.15.112:0/272501662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c01a89b0 con 0x7f04c010ddf0 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.934+0000 7f04b6ffd640 1 -- 172.21.15.112:0/272501662 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b802fd10 con 0x7f04c010ddf0 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.934+0000 7f04c801b640 1 -- 172.21.15.112:0/272501662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c01a8f40 con 0x7f04c010ddf0 2024-09-21T03:27:59.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.935+0000 7f04c801b640 1 -- 172.21.15.112:0/272501662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c0104c50 con 0x7f04c010ddf0 2024-09-21T03:27:59.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.936+0000 7f04b6ffd640 1 -- 172.21.15.112:0/272501662 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f04b8004040 con 0x7f04c010ddf0 2024-09-21T03:27:59.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.937+0000 7f04b6ffd640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f04ac0b6670 0x7f04ac0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:27:59.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.937+0000 7f04c5db7640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f04ac0b6670 0x7f04ac0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:27:59.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.937+0000 7f04b6ffd640 1 -- 172.21.15.112:0/272501662 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f04b8108d50 con 0x7f04c010ddf0 2024-09-21T03:27:59.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.937+0000 7f04c5db7640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f04ac0b6670 0x7f04ac0b8b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04c01041a0 tx=0x7f04bc00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:27:59.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:59.940+0000 7f04b6ffd640 1 -- 172.21.15.112:0/272501662 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f04b80cf3c0 con 0x7f04c010ddf0 2024-09-21T03:28:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.980+0000 7fd969535640 1 -- 172.21.15.112:0/1080970372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd964101da0 msgr2=0x7fd9641021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.980+0000 7fd969535640 1 --2- 172.21.15.112:0/1080970372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd964101da0 0x7fd9641021b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd958009900 tx=0x7fd95802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.980+0000 7fd969535640 1 -- 172.21.15.112:0/1080970372 shutdown_connections 2024-09-21T03:28:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.980+0000 7fd969535640 1 --2- 172.21.15.112:0/1080970372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd964102890 0x7fd964112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.980+0000 7fd969535640 1 --2- 172.21.15.112:0/1080970372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd964101da0 0x7fd9641021b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.980+0000 7fd969535640 1 --2- 172.21.15.112:0/1080970372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964104690 0x7fd964104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.980+0000 7fd969535640 1 -- 172.21.15.112:0/1080970372 >> 172.21.15.112:0/1080970372 conn(0x7fd964077c60 msgr2=0x7fd9640780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.980+0000 7fd969535640 1 -- 172.21.15.112:0/1080970372 shutdown_connections 2024-09-21T03:28:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.980+0000 7fd969535640 1 -- 172.21.15.112:0/1080970372 wait complete. 2024-09-21T03:28:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.981+0000 7fd969535640 1 Processor -- start 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.981+0000 7fd969535640 1 -- start start 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.981+0000 7fd969535640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd964101da0 0x7fd96419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.981+0000 7fd969535640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd964102890 0x7fd96419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.981+0000 7fd969535640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964104690 0x7fd9641a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.981+0000 7fd969535640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd964115c80 con 0x7fd964102890 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.981+0000 7fd969535640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd964115b00 con 0x7fd964101da0 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.981+0000 7fd969535640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd964115e00 con 0x7fd964104690 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd9627fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd964102890 0x7fd96419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd9627fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd964102890 0x7fd96419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40816/0 (socket says 172.21.15.112:40816) 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd9627fc640 1 -- 172.21.15.112:0/1374111258 learned_addr learned my addr 172.21.15.112:0/1374111258 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd962ffd640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd964101da0 0x7fd96419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd9637fe640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964104690 0x7fd9641a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd9627fc640 1 -- 172.21.15.112:0/1374111258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964104690 msgr2=0x7fd9641a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:00.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd9627fc640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964104690 0x7fd9641a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.047 INFO:teuthology.orchestra.run.smithi112.stdout:128849018939 2024-09-21T03:28:00.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd9627fc640 1 -- 172.21.15.112:0/1374111258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd964101da0 msgr2=0x7fd96419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:00.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd9627fc640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd964101da0 0x7fd96419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd9627fc640 1 -- 172.21.15.112:0/1374111258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9641a2600 con 0x7fd964102890 2024-09-21T03:28:00.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd9627fc640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd964102890 0x7fd96419cc30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd958006010 tx=0x7fd958004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:00.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd94bfff640 1 -- 172.21.15.112:0/1374111258 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95803f030 con 0x7fd964102890 2024-09-21T03:28:00.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd969535640 1 -- 172.21.15.112:0/1374111258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9641a2890 con 0x7fd964102890 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd94bfff640 1 -- 172.21.15.112:0/1374111258 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd958003aa0 con 0x7fd964102890 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd94bfff640 1 -- 172.21.15.112:0/1374111258 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958035070 con 0x7fd964102890 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.982+0000 7fd969535640 1 -- 172.21.15.112:0/1374111258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9641a9050 con 0x7fd964102890 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.984+0000 7fd969535640 1 -- 172.21.15.112:0/1374111258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9641021b0 con 0x7fd964102890 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.985+0000 7fd94bfff640 1 -- 172.21.15.112:0/1374111258 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fd958042030 con 0x7fd964102890 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.986+0000 7fd94bfff640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd94c0b65f0 0x7fd94c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.986+0000 7fd962ffd640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd94c0b65f0 0x7fd94c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.986+0000 7fd94bfff640 1 -- 172.21.15.112:0/1374111258 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd958107ea0 con 0x7fd964102890 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.986+0000 7fd962ffd640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd94c0b65f0 0x7fd94c0b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd954005fd0 tx=0x7fd95400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:52.989+0000 7fd94bfff640 1 -- 172.21.15.112:0/1374111258 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9580ce510 con 0x7fd964102890 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.175+0000 7fd969535640 1 -- 172.21.15.112:0/1374111258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd96410dc00 con 0x7fd964102890 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.176+0000 7fd94bfff640 1 -- 172.21.15.112:0/1374111258 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd9580efe20 con 0x7fd964102890 2024-09-21T03:28:00.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 -- 172.21.15.112:0/1374111258 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd94c0b65f0 msgr2=0x7fd94c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd94c0b65f0 0x7fd94c0b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd954005fd0 tx=0x7fd95400b390 comp rx=0 tx=0).stop 2024-09-21T03:28:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 -- 172.21.15.112:0/1374111258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd964102890 msgr2=0x7fd96419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd964102890 0x7fd96419cc30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd958006010 tx=0x7fd958004060 comp rx=0 tx=0).stop 2024-09-21T03:28:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 -- 172.21.15.112:0/1374111258 shutdown_connections 2024-09-21T03:28:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd94c0b65f0 0x7fd94c0b8b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd964104690 0x7fd9641a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd964102890 0x7fd96419cc30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 --2- 172.21.15.112:0/1374111258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd964101da0 0x7fd96419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 -- 172.21.15.112:0/1374111258 >> 172.21.15.112:0/1374111258 conn(0x7fd964077c60 msgr2=0x7fd9640ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 -- 172.21.15.112:0/1374111258 shutdown_connections 2024-09-21T03:28:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:53.178+0000 7fd969535640 1 -- 172.21.15.112:0/1374111258 wait complete. 2024-09-21T03:28:00.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.127+0000 7f04c801b640 1 -- 172.21.15.112:0/272501662 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f04c01a9310 con 0x7f04c010ddf0 2024-09-21T03:28:00.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.127+0000 7f04b6ffd640 1 -- 172.21.15.112:0/272501662 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f04b80d37a0 con 0x7f04c010ddf0 2024-09-21T03:28:00.129 INFO:teuthology.orchestra.run.smithi112.stdout:249108103202 2024-09-21T03:28:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 -- 172.21.15.112:0/272501662 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f04ac0b6670 msgr2=0x7f04ac0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f04ac0b6670 0x7f04ac0b8b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04c01041a0 tx=0x7f04bc00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 -- 172.21.15.112:0/272501662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c010ddf0 msgr2=0x7f04c01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c010ddf0 0x7f04c01031d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04b8005cc0 tx=0x7f04b802f470 comp rx=0 tx=0).stop 2024-09-21T03:28:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 -- 172.21.15.112:0/272501662 shutdown_connections 2024-09-21T03:28:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f04ac0b6670 0x7f04ac0b8b80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04c010ddf0 0x7f04c01031d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c0105270 0x7f04c0102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 --2- 172.21.15.112:0/272501662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c0104840 0x7f04c0102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 -- 172.21.15.112:0/272501662 >> 172.21.15.112:0/272501662 conn(0x7f04c0079730 msgr2=0x7f04c00ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 -- 172.21.15.112:0/272501662 shutdown_connections 2024-09-21T03:28:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:00.129+0000 7f04c801b640 1 -- 172.21.15.112:0/272501662 wait complete. 2024-09-21T03:28:00.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:59 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:00.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:59 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:00.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:59 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:00.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:59 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:00.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:59 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2769521536' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:28:00.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:27:59 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1541724139' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:28:00.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:59 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:00.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:59 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:00.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:59 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:00.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:59 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:00.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:59 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2769521536' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T03:28:00.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:27:59 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1541724139' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T03:28:01.160 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:00 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/272501662' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:28:01.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:00 smithi203 ceph-mon[34723]: pgmap v288: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:01.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:00 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/272501662' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:28:01.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:00 smithi187 ceph-mon[34603]: pgmap v288: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:01.195 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:00 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/272501662' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T03:28:01.195 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:00 smithi112 ceph-mon[31442]: pgmap v288: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:01.223 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103200 got 249108103202 for osd.9 2024-09-21T03:28:01.224 DEBUG:teuthology.parallel:result is None 2024-09-21T03:28:01.326 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018937 got 128849018939 for osd.4 2024-09-21T03:28:01.326 DEBUG:teuthology.parallel:result is None 2024-09-21T03:28:01.338 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411397 for osd.2 2024-09-21T03:28:01.339 DEBUG:teuthology.parallel:result is None 2024-09-21T03:28:01.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.265+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/3263725471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d68108cf0 msgr2=0x7f8d6810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:01.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.265+0000 7f8d6d6c2640 1 --2- 172.21.15.112:0/3263725471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d68108cf0 0x7f8d6810f520 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009900 tx=0x7f8d6002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.265+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/3263725471 shutdown_connections 2024-09-21T03:28:01.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.265+0000 7f8d6d6c2640 1 --2- 172.21.15.112:0/3263725471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d68108cf0 0x7f8d6810f520 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.265+0000 7f8d6d6c2640 1 --2- 172.21.15.112:0/3263725471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d681083a0 0x7f8d681087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.265+0000 7f8d6d6c2640 1 --2- 172.21.15.112:0/3263725471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d68102a60 0x7f8d68102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.265+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/3263725471 >> 172.21.15.112:0/3263725471 conn(0x7f8d68079730 msgr2=0x7f8d68079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:01.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.265+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/3263725471 shutdown_connections 2024-09-21T03:28:01.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.265+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/3263725471 wait complete. 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d6d6c2640 1 Processor -- start 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d6d6c2640 1 -- start start 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d6d6c2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68102a60 0x7f8d6819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d6d6c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d681083a0 0x7f8d6819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d6d6c2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d68108cf0 0x7f8d6819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d6d6c2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d681129c0 con 0x7f8d681083a0 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d6d6c2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8d68112840 con 0x7f8d68108cf0 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d6d6c2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8d68112b40 con 0x7f8d68102a60 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d667fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d681083a0 0x7f8d6819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d667fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d681083a0 0x7f8d6819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52282/0 (socket says 172.21.15.112:52282) 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d667fc640 1 -- 172.21.15.112:0/1727168720 learned_addr learned my addr 172.21.15.112:0/1727168720 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d677fe640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d68108cf0 0x7f8d6819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:01.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.266+0000 7f8d66ffd640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68102a60 0x7f8d6819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.267+0000 7f8d667fc640 1 -- 172.21.15.112:0/1727168720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68102a60 msgr2=0x7f8d6819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.267+0000 7f8d667fc640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68102a60 0x7f8d6819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.267+0000 7f8d667fc640 1 -- 172.21.15.112:0/1727168720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d68108cf0 msgr2=0x7f8d6819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.267+0000 7f8d667fc640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d68108cf0 0x7f8d6819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.267+0000 7f8d667fc640 1 -- 172.21.15.112:0/1727168720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d681a0370 con 0x7f8d681083a0 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.267+0000 7f8d667fc640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d681083a0 0x7f8d6819aa60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000c970 tx=0x7f8d5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.267+0000 7f8d5ffff640 1 -- 172.21.15.112:0/1727168720 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50014070 con 0x7f8d681083a0 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.267+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/1727168720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d681a0660 con 0x7f8d681083a0 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.267+0000 7f8d5ffff640 1 -- 172.21.15.112:0/1727168720 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d50004440 con 0x7f8d681083a0 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.267+0000 7f8d5ffff640 1 -- 172.21.15.112:0/1727168720 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50003e50 con 0x7f8d681083a0 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.267+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/1727168720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d681a6e80 con 0x7f8d681083a0 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.268+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/1727168720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d68102e70 con 0x7f8d681083a0 2024-09-21T03:28:01.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.269+0000 7f8d5ffff640 1 -- 172.21.15.112:0/1727168720 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f8d50017070 con 0x7f8d681083a0 2024-09-21T03:28:01.524 INFO:teuthology.orchestra.run.smithi112.stdout:107374182463 2024-09-21T03:28:01.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.270+0000 7f8d5ffff640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8d540b67c0 0x7f8d540b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:01.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.270+0000 7f8d66ffd640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8d540b67c0 0x7f8d540b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:01.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.270+0000 7f8d5ffff640 1 -- 172.21.15.112:0/1727168720 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f8d500e7020 con 0x7f8d681083a0 2024-09-21T03:28:01.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.271+0000 7f8d66ffd640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8d540b67c0 0x7f8d540b8cd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800a400 tx=0x7f8d5800d040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.274+0000 7f8d5ffff640 1 -- 172.21.15.112:0/1727168720 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8d5001e030 con 0x7f8d681083a0 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.465+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/1727168720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8d681a70e0 con 0x7f8d681083a0 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.465+0000 7f8d5ffff640 1 -- 172.21.15.112:0/1727168720 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8d500ad690 con 0x7f8d681083a0 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/1727168720 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8d540b67c0 msgr2=0x7f8d540b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8d540b67c0 0x7f8d540b8cd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800a400 tx=0x7f8d5800d040 comp rx=0 tx=0).stop 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/1727168720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d681083a0 msgr2=0x7f8d6819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d681083a0 0x7f8d6819aa60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000c970 tx=0x7f8d5000ce30 comp rx=0 tx=0).stop 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/1727168720 shutdown_connections 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8d540b67c0 0x7f8d540b8cd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d68108cf0 0x7f8d6819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d681083a0 0x7f8d6819aa60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 --2- 172.21.15.112:0/1727168720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68102a60 0x7f8d6819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/1727168720 >> 172.21.15.112:0/1727168720 conn(0x7f8d68079730 msgr2=0x7f8d68101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:01.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/1727168720 shutdown_connections 2024-09-21T03:28:01.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:27:55.468+0000 7f8d6d6c2640 1 -- 172.21.15.112:0/1727168720 wait complete. 2024-09-21T03:28:02.094 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182463 for osd.3 2024-09-21T03:28:02.094 DEBUG:teuthology.parallel:result is None 2024-09-21T03:28:02.733 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776150 got 292057776153 for osd.11 2024-09-21T03:28:02.733 DEBUG:teuthology.parallel:result is None 2024-09-21T03:28:03.145 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939673 got 270582939676 for osd.10 2024-09-21T03:28:03.145 DEBUG:teuthology.parallel:result is None 2024-09-21T03:28:03.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855411 got 150323855414 for osd.5 2024-09-21T03:28:03.565 DEBUG:teuthology.parallel:result is None 2024-09-21T03:28:03.655 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:03 smithi112 ceph-mon[31442]: pgmap v289: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:03.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:03 smithi203 ceph-mon[34723]: pgmap v289: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:03 smithi187 ceph-mon[34603]: pgmap v289: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:03.956 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462953 got 201863462957 for osd.7 2024-09-21T03:28:03.956 DEBUG:teuthology.parallel:result is None 2024-09-21T03:28:04.026 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:04.448 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659181 got 176093659184 for osd.6 2024-09-21T03:28:04.448 DEBUG:teuthology.parallel:result is None 2024-09-21T03:28:04.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.907+0000 7fe9d5545640 1 -- 172.21.15.112:0/3338538394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9d0108420 msgr2=0x7fe9d0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.907+0000 7fe9d5545640 1 --2- 172.21.15.112:0/3338538394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9d0108420 0x7fe9d0108830 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009900 tx=0x7fe9c402a410 comp rx=0 tx=0).stop 2024-09-21T03:28:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.907+0000 7fe9d5545640 1 -- 172.21.15.112:0/3338538394 shutdown_connections 2024-09-21T03:28:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.907+0000 7fe9d5545640 1 --2- 172.21.15.112:0/3338538394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9d0108d70 0x7fe9d010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.907+0000 7fe9d5545640 1 --2- 172.21.15.112:0/3338538394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9d0108420 0x7fe9d0108830 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.907+0000 7fe9d5545640 1 --2- 172.21.15.112:0/3338538394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d0102ae0 0x7fe9d0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.907+0000 7fe9d5545640 1 -- 172.21.15.112:0/3338538394 >> 172.21.15.112:0/3338538394 conn(0x7fe9d0079770 msgr2=0x7fe9d0079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.907+0000 7fe9d5545640 1 -- 172.21.15.112:0/3338538394 shutdown_connections 2024-09-21T03:28:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.908+0000 7fe9d5545640 1 -- 172.21.15.112:0/3338538394 wait complete. 2024-09-21T03:28:04.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.908+0000 7fe9d5545640 1 Processor -- start 2024-09-21T03:28:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.908+0000 7fe9d5545640 1 -- start start 2024-09-21T03:28:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9d5545640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d0102ae0 0x7fe9d019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9d5545640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9d0108420 0x7fe9d019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9d5545640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9d0108d70 0x7fe9d01a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9d5545640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d0112a50 con 0x7fe9d0108d70 2024-09-21T03:28:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9d5545640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01128d0 con 0x7fe9d0102ae0 2024-09-21T03:28:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9d5545640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d0112bd0 con 0x7fe9d0108420 2024-09-21T03:28:04.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9cf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9d0108d70 0x7fe9d01a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9cf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9d0108d70 0x7fe9d01a1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52428/0 (socket says 172.21.15.112:52428) 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9ce7fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9d0108420 0x7fe9d019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9cf7fe640 1 -- 172.21.15.112:0/1175103582 learned_addr learned my addr 172.21.15.112:0/1175103582 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9ceffd640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d0102ae0 0x7fe9d019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9cf7fe640 1 -- 172.21.15.112:0/1175103582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9d0108420 msgr2=0x7fe9d019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9cf7fe640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9d0108420 0x7fe9d019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9cf7fe640 1 -- 172.21.15.112:0/1175103582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d0102ae0 msgr2=0x7fe9d019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9cf7fe640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d0102ae0 0x7fe9d019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9cf7fe640 1 -- 172.21.15.112:0/1175103582 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d01a26f0 con 0x7fe9d0108d70 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9ceffd640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d0102ae0 0x7fe9d019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9cf7fe640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9d0108d70 0x7fe9d01a1fc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800b970 tx=0x7fe9c800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:04.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9b7fff640 1 -- 172.21.15.112:0/1175103582 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8019070 con 0x7fe9d0108d70 2024-09-21T03:28:04.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9b7fff640 1 -- 172.21.15.112:0/1175103582 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9c8014440 con 0x7fe9d0108d70 2024-09-21T03:28:04.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9d5545640 1 -- 172.21.15.112:0/1175103582 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d01a29e0 con 0x7fe9d0108d70 2024-09-21T03:28:04.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.909+0000 7fe9b7fff640 1 -- 172.21.15.112:0/1175103582 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80075d0 con 0x7fe9d0108d70 2024-09-21T03:28:04.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.910+0000 7fe9d5545640 1 -- 172.21.15.112:0/1175103582 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d01a9140 con 0x7fe9d0108d70 2024-09-21T03:28:04.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.910+0000 7fe9d5545640 1 -- 172.21.15.112:0/1175103582 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d0103d60 con 0x7fe9d0108d70 2024-09-21T03:28:04.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.912+0000 7fe9b7fff640 1 -- 172.21.15.112:0/1175103582 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fe9c801c070 con 0x7fe9d0108d70 2024-09-21T03:28:04.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.913+0000 7fe9b7fff640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b80b6670 0x7fe9b80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:04.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.913+0000 7fe9ceffd640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b80b6670 0x7fe9b80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:04.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.913+0000 7fe9b7fff640 1 -- 172.21.15.112:0/1175103582 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fe9c80e6e20 con 0x7fe9d0108d70 2024-09-21T03:28:04.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.913+0000 7fe9ceffd640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b80b6670 0x7fe9b80b8b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0005d00 tx=0x7fe9c0005ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:04.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:04.916+0000 7fe9b7fff640 1 -- 172.21.15.112:0/1175103582 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe9c80b1870 con 0x7fe9d0108d70 2024-09-21T03:28:05.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.109+0000 7fe9d5545640 1 -- 172.21.15.112:0/1175103582 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe9d019d8b0 con 0x7fe9d0108d70 2024-09-21T03:28:05.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.110+0000 7fe9b7fff640 1 -- 172.21.15.112:0/1175103582 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe9d019d8b0 con 0x7fe9d0108d70 2024-09-21T03:28:05.113 INFO:teuthology.orchestra.run.smithi112.stdout:223338299432 2024-09-21T03:28:05.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 -- 172.21.15.112:0/1175103582 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b80b6670 msgr2=0x7fe9b80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b80b6670 0x7fe9b80b8b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0005d00 tx=0x7fe9c0005ed0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 -- 172.21.15.112:0/1175103582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9d0108d70 msgr2=0x7fe9d01a1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9d0108d70 0x7fe9d01a1fc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800b970 tx=0x7fe9c800be30 comp rx=0 tx=0).stop 2024-09-21T03:28:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 -- 172.21.15.112:0/1175103582 shutdown_connections 2024-09-21T03:28:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b80b6670 0x7fe9b80b8b80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9d0108d70 0x7fe9d01a1fc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9d0108420 0x7fe9d019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 --2- 172.21.15.112:0/1175103582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d0102ae0 0x7fe9d019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 -- 172.21.15.112:0/1175103582 >> 172.21.15.112:0/1175103582 conn(0x7fe9d0079770 msgr2=0x7fe9d0106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 -- 172.21.15.112:0/1175103582 shutdown_connections 2024-09-21T03:28:05.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:05.112+0000 7fe9d5545640 1 -- 172.21.15.112:0/1175103582 wait complete. 2024-09-21T03:28:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:05 smithi203 ceph-mon[34723]: pgmap v290: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:05 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1175103582' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:28:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:05 smithi187 ceph-mon[34603]: pgmap v290: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:05 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1175103582' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:28:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:05 smithi112 ceph-mon[31442]: pgmap v290: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:05 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1175103582' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T03:28:05.867 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299428 got 223338299432 for osd.8 2024-09-21T03:28:05.867 DEBUG:teuthology.parallel:result is None 2024-09-21T03:28:05.867 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T03:28:05.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:28:06.456 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.934+0000 7fcc3e92d640 1 -- 172.21.15.112:0/104856095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38109e40 msgr2=0x7fcc38110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:06.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.934+0000 7fcc3e92d640 1 --2- 172.21.15.112:0/104856095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38109e40 0x7fcc38110670 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009ca0 tx=0x7fcc3002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.934+0000 7fcc3e92d640 1 -- 172.21.15.112:0/104856095 shutdown_connections 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.934+0000 7fcc3e92d640 1 --2- 172.21.15.112:0/104856095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38109e40 0x7fcc38110670 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.934+0000 7fcc3e92d640 1 --2- 172.21.15.112:0/104856095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc381094f0 0x7fcc38109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.934+0000 7fcc3e92d640 1 --2- 172.21.15.112:0/104856095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc38103bb0 0x7fcc38103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.934+0000 7fcc3e92d640 1 -- 172.21.15.112:0/104856095 >> 172.21.15.112:0/104856095 conn(0x7fcc380ff600 msgr2=0x7fcc38101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.935+0000 7fcc3e92d640 1 -- 172.21.15.112:0/104856095 shutdown_connections 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.935+0000 7fcc3e92d640 1 -- 172.21.15.112:0/104856095 wait complete. 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.935+0000 7fcc3e92d640 1 Processor -- start 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.935+0000 7fcc3e92d640 1 -- start start 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc3e92d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38103bb0 0x7fcc3819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc3e92d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc381094f0 0x7fcc3819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:06.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc3e92d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc38109e40 0x7fcc3819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc3e92d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38113b00 con 0x7fcc38103bb0 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc3e92d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38113980 con 0x7fcc38109e40 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc3e92d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38113c80 con 0x7fcc381094f0 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38103bb0 0x7fcc3819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38103bb0 0x7fcc3819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42022/0 (socket says 172.21.15.112:42022) 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc37fff640 1 -- 172.21.15.112:0/285623678 learned_addr learned my addr 172.21.15.112:0/285623678 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc3ceca640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc38109e40 0x7fcc3819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc377fe640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc381094f0 0x7fcc3819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc37fff640 1 -- 172.21.15.112:0/285623678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc381094f0 msgr2=0x7fcc3819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc37fff640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc381094f0 0x7fcc3819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc37fff640 1 -- 172.21.15.112:0/285623678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc38109e40 msgr2=0x7fcc3819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc37fff640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc38109e40 0x7fcc3819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:06.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc37fff640 1 -- 172.21.15.112:0/285623678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc381a03c0 con 0x7fcc38103bb0 2024-09-21T03:28:06.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.936+0000 7fcc37fff640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38103bb0 0x7fcc3819a450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800d770 tx=0x7fcc2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:06.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.937+0000 7fcc357fa640 1 -- 172.21.15.112:0/285623678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28017070 con 0x7fcc38103bb0 2024-09-21T03:28:06.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.937+0000 7fcc357fa640 1 -- 172.21.15.112:0/285623678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc280054d0 con 0x7fcc38103bb0 2024-09-21T03:28:06.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.937+0000 7fcc357fa640 1 -- 172.21.15.112:0/285623678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28004c30 con 0x7fcc38103bb0 2024-09-21T03:28:06.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.937+0000 7fcc3e92d640 1 -- 172.21.15.112:0/285623678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc381a06b0 con 0x7fcc38103bb0 2024-09-21T03:28:06.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.937+0000 7fcc3e92d640 1 -- 172.21.15.112:0/285623678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc381a6e10 con 0x7fcc38103bb0 2024-09-21T03:28:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.938+0000 7fcc3e92d640 1 -- 172.21.15.112:0/285623678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc38103fc0 con 0x7fcc38103bb0 2024-09-21T03:28:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.939+0000 7fcc357fa640 1 -- 172.21.15.112:0/285623678 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fcc28004dd0 con 0x7fcc38103bb0 2024-09-21T03:28:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.940+0000 7fcc357fa640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcc200b6670 0x7fcc200b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.940+0000 7fcc377fe640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcc200b6670 0x7fcc200b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.940+0000 7fcc357fa640 1 -- 172.21.15.112:0/285623678 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fcc280e7310 con 0x7fcc38103bb0 2024-09-21T03:28:06.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.941+0000 7fcc377fe640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcc200b6670 0x7fcc200b8b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc38102e50 tx=0x7fcc2c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:06.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:06.943+0000 7fcc357fa640 1 -- 172.21.15.112:0/285623678 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcc280b1de0 con 0x7fcc38103bb0 2024-09-21T03:28:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.168+0000 7fcc3e92d640 1 -- 172.21.15.112:0/285623678 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc38063800 con 0x7fcc200b6670 2024-09-21T03:28:07.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.174+0000 7fcc357fa640 1 -- 172.21.15.112:0/285623678 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258701 (secure 0 0 0) 0x7fcc38063800 con 0x7fcc200b6670 2024-09-21T03:28:07.178 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:07.179 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.176+0000 7fcc3e92d640 1 -- 172.21.15.112:0/285623678 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcc200b6670 msgr2=0x7fcc200b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.177+0000 7fcc3e92d640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcc200b6670 0x7fcc200b8b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc38102e50 tx=0x7fcc2c00b390 comp rx=0 tx=0).stop 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.177+0000 7fcc3e92d640 1 -- 172.21.15.112:0/285623678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38103bb0 msgr2=0x7fcc3819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.177+0000 7fcc3e92d640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38103bb0 0x7fcc3819a450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800d770 tx=0x7fcc2800dc30 comp rx=0 tx=0).stop 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.177+0000 7fcc3e92d640 1 -- 172.21.15.112:0/285623678 shutdown_connections 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.177+0000 7fcc3e92d640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcc200b6670 0x7fcc200b8b80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.177+0000 7fcc3e92d640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc38109e40 0x7fcc3819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.177+0000 7fcc3e92d640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc381094f0 0x7fcc3819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.177+0000 7fcc3e92d640 1 --2- 172.21.15.112:0/285623678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc38103bb0 0x7fcc3819a450 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.177+0000 7fcc3e92d640 1 -- 172.21.15.112:0/285623678 >> 172.21.15.112:0/285623678 conn(0x7fcc380ff600 msgr2=0x7fcc38108b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.177+0000 7fcc3e92d640 1 -- 172.21.15.112:0/285623678 shutdown_connections 2024-09-21T03:28:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:07.177+0000 7fcc3e92d640 1 -- 172.21.15.112:0/285623678 wait complete. 2024-09-21T03:28:07.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:07 smithi112 ceph-mon[31442]: pgmap v291: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:28:07.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:07 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:07.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:07 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:07 smithi187 ceph-mon[34603]: pgmap v291: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:28:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:07 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:07.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:07 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:07.856 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-21T03:28:06.349901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659360,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":421,"num_read_kb":309,"num_write":288,"num_write_kb":2267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952574,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365408,"kb_used_data":11928,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576416,"statfs":{"total":1151940427776,"available":1151566249984,"internally_reserved":0,"allocated":12214272,"data_stored":6929340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003489"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.426924+0000","last_change":"2024-09-21T03:27:01.339980+0000","last_active":"2024-09-21T03:27:33.426924+0000","last_peered":"2024-09-21T03:27:33.426924+0000","last_clean":"2024-09-21T03:27:33.426924+0000","last_became_active":"2024-09-21T03:27:01.339552+0000","last_became_peered":"2024-09-21T03:27:01.339552+0000","last_unstale":"2024-09-21T03:27:33.426924+0000","last_undegraded":"2024-09-21T03:27:33.426924+0000","last_fullsized":"2024-09-21T03:27:33.426924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:04.240862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428161+0000","last_change":"2024-09-21T03:27:01.334171+0000","last_active":"2024-09-21T03:27:33.428161+0000","last_peered":"2024-09-21T03:27:33.428161+0000","last_clean":"2024-09-21T03:27:33.428161+0000","last_became_active":"2024-09-21T03:27:01.333851+0000","last_became_peered":"2024-09-21T03:27:01.333851+0000","last_unstale":"2024-09-21T03:27:33.428161+0000","last_undegraded":"2024-09-21T03:27:33.428161+0000","last_fullsized":"2024-09-21T03:27:33.428161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:49.163142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428159+0000","last_change":"2024-09-21T03:27:01.339388+0000","last_active":"2024-09-21T03:27:33.428159+0000","last_peered":"2024-09-21T03:27:33.428159+0000","last_clean":"2024-09-21T03:27:33.428159+0000","last_became_active":"2024-09-21T03:27:01.339015+0000","last_became_peered":"2024-09-21T03:27:01.339015+0000","last_unstale":"2024-09-21T03:27:33.428159+0000","last_undegraded":"2024-09-21T03:27:33.428159+0000","last_fullsized":"2024-09-21T03:27:33.428159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:06.026397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.476437+0000","last_change":"2024-09-21T03:27:01.344700+0000","last_active":"2024-09-21T03:27:33.476437+0000","last_peered":"2024-09-21T03:27:33.476437+0000","last_clean":"2024-09-21T03:27:33.476437+0000","last_became_active":"2024-09-21T03:27:01.344349+0000","last_became_peered":"2024-09-21T03:27:01.344349+0000","last_unstale":"2024-09-21T03:27:33.476437+0000","last_undegraded":"2024-09-21T03:27:33.476437+0000","last_fullsized":"2024-09-21T03:27:33.476437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:00.271820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427539+0000","last_change":"2024-09-21T03:27:01.334904+0000","last_active":"2024-09-21T03:27:33.427539+0000","last_peered":"2024-09-21T03:27:33.427539+0000","last_clean":"2024-09-21T03:27:33.427539+0000","last_became_active":"2024-09-21T03:27:01.331916+0000","last_became_peered":"2024-09-21T03:27:01.331916+0000","last_unstale":"2024-09-21T03:27:33.427539+0000","last_undegraded":"2024-09-21T03:27:33.427539+0000","last_fullsized":"2024-09-21T03:27:33.427539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:49.376481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.431027+0000","last_change":"2024-09-21T03:27:01.343256+0000","last_active":"2024-09-21T03:27:33.431027+0000","last_peered":"2024-09-21T03:27:33.431027+0000","last_clean":"2024-09-21T03:27:33.431027+0000","last_became_active":"2024-09-21T03:27:01.342861+0000","last_became_peered":"2024-09-21T03:27:01.342861+0000","last_unstale":"2024-09-21T03:27:33.431027+0000","last_undegraded":"2024-09-21T03:27:33.431027+0000","last_fullsized":"2024-09-21T03:27:33.431027+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:00.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429136+0000","last_change":"2024-09-21T03:27:01.330618+0000","last_active":"2024-09-21T03:27:33.429136+0000","last_peered":"2024-09-21T03:27:33.429136+0000","last_clean":"2024-09-21T03:27:33.429136+0000","last_became_active":"2024-09-21T03:27:01.330018+0000","last_became_peered":"2024-09-21T03:27:01.330018+0000","last_unstale":"2024-09-21T03:27:33.429136+0000","last_undegraded":"2024-09-21T03:27:33.429136+0000","last_fullsized":"2024-09-21T03:27:33.429136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:03.291698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430088+0000","last_change":"2024-09-21T03:27:01.332840+0000","last_active":"2024-09-21T03:27:33.430088+0000","last_peered":"2024-09-21T03:27:33.430088+0000","last_clean":"2024-09-21T03:27:33.430088+0000","last_became_active":"2024-09-21T03:27:01.332513+0000","last_became_peered":"2024-09-21T03:27:01.332513+0000","last_unstale":"2024-09-21T03:27:33.430088+0000","last_undegraded":"2024-09-21T03:27:33.430088+0000","last_fullsized":"2024-09-21T03:27:33.430088+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:17.554934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592995+0000","last_change":"2024-09-21T03:27:01.338147+0000","last_active":"2024-09-21T03:27:33.592995+0000","last_peered":"2024-09-21T03:27:33.592995+0000","last_clean":"2024-09-21T03:27:33.592995+0000","last_became_active":"2024-09-21T03:27:01.337759+0000","last_became_peered":"2024-09-21T03:27:01.337759+0000","last_unstale":"2024-09-21T03:27:33.592995+0000","last_undegraded":"2024-09-21T03:27:33.592995+0000","last_fullsized":"2024-09-21T03:27:33.592995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:38.999593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.432240+0000","last_change":"2024-09-21T03:27:01.343373+0000","last_active":"2024-09-21T03:27:33.432240+0000","last_peered":"2024-09-21T03:27:33.432240+0000","last_clean":"2024-09-21T03:27:33.432240+0000","last_became_active":"2024-09-21T03:27:01.342924+0000","last_became_peered":"2024-09-21T03:27:01.342924+0000","last_unstale":"2024-09-21T03:27:33.432240+0000","last_undegraded":"2024-09-21T03:27:33.432240+0000","last_fullsized":"2024-09-21T03:27:33.432240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:33.869584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429631+0000","last_change":"2024-09-21T03:27:01.340120+0000","last_active":"2024-09-21T03:27:33.429631+0000","last_peered":"2024-09-21T03:27:33.429631+0000","last_clean":"2024-09-21T03:27:33.429631+0000","last_became_active":"2024-09-21T03:27:01.339708+0000","last_became_peered":"2024-09-21T03:27:01.339708+0000","last_unstale":"2024-09-21T03:27:33.429631+0000","last_undegraded":"2024-09-21T03:27:33.429631+0000","last_fullsized":"2024-09-21T03:27:33.429631+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:56.193494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.476452+0000","last_change":"2024-09-21T03:27:01.340205+0000","last_active":"2024-09-21T03:27:33.476452+0000","last_peered":"2024-09-21T03:27:33.476452+0000","last_clean":"2024-09-21T03:27:33.476452+0000","last_became_active":"2024-09-21T03:27:01.339839+0000","last_became_peered":"2024-09-21T03:27:01.339839+0000","last_unstale":"2024-09-21T03:27:33.476452+0000","last_undegraded":"2024-09-21T03:27:33.476452+0000","last_fullsized":"2024-09-21T03:27:33.476452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:05.248636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427451+0000","last_change":"2024-09-21T03:27:01.334715+0000","last_active":"2024-09-21T03:27:33.427451+0000","last_peered":"2024-09-21T03:27:33.427451+0000","last_clean":"2024-09-21T03:27:33.427451+0000","last_became_active":"2024-09-21T03:27:01.334392+0000","last_became_peered":"2024-09-21T03:27:01.334392+0000","last_unstale":"2024-09-21T03:27:33.427451+0000","last_undegraded":"2024-09-21T03:27:33.427451+0000","last_fullsized":"2024-09-21T03:27:33.427451+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:18.934063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427994+0000","last_change":"2024-09-21T03:27:01.337167+0000","last_active":"2024-09-21T03:27:33.427994+0000","last_peered":"2024-09-21T03:27:33.427994+0000","last_clean":"2024-09-21T03:27:33.427994+0000","last_became_active":"2024-09-21T03:27:01.336748+0000","last_became_peered":"2024-09-21T03:27:01.336748+0000","last_unstale":"2024-09-21T03:27:33.427994+0000","last_undegraded":"2024-09-21T03:27:33.427994+0000","last_fullsized":"2024-09-21T03:27:33.427994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:12.186727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427852+0000","last_change":"2024-09-21T03:27:01.336260+0000","last_active":"2024-09-21T03:27:33.427852+0000","last_peered":"2024-09-21T03:27:33.427852+0000","last_clean":"2024-09-21T03:27:33.427852+0000","last_became_active":"2024-09-21T03:27:01.335889+0000","last_became_peered":"2024-09-21T03:27:01.335889+0000","last_unstale":"2024-09-21T03:27:33.427852+0000","last_undegraded":"2024-09-21T03:27:33.427852+0000","last_fullsized":"2024-09-21T03:27:33.427852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:02.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592431+0000","last_change":"2024-09-21T03:27:01.337181+0000","last_active":"2024-09-21T03:27:33.592431+0000","last_peered":"2024-09-21T03:27:33.592431+0000","last_clean":"2024-09-21T03:27:33.592431+0000","last_became_active":"2024-09-21T03:27:01.336813+0000","last_became_peered":"2024-09-21T03:27:01.336813+0000","last_unstale":"2024-09-21T03:27:33.592431+0000","last_undegraded":"2024-09-21T03:27:33.592431+0000","last_fullsized":"2024-09-21T03:27:33.592431+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:04.052319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427074+0000","last_change":"2024-09-21T03:27:01.342079+0000","last_active":"2024-09-21T03:27:33.427074+0000","last_peered":"2024-09-21T03:27:33.427074+0000","last_clean":"2024-09-21T03:27:33.427074+0000","last_became_active":"2024-09-21T03:27:01.341430+0000","last_became_peered":"2024-09-21T03:27:01.341430+0000","last_unstale":"2024-09-21T03:27:33.427074+0000","last_undegraded":"2024-09-21T03:27:33.427074+0000","last_fullsized":"2024-09-21T03:27:33.427074+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:08.029978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424861+0000","last_change":"2024-09-21T03:27:01.338694+0000","last_active":"2024-09-21T03:27:33.424861+0000","last_peered":"2024-09-21T03:27:33.424861+0000","last_clean":"2024-09-21T03:27:33.424861+0000","last_became_active":"2024-09-21T03:27:01.338328+0000","last_became_peered":"2024-09-21T03:27:01.338328+0000","last_unstale":"2024-09-21T03:27:33.424861+0000","last_undegraded":"2024-09-21T03:27:33.424861+0000","last_fullsized":"2024-09-21T03:27:33.424861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:50.292238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429842+0000","last_change":"2024-09-21T03:27:01.336974+0000","last_active":"2024-09-21T03:27:33.429842+0000","last_peered":"2024-09-21T03:27:33.429842+0000","last_clean":"2024-09-21T03:27:33.429842+0000","last_became_active":"2024-09-21T03:27:01.336592+0000","last_became_peered":"2024-09-21T03:27:01.336592+0000","last_unstale":"2024-09-21T03:27:33.429842+0000","last_undegraded":"2024-09-21T03:27:33.429842+0000","last_fullsized":"2024-09-21T03:27:33.429842+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:09.399348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.431901+0000","last_change":"2024-09-21T03:27:33.431767+0000","last_active":"2024-09-21T03:27:33.431901+0000","last_peered":"2024-09-21T03:27:33.431901+0000","last_clean":"2024-09-21T03:27:33.431901+0000","last_became_active":"2024-09-21T03:27:33.431233+0000","last_became_peered":"2024-09-21T03:27:33.431233+0000","last_unstale":"2024-09-21T03:27:33.431901+0000","last_undegraded":"2024-09-21T03:27:33.431901+0000","last_fullsized":"2024-09-21T03:27:33.431901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:48.425375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427473+0000","last_change":"2024-09-21T03:27:01.331242+0000","last_active":"2024-09-21T03:27:33.427473+0000","last_peered":"2024-09-21T03:27:33.427473+0000","last_clean":"2024-09-21T03:27:33.427473+0000","last_became_active":"2024-09-21T03:27:01.330883+0000","last_became_peered":"2024-09-21T03:27:01.330883+0000","last_unstale":"2024-09-21T03:27:33.427473+0000","last_undegraded":"2024-09-21T03:27:33.427473+0000","last_fullsized":"2024-09-21T03:27:33.427473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:07.158448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592593+0000","last_change":"2024-09-21T03:27:01.335522+0000","last_active":"2024-09-21T03:27:33.592593+0000","last_peered":"2024-09-21T03:27:33.592593+0000","last_clean":"2024-09-21T03:27:33.592593+0000","last_became_active":"2024-09-21T03:27:01.335114+0000","last_became_peered":"2024-09-21T03:27:01.335114+0000","last_unstale":"2024-09-21T03:27:33.592593+0000","last_undegraded":"2024-09-21T03:27:33.592593+0000","last_fullsized":"2024-09-21T03:27:33.592593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:28.383410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430519+0000","last_change":"2024-09-21T03:27:01.329428+0000","last_active":"2024-09-21T03:27:33.430519+0000","last_peered":"2024-09-21T03:27:33.430519+0000","last_clean":"2024-09-21T03:27:33.430519+0000","last_became_active":"2024-09-21T03:27:01.328882+0000","last_became_peered":"2024-09-21T03:27:01.328882+0000","last_unstale":"2024-09-21T03:27:33.430519+0000","last_undegraded":"2024-09-21T03:27:33.430519+0000","last_fullsized":"2024-09-21T03:27:33.430519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:58.634995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.433587+0000","last_change":"2024-09-21T03:27:33.432442+0000","last_active":"2024-09-21T03:27:33.433587+0000","last_peered":"2024-09-21T03:27:33.433587+0000","last_clean":"2024-09-21T03:27:33.433587+0000","last_became_active":"2024-09-21T03:27:33.431881+0000","last_became_peered":"2024-09-21T03:27:33.431881+0000","last_unstale":"2024-09-21T03:27:33.433587+0000","last_undegraded":"2024-09-21T03:27:33.433587+0000","last_fullsized":"2024-09-21T03:27:33.433587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:46.356644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430661+0000","last_change":"2024-09-21T03:27:01.330988+0000","last_active":"2024-09-21T03:27:33.430661+0000","last_peered":"2024-09-21T03:27:33.430661+0000","last_clean":"2024-09-21T03:27:33.430661+0000","last_became_active":"2024-09-21T03:27:01.330574+0000","last_became_peered":"2024-09-21T03:27:01.330574+0000","last_unstale":"2024-09-21T03:27:33.430661+0000","last_undegraded":"2024-09-21T03:27:33.430661+0000","last_fullsized":"2024-09-21T03:27:33.430661+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:45.277947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.432050+0000","last_change":"2024-09-21T03:27:01.339606+0000","last_active":"2024-09-21T03:27:33.432050+0000","last_peered":"2024-09-21T03:27:33.432050+0000","last_clean":"2024-09-21T03:27:33.432050+0000","last_became_active":"2024-09-21T03:27:01.339208+0000","last_became_peered":"2024-09-21T03:27:01.339208+0000","last_unstale":"2024-09-21T03:27:33.432050+0000","last_undegraded":"2024-09-21T03:27:33.432050+0000","last_fullsized":"2024-09-21T03:27:33.432050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:31.593363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.426077+0000","last_change":"2024-09-21T03:27:01.343634+0000","last_active":"2024-09-21T03:27:33.426077+0000","last_peered":"2024-09-21T03:27:33.426077+0000","last_clean":"2024-09-21T03:27:33.426077+0000","last_became_active":"2024-09-21T03:27:01.342965+0000","last_became_peered":"2024-09-21T03:27:01.342965+0000","last_unstale":"2024-09-21T03:27:33.426077+0000","last_undegraded":"2024-09-21T03:27:33.426077+0000","last_fullsized":"2024-09-21T03:27:33.426077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:46.916198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427312+0000","last_change":"2024-09-21T03:27:01.324993+0000","last_active":"2024-09-21T03:27:33.427312+0000","last_peered":"2024-09-21T03:27:33.427312+0000","last_clean":"2024-09-21T03:27:33.427312+0000","last_became_active":"2024-09-21T03:27:01.324593+0000","last_became_peered":"2024-09-21T03:27:01.324593+0000","last_unstale":"2024-09-21T03:27:33.427312+0000","last_undegraded":"2024-09-21T03:27:33.427312+0000","last_fullsized":"2024-09-21T03:27:33.427312+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:13.211539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428165+0000","last_change":"2024-09-21T03:27:01.325232+0000","last_active":"2024-09-21T03:27:33.428165+0000","last_peered":"2024-09-21T03:27:33.428165+0000","last_clean":"2024-09-21T03:27:33.428165+0000","last_became_active":"2024-09-21T03:27:01.324570+0000","last_became_peered":"2024-09-21T03:27:01.324570+0000","last_unstale":"2024-09-21T03:27:33.428165+0000","last_undegraded":"2024-09-21T03:27:33.428165+0000","last_fullsized":"2024-09-21T03:27:33.428165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:30.843334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430574+0000","last_change":"2024-09-21T03:27:01.338789+0000","last_active":"2024-09-21T03:27:33.430574+0000","last_peered":"2024-09-21T03:27:33.430574+0000","last_clean":"2024-09-21T03:27:33.430574+0000","last_became_active":"2024-09-21T03:27:01.338282+0000","last_became_peered":"2024-09-21T03:27:01.338282+0000","last_unstale":"2024-09-21T03:27:33.430574+0000","last_undegraded":"2024-09-21T03:27:33.430574+0000","last_fullsized":"2024-09-21T03:27:33.430574+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:57.213343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.433346+0000","last_change":"2024-09-21T03:27:01.339521+0000","last_active":"2024-09-21T03:27:33.433346+0000","last_peered":"2024-09-21T03:27:33.433346+0000","last_clean":"2024-09-21T03:27:33.433346+0000","last_became_active":"2024-09-21T03:27:01.339175+0000","last_became_peered":"2024-09-21T03:27:01.339175+0000","last_unstale":"2024-09-21T03:27:33.433346+0000","last_undegraded":"2024-09-21T03:27:33.433346+0000","last_fullsized":"2024-09-21T03:27:33.433346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:46.163698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.426740+0000","last_change":"2024-09-21T03:27:01.338352+0000","last_active":"2024-09-21T03:27:33.426740+0000","last_peered":"2024-09-21T03:27:33.426740+0000","last_clean":"2024-09-21T03:27:33.426740+0000","last_became_active":"2024-09-21T03:27:01.337306+0000","last_became_peered":"2024-09-21T03:27:01.337306+0000","last_unstale":"2024-09-21T03:27:33.426740+0000","last_undegraded":"2024-09-21T03:27:33.426740+0000","last_fullsized":"2024-09-21T03:27:33.426740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.799039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427290+0000","last_change":"2024-09-21T03:27:01.332461+0000","last_active":"2024-09-21T03:27:33.427290+0000","last_peered":"2024-09-21T03:27:33.427290+0000","last_clean":"2024-09-21T03:27:33.427290+0000","last_became_active":"2024-09-21T03:27:01.331912+0000","last_became_peered":"2024-09-21T03:27:01.331912+0000","last_unstale":"2024-09-21T03:27:33.427290+0000","last_undegraded":"2024-09-21T03:27:33.427290+0000","last_fullsized":"2024-09-21T03:27:33.427290+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:34.771691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":96,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:03.524843+0000","last_change":"2024-09-21T03:27:33.482302+0000","last_active":"2024-09-21T03:28:03.524843+0000","last_peered":"2024-09-21T03:28:03.524843+0000","last_clean":"2024-09-21T03:28:03.524843+0000","last_became_active":"2024-09-21T03:27:33.478350+0000","last_became_peered":"2024-09-21T03:27:33.478350+0000","last_unstale":"2024-09-21T03:28:03.524843+0000","last_undegraded":"2024-09-21T03:28:03.524843+0000","last_fullsized":"2024-09-21T03:28:03.524843+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:46.916762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429267+0000","last_change":"2024-09-21T03:27:01.324364+0000","last_active":"2024-09-21T03:27:33.429267+0000","last_peered":"2024-09-21T03:27:33.429267+0000","last_clean":"2024-09-21T03:27:33.429267+0000","last_became_active":"2024-09-21T03:27:01.324018+0000","last_became_peered":"2024-09-21T03:27:01.324018+0000","last_unstale":"2024-09-21T03:27:33.429267+0000","last_undegraded":"2024-09-21T03:27:33.429267+0000","last_fullsized":"2024-09-21T03:27:33.429267+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:56.404893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.431029+0000","last_change":"2024-09-21T03:26:59.313289+0000","last_active":"2024-09-21T03:27:33.431029+0000","last_peered":"2024-09-21T03:27:33.431029+0000","last_clean":"2024-09-21T03:27:33.431029+0000","last_became_active":"2024-09-21T03:26:59.312877+0000","last_became_peered":"2024-09-21T03:26:59.312877+0000","last_unstale":"2024-09-21T03:27:33.431029+0000","last_undegraded":"2024-09-21T03:27:33.431029+0000","last_fullsized":"2024-09-21T03:27:33.431029+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:53.856541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427079+0000","last_change":"2024-09-21T03:26:59.307098+0000","last_active":"2024-09-21T03:27:33.427079+0000","last_peered":"2024-09-21T03:27:33.427079+0000","last_clean":"2024-09-21T03:27:33.427079+0000","last_became_active":"2024-09-21T03:26:59.306650+0000","last_became_peered":"2024-09-21T03:26:59.306650+0000","last_unstale":"2024-09-21T03:27:33.427079+0000","last_undegraded":"2024-09-21T03:27:33.427079+0000","last_fullsized":"2024-09-21T03:27:33.427079+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:44.033899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430151+0000","last_change":"2024-09-21T03:27:01.328443+0000","last_active":"2024-09-21T03:27:33.430151+0000","last_peered":"2024-09-21T03:27:33.430151+0000","last_clean":"2024-09-21T03:27:33.430151+0000","last_became_active":"2024-09-21T03:27:01.328107+0000","last_became_peered":"2024-09-21T03:27:01.328107+0000","last_unstale":"2024-09-21T03:27:33.430151+0000","last_undegraded":"2024-09-21T03:27:33.430151+0000","last_fullsized":"2024-09-21T03:27:33.430151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:47.798308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428946+0000","last_change":"2024-09-21T03:26:59.317934+0000","last_active":"2024-09-21T03:27:33.428946+0000","last_peered":"2024-09-21T03:27:33.428946+0000","last_clean":"2024-09-21T03:27:33.428946+0000","last_became_active":"2024-09-21T03:26:59.317166+0000","last_became_peered":"2024-09-21T03:26:59.317166+0000","last_unstale":"2024-09-21T03:27:33.428946+0000","last_undegraded":"2024-09-21T03:27:33.428946+0000","last_fullsized":"2024-09-21T03:27:33.428946+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:52.746406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424624+0000","last_change":"2024-09-21T03:27:01.331226+0000","last_active":"2024-09-21T03:27:33.424624+0000","last_peered":"2024-09-21T03:27:33.424624+0000","last_clean":"2024-09-21T03:27:33.424624+0000","last_became_active":"2024-09-21T03:27:01.330889+0000","last_became_peered":"2024-09-21T03:27:01.330889+0000","last_unstale":"2024-09-21T03:27:33.424624+0000","last_undegraded":"2024-09-21T03:27:33.424624+0000","last_fullsized":"2024-09-21T03:27:33.424624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:51.181554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'3","reported_seq":77,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:01.521435+0000","last_change":"2024-09-21T03:26:59.311671+0000","last_active":"2024-09-21T03:28:01.521435+0000","last_peered":"2024-09-21T03:28:01.521435+0000","last_clean":"2024-09-21T03:28:01.521435+0000","last_became_active":"2024-09-21T03:26:59.311149+0000","last_became_peered":"2024-09-21T03:26:59.311149+0000","last_unstale":"2024-09-21T03:28:01.521435+0000","last_undegraded":"2024-09-21T03:28:01.521435+0000","last_fullsized":"2024-09-21T03:28:01.521435+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:47.878274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.593019+0000","last_change":"2024-09-21T03:27:01.336142+0000","last_active":"2024-09-21T03:27:33.593019+0000","last_peered":"2024-09-21T03:27:33.593019+0000","last_clean":"2024-09-21T03:27:33.593019+0000","last_became_active":"2024-09-21T03:27:01.335795+0000","last_became_peered":"2024-09-21T03:27:01.335795+0000","last_unstale":"2024-09-21T03:27:33.593019+0000","last_undegraded":"2024-09-21T03:27:33.593019+0000","last_fullsized":"2024-09-21T03:27:33.593019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:29.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429365+0000","last_change":"2024-09-21T03:26:59.315077+0000","last_active":"2024-09-21T03:27:33.429365+0000","last_peered":"2024-09-21T03:27:33.429365+0000","last_clean":"2024-09-21T03:27:33.429365+0000","last_became_active":"2024-09-21T03:26:59.312347+0000","last_became_peered":"2024-09-21T03:26:59.312347+0000","last_unstale":"2024-09-21T03:27:33.429365+0000","last_undegraded":"2024-09-21T03:27:33.429365+0000","last_fullsized":"2024-09-21T03:27:33.429365+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:24.529066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428537+0000","last_change":"2024-09-21T03:27:01.324346+0000","last_active":"2024-09-21T03:27:33.428537+0000","last_peered":"2024-09-21T03:27:33.428537+0000","last_clean":"2024-09-21T03:27:33.428537+0000","last_became_active":"2024-09-21T03:27:01.323976+0000","last_became_peered":"2024-09-21T03:27:01.323976+0000","last_unstale":"2024-09-21T03:27:33.428537+0000","last_undegraded":"2024-09-21T03:27:33.428537+0000","last_fullsized":"2024-09-21T03:27:33.428537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:04.591595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429516+0000","last_change":"2024-09-21T03:26:59.326206+0000","last_active":"2024-09-21T03:27:33.429516+0000","last_peered":"2024-09-21T03:27:33.429516+0000","last_clean":"2024-09-21T03:27:33.429516+0000","last_became_active":"2024-09-21T03:26:59.325755+0000","last_became_peered":"2024-09-21T03:26:59.325755+0000","last_unstale":"2024-09-21T03:27:33.429516+0000","last_undegraded":"2024-09-21T03:27:33.429516+0000","last_fullsized":"2024-09-21T03:27:33.429516+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:23.220795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592656+0000","last_change":"2024-09-21T03:27:01.341501+0000","last_active":"2024-09-21T03:27:33.592656+0000","last_peered":"2024-09-21T03:27:33.592656+0000","last_clean":"2024-09-21T03:27:33.592656+0000","last_became_active":"2024-09-21T03:27:01.341039+0000","last_became_peered":"2024-09-21T03:27:01.341039+0000","last_unstale":"2024-09-21T03:27:33.592656+0000","last_undegraded":"2024-09-21T03:27:33.592656+0000","last_fullsized":"2024-09-21T03:27:33.592656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:15.132180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'28","reported_seq":162,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:02.522868+0000","last_change":"2024-09-21T03:26:59.313489+0000","last_active":"2024-09-21T03:28:02.522868+0000","last_peered":"2024-09-21T03:28:02.522868+0000","last_clean":"2024-09-21T03:28:02.522868+0000","last_became_active":"2024-09-21T03:26:59.313022+0000","last_became_peered":"2024-09-21T03:26:59.313022+0000","last_unstale":"2024-09-21T03:28:02.522868+0000","last_undegraded":"2024-09-21T03:28:02.522868+0000","last_fullsized":"2024-09-21T03:28:02.522868+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:08.541315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64757,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":111,"num_read_kb":117,"num_write":29,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429153+0000","last_change":"2024-09-21T03:27:01.339329+0000","last_active":"2024-09-21T03:27:33.429153+0000","last_peered":"2024-09-21T03:27:33.429153+0000","last_clean":"2024-09-21T03:27:33.429153+0000","last_became_active":"2024-09-21T03:27:01.338982+0000","last_became_peered":"2024-09-21T03:27:01.338982+0000","last_unstale":"2024-09-21T03:27:33.429153+0000","last_undegraded":"2024-09-21T03:27:33.429153+0000","last_fullsized":"2024-09-21T03:27:33.429153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:41.759450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'6","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430224+0000","last_change":"2024-09-21T03:26:59.313270+0000","last_active":"2024-09-21T03:27:33.430224+0000","last_peered":"2024-09-21T03:27:33.430224+0000","last_clean":"2024-09-21T03:27:33.430224+0000","last_became_active":"2024-09-21T03:26:59.312867+0000","last_became_peered":"2024-09-21T03:26:59.312867+0000","last_unstale":"2024-09-21T03:27:33.430224+0000","last_undegraded":"2024-09-21T03:27:33.430224+0000","last_fullsized":"2024-09-21T03:27:33.430224+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:53.798022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427146+0000","last_change":"2024-09-21T03:27:01.331268+0000","last_active":"2024-09-21T03:27:33.427146+0000","last_peered":"2024-09-21T03:27:33.427146+0000","last_clean":"2024-09-21T03:27:33.427146+0000","last_became_active":"2024-09-21T03:27:01.330922+0000","last_became_peered":"2024-09-21T03:27:01.330922+0000","last_unstale":"2024-09-21T03:27:33.427146+0000","last_undegraded":"2024-09-21T03:27:33.427146+0000","last_fullsized":"2024-09-21T03:27:33.427146+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428917+0000","last_change":"2024-09-21T03:26:59.318388+0000","last_active":"2024-09-21T03:27:33.428917+0000","last_peered":"2024-09-21T03:27:33.428917+0000","last_clean":"2024-09-21T03:27:33.428917+0000","last_became_active":"2024-09-21T03:26:59.317938+0000","last_became_peered":"2024-09-21T03:26:59.317938+0000","last_unstale":"2024-09-21T03:27:33.428917+0000","last_undegraded":"2024-09-21T03:27:33.428917+0000","last_fullsized":"2024-09-21T03:27:33.428917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.499699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.432373+0000","last_change":"2024-09-21T03:27:33.432191+0000","last_active":"2024-09-21T03:27:33.432373+0000","last_peered":"2024-09-21T03:27:33.432373+0000","last_clean":"2024-09-21T03:27:33.432373+0000","last_became_active":"2024-09-21T03:27:33.431667+0000","last_became_peered":"2024-09-21T03:27:33.431667+0000","last_unstale":"2024-09-21T03:27:33.432373+0000","last_undegraded":"2024-09-21T03:27:33.432373+0000","last_fullsized":"2024-09-21T03:27:33.432373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:13.293063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'4","reported_seq":84,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:03.477714+0000","last_change":"2024-09-21T03:26:59.311345+0000","last_active":"2024-09-21T03:28:03.477714+0000","last_peered":"2024-09-21T03:28:03.477714+0000","last_clean":"2024-09-21T03:28:03.477714+0000","last_became_active":"2024-09-21T03:26:59.310928+0000","last_became_peered":"2024-09-21T03:26:59.310928+0000","last_unstale":"2024-09-21T03:28:03.477714+0000","last_undegraded":"2024-09-21T03:28:03.477714+0000","last_fullsized":"2024-09-21T03:28:03.477714+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.614955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429104+0000","last_change":"2024-09-21T03:27:01.335785+0000","last_active":"2024-09-21T03:27:33.429104+0000","last_peered":"2024-09-21T03:27:33.429104+0000","last_clean":"2024-09-21T03:27:33.429104+0000","last_became_active":"2024-09-21T03:27:01.335396+0000","last_became_peered":"2024-09-21T03:27:01.335396+0000","last_unstale":"2024-09-21T03:27:33.429104+0000","last_undegraded":"2024-09-21T03:27:33.429104+0000","last_fullsized":"2024-09-21T03:27:33.429104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:52.785641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'15","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.448967+0000","last_change":"2024-09-21T03:27:33.448404+0000","last_active":"2024-09-21T03:27:33.448967+0000","last_peered":"2024-09-21T03:27:33.448967+0000","last_clean":"2024-09-21T03:27:33.448967+0000","last_became_active":"2024-09-21T03:27:33.434405+0000","last_became_peered":"2024-09-21T03:27:33.434405+0000","last_unstale":"2024-09-21T03:27:33.448967+0000","last_undegraded":"2024-09-21T03:27:33.448967+0000","last_fullsized":"2024-09-21T03:27:33.448967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:05.107391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429036+0000","last_change":"2024-09-21T03:27:01.342282+0000","last_active":"2024-09-21T03:27:33.429036+0000","last_peered":"2024-09-21T03:27:33.429036+0000","last_clean":"2024-09-21T03:27:33.429036+0000","last_became_active":"2024-09-21T03:27:01.341790+0000","last_became_peered":"2024-09-21T03:27:01.341790+0000","last_unstale":"2024-09-21T03:27:33.429036+0000","last_undegraded":"2024-09-21T03:27:33.429036+0000","last_fullsized":"2024-09-21T03:27:33.429036+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:05.432720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428084+0000","last_change":"2024-09-21T03:26:59.312442+0000","last_active":"2024-09-21T03:27:33.428084+0000","last_peered":"2024-09-21T03:27:33.428084+0000","last_clean":"2024-09-21T03:27:33.428084+0000","last_became_active":"2024-09-21T03:26:59.311601+0000","last_became_peered":"2024-09-21T03:26:59.311601+0000","last_unstale":"2024-09-21T03:27:33.428084+0000","last_undegraded":"2024-09-21T03:27:33.428084+0000","last_fullsized":"2024-09-21T03:27:33.428084+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:32.421228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.426347+0000","last_change":"2024-09-21T03:27:01.337917+0000","last_active":"2024-09-21T03:27:33.426347+0000","last_peered":"2024-09-21T03:27:33.426347+0000","last_clean":"2024-09-21T03:27:33.426347+0000","last_became_active":"2024-09-21T03:27:01.337457+0000","last_became_peered":"2024-09-21T03:27:01.337457+0000","last_unstale":"2024-09-21T03:27:33.426347+0000","last_undegraded":"2024-09-21T03:27:33.426347+0000","last_fullsized":"2024-09-21T03:27:33.426347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:49.011795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.597334+0000","last_change":"2024-09-21T03:27:33.597239+0000","last_active":"2024-09-21T03:27:33.597334+0000","last_peered":"2024-09-21T03:27:33.597334+0000","last_clean":"2024-09-21T03:27:33.597334+0000","last_became_active":"2024-09-21T03:27:33.593787+0000","last_became_peered":"2024-09-21T03:27:33.593787+0000","last_unstale":"2024-09-21T03:27:33.597334+0000","last_undegraded":"2024-09-21T03:27:33.597334+0000","last_fullsized":"2024-09-21T03:27:33.597334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:35.015446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429396+0000","last_change":"2024-09-21T03:27:01.338489+0000","last_active":"2024-09-21T03:27:33.429396+0000","last_peered":"2024-09-21T03:27:33.429396+0000","last_clean":"2024-09-21T03:27:33.429396+0000","last_became_active":"2024-09-21T03:27:01.338123+0000","last_became_peered":"2024-09-21T03:27:01.338123+0000","last_unstale":"2024-09-21T03:27:33.429396+0000","last_undegraded":"2024-09-21T03:27:33.429396+0000","last_fullsized":"2024-09-21T03:27:33.429396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:15.137546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429644+0000","last_change":"2024-09-21T03:26:59.315371+0000","last_active":"2024-09-21T03:27:33.429644+0000","last_peered":"2024-09-21T03:27:33.429644+0000","last_clean":"2024-09-21T03:27:33.429644+0000","last_became_active":"2024-09-21T03:26:59.314439+0000","last_became_peered":"2024-09-21T03:26:59.314439+0000","last_unstale":"2024-09-21T03:27:33.429644+0000","last_undegraded":"2024-09-21T03:27:33.429644+0000","last_fullsized":"2024-09-21T03:27:33.429644+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:13.656324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429627+0000","last_change":"2024-09-21T03:27:01.331086+0000","last_active":"2024-09-21T03:27:33.429627+0000","last_peered":"2024-09-21T03:27:33.429627+0000","last_clean":"2024-09-21T03:27:33.429627+0000","last_became_active":"2024-09-21T03:27:01.329113+0000","last_became_peered":"2024-09-21T03:27:01.329113+0000","last_unstale":"2024-09-21T03:27:33.429627+0000","last_undegraded":"2024-09-21T03:27:33.429627+0000","last_fullsized":"2024-09-21T03:27:33.429627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:25.514730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.433098+0000","last_change":"2024-09-21T03:26:59.554087+0000","last_active":"2024-09-21T03:27:33.433098+0000","last_peered":"2024-09-21T03:27:33.433098+0000","last_clean":"2024-09-21T03:27:33.433098+0000","last_became_active":"2024-09-21T03:26:59.553610+0000","last_became_peered":"2024-09-21T03:26:59.553610+0000","last_unstale":"2024-09-21T03:27:33.433098+0000","last_undegraded":"2024-09-21T03:27:33.433098+0000","last_fullsized":"2024-09-21T03:27:33.433098+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:22.048057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429773+0000","last_change":"2024-09-21T03:27:01.339987+0000","last_active":"2024-09-21T03:27:33.429773+0000","last_peered":"2024-09-21T03:27:33.429773+0000","last_clean":"2024-09-21T03:27:33.429773+0000","last_became_active":"2024-09-21T03:27:01.339060+0000","last_became_peered":"2024-09-21T03:27:01.339060+0000","last_unstale":"2024-09-21T03:27:33.429773+0000","last_undegraded":"2024-09-21T03:27:33.429773+0000","last_fullsized":"2024-09-21T03:27:33.429773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.901337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429688+0000","last_change":"2024-09-21T03:25:50.381920+0000","last_active":"2024-09-21T03:27:33.429688+0000","last_peered":"2024-09-21T03:27:33.429688+0000","last_clean":"2024-09-21T03:27:33.429688+0000","last_became_active":"2024-09-21T03:25:50.376170+0000","last_became_peered":"2024-09-21T03:25:50.376170+0000","last_unstale":"2024-09-21T03:27:33.429688+0000","last_undegraded":"2024-09-21T03:27:33.429688+0000","last_fullsized":"2024-09-21T03:27:33.429688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_clean_scrub_stamp":"2024-09-21T03:22:31.269883+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:57.281150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'3","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.477054+0000","last_change":"2024-09-21T03:26:59.313612+0000","last_active":"2024-09-21T03:27:33.477054+0000","last_peered":"2024-09-21T03:27:33.477054+0000","last_clean":"2024-09-21T03:27:33.477054+0000","last_became_active":"2024-09-21T03:26:59.313060+0000","last_became_peered":"2024-09-21T03:26:59.313060+0000","last_unstale":"2024-09-21T03:27:33.477054+0000","last_undegraded":"2024-09-21T03:27:33.477054+0000","last_fullsized":"2024-09-21T03:27:33.477054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:06.193526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430283+0000","last_change":"2024-09-21T03:27:01.337784+0000","last_active":"2024-09-21T03:27:33.430283+0000","last_peered":"2024-09-21T03:27:33.430283+0000","last_clean":"2024-09-21T03:27:33.430283+0000","last_became_active":"2024-09-21T03:27:01.337415+0000","last_became_peered":"2024-09-21T03:27:01.337415+0000","last_unstale":"2024-09-21T03:27:33.430283+0000","last_undegraded":"2024-09-21T03:27:33.430283+0000","last_fullsized":"2024-09-21T03:27:33.430283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:40.482054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'3","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.436960+0000","last_change":"2024-09-21T03:27:33.436319+0000","last_active":"2024-09-21T03:27:33.436960+0000","last_peered":"2024-09-21T03:27:33.436960+0000","last_clean":"2024-09-21T03:27:33.436960+0000","last_became_active":"2024-09-21T03:27:33.431555+0000","last_became_peered":"2024-09-21T03:27:33.431555+0000","last_unstale":"2024-09-21T03:27:33.436960+0000","last_undegraded":"2024-09-21T03:27:33.436960+0000","last_fullsized":"2024-09-21T03:27:33.436960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:16.073988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427062+0000","last_change":"2024-09-21T03:27:01.333769+0000","last_active":"2024-09-21T03:27:33.427062+0000","last_peered":"2024-09-21T03:27:33.427062+0000","last_clean":"2024-09-21T03:27:33.427062+0000","last_became_active":"2024-09-21T03:27:01.333436+0000","last_became_peered":"2024-09-21T03:27:01.333436+0000","last_unstale":"2024-09-21T03:27:33.427062+0000","last_undegraded":"2024-09-21T03:27:33.427062+0000","last_fullsized":"2024-09-21T03:27:33.427062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:30.395916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'4","reported_seq":81,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:03.476911+0000","last_change":"2024-09-21T03:26:59.325314+0000","last_active":"2024-09-21T03:28:03.476911+0000","last_peered":"2024-09-21T03:28:03.476911+0000","last_clean":"2024-09-21T03:28:03.476911+0000","last_became_active":"2024-09-21T03:26:59.324752+0000","last_became_peered":"2024-09-21T03:26:59.324752+0000","last_unstale":"2024-09-21T03:28:03.476911+0000","last_undegraded":"2024-09-21T03:28:03.476911+0000","last_fullsized":"2024-09-21T03:28:03.476911+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.252596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428802+0000","last_change":"2024-09-21T03:27:01.335154+0000","last_active":"2024-09-21T03:27:33.428802+0000","last_peered":"2024-09-21T03:27:33.428802+0000","last_clean":"2024-09-21T03:27:33.428802+0000","last_became_active":"2024-09-21T03:27:01.334633+0000","last_became_peered":"2024-09-21T03:27:01.334633+0000","last_unstale":"2024-09-21T03:27:33.428802+0000","last_undegraded":"2024-09-21T03:27:33.428802+0000","last_fullsized":"2024-09-21T03:27:33.428802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:27.992858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429757+0000","last_change":"2024-09-21T03:26:59.314019+0000","last_active":"2024-09-21T03:27:33.429757+0000","last_peered":"2024-09-21T03:27:33.429757+0000","last_clean":"2024-09-21T03:27:33.429757+0000","last_became_active":"2024-09-21T03:26:59.311915+0000","last_became_peered":"2024-09-21T03:26:59.311915+0000","last_unstale":"2024-09-21T03:27:33.429757+0000","last_undegraded":"2024-09-21T03:27:33.429757+0000","last_fullsized":"2024-09-21T03:27:33.429757+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:51.412303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.433313+0000","last_change":"2024-09-21T03:27:33.433193+0000","last_active":"2024-09-21T03:27:33.433313+0000","last_peered":"2024-09-21T03:27:33.433313+0000","last_clean":"2024-09-21T03:27:33.433313+0000","last_became_active":"2024-09-21T03:27:33.432729+0000","last_became_peered":"2024-09-21T03:27:33.432729+0000","last_unstale":"2024-09-21T03:27:33.433313+0000","last_undegraded":"2024-09-21T03:27:33.433313+0000","last_fullsized":"2024-09-21T03:27:33.433313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:37.575110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424782+0000","last_change":"2024-09-21T03:26:59.325943+0000","last_active":"2024-09-21T03:27:33.424782+0000","last_peered":"2024-09-21T03:27:33.424782+0000","last_clean":"2024-09-21T03:27:33.424782+0000","last_became_active":"2024-09-21T03:26:59.325524+0000","last_became_peered":"2024-09-21T03:26:59.325524+0000","last_unstale":"2024-09-21T03:27:33.424782+0000","last_undegraded":"2024-09-21T03:27:33.424782+0000","last_fullsized":"2024-09-21T03:27:33.424782+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:45.111480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.431730+0000","last_change":"2024-09-21T03:27:01.335369+0000","last_active":"2024-09-21T03:27:33.431730+0000","last_peered":"2024-09-21T03:27:33.431730+0000","last_clean":"2024-09-21T03:27:33.431730+0000","last_became_active":"2024-09-21T03:27:01.334997+0000","last_became_peered":"2024-09-21T03:27:01.334997+0000","last_unstale":"2024-09-21T03:27:33.431730+0000","last_undegraded":"2024-09-21T03:27:33.431730+0000","last_fullsized":"2024-09-21T03:27:33.431730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:10.257451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429553+0000","last_change":"2024-09-21T03:26:59.326204+0000","last_active":"2024-09-21T03:27:33.429553+0000","last_peered":"2024-09-21T03:27:33.429553+0000","last_clean":"2024-09-21T03:27:33.429553+0000","last_became_active":"2024-09-21T03:26:59.325431+0000","last_became_peered":"2024-09-21T03:26:59.325431+0000","last_unstale":"2024-09-21T03:27:33.429553+0000","last_undegraded":"2024-09-21T03:27:33.429553+0000","last_fullsized":"2024-09-21T03:27:33.429553+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:27.863217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427579+0000","last_change":"2024-09-21T03:27:01.333861+0000","last_active":"2024-09-21T03:27:33.427579+0000","last_peered":"2024-09-21T03:27:33.427579+0000","last_clean":"2024-09-21T03:27:33.427579+0000","last_became_active":"2024-09-21T03:27:01.333485+0000","last_became_peered":"2024-09-21T03:27:01.333485+0000","last_unstale":"2024-09-21T03:27:33.427579+0000","last_undegraded":"2024-09-21T03:27:33.427579+0000","last_fullsized":"2024-09-21T03:27:33.427579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:04.475811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"74'11","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424867+0000","last_change":"2024-09-21T03:26:59.309767+0000","last_active":"2024-09-21T03:27:33.424867+0000","last_peered":"2024-09-21T03:27:33.424867+0000","last_clean":"2024-09-21T03:27:33.424867+0000","last_became_active":"2024-09-21T03:26:59.309354+0000","last_became_peered":"2024-09-21T03:26:59.309354+0000","last_unstale":"2024-09-21T03:27:33.424867+0000","last_undegraded":"2024-09-21T03:27:33.424867+0000","last_fullsized":"2024-09-21T03:27:33.424867+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:26.728719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430664+0000","last_change":"2024-09-21T03:27:01.333097+0000","last_active":"2024-09-21T03:27:33.430664+0000","last_peered":"2024-09-21T03:27:33.430664+0000","last_clean":"2024-09-21T03:27:33.430664+0000","last_became_active":"2024-09-21T03:27:01.332743+0000","last_became_peered":"2024-09-21T03:27:01.332743+0000","last_unstale":"2024-09-21T03:27:33.430664+0000","last_undegraded":"2024-09-21T03:27:33.430664+0000","last_fullsized":"2024-09-21T03:27:33.430664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:41.004213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.476588+0000","last_change":"2024-09-21T03:26:59.553974+0000","last_active":"2024-09-21T03:27:33.476588+0000","last_peered":"2024-09-21T03:27:33.476588+0000","last_clean":"2024-09-21T03:27:33.476588+0000","last_became_active":"2024-09-21T03:26:59.553647+0000","last_became_peered":"2024-09-21T03:26:59.553647+0000","last_unstale":"2024-09-21T03:27:33.476588+0000","last_undegraded":"2024-09-21T03:27:33.476588+0000","last_fullsized":"2024-09-21T03:27:33.476588+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:00.490951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428356+0000","last_change":"2024-09-21T03:27:01.325308+0000","last_active":"2024-09-21T03:27:33.428356+0000","last_peered":"2024-09-21T03:27:33.428356+0000","last_clean":"2024-09-21T03:27:33.428356+0000","last_became_active":"2024-09-21T03:27:01.324977+0000","last_became_peered":"2024-09-21T03:27:01.324977+0000","last_unstale":"2024-09-21T03:27:33.428356+0000","last_undegraded":"2024-09-21T03:27:33.428356+0000","last_fullsized":"2024-09-21T03:27:33.428356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:10.093856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592537+0000","last_change":"2024-09-21T03:26:59.326017+0000","last_active":"2024-09-21T03:27:33.592537+0000","last_peered":"2024-09-21T03:27:33.592537+0000","last_clean":"2024-09-21T03:27:33.592537+0000","last_became_active":"2024-09-21T03:26:59.325588+0000","last_became_peered":"2024-09-21T03:26:59.325588+0000","last_unstale":"2024-09-21T03:27:33.592537+0000","last_undegraded":"2024-09-21T03:27:33.592537+0000","last_fullsized":"2024-09-21T03:27:33.592537+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:39.740449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427052+0000","last_change":"2024-09-21T03:27:01.336871+0000","last_active":"2024-09-21T03:27:33.427052+0000","last_peered":"2024-09-21T03:27:33.427052+0000","last_clean":"2024-09-21T03:27:33.427052+0000","last_became_active":"2024-09-21T03:27:01.336507+0000","last_became_peered":"2024-09-21T03:27:01.336507+0000","last_unstale":"2024-09-21T03:27:33.427052+0000","last_undegraded":"2024-09-21T03:27:33.427052+0000","last_fullsized":"2024-09-21T03:27:33.427052+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:58.674375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430477+0000","last_change":"2024-09-21T03:26:59.308997+0000","last_active":"2024-09-21T03:27:33.430477+0000","last_peered":"2024-09-21T03:27:33.430477+0000","last_clean":"2024-09-21T03:27:33.430477+0000","last_became_active":"2024-09-21T03:26:59.308338+0000","last_became_peered":"2024-09-21T03:26:59.308338+0000","last_unstale":"2024-09-21T03:27:33.430477+0000","last_undegraded":"2024-09-21T03:27:33.430477+0000","last_fullsized":"2024-09-21T03:27:33.430477+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:15.328468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.426751+0000","last_change":"2024-09-21T03:27:01.338808+0000","last_active":"2024-09-21T03:27:33.426751+0000","last_peered":"2024-09-21T03:27:33.426751+0000","last_clean":"2024-09-21T03:27:33.426751+0000","last_became_active":"2024-09-21T03:27:01.337987+0000","last_became_peered":"2024-09-21T03:27:01.337987+0000","last_unstale":"2024-09-21T03:27:33.426751+0000","last_undegraded":"2024-09-21T03:27:33.426751+0000","last_fullsized":"2024-09-21T03:27:33.426751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:56.265014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430705+0000","last_change":"2024-09-21T03:26:59.312684+0000","last_active":"2024-09-21T03:27:33.430705+0000","last_peered":"2024-09-21T03:27:33.430705+0000","last_clean":"2024-09-21T03:27:33.430705+0000","last_became_active":"2024-09-21T03:26:59.312038+0000","last_became_peered":"2024-09-21T03:26:59.312038+0000","last_unstale":"2024-09-21T03:27:33.430705+0000","last_undegraded":"2024-09-21T03:27:33.430705+0000","last_fullsized":"2024-09-21T03:27:33.430705+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:02.672255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430351+0000","last_change":"2024-09-21T03:27:01.333080+0000","last_active":"2024-09-21T03:27:33.430351+0000","last_peered":"2024-09-21T03:27:33.430351+0000","last_clean":"2024-09-21T03:27:33.430351+0000","last_became_active":"2024-09-21T03:27:01.332663+0000","last_became_peered":"2024-09-21T03:27:01.332663+0000","last_unstale":"2024-09-21T03:27:33.430351+0000","last_undegraded":"2024-09-21T03:27:33.430351+0000","last_fullsized":"2024-09-21T03:27:33.430351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:34.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427673+0000","last_change":"2024-09-21T03:26:59.315490+0000","last_active":"2024-09-21T03:27:33.427673+0000","last_peered":"2024-09-21T03:27:33.427673+0000","last_clean":"2024-09-21T03:27:33.427673+0000","last_became_active":"2024-09-21T03:26:59.313789+0000","last_became_peered":"2024-09-21T03:26:59.313789+0000","last_unstale":"2024-09-21T03:27:33.427673+0000","last_undegraded":"2024-09-21T03:27:33.427673+0000","last_fullsized":"2024-09-21T03:27:33.427673+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:10.140711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429489+0000","last_change":"2024-09-21T03:27:01.341019+0000","last_active":"2024-09-21T03:27:33.429489+0000","last_peered":"2024-09-21T03:27:33.429489+0000","last_clean":"2024-09-21T03:27:33.429489+0000","last_became_active":"2024-09-21T03:27:01.340660+0000","last_became_peered":"2024-09-21T03:27:01.340660+0000","last_unstale":"2024-09-21T03:27:33.429489+0000","last_undegraded":"2024-09-21T03:27:33.429489+0000","last_fullsized":"2024-09-21T03:27:33.429489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:16.736993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429385+0000","last_change":"2024-09-21T03:27:01.341165+0000","last_active":"2024-09-21T03:27:33.429385+0000","last_peered":"2024-09-21T03:27:33.429385+0000","last_clean":"2024-09-21T03:27:33.429385+0000","last_became_active":"2024-09-21T03:27:01.340765+0000","last_became_peered":"2024-09-21T03:27:01.340765+0000","last_unstale":"2024-09-21T03:27:33.429385+0000","last_undegraded":"2024-09-21T03:27:33.429385+0000","last_fullsized":"2024-09-21T03:27:33.429385+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:15.455944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429107+0000","last_change":"2024-09-21T03:26:59.313410+0000","last_active":"2024-09-21T03:27:33.429107+0000","last_peered":"2024-09-21T03:27:33.429107+0000","last_clean":"2024-09-21T03:27:33.429107+0000","last_became_active":"2024-09-21T03:26:59.312233+0000","last_became_peered":"2024-09-21T03:26:59.312233+0000","last_unstale":"2024-09-21T03:27:33.429107+0000","last_undegraded":"2024-09-21T03:27:33.429107+0000","last_fullsized":"2024-09-21T03:27:33.429107+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:26.089611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592537+0000","last_change":"2024-09-21T03:27:01.335332+0000","last_active":"2024-09-21T03:27:33.592537+0000","last_peered":"2024-09-21T03:27:33.592537+0000","last_clean":"2024-09-21T03:27:33.592537+0000","last_became_active":"2024-09-21T03:27:01.334872+0000","last_became_peered":"2024-09-21T03:27:01.334872+0000","last_unstale":"2024-09-21T03:27:33.592537+0000","last_undegraded":"2024-09-21T03:27:33.592537+0000","last_fullsized":"2024-09-21T03:27:33.592537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:39.631246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429421+0000","last_change":"2024-09-21T03:26:59.314250+0000","last_active":"2024-09-21T03:27:33.429421+0000","last_peered":"2024-09-21T03:27:33.429421+0000","last_clean":"2024-09-21T03:27:33.429421+0000","last_became_active":"2024-09-21T03:26:59.313707+0000","last_became_peered":"2024-09-21T03:26:59.313707+0000","last_unstale":"2024-09-21T03:27:33.429421+0000","last_undegraded":"2024-09-21T03:27:33.429421+0000","last_fullsized":"2024-09-21T03:27:33.429421+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:22.935292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428509+0000","last_change":"2024-09-21T03:27:01.339230+0000","last_active":"2024-09-21T03:27:33.428509+0000","last_peered":"2024-09-21T03:27:33.428509+0000","last_clean":"2024-09-21T03:27:33.428509+0000","last_became_active":"2024-09-21T03:27:01.338881+0000","last_became_peered":"2024-09-21T03:27:01.338881+0000","last_unstale":"2024-09-21T03:27:33.428509+0000","last_undegraded":"2024-09-21T03:27:33.428509+0000","last_fullsized":"2024-09-21T03:27:33.428509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:49.189180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430888+0000","last_change":"2024-09-21T03:26:59.326549+0000","last_active":"2024-09-21T03:27:33.430888+0000","last_peered":"2024-09-21T03:27:33.430888+0000","last_clean":"2024-09-21T03:27:33.430888+0000","last_became_active":"2024-09-21T03:26:59.326091+0000","last_became_peered":"2024-09-21T03:26:59.326091+0000","last_unstale":"2024-09-21T03:27:33.430888+0000","last_undegraded":"2024-09-21T03:27:33.430888+0000","last_fullsized":"2024-09-21T03:27:33.430888+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:13.507716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430739+0000","last_change":"2024-09-21T03:27:01.331367+0000","last_active":"2024-09-21T03:27:33.430739+0000","last_peered":"2024-09-21T03:27:33.430739+0000","last_clean":"2024-09-21T03:27:33.430739+0000","last_became_active":"2024-09-21T03:27:01.330604+0000","last_became_peered":"2024-09-21T03:27:01.330604+0000","last_unstale":"2024-09-21T03:27:33.430739+0000","last_undegraded":"2024-09-21T03:27:33.430739+0000","last_fullsized":"2024-09-21T03:27:33.430739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:32.207561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'9","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424843+0000","last_change":"2024-09-21T03:26:59.311912+0000","last_active":"2024-09-21T03:27:33.424843+0000","last_peered":"2024-09-21T03:27:33.424843+0000","last_clean":"2024-09-21T03:27:33.424843+0000","last_became_active":"2024-09-21T03:26:59.311571+0000","last_became_peered":"2024-09-21T03:26:59.311571+0000","last_unstale":"2024-09-21T03:27:33.424843+0000","last_undegraded":"2024-09-21T03:27:33.424843+0000","last_fullsized":"2024-09-21T03:27:33.424843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:55.946494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68758,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":340,"num_read_kb":240,"num_write":163,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":206274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776155,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":843776,"data_stored":413299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939678,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1536000,"data_stored":1069816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":851968,"data_stored":413423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299433,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":843776,"data_stored":413231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462958,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1445888,"data_stored":1005273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659186,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":412812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855416,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1458176,"data_stored":1004566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":851968,"data_stored":414262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":856064,"data_stored":413123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":478090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542220,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":905216,"data_stored":477362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705745,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":860160,"data_stored":414083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:28:07.860 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T03:28:07.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:07 smithi203 ceph-mon[34723]: pgmap v291: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:28:07.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:07 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:07.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:07 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:08.104 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:08.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:08 smithi112 ceph-mon[31442]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:08.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:08.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:08.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:08.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.586+0000 7f631e10e640 1 -- 172.21.15.112:0/2955447661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6318108d30 msgr2=0x7f631810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:08.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.586+0000 7f631e10e640 1 --2- 172.21.15.112:0/2955447661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6318108d30 0x7f631810f560 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6310009ca0 tx=0x7f631002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:08.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.586+0000 7f631e10e640 1 -- 172.21.15.112:0/2955447661 shutdown_connections 2024-09-21T03:28:08.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.586+0000 7f631e10e640 1 --2- 172.21.15.112:0/2955447661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6318108d30 0x7f631810f560 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:08.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.586+0000 7f631e10e640 1 --2- 172.21.15.112:0/2955447661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63181083e0 0x7f63181087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:08.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.586+0000 7f631e10e640 1 --2- 172.21.15.112:0/2955447661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6318102aa0 0x7f6318102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:08.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.586+0000 7f631e10e640 1 -- 172.21.15.112:0/2955447661 >> 172.21.15.112:0/2955447661 conn(0x7f63180797e0 msgr2=0x7f6318079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:08.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.586+0000 7f631e10e640 1 -- 172.21.15.112:0/2955447661 shutdown_connections 2024-09-21T03:28:08.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.586+0000 7f631e10e640 1 -- 172.21.15.112:0/2955447661 wait complete. 2024-09-21T03:28:08.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f631e10e640 1 Processor -- start 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f631e10e640 1 -- start start 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f631e10e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6318102aa0 0x7f631819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f631e10e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63181083e0 0x7f631819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f631e10e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6318108d30 0x7f631819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f631e10e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6318113b10 con 0x7f6318102aa0 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f631e10e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6318113990 con 0x7f63181083e0 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f631e10e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6318113c90 con 0x7f6318108d30 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f63177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6318102aa0 0x7f631819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f63177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6318102aa0 0x7f631819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42040/0 (socket says 172.21.15.112:42040) 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f63177fe640 1 -- 172.21.15.112:0/3836227223 learned_addr learned my addr 172.21.15.112:0/3836227223 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.587+0000 7f630ffff640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63181083e0 0x7f631819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f6317fff640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6318108d30 0x7f631819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:08.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f63177fe640 1 -- 172.21.15.112:0/3836227223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6318108d30 msgr2=0x7f631819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f63177fe640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6318108d30 0x7f631819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f63177fe640 1 -- 172.21.15.112:0/3836227223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63181083e0 msgr2=0x7f631819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f63177fe640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63181083e0 0x7f631819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f63177fe640 1 -- 172.21.15.112:0/3836227223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63181a0410 con 0x7f6318102aa0 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f6317fff640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6318108d30 0x7f631819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f630ffff640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63181083e0 0x7f631819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f63177fe640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6318102aa0 0x7f631819a4a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f630800e9f0 tx=0x7f630800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f63157fa640 1 -- 172.21.15.112:0/3836227223 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630800cd80 con 0x7f6318102aa0 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f63157fa640 1 -- 172.21.15.112:0/3836227223 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63080108e0 con 0x7f6318102aa0 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f63157fa640 1 -- 172.21.15.112:0/3836227223 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308018980 con 0x7f6318102aa0 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f631e10e640 1 -- 172.21.15.112:0/3836227223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63181a0700 con 0x7f6318102aa0 2024-09-21T03:28:08.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.588+0000 7f631e10e640 1 -- 172.21.15.112:0/3836227223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63181a6e60 con 0x7f6318102aa0 2024-09-21T03:28:08.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.589+0000 7f631e10e640 1 -- 172.21.15.112:0/3836227223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6318102eb0 con 0x7f6318102aa0 2024-09-21T03:28:08.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.591+0000 7f63157fa640 1 -- 172.21.15.112:0/3836227223 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f6308010430 con 0x7f6318102aa0 2024-09-21T03:28:08.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.591+0000 7f63157fa640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63040b6670 0x7f63040b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:08.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.591+0000 7f630ffff640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63040b6670 0x7f63040b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:08.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.591+0000 7f63157fa640 1 -- 172.21.15.112:0/3836227223 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f63080e7e50 con 0x7f6318102aa0 2024-09-21T03:28:08.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.592+0000 7f630ffff640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63040b6670 0x7f63040b8b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6318107aa0 tx=0x7f6300008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:08.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.594+0000 7f63157fa640 1 -- 172.21.15.112:0/3836227223 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f63080ae4c0 con 0x7f6318102aa0 2024-09-21T03:28:08.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:08 smithi203 ceph-mon[34723]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:08.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:08.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:08.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:08.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:08 smithi187 ceph-mon[34603]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:08.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:28:08.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:28:08.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:28:08.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.769+0000 7f631e10e640 1 -- 172.21.15.112:0/3836227223 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f631802e670 con 0x7f63040b6670 2024-09-21T03:28:08.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.774+0000 7f63157fa640 1 -- 172.21.15.112:0/3836227223 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258701 (secure 0 0 0) 0x7f631802e670 con 0x7f63040b6670 2024-09-21T03:28:08.776 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:08.783 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T03:28:08.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 -- 172.21.15.112:0/3836227223 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63040b6670 msgr2=0x7f63040b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:08.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63040b6670 0x7f63040b8b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6318107aa0 tx=0x7f6300008040 comp rx=0 tx=0).stop 2024-09-21T03:28:08.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 -- 172.21.15.112:0/3836227223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6318102aa0 msgr2=0x7f631819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:08.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6318102aa0 0x7f631819a4a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f630800e9f0 tx=0x7f630800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:28:08.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 -- 172.21.15.112:0/3836227223 shutdown_connections 2024-09-21T03:28:08.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63040b6670 0x7f63040b8b80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6318108d30 0x7f631819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63181083e0 0x7f631819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 --2- 172.21.15.112:0/3836227223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6318102aa0 0x7f631819a4a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 -- 172.21.15.112:0/3836227223 >> 172.21.15.112:0/3836227223 conn(0x7f63180797e0 msgr2=0x7f6318105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 -- 172.21.15.112:0/3836227223 shutdown_connections 2024-09-21T03:28:08.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:08.777+0000 7f631e10e640 1 -- 172.21.15.112:0/3836227223 wait complete. 2024-09-21T03:28:09.434 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-21T03:28:08.350561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659360,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":421,"num_read_kb":309,"num_write":288,"num_write_kb":2267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952574,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365408,"kb_used_data":11928,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576416,"statfs":{"total":1151940427776,"available":1151566249984,"internally_reserved":0,"allocated":12214272,"data_stored":6929340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003689"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.426924+0000","last_change":"2024-09-21T03:27:01.339980+0000","last_active":"2024-09-21T03:27:33.426924+0000","last_peered":"2024-09-21T03:27:33.426924+0000","last_clean":"2024-09-21T03:27:33.426924+0000","last_became_active":"2024-09-21T03:27:01.339552+0000","last_became_peered":"2024-09-21T03:27:01.339552+0000","last_unstale":"2024-09-21T03:27:33.426924+0000","last_undegraded":"2024-09-21T03:27:33.426924+0000","last_fullsized":"2024-09-21T03:27:33.426924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:04.240862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428161+0000","last_change":"2024-09-21T03:27:01.334171+0000","last_active":"2024-09-21T03:27:33.428161+0000","last_peered":"2024-09-21T03:27:33.428161+0000","last_clean":"2024-09-21T03:27:33.428161+0000","last_became_active":"2024-09-21T03:27:01.333851+0000","last_became_peered":"2024-09-21T03:27:01.333851+0000","last_unstale":"2024-09-21T03:27:33.428161+0000","last_undegraded":"2024-09-21T03:27:33.428161+0000","last_fullsized":"2024-09-21T03:27:33.428161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:49.163142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428159+0000","last_change":"2024-09-21T03:27:01.339388+0000","last_active":"2024-09-21T03:27:33.428159+0000","last_peered":"2024-09-21T03:27:33.428159+0000","last_clean":"2024-09-21T03:27:33.428159+0000","last_became_active":"2024-09-21T03:27:01.339015+0000","last_became_peered":"2024-09-21T03:27:01.339015+0000","last_unstale":"2024-09-21T03:27:33.428159+0000","last_undegraded":"2024-09-21T03:27:33.428159+0000","last_fullsized":"2024-09-21T03:27:33.428159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:06.026397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.476437+0000","last_change":"2024-09-21T03:27:01.344700+0000","last_active":"2024-09-21T03:27:33.476437+0000","last_peered":"2024-09-21T03:27:33.476437+0000","last_clean":"2024-09-21T03:27:33.476437+0000","last_became_active":"2024-09-21T03:27:01.344349+0000","last_became_peered":"2024-09-21T03:27:01.344349+0000","last_unstale":"2024-09-21T03:27:33.476437+0000","last_undegraded":"2024-09-21T03:27:33.476437+0000","last_fullsized":"2024-09-21T03:27:33.476437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:00.271820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427539+0000","last_change":"2024-09-21T03:27:01.334904+0000","last_active":"2024-09-21T03:27:33.427539+0000","last_peered":"2024-09-21T03:27:33.427539+0000","last_clean":"2024-09-21T03:27:33.427539+0000","last_became_active":"2024-09-21T03:27:01.331916+0000","last_became_peered":"2024-09-21T03:27:01.331916+0000","last_unstale":"2024-09-21T03:27:33.427539+0000","last_undegraded":"2024-09-21T03:27:33.427539+0000","last_fullsized":"2024-09-21T03:27:33.427539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:49.376481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.431027+0000","last_change":"2024-09-21T03:27:01.343256+0000","last_active":"2024-09-21T03:27:33.431027+0000","last_peered":"2024-09-21T03:27:33.431027+0000","last_clean":"2024-09-21T03:27:33.431027+0000","last_became_active":"2024-09-21T03:27:01.342861+0000","last_became_peered":"2024-09-21T03:27:01.342861+0000","last_unstale":"2024-09-21T03:27:33.431027+0000","last_undegraded":"2024-09-21T03:27:33.431027+0000","last_fullsized":"2024-09-21T03:27:33.431027+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:00.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429136+0000","last_change":"2024-09-21T03:27:01.330618+0000","last_active":"2024-09-21T03:27:33.429136+0000","last_peered":"2024-09-21T03:27:33.429136+0000","last_clean":"2024-09-21T03:27:33.429136+0000","last_became_active":"2024-09-21T03:27:01.330018+0000","last_became_peered":"2024-09-21T03:27:01.330018+0000","last_unstale":"2024-09-21T03:27:33.429136+0000","last_undegraded":"2024-09-21T03:27:33.429136+0000","last_fullsized":"2024-09-21T03:27:33.429136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:03.291698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430088+0000","last_change":"2024-09-21T03:27:01.332840+0000","last_active":"2024-09-21T03:27:33.430088+0000","last_peered":"2024-09-21T03:27:33.430088+0000","last_clean":"2024-09-21T03:27:33.430088+0000","last_became_active":"2024-09-21T03:27:01.332513+0000","last_became_peered":"2024-09-21T03:27:01.332513+0000","last_unstale":"2024-09-21T03:27:33.430088+0000","last_undegraded":"2024-09-21T03:27:33.430088+0000","last_fullsized":"2024-09-21T03:27:33.430088+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:17.554934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592995+0000","last_change":"2024-09-21T03:27:01.338147+0000","last_active":"2024-09-21T03:27:33.592995+0000","last_peered":"2024-09-21T03:27:33.592995+0000","last_clean":"2024-09-21T03:27:33.592995+0000","last_became_active":"2024-09-21T03:27:01.337759+0000","last_became_peered":"2024-09-21T03:27:01.337759+0000","last_unstale":"2024-09-21T03:27:33.592995+0000","last_undegraded":"2024-09-21T03:27:33.592995+0000","last_fullsized":"2024-09-21T03:27:33.592995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:38.999593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.432240+0000","last_change":"2024-09-21T03:27:01.343373+0000","last_active":"2024-09-21T03:27:33.432240+0000","last_peered":"2024-09-21T03:27:33.432240+0000","last_clean":"2024-09-21T03:27:33.432240+0000","last_became_active":"2024-09-21T03:27:01.342924+0000","last_became_peered":"2024-09-21T03:27:01.342924+0000","last_unstale":"2024-09-21T03:27:33.432240+0000","last_undegraded":"2024-09-21T03:27:33.432240+0000","last_fullsized":"2024-09-21T03:27:33.432240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:33.869584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429631+0000","last_change":"2024-09-21T03:27:01.340120+0000","last_active":"2024-09-21T03:27:33.429631+0000","last_peered":"2024-09-21T03:27:33.429631+0000","last_clean":"2024-09-21T03:27:33.429631+0000","last_became_active":"2024-09-21T03:27:01.339708+0000","last_became_peered":"2024-09-21T03:27:01.339708+0000","last_unstale":"2024-09-21T03:27:33.429631+0000","last_undegraded":"2024-09-21T03:27:33.429631+0000","last_fullsized":"2024-09-21T03:27:33.429631+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:56.193494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.476452+0000","last_change":"2024-09-21T03:27:01.340205+0000","last_active":"2024-09-21T03:27:33.476452+0000","last_peered":"2024-09-21T03:27:33.476452+0000","last_clean":"2024-09-21T03:27:33.476452+0000","last_became_active":"2024-09-21T03:27:01.339839+0000","last_became_peered":"2024-09-21T03:27:01.339839+0000","last_unstale":"2024-09-21T03:27:33.476452+0000","last_undegraded":"2024-09-21T03:27:33.476452+0000","last_fullsized":"2024-09-21T03:27:33.476452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:05.248636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427451+0000","last_change":"2024-09-21T03:27:01.334715+0000","last_active":"2024-09-21T03:27:33.427451+0000","last_peered":"2024-09-21T03:27:33.427451+0000","last_clean":"2024-09-21T03:27:33.427451+0000","last_became_active":"2024-09-21T03:27:01.334392+0000","last_became_peered":"2024-09-21T03:27:01.334392+0000","last_unstale":"2024-09-21T03:27:33.427451+0000","last_undegraded":"2024-09-21T03:27:33.427451+0000","last_fullsized":"2024-09-21T03:27:33.427451+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:18.934063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427994+0000","last_change":"2024-09-21T03:27:01.337167+0000","last_active":"2024-09-21T03:27:33.427994+0000","last_peered":"2024-09-21T03:27:33.427994+0000","last_clean":"2024-09-21T03:27:33.427994+0000","last_became_active":"2024-09-21T03:27:01.336748+0000","last_became_peered":"2024-09-21T03:27:01.336748+0000","last_unstale":"2024-09-21T03:27:33.427994+0000","last_undegraded":"2024-09-21T03:27:33.427994+0000","last_fullsized":"2024-09-21T03:27:33.427994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:12.186727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427852+0000","last_change":"2024-09-21T03:27:01.336260+0000","last_active":"2024-09-21T03:27:33.427852+0000","last_peered":"2024-09-21T03:27:33.427852+0000","last_clean":"2024-09-21T03:27:33.427852+0000","last_became_active":"2024-09-21T03:27:01.335889+0000","last_became_peered":"2024-09-21T03:27:01.335889+0000","last_unstale":"2024-09-21T03:27:33.427852+0000","last_undegraded":"2024-09-21T03:27:33.427852+0000","last_fullsized":"2024-09-21T03:27:33.427852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:02.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592431+0000","last_change":"2024-09-21T03:27:01.337181+0000","last_active":"2024-09-21T03:27:33.592431+0000","last_peered":"2024-09-21T03:27:33.592431+0000","last_clean":"2024-09-21T03:27:33.592431+0000","last_became_active":"2024-09-21T03:27:01.336813+0000","last_became_peered":"2024-09-21T03:27:01.336813+0000","last_unstale":"2024-09-21T03:27:33.592431+0000","last_undegraded":"2024-09-21T03:27:33.592431+0000","last_fullsized":"2024-09-21T03:27:33.592431+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:04.052319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427074+0000","last_change":"2024-09-21T03:27:01.342079+0000","last_active":"2024-09-21T03:27:33.427074+0000","last_peered":"2024-09-21T03:27:33.427074+0000","last_clean":"2024-09-21T03:27:33.427074+0000","last_became_active":"2024-09-21T03:27:01.341430+0000","last_became_peered":"2024-09-21T03:27:01.341430+0000","last_unstale":"2024-09-21T03:27:33.427074+0000","last_undegraded":"2024-09-21T03:27:33.427074+0000","last_fullsized":"2024-09-21T03:27:33.427074+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:08.029978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424861+0000","last_change":"2024-09-21T03:27:01.338694+0000","last_active":"2024-09-21T03:27:33.424861+0000","last_peered":"2024-09-21T03:27:33.424861+0000","last_clean":"2024-09-21T03:27:33.424861+0000","last_became_active":"2024-09-21T03:27:01.338328+0000","last_became_peered":"2024-09-21T03:27:01.338328+0000","last_unstale":"2024-09-21T03:27:33.424861+0000","last_undegraded":"2024-09-21T03:27:33.424861+0000","last_fullsized":"2024-09-21T03:27:33.424861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:50.292238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429842+0000","last_change":"2024-09-21T03:27:01.336974+0000","last_active":"2024-09-21T03:27:33.429842+0000","last_peered":"2024-09-21T03:27:33.429842+0000","last_clean":"2024-09-21T03:27:33.429842+0000","last_became_active":"2024-09-21T03:27:01.336592+0000","last_became_peered":"2024-09-21T03:27:01.336592+0000","last_unstale":"2024-09-21T03:27:33.429842+0000","last_undegraded":"2024-09-21T03:27:33.429842+0000","last_fullsized":"2024-09-21T03:27:33.429842+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:09.399348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.431901+0000","last_change":"2024-09-21T03:27:33.431767+0000","last_active":"2024-09-21T03:27:33.431901+0000","last_peered":"2024-09-21T03:27:33.431901+0000","last_clean":"2024-09-21T03:27:33.431901+0000","last_became_active":"2024-09-21T03:27:33.431233+0000","last_became_peered":"2024-09-21T03:27:33.431233+0000","last_unstale":"2024-09-21T03:27:33.431901+0000","last_undegraded":"2024-09-21T03:27:33.431901+0000","last_fullsized":"2024-09-21T03:27:33.431901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:48.425375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427473+0000","last_change":"2024-09-21T03:27:01.331242+0000","last_active":"2024-09-21T03:27:33.427473+0000","last_peered":"2024-09-21T03:27:33.427473+0000","last_clean":"2024-09-21T03:27:33.427473+0000","last_became_active":"2024-09-21T03:27:01.330883+0000","last_became_peered":"2024-09-21T03:27:01.330883+0000","last_unstale":"2024-09-21T03:27:33.427473+0000","last_undegraded":"2024-09-21T03:27:33.427473+0000","last_fullsized":"2024-09-21T03:27:33.427473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:07.158448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592593+0000","last_change":"2024-09-21T03:27:01.335522+0000","last_active":"2024-09-21T03:27:33.592593+0000","last_peered":"2024-09-21T03:27:33.592593+0000","last_clean":"2024-09-21T03:27:33.592593+0000","last_became_active":"2024-09-21T03:27:01.335114+0000","last_became_peered":"2024-09-21T03:27:01.335114+0000","last_unstale":"2024-09-21T03:27:33.592593+0000","last_undegraded":"2024-09-21T03:27:33.592593+0000","last_fullsized":"2024-09-21T03:27:33.592593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:28.383410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430519+0000","last_change":"2024-09-21T03:27:01.329428+0000","last_active":"2024-09-21T03:27:33.430519+0000","last_peered":"2024-09-21T03:27:33.430519+0000","last_clean":"2024-09-21T03:27:33.430519+0000","last_became_active":"2024-09-21T03:27:01.328882+0000","last_became_peered":"2024-09-21T03:27:01.328882+0000","last_unstale":"2024-09-21T03:27:33.430519+0000","last_undegraded":"2024-09-21T03:27:33.430519+0000","last_fullsized":"2024-09-21T03:27:33.430519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:58.634995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.433587+0000","last_change":"2024-09-21T03:27:33.432442+0000","last_active":"2024-09-21T03:27:33.433587+0000","last_peered":"2024-09-21T03:27:33.433587+0000","last_clean":"2024-09-21T03:27:33.433587+0000","last_became_active":"2024-09-21T03:27:33.431881+0000","last_became_peered":"2024-09-21T03:27:33.431881+0000","last_unstale":"2024-09-21T03:27:33.433587+0000","last_undegraded":"2024-09-21T03:27:33.433587+0000","last_fullsized":"2024-09-21T03:27:33.433587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:46.356644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430661+0000","last_change":"2024-09-21T03:27:01.330988+0000","last_active":"2024-09-21T03:27:33.430661+0000","last_peered":"2024-09-21T03:27:33.430661+0000","last_clean":"2024-09-21T03:27:33.430661+0000","last_became_active":"2024-09-21T03:27:01.330574+0000","last_became_peered":"2024-09-21T03:27:01.330574+0000","last_unstale":"2024-09-21T03:27:33.430661+0000","last_undegraded":"2024-09-21T03:27:33.430661+0000","last_fullsized":"2024-09-21T03:27:33.430661+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:45.277947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.432050+0000","last_change":"2024-09-21T03:27:01.339606+0000","last_active":"2024-09-21T03:27:33.432050+0000","last_peered":"2024-09-21T03:27:33.432050+0000","last_clean":"2024-09-21T03:27:33.432050+0000","last_became_active":"2024-09-21T03:27:01.339208+0000","last_became_peered":"2024-09-21T03:27:01.339208+0000","last_unstale":"2024-09-21T03:27:33.432050+0000","last_undegraded":"2024-09-21T03:27:33.432050+0000","last_fullsized":"2024-09-21T03:27:33.432050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:31.593363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.426077+0000","last_change":"2024-09-21T03:27:01.343634+0000","last_active":"2024-09-21T03:27:33.426077+0000","last_peered":"2024-09-21T03:27:33.426077+0000","last_clean":"2024-09-21T03:27:33.426077+0000","last_became_active":"2024-09-21T03:27:01.342965+0000","last_became_peered":"2024-09-21T03:27:01.342965+0000","last_unstale":"2024-09-21T03:27:33.426077+0000","last_undegraded":"2024-09-21T03:27:33.426077+0000","last_fullsized":"2024-09-21T03:27:33.426077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:46.916198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427312+0000","last_change":"2024-09-21T03:27:01.324993+0000","last_active":"2024-09-21T03:27:33.427312+0000","last_peered":"2024-09-21T03:27:33.427312+0000","last_clean":"2024-09-21T03:27:33.427312+0000","last_became_active":"2024-09-21T03:27:01.324593+0000","last_became_peered":"2024-09-21T03:27:01.324593+0000","last_unstale":"2024-09-21T03:27:33.427312+0000","last_undegraded":"2024-09-21T03:27:33.427312+0000","last_fullsized":"2024-09-21T03:27:33.427312+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:13.211539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428165+0000","last_change":"2024-09-21T03:27:01.325232+0000","last_active":"2024-09-21T03:27:33.428165+0000","last_peered":"2024-09-21T03:27:33.428165+0000","last_clean":"2024-09-21T03:27:33.428165+0000","last_became_active":"2024-09-21T03:27:01.324570+0000","last_became_peered":"2024-09-21T03:27:01.324570+0000","last_unstale":"2024-09-21T03:27:33.428165+0000","last_undegraded":"2024-09-21T03:27:33.428165+0000","last_fullsized":"2024-09-21T03:27:33.428165+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:30.843334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430574+0000","last_change":"2024-09-21T03:27:01.338789+0000","last_active":"2024-09-21T03:27:33.430574+0000","last_peered":"2024-09-21T03:27:33.430574+0000","last_clean":"2024-09-21T03:27:33.430574+0000","last_became_active":"2024-09-21T03:27:01.338282+0000","last_became_peered":"2024-09-21T03:27:01.338282+0000","last_unstale":"2024-09-21T03:27:33.430574+0000","last_undegraded":"2024-09-21T03:27:33.430574+0000","last_fullsized":"2024-09-21T03:27:33.430574+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:57.213343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.433346+0000","last_change":"2024-09-21T03:27:01.339521+0000","last_active":"2024-09-21T03:27:33.433346+0000","last_peered":"2024-09-21T03:27:33.433346+0000","last_clean":"2024-09-21T03:27:33.433346+0000","last_became_active":"2024-09-21T03:27:01.339175+0000","last_became_peered":"2024-09-21T03:27:01.339175+0000","last_unstale":"2024-09-21T03:27:33.433346+0000","last_undegraded":"2024-09-21T03:27:33.433346+0000","last_fullsized":"2024-09-21T03:27:33.433346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:46.163698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.426740+0000","last_change":"2024-09-21T03:27:01.338352+0000","last_active":"2024-09-21T03:27:33.426740+0000","last_peered":"2024-09-21T03:27:33.426740+0000","last_clean":"2024-09-21T03:27:33.426740+0000","last_became_active":"2024-09-21T03:27:01.337306+0000","last_became_peered":"2024-09-21T03:27:01.337306+0000","last_unstale":"2024-09-21T03:27:33.426740+0000","last_undegraded":"2024-09-21T03:27:33.426740+0000","last_fullsized":"2024-09-21T03:27:33.426740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.799039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427290+0000","last_change":"2024-09-21T03:27:01.332461+0000","last_active":"2024-09-21T03:27:33.427290+0000","last_peered":"2024-09-21T03:27:33.427290+0000","last_clean":"2024-09-21T03:27:33.427290+0000","last_became_active":"2024-09-21T03:27:01.331912+0000","last_became_peered":"2024-09-21T03:27:01.331912+0000","last_unstale":"2024-09-21T03:27:33.427290+0000","last_undegraded":"2024-09-21T03:27:33.427290+0000","last_fullsized":"2024-09-21T03:27:33.427290+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:34.771691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":96,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:03.524843+0000","last_change":"2024-09-21T03:27:33.482302+0000","last_active":"2024-09-21T03:28:03.524843+0000","last_peered":"2024-09-21T03:28:03.524843+0000","last_clean":"2024-09-21T03:28:03.524843+0000","last_became_active":"2024-09-21T03:27:33.478350+0000","last_became_peered":"2024-09-21T03:27:33.478350+0000","last_unstale":"2024-09-21T03:28:03.524843+0000","last_undegraded":"2024-09-21T03:28:03.524843+0000","last_fullsized":"2024-09-21T03:28:03.524843+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:46.916762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429267+0000","last_change":"2024-09-21T03:27:01.324364+0000","last_active":"2024-09-21T03:27:33.429267+0000","last_peered":"2024-09-21T03:27:33.429267+0000","last_clean":"2024-09-21T03:27:33.429267+0000","last_became_active":"2024-09-21T03:27:01.324018+0000","last_became_peered":"2024-09-21T03:27:01.324018+0000","last_unstale":"2024-09-21T03:27:33.429267+0000","last_undegraded":"2024-09-21T03:27:33.429267+0000","last_fullsized":"2024-09-21T03:27:33.429267+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:56.404893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.431029+0000","last_change":"2024-09-21T03:26:59.313289+0000","last_active":"2024-09-21T03:27:33.431029+0000","last_peered":"2024-09-21T03:27:33.431029+0000","last_clean":"2024-09-21T03:27:33.431029+0000","last_became_active":"2024-09-21T03:26:59.312877+0000","last_became_peered":"2024-09-21T03:26:59.312877+0000","last_unstale":"2024-09-21T03:27:33.431029+0000","last_undegraded":"2024-09-21T03:27:33.431029+0000","last_fullsized":"2024-09-21T03:27:33.431029+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:53.856541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427079+0000","last_change":"2024-09-21T03:26:59.307098+0000","last_active":"2024-09-21T03:27:33.427079+0000","last_peered":"2024-09-21T03:27:33.427079+0000","last_clean":"2024-09-21T03:27:33.427079+0000","last_became_active":"2024-09-21T03:26:59.306650+0000","last_became_peered":"2024-09-21T03:26:59.306650+0000","last_unstale":"2024-09-21T03:27:33.427079+0000","last_undegraded":"2024-09-21T03:27:33.427079+0000","last_fullsized":"2024-09-21T03:27:33.427079+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:44.033899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430151+0000","last_change":"2024-09-21T03:27:01.328443+0000","last_active":"2024-09-21T03:27:33.430151+0000","last_peered":"2024-09-21T03:27:33.430151+0000","last_clean":"2024-09-21T03:27:33.430151+0000","last_became_active":"2024-09-21T03:27:01.328107+0000","last_became_peered":"2024-09-21T03:27:01.328107+0000","last_unstale":"2024-09-21T03:27:33.430151+0000","last_undegraded":"2024-09-21T03:27:33.430151+0000","last_fullsized":"2024-09-21T03:27:33.430151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:47.798308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428946+0000","last_change":"2024-09-21T03:26:59.317934+0000","last_active":"2024-09-21T03:27:33.428946+0000","last_peered":"2024-09-21T03:27:33.428946+0000","last_clean":"2024-09-21T03:27:33.428946+0000","last_became_active":"2024-09-21T03:26:59.317166+0000","last_became_peered":"2024-09-21T03:26:59.317166+0000","last_unstale":"2024-09-21T03:27:33.428946+0000","last_undegraded":"2024-09-21T03:27:33.428946+0000","last_fullsized":"2024-09-21T03:27:33.428946+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:52.746406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424624+0000","last_change":"2024-09-21T03:27:01.331226+0000","last_active":"2024-09-21T03:27:33.424624+0000","last_peered":"2024-09-21T03:27:33.424624+0000","last_clean":"2024-09-21T03:27:33.424624+0000","last_became_active":"2024-09-21T03:27:01.330889+0000","last_became_peered":"2024-09-21T03:27:01.330889+0000","last_unstale":"2024-09-21T03:27:33.424624+0000","last_undegraded":"2024-09-21T03:27:33.424624+0000","last_fullsized":"2024-09-21T03:27:33.424624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:51.181554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'3","reported_seq":77,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:01.521435+0000","last_change":"2024-09-21T03:26:59.311671+0000","last_active":"2024-09-21T03:28:01.521435+0000","last_peered":"2024-09-21T03:28:01.521435+0000","last_clean":"2024-09-21T03:28:01.521435+0000","last_became_active":"2024-09-21T03:26:59.311149+0000","last_became_peered":"2024-09-21T03:26:59.311149+0000","last_unstale":"2024-09-21T03:28:01.521435+0000","last_undegraded":"2024-09-21T03:28:01.521435+0000","last_fullsized":"2024-09-21T03:28:01.521435+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:47.878274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.593019+0000","last_change":"2024-09-21T03:27:01.336142+0000","last_active":"2024-09-21T03:27:33.593019+0000","last_peered":"2024-09-21T03:27:33.593019+0000","last_clean":"2024-09-21T03:27:33.593019+0000","last_became_active":"2024-09-21T03:27:01.335795+0000","last_became_peered":"2024-09-21T03:27:01.335795+0000","last_unstale":"2024-09-21T03:27:33.593019+0000","last_undegraded":"2024-09-21T03:27:33.593019+0000","last_fullsized":"2024-09-21T03:27:33.593019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:29.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429365+0000","last_change":"2024-09-21T03:26:59.315077+0000","last_active":"2024-09-21T03:27:33.429365+0000","last_peered":"2024-09-21T03:27:33.429365+0000","last_clean":"2024-09-21T03:27:33.429365+0000","last_became_active":"2024-09-21T03:26:59.312347+0000","last_became_peered":"2024-09-21T03:26:59.312347+0000","last_unstale":"2024-09-21T03:27:33.429365+0000","last_undegraded":"2024-09-21T03:27:33.429365+0000","last_fullsized":"2024-09-21T03:27:33.429365+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:24.529066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428537+0000","last_change":"2024-09-21T03:27:01.324346+0000","last_active":"2024-09-21T03:27:33.428537+0000","last_peered":"2024-09-21T03:27:33.428537+0000","last_clean":"2024-09-21T03:27:33.428537+0000","last_became_active":"2024-09-21T03:27:01.323976+0000","last_became_peered":"2024-09-21T03:27:01.323976+0000","last_unstale":"2024-09-21T03:27:33.428537+0000","last_undegraded":"2024-09-21T03:27:33.428537+0000","last_fullsized":"2024-09-21T03:27:33.428537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:04.591595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429516+0000","last_change":"2024-09-21T03:26:59.326206+0000","last_active":"2024-09-21T03:27:33.429516+0000","last_peered":"2024-09-21T03:27:33.429516+0000","last_clean":"2024-09-21T03:27:33.429516+0000","last_became_active":"2024-09-21T03:26:59.325755+0000","last_became_peered":"2024-09-21T03:26:59.325755+0000","last_unstale":"2024-09-21T03:27:33.429516+0000","last_undegraded":"2024-09-21T03:27:33.429516+0000","last_fullsized":"2024-09-21T03:27:33.429516+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:23.220795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592656+0000","last_change":"2024-09-21T03:27:01.341501+0000","last_active":"2024-09-21T03:27:33.592656+0000","last_peered":"2024-09-21T03:27:33.592656+0000","last_clean":"2024-09-21T03:27:33.592656+0000","last_became_active":"2024-09-21T03:27:01.341039+0000","last_became_peered":"2024-09-21T03:27:01.341039+0000","last_unstale":"2024-09-21T03:27:33.592656+0000","last_undegraded":"2024-09-21T03:27:33.592656+0000","last_fullsized":"2024-09-21T03:27:33.592656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:15.132180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'28","reported_seq":162,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:02.522868+0000","last_change":"2024-09-21T03:26:59.313489+0000","last_active":"2024-09-21T03:28:02.522868+0000","last_peered":"2024-09-21T03:28:02.522868+0000","last_clean":"2024-09-21T03:28:02.522868+0000","last_became_active":"2024-09-21T03:26:59.313022+0000","last_became_peered":"2024-09-21T03:26:59.313022+0000","last_unstale":"2024-09-21T03:28:02.522868+0000","last_undegraded":"2024-09-21T03:28:02.522868+0000","last_fullsized":"2024-09-21T03:28:02.522868+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:08.541315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64757,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":111,"num_read_kb":117,"num_write":29,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429153+0000","last_change":"2024-09-21T03:27:01.339329+0000","last_active":"2024-09-21T03:27:33.429153+0000","last_peered":"2024-09-21T03:27:33.429153+0000","last_clean":"2024-09-21T03:27:33.429153+0000","last_became_active":"2024-09-21T03:27:01.338982+0000","last_became_peered":"2024-09-21T03:27:01.338982+0000","last_unstale":"2024-09-21T03:27:33.429153+0000","last_undegraded":"2024-09-21T03:27:33.429153+0000","last_fullsized":"2024-09-21T03:27:33.429153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:41.759450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'6","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430224+0000","last_change":"2024-09-21T03:26:59.313270+0000","last_active":"2024-09-21T03:27:33.430224+0000","last_peered":"2024-09-21T03:27:33.430224+0000","last_clean":"2024-09-21T03:27:33.430224+0000","last_became_active":"2024-09-21T03:26:59.312867+0000","last_became_peered":"2024-09-21T03:26:59.312867+0000","last_unstale":"2024-09-21T03:27:33.430224+0000","last_undegraded":"2024-09-21T03:27:33.430224+0000","last_fullsized":"2024-09-21T03:27:33.430224+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:53.798022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427146+0000","last_change":"2024-09-21T03:27:01.331268+0000","last_active":"2024-09-21T03:27:33.427146+0000","last_peered":"2024-09-21T03:27:33.427146+0000","last_clean":"2024-09-21T03:27:33.427146+0000","last_became_active":"2024-09-21T03:27:01.330922+0000","last_became_peered":"2024-09-21T03:27:01.330922+0000","last_unstale":"2024-09-21T03:27:33.427146+0000","last_undegraded":"2024-09-21T03:27:33.427146+0000","last_fullsized":"2024-09-21T03:27:33.427146+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428917+0000","last_change":"2024-09-21T03:26:59.318388+0000","last_active":"2024-09-21T03:27:33.428917+0000","last_peered":"2024-09-21T03:27:33.428917+0000","last_clean":"2024-09-21T03:27:33.428917+0000","last_became_active":"2024-09-21T03:26:59.317938+0000","last_became_peered":"2024-09-21T03:26:59.317938+0000","last_unstale":"2024-09-21T03:27:33.428917+0000","last_undegraded":"2024-09-21T03:27:33.428917+0000","last_fullsized":"2024-09-21T03:27:33.428917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.499699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.432373+0000","last_change":"2024-09-21T03:27:33.432191+0000","last_active":"2024-09-21T03:27:33.432373+0000","last_peered":"2024-09-21T03:27:33.432373+0000","last_clean":"2024-09-21T03:27:33.432373+0000","last_became_active":"2024-09-21T03:27:33.431667+0000","last_became_peered":"2024-09-21T03:27:33.431667+0000","last_unstale":"2024-09-21T03:27:33.432373+0000","last_undegraded":"2024-09-21T03:27:33.432373+0000","last_fullsized":"2024-09-21T03:27:33.432373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:13.293063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'4","reported_seq":84,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:03.477714+0000","last_change":"2024-09-21T03:26:59.311345+0000","last_active":"2024-09-21T03:28:03.477714+0000","last_peered":"2024-09-21T03:28:03.477714+0000","last_clean":"2024-09-21T03:28:03.477714+0000","last_became_active":"2024-09-21T03:26:59.310928+0000","last_became_peered":"2024-09-21T03:26:59.310928+0000","last_unstale":"2024-09-21T03:28:03.477714+0000","last_undegraded":"2024-09-21T03:28:03.477714+0000","last_fullsized":"2024-09-21T03:28:03.477714+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.614955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429104+0000","last_change":"2024-09-21T03:27:01.335785+0000","last_active":"2024-09-21T03:27:33.429104+0000","last_peered":"2024-09-21T03:27:33.429104+0000","last_clean":"2024-09-21T03:27:33.429104+0000","last_became_active":"2024-09-21T03:27:01.335396+0000","last_became_peered":"2024-09-21T03:27:01.335396+0000","last_unstale":"2024-09-21T03:27:33.429104+0000","last_undegraded":"2024-09-21T03:27:33.429104+0000","last_fullsized":"2024-09-21T03:27:33.429104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:52.785641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'15","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.448967+0000","last_change":"2024-09-21T03:27:33.448404+0000","last_active":"2024-09-21T03:27:33.448967+0000","last_peered":"2024-09-21T03:27:33.448967+0000","last_clean":"2024-09-21T03:27:33.448967+0000","last_became_active":"2024-09-21T03:27:33.434405+0000","last_became_peered":"2024-09-21T03:27:33.434405+0000","last_unstale":"2024-09-21T03:27:33.448967+0000","last_undegraded":"2024-09-21T03:27:33.448967+0000","last_fullsized":"2024-09-21T03:27:33.448967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:05.107391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429036+0000","last_change":"2024-09-21T03:27:01.342282+0000","last_active":"2024-09-21T03:27:33.429036+0000","last_peered":"2024-09-21T03:27:33.429036+0000","last_clean":"2024-09-21T03:27:33.429036+0000","last_became_active":"2024-09-21T03:27:01.341790+0000","last_became_peered":"2024-09-21T03:27:01.341790+0000","last_unstale":"2024-09-21T03:27:33.429036+0000","last_undegraded":"2024-09-21T03:27:33.429036+0000","last_fullsized":"2024-09-21T03:27:33.429036+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:05.432720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428084+0000","last_change":"2024-09-21T03:26:59.312442+0000","last_active":"2024-09-21T03:27:33.428084+0000","last_peered":"2024-09-21T03:27:33.428084+0000","last_clean":"2024-09-21T03:27:33.428084+0000","last_became_active":"2024-09-21T03:26:59.311601+0000","last_became_peered":"2024-09-21T03:26:59.311601+0000","last_unstale":"2024-09-21T03:27:33.428084+0000","last_undegraded":"2024-09-21T03:27:33.428084+0000","last_fullsized":"2024-09-21T03:27:33.428084+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:32.421228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.426347+0000","last_change":"2024-09-21T03:27:01.337917+0000","last_active":"2024-09-21T03:27:33.426347+0000","last_peered":"2024-09-21T03:27:33.426347+0000","last_clean":"2024-09-21T03:27:33.426347+0000","last_became_active":"2024-09-21T03:27:01.337457+0000","last_became_peered":"2024-09-21T03:27:01.337457+0000","last_unstale":"2024-09-21T03:27:33.426347+0000","last_undegraded":"2024-09-21T03:27:33.426347+0000","last_fullsized":"2024-09-21T03:27:33.426347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:49.011795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.597334+0000","last_change":"2024-09-21T03:27:33.597239+0000","last_active":"2024-09-21T03:27:33.597334+0000","last_peered":"2024-09-21T03:27:33.597334+0000","last_clean":"2024-09-21T03:27:33.597334+0000","last_became_active":"2024-09-21T03:27:33.593787+0000","last_became_peered":"2024-09-21T03:27:33.593787+0000","last_unstale":"2024-09-21T03:27:33.597334+0000","last_undegraded":"2024-09-21T03:27:33.597334+0000","last_fullsized":"2024-09-21T03:27:33.597334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:35.015446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429396+0000","last_change":"2024-09-21T03:27:01.338489+0000","last_active":"2024-09-21T03:27:33.429396+0000","last_peered":"2024-09-21T03:27:33.429396+0000","last_clean":"2024-09-21T03:27:33.429396+0000","last_became_active":"2024-09-21T03:27:01.338123+0000","last_became_peered":"2024-09-21T03:27:01.338123+0000","last_unstale":"2024-09-21T03:27:33.429396+0000","last_undegraded":"2024-09-21T03:27:33.429396+0000","last_fullsized":"2024-09-21T03:27:33.429396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:15.137546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429644+0000","last_change":"2024-09-21T03:26:59.315371+0000","last_active":"2024-09-21T03:27:33.429644+0000","last_peered":"2024-09-21T03:27:33.429644+0000","last_clean":"2024-09-21T03:27:33.429644+0000","last_became_active":"2024-09-21T03:26:59.314439+0000","last_became_peered":"2024-09-21T03:26:59.314439+0000","last_unstale":"2024-09-21T03:27:33.429644+0000","last_undegraded":"2024-09-21T03:27:33.429644+0000","last_fullsized":"2024-09-21T03:27:33.429644+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:13.656324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429627+0000","last_change":"2024-09-21T03:27:01.331086+0000","last_active":"2024-09-21T03:27:33.429627+0000","last_peered":"2024-09-21T03:27:33.429627+0000","last_clean":"2024-09-21T03:27:33.429627+0000","last_became_active":"2024-09-21T03:27:01.329113+0000","last_became_peered":"2024-09-21T03:27:01.329113+0000","last_unstale":"2024-09-21T03:27:33.429627+0000","last_undegraded":"2024-09-21T03:27:33.429627+0000","last_fullsized":"2024-09-21T03:27:33.429627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:25.514730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.433098+0000","last_change":"2024-09-21T03:26:59.554087+0000","last_active":"2024-09-21T03:27:33.433098+0000","last_peered":"2024-09-21T03:27:33.433098+0000","last_clean":"2024-09-21T03:27:33.433098+0000","last_became_active":"2024-09-21T03:26:59.553610+0000","last_became_peered":"2024-09-21T03:26:59.553610+0000","last_unstale":"2024-09-21T03:27:33.433098+0000","last_undegraded":"2024-09-21T03:27:33.433098+0000","last_fullsized":"2024-09-21T03:27:33.433098+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:22.048057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429773+0000","last_change":"2024-09-21T03:27:01.339987+0000","last_active":"2024-09-21T03:27:33.429773+0000","last_peered":"2024-09-21T03:27:33.429773+0000","last_clean":"2024-09-21T03:27:33.429773+0000","last_became_active":"2024-09-21T03:27:01.339060+0000","last_became_peered":"2024-09-21T03:27:01.339060+0000","last_unstale":"2024-09-21T03:27:33.429773+0000","last_undegraded":"2024-09-21T03:27:33.429773+0000","last_fullsized":"2024-09-21T03:27:33.429773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.901337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429688+0000","last_change":"2024-09-21T03:25:50.381920+0000","last_active":"2024-09-21T03:27:33.429688+0000","last_peered":"2024-09-21T03:27:33.429688+0000","last_clean":"2024-09-21T03:27:33.429688+0000","last_became_active":"2024-09-21T03:25:50.376170+0000","last_became_peered":"2024-09-21T03:25:50.376170+0000","last_unstale":"2024-09-21T03:27:33.429688+0000","last_undegraded":"2024-09-21T03:27:33.429688+0000","last_fullsized":"2024-09-21T03:27:33.429688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_clean_scrub_stamp":"2024-09-21T03:22:31.269883+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:57.281150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'3","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.477054+0000","last_change":"2024-09-21T03:26:59.313612+0000","last_active":"2024-09-21T03:27:33.477054+0000","last_peered":"2024-09-21T03:27:33.477054+0000","last_clean":"2024-09-21T03:27:33.477054+0000","last_became_active":"2024-09-21T03:26:59.313060+0000","last_became_peered":"2024-09-21T03:26:59.313060+0000","last_unstale":"2024-09-21T03:27:33.477054+0000","last_undegraded":"2024-09-21T03:27:33.477054+0000","last_fullsized":"2024-09-21T03:27:33.477054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:06.193526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430283+0000","last_change":"2024-09-21T03:27:01.337784+0000","last_active":"2024-09-21T03:27:33.430283+0000","last_peered":"2024-09-21T03:27:33.430283+0000","last_clean":"2024-09-21T03:27:33.430283+0000","last_became_active":"2024-09-21T03:27:01.337415+0000","last_became_peered":"2024-09-21T03:27:01.337415+0000","last_unstale":"2024-09-21T03:27:33.430283+0000","last_undegraded":"2024-09-21T03:27:33.430283+0000","last_fullsized":"2024-09-21T03:27:33.430283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:40.482054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'3","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.436960+0000","last_change":"2024-09-21T03:27:33.436319+0000","last_active":"2024-09-21T03:27:33.436960+0000","last_peered":"2024-09-21T03:27:33.436960+0000","last_clean":"2024-09-21T03:27:33.436960+0000","last_became_active":"2024-09-21T03:27:33.431555+0000","last_became_peered":"2024-09-21T03:27:33.431555+0000","last_unstale":"2024-09-21T03:27:33.436960+0000","last_undegraded":"2024-09-21T03:27:33.436960+0000","last_fullsized":"2024-09-21T03:27:33.436960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:16.073988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427062+0000","last_change":"2024-09-21T03:27:01.333769+0000","last_active":"2024-09-21T03:27:33.427062+0000","last_peered":"2024-09-21T03:27:33.427062+0000","last_clean":"2024-09-21T03:27:33.427062+0000","last_became_active":"2024-09-21T03:27:01.333436+0000","last_became_peered":"2024-09-21T03:27:01.333436+0000","last_unstale":"2024-09-21T03:27:33.427062+0000","last_undegraded":"2024-09-21T03:27:33.427062+0000","last_fullsized":"2024-09-21T03:27:33.427062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:30.395916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'4","reported_seq":81,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:03.476911+0000","last_change":"2024-09-21T03:26:59.325314+0000","last_active":"2024-09-21T03:28:03.476911+0000","last_peered":"2024-09-21T03:28:03.476911+0000","last_clean":"2024-09-21T03:28:03.476911+0000","last_became_active":"2024-09-21T03:26:59.324752+0000","last_became_peered":"2024-09-21T03:26:59.324752+0000","last_unstale":"2024-09-21T03:28:03.476911+0000","last_undegraded":"2024-09-21T03:28:03.476911+0000","last_fullsized":"2024-09-21T03:28:03.476911+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.252596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428802+0000","last_change":"2024-09-21T03:27:01.335154+0000","last_active":"2024-09-21T03:27:33.428802+0000","last_peered":"2024-09-21T03:27:33.428802+0000","last_clean":"2024-09-21T03:27:33.428802+0000","last_became_active":"2024-09-21T03:27:01.334633+0000","last_became_peered":"2024-09-21T03:27:01.334633+0000","last_unstale":"2024-09-21T03:27:33.428802+0000","last_undegraded":"2024-09-21T03:27:33.428802+0000","last_fullsized":"2024-09-21T03:27:33.428802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:27.992858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429757+0000","last_change":"2024-09-21T03:26:59.314019+0000","last_active":"2024-09-21T03:27:33.429757+0000","last_peered":"2024-09-21T03:27:33.429757+0000","last_clean":"2024-09-21T03:27:33.429757+0000","last_became_active":"2024-09-21T03:26:59.311915+0000","last_became_peered":"2024-09-21T03:26:59.311915+0000","last_unstale":"2024-09-21T03:27:33.429757+0000","last_undegraded":"2024-09-21T03:27:33.429757+0000","last_fullsized":"2024-09-21T03:27:33.429757+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:51.412303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.433313+0000","last_change":"2024-09-21T03:27:33.433193+0000","last_active":"2024-09-21T03:27:33.433313+0000","last_peered":"2024-09-21T03:27:33.433313+0000","last_clean":"2024-09-21T03:27:33.433313+0000","last_became_active":"2024-09-21T03:27:33.432729+0000","last_became_peered":"2024-09-21T03:27:33.432729+0000","last_unstale":"2024-09-21T03:27:33.433313+0000","last_undegraded":"2024-09-21T03:27:33.433313+0000","last_fullsized":"2024-09-21T03:27:33.433313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:37.575110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424782+0000","last_change":"2024-09-21T03:26:59.325943+0000","last_active":"2024-09-21T03:27:33.424782+0000","last_peered":"2024-09-21T03:27:33.424782+0000","last_clean":"2024-09-21T03:27:33.424782+0000","last_became_active":"2024-09-21T03:26:59.325524+0000","last_became_peered":"2024-09-21T03:26:59.325524+0000","last_unstale":"2024-09-21T03:27:33.424782+0000","last_undegraded":"2024-09-21T03:27:33.424782+0000","last_fullsized":"2024-09-21T03:27:33.424782+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:45.111480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.431730+0000","last_change":"2024-09-21T03:27:01.335369+0000","last_active":"2024-09-21T03:27:33.431730+0000","last_peered":"2024-09-21T03:27:33.431730+0000","last_clean":"2024-09-21T03:27:33.431730+0000","last_became_active":"2024-09-21T03:27:01.334997+0000","last_became_peered":"2024-09-21T03:27:01.334997+0000","last_unstale":"2024-09-21T03:27:33.431730+0000","last_undegraded":"2024-09-21T03:27:33.431730+0000","last_fullsized":"2024-09-21T03:27:33.431730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:10.257451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429553+0000","last_change":"2024-09-21T03:26:59.326204+0000","last_active":"2024-09-21T03:27:33.429553+0000","last_peered":"2024-09-21T03:27:33.429553+0000","last_clean":"2024-09-21T03:27:33.429553+0000","last_became_active":"2024-09-21T03:26:59.325431+0000","last_became_peered":"2024-09-21T03:26:59.325431+0000","last_unstale":"2024-09-21T03:27:33.429553+0000","last_undegraded":"2024-09-21T03:27:33.429553+0000","last_fullsized":"2024-09-21T03:27:33.429553+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:27.863217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427579+0000","last_change":"2024-09-21T03:27:01.333861+0000","last_active":"2024-09-21T03:27:33.427579+0000","last_peered":"2024-09-21T03:27:33.427579+0000","last_clean":"2024-09-21T03:27:33.427579+0000","last_became_active":"2024-09-21T03:27:01.333485+0000","last_became_peered":"2024-09-21T03:27:01.333485+0000","last_unstale":"2024-09-21T03:27:33.427579+0000","last_undegraded":"2024-09-21T03:27:33.427579+0000","last_fullsized":"2024-09-21T03:27:33.427579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:04.475811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"74'11","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424867+0000","last_change":"2024-09-21T03:26:59.309767+0000","last_active":"2024-09-21T03:27:33.424867+0000","last_peered":"2024-09-21T03:27:33.424867+0000","last_clean":"2024-09-21T03:27:33.424867+0000","last_became_active":"2024-09-21T03:26:59.309354+0000","last_became_peered":"2024-09-21T03:26:59.309354+0000","last_unstale":"2024-09-21T03:27:33.424867+0000","last_undegraded":"2024-09-21T03:27:33.424867+0000","last_fullsized":"2024-09-21T03:27:33.424867+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:26.728719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430664+0000","last_change":"2024-09-21T03:27:01.333097+0000","last_active":"2024-09-21T03:27:33.430664+0000","last_peered":"2024-09-21T03:27:33.430664+0000","last_clean":"2024-09-21T03:27:33.430664+0000","last_became_active":"2024-09-21T03:27:01.332743+0000","last_became_peered":"2024-09-21T03:27:01.332743+0000","last_unstale":"2024-09-21T03:27:33.430664+0000","last_undegraded":"2024-09-21T03:27:33.430664+0000","last_fullsized":"2024-09-21T03:27:33.430664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:41.004213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.476588+0000","last_change":"2024-09-21T03:26:59.553974+0000","last_active":"2024-09-21T03:27:33.476588+0000","last_peered":"2024-09-21T03:27:33.476588+0000","last_clean":"2024-09-21T03:27:33.476588+0000","last_became_active":"2024-09-21T03:26:59.553647+0000","last_became_peered":"2024-09-21T03:26:59.553647+0000","last_unstale":"2024-09-21T03:27:33.476588+0000","last_undegraded":"2024-09-21T03:27:33.476588+0000","last_fullsized":"2024-09-21T03:27:33.476588+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:00.490951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428356+0000","last_change":"2024-09-21T03:27:01.325308+0000","last_active":"2024-09-21T03:27:33.428356+0000","last_peered":"2024-09-21T03:27:33.428356+0000","last_clean":"2024-09-21T03:27:33.428356+0000","last_became_active":"2024-09-21T03:27:01.324977+0000","last_became_peered":"2024-09-21T03:27:01.324977+0000","last_unstale":"2024-09-21T03:27:33.428356+0000","last_undegraded":"2024-09-21T03:27:33.428356+0000","last_fullsized":"2024-09-21T03:27:33.428356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:10.093856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592537+0000","last_change":"2024-09-21T03:26:59.326017+0000","last_active":"2024-09-21T03:27:33.592537+0000","last_peered":"2024-09-21T03:27:33.592537+0000","last_clean":"2024-09-21T03:27:33.592537+0000","last_became_active":"2024-09-21T03:26:59.325588+0000","last_became_peered":"2024-09-21T03:26:59.325588+0000","last_unstale":"2024-09-21T03:27:33.592537+0000","last_undegraded":"2024-09-21T03:27:33.592537+0000","last_fullsized":"2024-09-21T03:27:33.592537+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:39.740449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427052+0000","last_change":"2024-09-21T03:27:01.336871+0000","last_active":"2024-09-21T03:27:33.427052+0000","last_peered":"2024-09-21T03:27:33.427052+0000","last_clean":"2024-09-21T03:27:33.427052+0000","last_became_active":"2024-09-21T03:27:01.336507+0000","last_became_peered":"2024-09-21T03:27:01.336507+0000","last_unstale":"2024-09-21T03:27:33.427052+0000","last_undegraded":"2024-09-21T03:27:33.427052+0000","last_fullsized":"2024-09-21T03:27:33.427052+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:58.674375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430477+0000","last_change":"2024-09-21T03:26:59.308997+0000","last_active":"2024-09-21T03:27:33.430477+0000","last_peered":"2024-09-21T03:27:33.430477+0000","last_clean":"2024-09-21T03:27:33.430477+0000","last_became_active":"2024-09-21T03:26:59.308338+0000","last_became_peered":"2024-09-21T03:26:59.308338+0000","last_unstale":"2024-09-21T03:27:33.430477+0000","last_undegraded":"2024-09-21T03:27:33.430477+0000","last_fullsized":"2024-09-21T03:27:33.430477+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:15.328468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.426751+0000","last_change":"2024-09-21T03:27:01.338808+0000","last_active":"2024-09-21T03:27:33.426751+0000","last_peered":"2024-09-21T03:27:33.426751+0000","last_clean":"2024-09-21T03:27:33.426751+0000","last_became_active":"2024-09-21T03:27:01.337987+0000","last_became_peered":"2024-09-21T03:27:01.337987+0000","last_unstale":"2024-09-21T03:27:33.426751+0000","last_undegraded":"2024-09-21T03:27:33.426751+0000","last_fullsized":"2024-09-21T03:27:33.426751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:56.265014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430705+0000","last_change":"2024-09-21T03:26:59.312684+0000","last_active":"2024-09-21T03:27:33.430705+0000","last_peered":"2024-09-21T03:27:33.430705+0000","last_clean":"2024-09-21T03:27:33.430705+0000","last_became_active":"2024-09-21T03:26:59.312038+0000","last_became_peered":"2024-09-21T03:26:59.312038+0000","last_unstale":"2024-09-21T03:27:33.430705+0000","last_undegraded":"2024-09-21T03:27:33.430705+0000","last_fullsized":"2024-09-21T03:27:33.430705+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:02.672255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430351+0000","last_change":"2024-09-21T03:27:01.333080+0000","last_active":"2024-09-21T03:27:33.430351+0000","last_peered":"2024-09-21T03:27:33.430351+0000","last_clean":"2024-09-21T03:27:33.430351+0000","last_became_active":"2024-09-21T03:27:01.332663+0000","last_became_peered":"2024-09-21T03:27:01.332663+0000","last_unstale":"2024-09-21T03:27:33.430351+0000","last_undegraded":"2024-09-21T03:27:33.430351+0000","last_fullsized":"2024-09-21T03:27:33.430351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:34.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427673+0000","last_change":"2024-09-21T03:26:59.315490+0000","last_active":"2024-09-21T03:27:33.427673+0000","last_peered":"2024-09-21T03:27:33.427673+0000","last_clean":"2024-09-21T03:27:33.427673+0000","last_became_active":"2024-09-21T03:26:59.313789+0000","last_became_peered":"2024-09-21T03:26:59.313789+0000","last_unstale":"2024-09-21T03:27:33.427673+0000","last_undegraded":"2024-09-21T03:27:33.427673+0000","last_fullsized":"2024-09-21T03:27:33.427673+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:10.140711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429489+0000","last_change":"2024-09-21T03:27:01.341019+0000","last_active":"2024-09-21T03:27:33.429489+0000","last_peered":"2024-09-21T03:27:33.429489+0000","last_clean":"2024-09-21T03:27:33.429489+0000","last_became_active":"2024-09-21T03:27:01.340660+0000","last_became_peered":"2024-09-21T03:27:01.340660+0000","last_unstale":"2024-09-21T03:27:33.429489+0000","last_undegraded":"2024-09-21T03:27:33.429489+0000","last_fullsized":"2024-09-21T03:27:33.429489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:16.736993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429385+0000","last_change":"2024-09-21T03:27:01.341165+0000","last_active":"2024-09-21T03:27:33.429385+0000","last_peered":"2024-09-21T03:27:33.429385+0000","last_clean":"2024-09-21T03:27:33.429385+0000","last_became_active":"2024-09-21T03:27:01.340765+0000","last_became_peered":"2024-09-21T03:27:01.340765+0000","last_unstale":"2024-09-21T03:27:33.429385+0000","last_undegraded":"2024-09-21T03:27:33.429385+0000","last_fullsized":"2024-09-21T03:27:33.429385+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:15.455944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429107+0000","last_change":"2024-09-21T03:26:59.313410+0000","last_active":"2024-09-21T03:27:33.429107+0000","last_peered":"2024-09-21T03:27:33.429107+0000","last_clean":"2024-09-21T03:27:33.429107+0000","last_became_active":"2024-09-21T03:26:59.312233+0000","last_became_peered":"2024-09-21T03:26:59.312233+0000","last_unstale":"2024-09-21T03:27:33.429107+0000","last_undegraded":"2024-09-21T03:27:33.429107+0000","last_fullsized":"2024-09-21T03:27:33.429107+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:26.089611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592537+0000","last_change":"2024-09-21T03:27:01.335332+0000","last_active":"2024-09-21T03:27:33.592537+0000","last_peered":"2024-09-21T03:27:33.592537+0000","last_clean":"2024-09-21T03:27:33.592537+0000","last_became_active":"2024-09-21T03:27:01.334872+0000","last_became_peered":"2024-09-21T03:27:01.334872+0000","last_unstale":"2024-09-21T03:27:33.592537+0000","last_undegraded":"2024-09-21T03:27:33.592537+0000","last_fullsized":"2024-09-21T03:27:33.592537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:39.631246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429421+0000","last_change":"2024-09-21T03:26:59.314250+0000","last_active":"2024-09-21T03:27:33.429421+0000","last_peered":"2024-09-21T03:27:33.429421+0000","last_clean":"2024-09-21T03:27:33.429421+0000","last_became_active":"2024-09-21T03:26:59.313707+0000","last_became_peered":"2024-09-21T03:26:59.313707+0000","last_unstale":"2024-09-21T03:27:33.429421+0000","last_undegraded":"2024-09-21T03:27:33.429421+0000","last_fullsized":"2024-09-21T03:27:33.429421+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:22.935292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428509+0000","last_change":"2024-09-21T03:27:01.339230+0000","last_active":"2024-09-21T03:27:33.428509+0000","last_peered":"2024-09-21T03:27:33.428509+0000","last_clean":"2024-09-21T03:27:33.428509+0000","last_became_active":"2024-09-21T03:27:01.338881+0000","last_became_peered":"2024-09-21T03:27:01.338881+0000","last_unstale":"2024-09-21T03:27:33.428509+0000","last_undegraded":"2024-09-21T03:27:33.428509+0000","last_fullsized":"2024-09-21T03:27:33.428509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:49.189180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430888+0000","last_change":"2024-09-21T03:26:59.326549+0000","last_active":"2024-09-21T03:27:33.430888+0000","last_peered":"2024-09-21T03:27:33.430888+0000","last_clean":"2024-09-21T03:27:33.430888+0000","last_became_active":"2024-09-21T03:26:59.326091+0000","last_became_peered":"2024-09-21T03:26:59.326091+0000","last_unstale":"2024-09-21T03:27:33.430888+0000","last_undegraded":"2024-09-21T03:27:33.430888+0000","last_fullsized":"2024-09-21T03:27:33.430888+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:13.507716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430739+0000","last_change":"2024-09-21T03:27:01.331367+0000","last_active":"2024-09-21T03:27:33.430739+0000","last_peered":"2024-09-21T03:27:33.430739+0000","last_clean":"2024-09-21T03:27:33.430739+0000","last_became_active":"2024-09-21T03:27:01.330604+0000","last_became_peered":"2024-09-21T03:27:01.330604+0000","last_unstale":"2024-09-21T03:27:33.430739+0000","last_undegraded":"2024-09-21T03:27:33.430739+0000","last_fullsized":"2024-09-21T03:27:33.430739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:32.207561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'9","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424843+0000","last_change":"2024-09-21T03:26:59.311912+0000","last_active":"2024-09-21T03:27:33.424843+0000","last_peered":"2024-09-21T03:27:33.424843+0000","last_clean":"2024-09-21T03:27:33.424843+0000","last_became_active":"2024-09-21T03:26:59.311571+0000","last_became_peered":"2024-09-21T03:26:59.311571+0000","last_unstale":"2024-09-21T03:27:33.424843+0000","last_undegraded":"2024-09-21T03:27:33.424843+0000","last_fullsized":"2024-09-21T03:27:33.424843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:55.946494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68758,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":340,"num_read_kb":240,"num_write":163,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":206274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776155,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":843776,"data_stored":413299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939678,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1536000,"data_stored":1069816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":851968,"data_stored":413423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299433,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":843776,"data_stored":413231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462959,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1445888,"data_stored":1005273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659186,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":412812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855416,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1458176,"data_stored":1004566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":851968,"data_stored":414262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":856064,"data_stored":413123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":478090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542221,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":905216,"data_stored":477362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705745,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":860160,"data_stored":414083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:28:09.438 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T03:28:09.438 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T03:28:09.438 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T03:28:09.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T03:28:09.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:09 smithi187 ceph-mon[34603]: pgmap v292: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:28:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:09.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:09 smithi112 ceph-mon[31442]: pgmap v292: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:28:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:09 smithi203 ceph-mon[34723]: pgmap v292: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:28:10.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.158+0000 7f8eed659640 1 -- 172.21.15.112:0/840518149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8103b60 msgr2=0x7f8ee8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:10.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.158+0000 7f8eed659640 1 --2- 172.21.15.112:0/840518149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8103b60 0x7f8ee8103f70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009900 tx=0x7f8ed802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:10.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.158+0000 7f8eed659640 1 -- 172.21.15.112:0/840518149 shutdown_connections 2024-09-21T03:28:10.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.158+0000 7f8eed659640 1 --2- 172.21.15.112:0/840518149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee8109df0 0x7f8ee8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:10.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.158+0000 7f8eed659640 1 --2- 172.21.15.112:0/840518149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee81094a0 0x7f8ee81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:10.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.158+0000 7f8eed659640 1 --2- 172.21.15.112:0/840518149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8103b60 0x7f8ee8103f70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:10.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.158+0000 7f8eed659640 1 -- 172.21.15.112:0/840518149 >> 172.21.15.112:0/840518149 conn(0x7f8ee807b850 msgr2=0x7f8ee807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:10.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.158+0000 7f8eed659640 1 -- 172.21.15.112:0/840518149 shutdown_connections 2024-09-21T03:28:10.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.158+0000 7f8eed659640 1 -- 172.21.15.112:0/840518149 wait complete. 2024-09-21T03:28:10.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.159+0000 7f8eed659640 1 Processor -- start 2024-09-21T03:28:10.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.159+0000 7f8eed659640 1 -- start start 2024-09-21T03:28:10.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.159+0000 7f8eed659640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8103b60 0x7f8ee8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:10.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.159+0000 7f8eed659640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81094a0 0x7f8ee8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:10.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8eed659640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8109df0 0x7f8ee8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:10.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8eed659640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8113ab0 con 0x7f8ee8109df0 2024-09-21T03:28:10.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8eed659640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8113930 con 0x7f8ee8103b60 2024-09-21T03:28:10.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8eed659640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8113c30 con 0x7f8ee81094a0 2024-09-21T03:28:10.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8109df0 0x7f8ee8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:10.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8109df0 0x7f8ee8076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42068/0 (socket says 172.21.15.112:42068) 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee77fe640 1 -- 172.21.15.112:0/2689263740 learned_addr learned my addr 172.21.15.112:0/2689263740 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee67fc640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81094a0 0x7f8ee8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee6ffd640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8103b60 0x7f8ee8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee77fe640 1 -- 172.21.15.112:0/2689263740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81094a0 msgr2=0x7f8ee8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee77fe640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81094a0 0x7f8ee8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee77fe640 1 -- 172.21.15.112:0/2689263740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8103b60 msgr2=0x7f8ee8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee77fe640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8103b60 0x7f8ee8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee77fe640 1 -- 172.21.15.112:0/2689263740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee81a8760 con 0x7f8ee8109df0 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee6ffd640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8103b60 0x7f8ee8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ee77fe640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8109df0 0x7f8ee8076190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000eaa0 tx=0x7f8ee000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ecffff640 1 -- 172.21.15.112:0/2689263740 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000cdb0 con 0x7f8ee8109df0 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ecffff640 1 -- 172.21.15.112:0/2689263740 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee0005020 con 0x7f8ee8109df0 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8eed659640 1 -- 172.21.15.112:0/2689263740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee81a8a50 con 0x7f8ee8109df0 2024-09-21T03:28:10.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8ecffff640 1 -- 172.21.15.112:0/2689263740 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00116d0 con 0x7f8ee8109df0 2024-09-21T03:28:10.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.160+0000 7f8eed659640 1 -- 172.21.15.112:0/2689263740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee81a8f40 con 0x7f8ee8109df0 2024-09-21T03:28:10.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.162+0000 7f8eed659640 1 -- 172.21.15.112:0/2689263740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee8103f70 con 0x7f8ee8109df0 2024-09-21T03:28:10.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.163+0000 7f8ecffff640 1 -- 172.21.15.112:0/2689263740 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f8ee00051d0 con 0x7f8ee8109df0 2024-09-21T03:28:10.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.163+0000 7f8ecffff640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed00b65f0 0x7f8ed00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:10.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.164+0000 7f8ee6ffd640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed00b65f0 0x7f8ed00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:10.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.164+0000 7f8ecffff640 1 -- 172.21.15.112:0/2689263740 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f8ee00e7870 con 0x7f8ee8109df0 2024-09-21T03:28:10.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.164+0000 7f8ee6ffd640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed00b65f0 0x7f8ed00b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8000c00 tx=0x7f8ed8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:10.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.168+0000 7f8ecffff640 1 -- 172.21.15.112:0/2689263740 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8ee0016030 con 0x7f8ee8109df0 2024-09-21T03:28:10.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.384+0000 7f8eed659640 1 -- 172.21.15.112:0/2689263740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8ee8076a10 con 0x7f8ee8109df0 2024-09-21T03:28:10.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.384+0000 7f8ecffff640 1 -- 172.21.15.112:0/2689263740 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8ee00adee0 con 0x7f8ee8109df0 2024-09-21T03:28:10.386 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:10.386 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T03:28:10.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 -- 172.21.15.112:0/2689263740 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed00b65f0 msgr2=0x7f8ed00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:10.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed00b65f0 0x7f8ed00b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8000c00 tx=0x7f8ed8031400 comp rx=0 tx=0).stop 2024-09-21T03:28:10.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 -- 172.21.15.112:0/2689263740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8109df0 msgr2=0x7f8ee8076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:10.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8109df0 0x7f8ee8076190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000eaa0 tx=0x7f8ee000ef60 comp rx=0 tx=0).stop 2024-09-21T03:28:10.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 -- 172.21.15.112:0/2689263740 shutdown_connections 2024-09-21T03:28:10.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed00b65f0 0x7f8ed00b8b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:10.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8109df0 0x7f8ee8076190 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:10.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81094a0 0x7f8ee8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:10.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 --2- 172.21.15.112:0/2689263740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8103b60 0x7f8ee8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:10.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 -- 172.21.15.112:0/2689263740 >> 172.21.15.112:0/2689263740 conn(0x7f8ee807b850 msgr2=0x7f8ee8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:10.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 -- 172.21.15.112:0/2689263740 shutdown_connections 2024-09-21T03:28:10.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:10.387+0000 7f8eed659640 1 -- 172.21.15.112:0/2689263740 wait complete. 2024-09-21T03:28:10.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:10 smithi187 ceph-mon[34603]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:10.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:10 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2689263740' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:28:10.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:10 smithi112 ceph-mon[31442]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:10.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:10 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2689263740' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:28:10.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:10 smithi203 ceph-mon[34723]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:10.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:10 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2689263740' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T03:28:11.005 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T03:28:11.006 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:11.254 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:11.647 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:11 smithi112 ceph-mon[31442]: pgmap v293: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:11 smithi187 ceph-mon[34603]: pgmap v293: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:11.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.734+0000 7f2a506b3640 1 -- 172.21.15.112:0/403817012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a481094c0 msgr2=0x7f2a481098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:11.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.734+0000 7f2a506b3640 1 --2- 172.21.15.112:0/403817012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a481094c0 0x7f2a481098d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009900 tx=0x7f2a3802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.734+0000 7f2a506b3640 1 -- 172.21.15.112:0/403817012 shutdown_connections 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.734+0000 7f2a506b3640 1 --2- 172.21.15.112:0/403817012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a48109e10 0x7f2a48110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.734+0000 7f2a506b3640 1 --2- 172.21.15.112:0/403817012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a481094c0 0x7f2a481098d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.734+0000 7f2a506b3640 1 --2- 172.21.15.112:0/403817012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a48103b80 0x7f2a48103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.734+0000 7f2a506b3640 1 -- 172.21.15.112:0/403817012 >> 172.21.15.112:0/403817012 conn(0x7f2a480ff610 msgr2=0x7f2a48101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.734+0000 7f2a506b3640 1 -- 172.21.15.112:0/403817012 shutdown_connections 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.734+0000 7f2a506b3640 1 -- 172.21.15.112:0/403817012 wait complete. 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a506b3640 1 Processor -- start 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a506b3640 1 -- start start 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a506b3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a48103b80 0x7f2a4819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a506b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a481094c0 0x7f2a4819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a506b3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a48109e10 0x7f2a4819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:11.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a506b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48113af0 con 0x7f2a481094c0 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a506b3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48113970 con 0x7f2a48109e10 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a506b3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48113c70 con 0x7f2a48103b80 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a4dc4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a481094c0 0x7f2a4819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a4dc4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a481094c0 0x7f2a4819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42084/0 (socket says 172.21.15.112:42084) 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a4dc4e640 1 -- 172.21.15.112:0/795663206 learned_addr learned my addr 172.21.15.112:0/795663206 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a4ec50640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a48109e10 0x7f2a4819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a4dc4e640 1 -- 172.21.15.112:0/795663206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a48103b80 msgr2=0x7f2a4819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a4dc4e640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a48103b80 0x7f2a4819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a4dc4e640 1 -- 172.21.15.112:0/795663206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a48109e10 msgr2=0x7f2a4819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a4dc4e640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a48109e10 0x7f2a4819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a4dc4e640 1 -- 172.21.15.112:0/795663206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a481a0450 con 0x7f2a481094c0 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.735+0000 7f2a4ec50640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a48109e10 0x7f2a4819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:11.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.736+0000 7f2a4dc4e640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a481094c0 0x7f2a4819aa80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2a3802a930 tx=0x7f2a38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.736+0000 7f2a3f7fe640 1 -- 172.21.15.112:0/795663206 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3803f030 con 0x7f2a481094c0 2024-09-21T03:28:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.736+0000 7f2a506b3640 1 -- 172.21.15.112:0/795663206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a481a06e0 con 0x7f2a481094c0 2024-09-21T03:28:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.736+0000 7f2a3f7fe640 1 -- 172.21.15.112:0/795663206 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a38003cb0 con 0x7f2a481094c0 2024-09-21T03:28:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.736+0000 7f2a3f7fe640 1 -- 172.21.15.112:0/795663206 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38035070 con 0x7f2a481094c0 2024-09-21T03:28:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.736+0000 7f2a506b3640 1 -- 172.21.15.112:0/795663206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a481a6ea0 con 0x7f2a481094c0 2024-09-21T03:28:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.737+0000 7f2a506b3640 1 -- 172.21.15.112:0/795663206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a48103f90 con 0x7f2a481094c0 2024-09-21T03:28:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.739+0000 7f2a3f7fe640 1 -- 172.21.15.112:0/795663206 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f2a38042030 con 0x7f2a481094c0 2024-09-21T03:28:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.739+0000 7f2a3f7fe640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2a340b65f0 0x7f2a340b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.739+0000 7f2a4e44f640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2a340b65f0 0x7f2a340b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.739+0000 7f2a3f7fe640 1 -- 172.21.15.112:0/795663206 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f2a381080c0 con 0x7f2a481094c0 2024-09-21T03:28:11.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.740+0000 7f2a4e44f640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2a340b65f0 0x7f2a340b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400a8f0 tx=0x7f2a44005bf0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:11.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.743+0000 7f2a3f7fe640 1 -- 172.21.15.112:0/795663206 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2a380ce730 con 0x7f2a481094c0 2024-09-21T03:28:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:11 smithi203 ceph-mon[34723]: pgmap v293: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:11.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.957+0000 7f2a506b3640 1 -- 172.21.15.112:0/795663206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a4819b610 con 0x7f2a481094c0 2024-09-21T03:28:11.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.958+0000 7f2a3f7fe640 1 -- 172.21.15.112:0/795663206 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11929 (secure 0 0 0) 0x7f2a4819b610 con 0x7f2a481094c0 2024-09-21T03:28:11.960 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:11.960 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"btime":"2024-09-21T03:27:13:388843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:27:13.104308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:28:11.960 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-09-21T03:28:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.960+0000 7f2a506b3640 1 -- 172.21.15.112:0/795663206 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2a340b65f0 msgr2=0x7f2a340b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.960+0000 7f2a506b3640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2a340b65f0 0x7f2a340b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400a8f0 tx=0x7f2a44005bf0 comp rx=0 tx=0).stop 2024-09-21T03:28:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.960+0000 7f2a506b3640 1 -- 172.21.15.112:0/795663206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a481094c0 msgr2=0x7f2a4819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.960+0000 7f2a506b3640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a481094c0 0x7f2a4819aa80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2a3802a930 tx=0x7f2a38004270 comp rx=0 tx=0).stop 2024-09-21T03:28:11.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.960+0000 7f2a506b3640 1 -- 172.21.15.112:0/795663206 shutdown_connections 2024-09-21T03:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.960+0000 7f2a506b3640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2a340b65f0 0x7f2a340b8b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.960+0000 7f2a506b3640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a48109e10 0x7f2a4819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.960+0000 7f2a506b3640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a481094c0 0x7f2a4819aa80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.960+0000 7f2a506b3640 1 --2- 172.21.15.112:0/795663206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a48103b80 0x7f2a4819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.960+0000 7f2a506b3640 1 -- 172.21.15.112:0/795663206 >> 172.21.15.112:0/795663206 conn(0x7f2a480ff610 msgr2=0x7f2a48101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.961+0000 7f2a506b3640 1 -- 172.21.15.112:0/795663206 shutdown_connections 2024-09-21T03:28:11.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:11.961+0000 7f2a506b3640 1 -- 172.21.15.112:0/795663206 wait complete. 2024-09-21T03:28:12.603 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:12.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:12 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/795663206' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:12.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:12 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/795663206' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:12.846 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:12.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:12 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/795663206' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:13.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.283+0000 7ffb840c4640 1 -- 172.21.15.112:0/4153673955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c1031f0 msgr2=0x7ffb7c103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:13.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.283+0000 7ffb840c4640 1 --2- 172.21.15.112:0/4153673955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c1031f0 0x7ffb7c103600 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffb78009900 tx=0x7ffb7802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:13.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.283+0000 7ffb840c4640 1 -- 172.21.15.112:0/4153673955 shutdown_connections 2024-09-21T03:28:13.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.283+0000 7ffb840c4640 1 --2- 172.21.15.112:0/4153673955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c108db0 0x7ffb7c10f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.283+0000 7ffb840c4640 1 --2- 172.21.15.112:0/4153673955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb7c108460 0x7ffb7c108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.283+0000 7ffb840c4640 1 --2- 172.21.15.112:0/4153673955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c1031f0 0x7ffb7c103600 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.283+0000 7ffb840c4640 1 -- 172.21.15.112:0/4153673955 >> 172.21.15.112:0/4153673955 conn(0x7ffb7c079ec0 msgr2=0x7ffb7c07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:13.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.284+0000 7ffb840c4640 1 -- 172.21.15.112:0/4153673955 shutdown_connections 2024-09-21T03:28:13.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.284+0000 7ffb840c4640 1 -- 172.21.15.112:0/4153673955 wait complete. 2024-09-21T03:28:13.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.284+0000 7ffb840c4640 1 Processor -- start 2024-09-21T03:28:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.284+0000 7ffb840c4640 1 -- start start 2024-09-21T03:28:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb840c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c1031f0 0x7ffb7c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb840c4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb7c108460 0x7ffb7c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb840c4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c108db0 0x7ffb7c19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb840c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c112a70 con 0x7ffb7c1031f0 2024-09-21T03:28:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb840c4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c1128f0 con 0x7ffb7c108db0 2024-09-21T03:28:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb840c4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c112bf0 con 0x7ffb7c108460 2024-09-21T03:28:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb81e60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c1031f0 0x7ffb7c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb81e60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c1031f0 0x7ffb7c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42106/0 (socket says 172.21.15.112:42106) 2024-09-21T03:28:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb81e60640 1 -- 172.21.15.112:0/4281285855 learned_addr learned my addr 172.21.15.112:0/4281285855 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb8165f640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb7c108460 0x7ffb7c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb82661640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c108db0 0x7ffb7c19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb81e60640 1 -- 172.21.15.112:0/4281285855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb7c108460 msgr2=0x7ffb7c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb81e60640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb7c108460 0x7ffb7c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb81e60640 1 -- 172.21.15.112:0/4281285855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c108db0 msgr2=0x7ffb7c19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb81e60640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c108db0 0x7ffb7c19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb81e60640 1 -- 172.21.15.112:0/4281285855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c1a0520 con 0x7ffb7c1031f0 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb82661640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c108db0 0x7ffb7c19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb81e60640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c1031f0 0x7ffb7c19a5b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffb78005940 tx=0x7ffb780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb72ffd640 1 -- 172.21.15.112:0/4281285855 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7803f030 con 0x7ffb7c1031f0 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb840c4640 1 -- 172.21.15.112:0/4281285855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c1a07b0 con 0x7ffb7c1031f0 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb72ffd640 1 -- 172.21.15.112:0/4281285855 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb78003cb0 con 0x7ffb7c1031f0 2024-09-21T03:28:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb72ffd640 1 -- 172.21.15.112:0/4281285855 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78035070 con 0x7ffb7c1031f0 2024-09-21T03:28:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.285+0000 7ffb840c4640 1 -- 172.21.15.112:0/4281285855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c1a6f70 con 0x7ffb7c1031f0 2024-09-21T03:28:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.287+0000 7ffb840c4640 1 -- 172.21.15.112:0/4281285855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb7c103f70 con 0x7ffb7c1031f0 2024-09-21T03:28:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.288+0000 7ffb72ffd640 1 -- 172.21.15.112:0/4281285855 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7ffb78042030 con 0x7ffb7c1031f0 2024-09-21T03:28:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.289+0000 7ffb72ffd640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ffb680b65f0 0x7ffb680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.289+0000 7ffb8165f640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ffb680b65f0 0x7ffb680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.289+0000 7ffb72ffd640 1 -- 172.21.15.112:0/4281285855 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ffb78108180 con 0x7ffb7c1031f0 2024-09-21T03:28:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.290+0000 7ffb8165f640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ffb680b65f0 0x7ffb680b8b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005fd0 tx=0x7ffb6c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.293+0000 7ffb72ffd640 1 -- 172.21.15.112:0/4281285855 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffb780ce7f0 con 0x7ffb7c1031f0 2024-09-21T03:28:13.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.514+0000 7ffb840c4640 1 -- 172.21.15.112:0/4281285855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb7c103600 con 0x7ffb7c1031f0 2024-09-21T03:28:13.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.514+0000 7ffb72ffd640 1 -- 172.21.15.112:0/4281285855 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11929 (secure 0 0 0) 0x7ffb780d2bd0 con 0x7ffb7c1031f0 2024-09-21T03:28:13.517 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:13.517 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"btime":"2024-09-21T03:27:13:388843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:27:13.104308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:28:13.517 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-09-21T03:28:13.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 -- 172.21.15.112:0/4281285855 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ffb680b65f0 msgr2=0x7ffb680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:13.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ffb680b65f0 0x7ffb680b8b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005fd0 tx=0x7ffb6c005850 comp rx=0 tx=0).stop 2024-09-21T03:28:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 -- 172.21.15.112:0/4281285855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c1031f0 msgr2=0x7ffb7c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c1031f0 0x7ffb7c19a5b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffb78005940 tx=0x7ffb780042f0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 -- 172.21.15.112:0/4281285855 shutdown_connections 2024-09-21T03:28:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ffb680b65f0 0x7ffb680b8b00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb7c108db0 0x7ffb7c19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb7c108460 0x7ffb7c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 --2- 172.21.15.112:0/4281285855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb7c1031f0 0x7ffb7c19a5b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 -- 172.21.15.112:0/4281285855 >> 172.21.15.112:0/4281285855 conn(0x7ffb7c079ec0 msgr2=0x7ffb7c1013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 -- 172.21.15.112:0/4281285855 shutdown_connections 2024-09-21T03:28:13.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:13.517+0000 7ffb840c4640 1 -- 172.21.15.112:0/4281285855 wait complete. 2024-09-21T03:28:13.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:13 smithi112 ceph-mon[31442]: pgmap v294: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:13 smithi187 ceph-mon[34603]: pgmap v294: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:13.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:13 smithi203 ceph-mon[34723]: pgmap v294: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:14.298 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:28:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:14 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/4281285855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:14 smithi112 ceph-mon[31442]: pgmap v295: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:14.564 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:14.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:14 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/4281285855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:14.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:14 smithi203 ceph-mon[34723]: pgmap v295: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:14 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/4281285855' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:14 smithi187 ceph-mon[34603]: pgmap v295: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:15.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.001+0000 7f08848bb640 1 -- 172.21.15.112:0/899685193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0880108420 msgr2=0x7f0880108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:15.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.001+0000 7f08848bb640 1 --2- 172.21.15.112:0/899685193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0880108420 0x7f0880108830 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0874009900 tx=0x7f087402a410 comp rx=0 tx=0).stop 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.001+0000 7f08848bb640 1 -- 172.21.15.112:0/899685193 shutdown_connections 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.001+0000 7f08848bb640 1 --2- 172.21.15.112:0/899685193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0880108d70 0x7f088010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.001+0000 7f08848bb640 1 --2- 172.21.15.112:0/899685193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0880108420 0x7f0880108830 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.001+0000 7f08848bb640 1 --2- 172.21.15.112:0/899685193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0880102ae0 0x7f0880102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.001+0000 7f08848bb640 1 -- 172.21.15.112:0/899685193 >> 172.21.15.112:0/899685193 conn(0x7f0880079770 msgr2=0x7f0880079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.001+0000 7f08848bb640 1 -- 172.21.15.112:0/899685193 shutdown_connections 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.001+0000 7f08848bb640 1 -- 172.21.15.112:0/899685193 wait complete. 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f08848bb640 1 Processor -- start 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f08848bb640 1 -- start start 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f08848bb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0880102ae0 0x7f088019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f08848bb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0880108420 0x7f088019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f08848bb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0880108d70 0x7f088019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:15.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f08848bb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0880112a30 con 0x7f0880102ae0 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f08848bb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08801128b0 con 0x7f0880108d70 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f08848bb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0880112bb0 con 0x7f0880108420 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f087e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0880102ae0 0x7f088019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f087e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0880102ae0 0x7f088019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42118/0 (socket says 172.21.15.112:42118) 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f087e59c640 1 -- 172.21.15.112:0/4003016230 learned_addr learned my addr 172.21.15.112:0/4003016230 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f087ed9d640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0880108d70 0x7f088019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.002+0000 7f087dd9b640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0880108420 0x7f088019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.003+0000 7f087e59c640 1 -- 172.21.15.112:0/4003016230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0880108420 msgr2=0x7f088019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.003+0000 7f087e59c640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0880108420 0x7f088019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.003+0000 7f087e59c640 1 -- 172.21.15.112:0/4003016230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0880108d70 msgr2=0x7f088019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.003+0000 7f087e59c640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0880108d70 0x7f088019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.003+0000 7f087e59c640 1 -- 172.21.15.112:0/4003016230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08801a0500 con 0x7f0880102ae0 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.003+0000 7f087e59c640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0880102ae0 0x7f088019a590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f087000c890 tx=0x7f087000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:15.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.003+0000 7f086f7fe640 1 -- 172.21.15.112:0/4003016230 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870004220 con 0x7f0880102ae0 2024-09-21T03:28:15.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.003+0000 7f086f7fe640 1 -- 172.21.15.112:0/4003016230 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08700043c0 con 0x7f0880102ae0 2024-09-21T03:28:15.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.003+0000 7f08848bb640 1 -- 172.21.15.112:0/4003016230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08801a07f0 con 0x7f0880102ae0 2024-09-21T03:28:15.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.003+0000 7f086f7fe640 1 -- 172.21.15.112:0/4003016230 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870004a00 con 0x7f0880102ae0 2024-09-21T03:28:15.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.003+0000 7f08848bb640 1 -- 172.21.15.112:0/4003016230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08801a6f50 con 0x7f0880102ae0 2024-09-21T03:28:15.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.004+0000 7f08848bb640 1 -- 172.21.15.112:0/4003016230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0880103d60 con 0x7f0880102ae0 2024-09-21T03:28:15.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.006+0000 7f086f7fe640 1 -- 172.21.15.112:0/4003016230 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f08700079b0 con 0x7f0880102ae0 2024-09-21T03:28:15.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.006+0000 7f086f7fe640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08680b65f0 0x7f08680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:15.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.007+0000 7f086f7fe640 1 -- 172.21.15.112:0/4003016230 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f08700e7680 con 0x7f0880102ae0 2024-09-21T03:28:15.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.007+0000 7f087dd9b640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08680b65f0 0x7f08680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:15.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.007+0000 7f087dd9b640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08680b65f0 0x7f08680b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0874009230 tx=0x7f0874031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:15.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.010+0000 7f086f7fe640 1 -- 172.21.15.112:0/4003016230 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0870014030 con 0x7f0880102ae0 2024-09-21T03:28:15.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.190+0000 7f08848bb640 1 -- 172.21.15.112:0/4003016230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f088010bbd0 con 0x7f0880102ae0 2024-09-21T03:28:15.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.191+0000 7f086f7fe640 1 -- 172.21.15.112:0/4003016230 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23065 (secure 0 0 0) 0x7f08700b20d0 con 0x7f0880102ae0 2024-09-21T03:28:15.193 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:15.193 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":76,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:27:33.414969+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:26:58.087349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:26:59.970444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:28:15.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 -- 172.21.15.112:0/4003016230 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08680b65f0 msgr2=0x7f08680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:15.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08680b65f0 0x7f08680b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0874009230 tx=0x7f0874031400 comp rx=0 tx=0).stop 2024-09-21T03:28:15.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 -- 172.21.15.112:0/4003016230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0880102ae0 msgr2=0x7f088019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:15.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0880102ae0 0x7f088019a590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f087000c890 tx=0x7f087000cd50 comp rx=0 tx=0).stop 2024-09-21T03:28:15.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 -- 172.21.15.112:0/4003016230 shutdown_connections 2024-09-21T03:28:15.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f08680b65f0 0x7f08680b8b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:15.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0880108d70 0x7f088019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:15.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0880108420 0x7f088019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:15.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 --2- 172.21.15.112:0/4003016230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0880102ae0 0x7f088019a590 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:15.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 -- 172.21.15.112:0/4003016230 >> 172.21.15.112:0/4003016230 conn(0x7f0880079770 msgr2=0x7f0880101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:15.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 -- 172.21.15.112:0/4003016230 shutdown_connections 2024-09-21T03:28:15.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:15.193+0000 7f08848bb640 1 -- 172.21.15.112:0/4003016230 wait complete. 2024-09-21T03:28:15.566 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:15 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/4003016230' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:15.812 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:15 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/4003016230' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:15.831 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:15 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/4003016230' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:15.871 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:16.119 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:16.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:16 smithi112 ceph-mon[31442]: pgmap v296: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:28:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.598+0000 7f40c86e8640 1 -- 172.21.15.112:0/3956612128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c0103b60 msgr2=0x7f40c0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.598+0000 7f40c86e8640 1 --2- 172.21.15.112:0/3956612128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c0103b60 0x7f40c0103f70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009900 tx=0x7f40bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.598+0000 7f40c86e8640 1 -- 172.21.15.112:0/3956612128 shutdown_connections 2024-09-21T03:28:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.598+0000 7f40c86e8640 1 --2- 172.21.15.112:0/3956612128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c0109df0 0x7f40c0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.598+0000 7f40c86e8640 1 --2- 172.21.15.112:0/3956612128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c01094a0 0x7f40c01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.598+0000 7f40c86e8640 1 --2- 172.21.15.112:0/3956612128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c0103b60 0x7f40c0103f70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.598+0000 7f40c86e8640 1 -- 172.21.15.112:0/3956612128 >> 172.21.15.112:0/3956612128 conn(0x7f40c007b850 msgr2=0x7f40c007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.598+0000 7f40c86e8640 1 -- 172.21.15.112:0/3956612128 shutdown_connections 2024-09-21T03:28:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.598+0000 7f40c86e8640 1 -- 172.21.15.112:0/3956612128 wait complete. 2024-09-21T03:28:16.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.598+0000 7f40c86e8640 1 Processor -- start 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c86e8640 1 -- start start 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c86e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c0103b60 0x7f40c019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c86e8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c01094a0 0x7f40c019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c86e8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c0109df0 0x7f40c019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c86e8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0113ab0 con 0x7f40c0103b60 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c86e8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0113930 con 0x7f40c0109df0 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c86e8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0113c30 con 0x7f40c01094a0 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c6484640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c0103b60 0x7f40c019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c6484640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c0103b60 0x7f40c019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40078/0 (socket says 172.21.15.112:40078) 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c6484640 1 -- 172.21.15.112:0/3197557610 learned_addr learned my addr 172.21.15.112:0/3197557610 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c5c83640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c01094a0 0x7f40c019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c6c85640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c0109df0 0x7f40c019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:16.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c6484640 1 -- 172.21.15.112:0/3197557610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c01094a0 msgr2=0x7f40c019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c6484640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c01094a0 0x7f40c019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c6484640 1 -- 172.21.15.112:0/3197557610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c0109df0 msgr2=0x7f40c019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c6484640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c0109df0 0x7f40c019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c6484640 1 -- 172.21.15.112:0/3197557610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c01a0410 con 0x7f40c0103b60 2024-09-21T03:28:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40c6484640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c0103b60 0x7f40c019a4a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f40bc005940 tx=0x7f40bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.599+0000 7f40b77fe640 1 -- 172.21.15.112:0/3197557610 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc03f030 con 0x7f40c0103b60 2024-09-21T03:28:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.600+0000 7f40b77fe640 1 -- 172.21.15.112:0/3197557610 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40bc003cb0 con 0x7f40c0103b60 2024-09-21T03:28:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.600+0000 7f40b77fe640 1 -- 172.21.15.112:0/3197557610 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc035070 con 0x7f40c0103b60 2024-09-21T03:28:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.600+0000 7f40c86e8640 1 -- 172.21.15.112:0/3197557610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c01a06a0 con 0x7f40c0103b60 2024-09-21T03:28:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.600+0000 7f40c86e8640 1 -- 172.21.15.112:0/3197557610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c01a6e60 con 0x7f40c0103b60 2024-09-21T03:28:16.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.601+0000 7f40c86e8640 1 -- 172.21.15.112:0/3197557610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c0104de0 con 0x7f40c0103b60 2024-09-21T03:28:16.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.602+0000 7f40b77fe640 1 -- 172.21.15.112:0/3197557610 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f40bc042030 con 0x7f40c0103b60 2024-09-21T03:28:16.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.603+0000 7f40b77fe640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f40ac0b6670 0x7f40ac0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:16.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.603+0000 7f40c5c83640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f40ac0b6670 0x7f40ac0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:16.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.603+0000 7f40b77fe640 1 -- 172.21.15.112:0/3197557610 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f40bc108050 con 0x7f40c0103b60 2024-09-21T03:28:16.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.604+0000 7f40c5c83640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f40ac0b6670 0x7f40ac0b8b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40c0106530 tx=0x7f40b000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:16.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.606+0000 7f40b77fe640 1 -- 172.21.15.112:0/3197557610 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f40bc0ce6c0 con 0x7f40c0103b60 2024-09-21T03:28:16.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.821+0000 7f40c86e8640 1 -- 172.21.15.112:0/3197557610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40c010bbc0 con 0x7f40c0103b60 2024-09-21T03:28:16.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.822+0000 7f40b77fe640 1 -- 172.21.15.112:0/3197557610 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11929 (secure 0 0 0) 0x7f40bc0d5e20 con 0x7f40c0103b60 2024-09-21T03:28:16.824 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:16.824 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"btime":"2024-09-21T03:27:13:388843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:27:13.104308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:28:16.824 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-09-21T03:28:16.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 -- 172.21.15.112:0/3197557610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f40ac0b6670 msgr2=0x7f40ac0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:16.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f40ac0b6670 0x7f40ac0b8b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40c0106530 tx=0x7f40b000b410 comp rx=0 tx=0).stop 2024-09-21T03:28:16.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 -- 172.21.15.112:0/3197557610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c0103b60 msgr2=0x7f40c019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:16.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c0103b60 0x7f40c019a4a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f40bc005940 tx=0x7f40bc0042f0 comp rx=0 tx=0).stop 2024-09-21T03:28:16.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 -- 172.21.15.112:0/3197557610 shutdown_connections 2024-09-21T03:28:16.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f40ac0b6670 0x7f40ac0b8b80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:16.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c0109df0 0x7f40c019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:16.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c01094a0 0x7f40c019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:16.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 --2- 172.21.15.112:0/3197557610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c0103b60 0x7f40c019a4a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:16.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 -- 172.21.15.112:0/3197557610 >> 172.21.15.112:0/3197557610 conn(0x7f40c007b850 msgr2=0x7f40c0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:16.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 -- 172.21.15.112:0/3197557610 shutdown_connections 2024-09-21T03:28:16.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:16.824+0000 7f40c86e8640 1 -- 172.21.15.112:0/3197557610 wait complete. 2024-09-21T03:28:16.844 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:16 smithi187 ceph-mon[34603]: pgmap v296: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:28:16.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:16 smithi203 ceph-mon[34723]: pgmap v296: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:28:17.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:27:01.886242+0000', 'modified': '2024-09-21T03:27:13.104308+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24685, 'mds_1': 24674, 'mds_2': 24667}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24626': {'gid': 24626, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6835/2552593896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 2552593896}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 2552593896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24647': {'gid': 24647, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6837/2217535204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6836', 'nonce': 2217535204}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 2217535204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24658': {'gid': 24658, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/3541929468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 3541929468}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 3541929468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24667': {'gid': 24667, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.203:6837/4062574800', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 4062574800}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 4062574800}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24674': {'gid': 24674, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.187:6839/910214120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6838', 'nonce': 910214120}, {'type': 'v1', 'addr': '172.21.15.187:6839', 'nonce': 910214120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24685': {'gid': 24685, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.203:6839/482815172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 482815172}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 482815172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24685, 'qdb_cluster': [24667, 24674, 24685]} 2024-09-21T03:28:17.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T03:28:17.511 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T03:28:17.758 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:17 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3197557610' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:17.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:17 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3197557610' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:17.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:17 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3197557610' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:18.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.256+0000 7f7f45582640 1 -- 172.21.15.112:0/947686031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f40103b60 msgr2=0x7f7f40103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:18.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.256+0000 7f7f45582640 1 --2- 172.21.15.112:0/947686031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f40103b60 0x7f7f40103f70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009900 tx=0x7f7f3002a410 comp rx=0 tx=0).stop 2024-09-21T03:28:18.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.256+0000 7f7f45582640 1 -- 172.21.15.112:0/947686031 shutdown_connections 2024-09-21T03:28:18.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.256+0000 7f7f45582640 1 --2- 172.21.15.112:0/947686031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f40109df0 0x7f7f40110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.256+0000 7f7f45582640 1 --2- 172.21.15.112:0/947686031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f401094a0 0x7f7f401098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.256+0000 7f7f45582640 1 --2- 172.21.15.112:0/947686031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f40103b60 0x7f7f40103f70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.256+0000 7f7f45582640 1 -- 172.21.15.112:0/947686031 >> 172.21.15.112:0/947686031 conn(0x7f7f4007b850 msgr2=0x7f7f4007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:18.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.256+0000 7f7f45582640 1 -- 172.21.15.112:0/947686031 shutdown_connections 2024-09-21T03:28:18.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.256+0000 7f7f45582640 1 -- 172.21.15.112:0/947686031 wait complete. 2024-09-21T03:28:18.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.257+0000 7f7f45582640 1 Processor -- start 2024-09-21T03:28:18.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.257+0000 7f7f45582640 1 -- start start 2024-09-21T03:28:18.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.257+0000 7f7f45582640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f40103b60 0x7f7f4019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.257+0000 7f7f45582640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f401094a0 0x7f7f4019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f45582640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f40109df0 0x7f7f4019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f45582640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40113b60 con 0x7f7f40109df0 2024-09-21T03:28:18.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f45582640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7f401139e0 con 0x7f7f40103b60 2024-09-21T03:28:18.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f45582640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40113ce0 con 0x7f7f401094a0 2024-09-21T03:28:18.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f40109df0 0x7f7f4019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f40103b60 0x7f7f4019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f40103b60 0x7f7f4019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:59888/0 (socket says 172.21.15.112:59888) 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3effd640 1 -- 172.21.15.112:0/1131017968 learned_addr learned my addr 172.21.15.112:0/1131017968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3e7fc640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f401094a0 0x7f7f4019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/1131017968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f401094a0 msgr2=0x7f7f4019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f401094a0 0x7f7f4019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/1131017968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f40103b60 msgr2=0x7f7f4019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f40103b60 0x7f7f4019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/1131017968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f401a04e0 con 0x7f7f40109df0 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3effd640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f40103b60 0x7f7f4019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f40109df0 0x7f7f4019fdb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800be20 tx=0x7f7f3800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f27fff640 1 -- 172.21.15.112:0/1131017968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38019070 con 0x7f7f40109df0 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f45582640 1 -- 172.21.15.112:0/1131017968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f401a07d0 con 0x7f7f40109df0 2024-09-21T03:28:18.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f27fff640 1 -- 172.21.15.112:0/1131017968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f380148f0 con 0x7f7f40109df0 2024-09-21T03:28:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f27fff640 1 -- 172.21.15.112:0/1131017968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38012840 con 0x7f7f40109df0 2024-09-21T03:28:18.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.258+0000 7f7f45582640 1 -- 172.21.15.112:0/1131017968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f401a6fb0 con 0x7f7f40109df0 2024-09-21T03:28:18.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.261+0000 7f7f4baf6740 1 Processor -- start 2024-09-21T03:28:18.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.261+0000 7f7f27fff640 1 -- 172.21.15.112:0/1131017968 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f7f3801c070 con 0x7f7f40109df0 2024-09-21T03:28:18.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.261+0000 7f7f4baf6740 1 -- start start 2024-09-21T03:28:18.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.261+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28343f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.261+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f750a0 0x55fb28344440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.261+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f7d280 0x55fb28344980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fb282bbc10 con 0x55fb27f9df60 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fb282ab040 con 0x55fb27f750a0 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fb282aee50 con 0x55fb27f7d280 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28343f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28343f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40100/0 (socket says 172.21.15.112:40100) 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3e7fc640 1 -- 172.21.15.112:0/1670014296 learned_addr learned my addr 172.21.15.112:0/1670014296 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3effd640 1 --2- 172.21.15.112:0/1670014296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f750a0 0x55fb28344440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3e7fc640 1 --2- 172.21.15.112:0/1670014296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f7d280 0x55fb28344980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3e7fc640 1 -- 172.21.15.112:0/1670014296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f7d280 msgr2=0x55fb28344980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3e7fc640 1 --2- 172.21.15.112:0/1670014296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f7d280 0x55fb28344980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3e7fc640 1 -- 172.21.15.112:0/1670014296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f750a0 msgr2=0x55fb28344440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3e7fc640 1 --2- 172.21.15.112:0/1670014296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f750a0 0x55fb28344440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3e7fc640 1 -- 172.21.15.112:0/1670014296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb28009540 con 0x55fb27f9df60 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f27fff640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7f280b65f0 0x7f7f280b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3e7fc640 1 --2- 172.21.15.112:0/1670014296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28343f00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7f34005d00 tx=0x7f7f34009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293999c964593e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f257fa640 1 -- 172.21.15.112:0/1670014296 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34005020 con 0x55fb27f9df60 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f3effd640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7f280b65f0 0x7f7f280b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f257fa640 1 -- 172.21.15.112:0/1670014296 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f34005670 con 0x55fb27f9df60 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f257fa640 1 -- 172.21.15.112:0/1670014296 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34004e60 con 0x55fb27f9df60 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.262+0000 7f7f27fff640 1 -- 172.21.15.112:0/1131017968 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f7f380e6c30 con 0x7f7f40109df0 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.263+0000 7f7f4baf6740 1 -- 172.21.15.112:0/1670014296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 msgr2=0x55fb28343f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.263+0000 7f7f3effd640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7f280b65f0 0x7f7f280b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55fb28342b80 tx=0x7f7f30005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.263+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/1670014296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28343f00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7f34005d00 tx=0x7f7f34009510 comp rx=0 tx=0).stop 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.263+0000 7f7f4baf6740 1 -- 172.21.15.112:0/1670014296 shutdown_connections 2024-09-21T03:28:18.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.263+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/1670014296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f7d280 0x55fb28344980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.263+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/1670014296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f750a0 0x55fb28344440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.263+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/1670014296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28343f00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.263+0000 7f7f4baf6740 1 -- 172.21.15.112:0/1670014296 >> 172.21.15.112:0/1670014296 conn(0x55fb27f92350 msgr2=0x55fb28341df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:18.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.263+0000 7f7f4baf6740 1 -- 172.21.15.112:0/1670014296 shutdown_connections 2024-09-21T03:28:18.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.263+0000 7f7f4baf6740 1 -- 172.21.15.112:0/1670014296 wait complete. 2024-09-21T03:28:18.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f4baf6740 1 -- start start 2024-09-21T03:28:18.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f4baf6740 1 Processor -- start 2024-09-21T03:28:18.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f750a0 0x55fb2834f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f7d280 0x55fb2834fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fb2810c5f0 con 0x55fb27f9df60 2024-09-21T03:28:18.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fb28246010 con 0x55fb27f7d280 2024-09-21T03:28:18.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fb28352680 con 0x55fb27f750a0 2024-09-21T03:28:18.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f4baf6740 10 client.0 _ll_register_callbacks cb 0x55fb282a5340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40102/0 (socket says 172.21.15.112:40102) 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/4102025203 learned_addr learned my addr 172.21.15.112:0/4102025203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.264+0000 7f7f3e7fc640 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f7d280 0x55fb2834fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/4102025203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f750a0 msgr2=0x55fb2834f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f750a0 0x55fb2834f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/4102025203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f7d280 msgr2=0x55fb2834fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f7d280 0x55fb2834fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/4102025203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb28273a00 con 0x55fb27f9df60 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835b9a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7f380a9220 tx=0x7f7f380acb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11ae717f60ad7cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f22ff5640 1 -- 172.21.15.112:0/4102025203 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f380940b0 con 0x55fb27f9df60 2024-09-21T03:28:18.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f22ff5640 1 -- 172.21.15.112:0/4102025203 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f7f380cebb0 con 0x55fb27f9df60 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f4baf6740 1 -- 172.21.15.112:0/4102025203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f34005670 con 0x55fb27f9df60 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f22ff5640 1 -- 172.21.15.112:0/4102025203 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f380b1680 con 0x55fb27f9df60 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f4baf6740 10 client.15207 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f4baf6740 1 -- 172.21.15.112:0/4102025203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fb2835e5b0 con 0x55fb27f9df60 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.265+0000 7f7f22ff5640 1 -- 172.21.15.112:0/4102025203 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f34005670 con 0x55fb27f9df60 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.266+0000 7f7f22ff5640 1 -- 172.21.15.112:0/4102025203 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7f380e4b30 con 0x55fb27f9df60 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.266+0000 7f7f4baf6740 10 client.15207 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.266+0000 7f7f4baf6740 10 client.15207 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.266+0000 7f7f4baf6740 20 client.15207 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.266+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fb27f5c8e0 0x55fb28362030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.266+0000 7f7f4baf6740 4 client.15207 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.266+0000 7f7f4baf6740 1 -- 172.21.15.112:0/4102025203 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb282bdb60 con 0x55fb27f5c8e0 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.266+0000 7f7f3effd640 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fb27f5c8e0 0x55fb28362030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.266+0000 7f7f22ff5640 10 client.15207 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.266+0000 7f7f3effd640 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fb27f5c8e0 0x55fb28362030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca54bccf59202283 server_cookie=a4bf48cd81923f9 in_seq=0 out_seq=0 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.267+0000 7f7f22ff5640 1 -- 172.21.15.112:0/4102025203 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb282bdb60 con 0x55fb27f5c8e0 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.267+0000 7f7f22ff5640 10 client.15207 handle_command_reply: tid=0 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.267+0000 7f7f4baf6740 1 client.15207 shutdown 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.267+0000 7f7f4baf6740 1 -- 172.21.15.112:0/4102025203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 msgr2=0x55fb2835b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.267+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835b9a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7f380a9220 tx=0x7f7f380acb50 comp rx=0 tx=0).stop 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.268+0000 7f7f4baf6740 1 -- 172.21.15.112:0/4102025203 shutdown_connections 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.268+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fb27f5c8e0 0x55fb28362030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.268+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835b9a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.268+0000 7f7f22ff5640 0 client.15207 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.268+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f7d280 0x55fb2834fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.268+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/4102025203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f750a0 0x55fb2834f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.268+0000 7f7f4baf6740 1 -- 172.21.15.112:0/4102025203 >> 172.21.15.112:0/4102025203 conn(0x55fb27f92350 msgr2=0x55fb283470e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:18.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.268+0000 7f7f4baf6740 1 -- 172.21.15.112:0/4102025203 shutdown_connections 2024-09-21T03:28:18.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.268+0000 7f7f4baf6740 1 -- 172.21.15.112:0/4102025203 wait complete. 2024-09-21T03:28:18.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.268+0000 7f7f4baf6740 20 client.15207 trim_cache size 0 max 16384 2024-09-21T03:28:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f4baf6740 1 Processor -- start 2024-09-21T03:28:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f4baf6740 1 -- start start 2024-09-21T03:28:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f750a0 0x55fb2834a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f7d280 0x55fb2835b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fb28360370 con 0x55fb27f9df60 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fb28360a40 con 0x55fb27f750a0 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fb28361110 con 0x55fb27f7d280 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40118/0 (socket says 172.21.15.112:40118) 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/1880603675 learned_addr learned my addr 172.21.15.112:0/1880603675 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/1880603675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f7d280 msgr2=0x55fb2835b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/1880603675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f7d280 0x55fb2835b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/1880603675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f750a0 msgr2=0x55fb2834a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/1880603675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f750a0 0x55fb2834a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/1880603675 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb28350a60 con 0x55fb27f9df60 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/1880603675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835be50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f380b1840 tx=0x7f7f38014cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47184501f855481a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.286+0000 7f7f24ff9640 1 -- 172.21.15.112:0/1880603675 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f380ac5a0 con 0x55fb27f9df60 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.287+0000 7f7f24ff9640 1 -- 172.21.15.112:0/1880603675 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f380940b0 con 0x55fb27f9df60 2024-09-21T03:28:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.287+0000 7f7f24ff9640 1 -- 172.21.15.112:0/1880603675 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f380b11c0 con 0x55fb27f9df60 2024-09-21T03:28:18.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.287+0000 7f7f4baf6740 1 -- 172.21.15.112:0/1880603675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 msgr2=0x55fb2835be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:18.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.287+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/1880603675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835be50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f380b1840 tx=0x7f7f38014cb0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.287+0000 7f7f4baf6740 1 -- 172.21.15.112:0/1880603675 shutdown_connections 2024-09-21T03:28:18.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.287+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/1880603675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb2835be50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.287+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/1880603675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f7d280 0x55fb2835b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.287+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/1880603675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f750a0 0x55fb2834a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.287+0000 7f7f4baf6740 1 -- 172.21.15.112:0/1880603675 >> 172.21.15.112:0/1880603675 conn(0x55fb27f92350 msgr2=0x55fb2834e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:18.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.287+0000 7f7f4baf6740 1 -- 172.21.15.112:0/1880603675 shutdown_connections 2024-09-21T03:28:18.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.287+0000 7f7f4baf6740 1 -- 172.21.15.112:0/1880603675 wait complete. 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f4baf6740 1 -- start start 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f4baf6740 1 Processor -- start 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f750a0 0x55fb28364580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f7d280 0x55fb28364ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f4baf6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28369d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fb28361eb0 con 0x55fb27f9df60 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fb283617e0 con 0x55fb27f7d280 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f4baf6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fb28362580 con 0x55fb27f750a0 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f4baf6740 10 client.0 _ll_register_callbacks cb 0x7f7f380a92a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28369d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f3f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28369d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40130/0 (socket says 172.21.15.112:40130) 2024-09-21T03:28:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.288+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/2605939916 learned_addr learned my addr 172.21.15.112:0/2605939916 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/2605939916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f750a0 msgr2=0x55fb28364580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/2605939916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f750a0 0x55fb28364580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/2605939916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f7d280 msgr2=0x55fb28364ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/2605939916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f7d280 0x55fb28364ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f3f7fe640 1 -- 172.21.15.112:0/2605939916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb28365340 con 0x55fb27f9df60 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f3f7fe640 1 --2- 172.21.15.112:0/2605939916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28369d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7f380e4aa0 tx=0x7f7f380acb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936470dc00cb9653 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f22ff5640 1 -- 172.21.15.112:0/2605939916 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f380ac5a0 con 0x55fb27f9df60 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f22ff5640 1 -- 172.21.15.112:0/2605939916 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f7f380cedb0 con 0x55fb27f9df60 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f4baf6740 1 -- 172.21.15.112:0/2605939916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fb282b9610 con 0x55fb27f9df60 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f22ff5640 1 -- 172.21.15.112:0/2605939916 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f380b65c0 con 0x55fb27f9df60 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f4baf6740 10 client.15213 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f4baf6740 1 -- 172.21.15.112:0/2605939916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fb283cda40 con 0x55fb27f9df60 2024-09-21T03:28:18.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f22ff5640 1 -- 172.21.15.112:0/2605939916 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fb282b9610 con 0x55fb27f9df60 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f22ff5640 1 -- 172.21.15.112:0/2605939916 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7f380ab920 con 0x55fb27f9df60 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.289+0000 7f7f4baf6740 10 client.15213 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.290+0000 7f7f4baf6740 10 client.15213 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.290+0000 7f7f4baf6740 20 client.15213 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.290+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/2605939916 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fb27f31c90 0x55fb283d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.290+0000 7f7f4baf6740 4 client.15213 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.290+0000 7f7f4baf6740 1 -- 172.21.15.112:0/2605939916 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "status"}) -- 0x55fb282b6a40 con 0x55fb27f31c90 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.290+0000 7f7f3effd640 1 --2- 172.21.15.112:0/2605939916 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fb27f31c90 0x55fb283d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.290+0000 7f7f22ff5640 10 client.15213 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.290+0000 7f7f3effd640 1 --2- 172.21.15.112:0/2605939916 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fb27f31c90 0x55fb283d0df0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f57c3a1887a72286 server_cookie=eb9f9bd3c8e40976 in_seq=0 out_seq=0 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.290+0000 7f7f22ff5640 1 -- 172.21.15.112:0/2605939916 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fb282b6a40 con 0x55fb27f31c90 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.290+0000 7f7f22ff5640 10 client.15213 handle_command_reply: tid=0 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.290+0000 7f7f4baf6740 1 client.15213 shutdown 2024-09-21T03:28:18.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f4baf6740 1 -- 172.21.15.112:0/2605939916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 msgr2=0x55fb28369d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/2605939916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28369d10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7f380e4aa0 tx=0x7f7f380acb50 comp rx=0 tx=0).stop 2024-09-21T03:28:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f4baf6740 1 -- 172.21.15.112:0/2605939916 shutdown_connections 2024-09-21T03:28:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/2605939916 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fb27f31c90 0x55fb283d0df0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/2605939916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb27f9df60 0x55fb28369d10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/2605939916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fb27f7d280 0x55fb28364ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f4baf6740 1 --2- 172.21.15.112:0/2605939916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fb27f750a0 0x55fb28364580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f4baf6740 1 -- 172.21.15.112:0/2605939916 >> 172.21.15.112:0/2605939916 conn(0x55fb27f92350 msgr2=0x55fb2834f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f22ff5640 0 client.15213 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:28:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f4baf6740 1 -- 172.21.15.112:0/2605939916 shutdown_connections 2024-09-21T03:28:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f4baf6740 1 -- 172.21.15.112:0/2605939916 wait complete. 2024-09-21T03:28:18.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f4baf6740 20 client.15213 trim_cache size 0 max 16384 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "063b32f6-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24667, 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 67.912606088999993, 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 76, 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 99.271134708999995 2024-09-21T03:28:18.298 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:28:18.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f45582640 1 -- 172.21.15.112:0/1131017968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7f280b65f0 msgr2=0x7f7f280b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:18.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f45582640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7f280b65f0 0x7f7f280b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55fb28342b80 tx=0x7f7f30005a20 comp rx=0 tx=0).stop 2024-09-21T03:28:18.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f45582640 1 -- 172.21.15.112:0/1131017968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f40109df0 msgr2=0x7f7f4019fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:18.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f45582640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f40109df0 0x7f7f4019fdb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800be20 tx=0x7f7f3800bf10 comp rx=0 tx=0).stop 2024-09-21T03:28:18.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f45582640 1 -- 172.21.15.112:0/1131017968 shutdown_connections 2024-09-21T03:28:18.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f45582640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7f280b65f0 0x7f7f280b8b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f45582640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f40109df0 0x7f7f4019fdb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f45582640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f401094a0 0x7f7f4019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f45582640 1 --2- 172.21.15.112:0/1131017968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f40103b60 0x7f7f4019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:18.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.291+0000 7f7f45582640 1 -- 172.21.15.112:0/1131017968 >> 172.21.15.112:0/1131017968 conn(0x7f7f4007b850 msgr2=0x7f7f40107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:18.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.292+0000 7f7f45582640 1 -- 172.21.15.112:0/1131017968 shutdown_connections 2024-09-21T03:28:18.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:18.292+0000 7f7f45582640 1 -- 172.21.15.112:0/1131017968 wait complete. 2024-09-21T03:28:18.789 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:18 smithi112 ceph-mon[31442]: pgmap v297: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:28:18.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:18 smithi203 ceph-mon[34723]: pgmap v297: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:28:18.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:18 smithi187 ceph-mon[34603]: pgmap v297: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:28:19.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:28:19.300 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:19.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.723+0000 7fc12a7e1640 1 -- 172.21.15.112:0/3657470733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124108cf0 msgr2=0x7fc12410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:19.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.723+0000 7fc12a7e1640 1 --2- 172.21.15.112:0/3657470733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124108cf0 0x7fc12410f520 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009ca0 tx=0x7fc11c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.723+0000 7fc12a7e1640 1 -- 172.21.15.112:0/3657470733 shutdown_connections 2024-09-21T03:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.723+0000 7fc12a7e1640 1 --2- 172.21.15.112:0/3657470733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124108cf0 0x7fc12410f520 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.723+0000 7fc12a7e1640 1 --2- 172.21.15.112:0/3657470733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1241083a0 0x7fc1241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.723+0000 7fc12a7e1640 1 --2- 172.21.15.112:0/3657470733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc124102a60 0x7fc124102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.723+0000 7fc12a7e1640 1 -- 172.21.15.112:0/3657470733 >> 172.21.15.112:0/3657470733 conn(0x7fc124079730 msgr2=0x7fc124079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.723+0000 7fc12a7e1640 1 -- 172.21.15.112:0/3657470733 shutdown_connections 2024-09-21T03:28:19.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.723+0000 7fc12a7e1640 1 -- 172.21.15.112:0/3657470733 wait complete. 2024-09-21T03:28:19.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.723+0000 7fc12a7e1640 1 Processor -- start 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc12a7e1640 1 -- start start 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc12a7e1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc124102a60 0x7fc12419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc12a7e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1241083a0 0x7fc12419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc12a7e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc124108cf0 0x7fc12419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc12a7e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc124113a80 con 0x7fc1241083a0 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc12a7e1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc124113900 con 0x7fc124102a60 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc12a7e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc124113c00 con 0x7fc124108cf0 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1241083a0 0x7fc12419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1241083a0 0x7fc12419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40150/0 (socket says 172.21.15.112:40150) 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc11bfff640 1 -- 172.21.15.112:0/1743747681 learned_addr learned my addr 172.21.15.112:0/1743747681 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc123fff640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc124102a60 0x7fc12419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc128d7e640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc124108cf0 0x7fc12419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc11bfff640 1 -- 172.21.15.112:0/1743747681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc124108cf0 msgr2=0x7fc12419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc11bfff640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc124108cf0 0x7fc12419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc11bfff640 1 -- 172.21.15.112:0/1743747681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc124102a60 msgr2=0x7fc12419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc11bfff640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc124102a60 0x7fc12419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.724+0000 7fc11bfff640 1 -- 172.21.15.112:0/1743747681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1241a03e0 con 0x7fc1241083a0 2024-09-21T03:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.725+0000 7fc11bfff640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1241083a0 0x7fc12419aa10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00d770 tx=0x7fc10c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.725+0000 7fc121ffb640 1 -- 172.21.15.112:0/1743747681 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c017070 con 0x7fc1241083a0 2024-09-21T03:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.725+0000 7fc121ffb640 1 -- 172.21.15.112:0/1743747681 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc10c009c30 con 0x7fc1241083a0 2024-09-21T03:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.725+0000 7fc121ffb640 1 -- 172.21.15.112:0/1743747681 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c00f620 con 0x7fc1241083a0 2024-09-21T03:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.725+0000 7fc12a7e1640 1 -- 172.21.15.112:0/1743747681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1241a06d0 con 0x7fc1241083a0 2024-09-21T03:28:19.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.725+0000 7fc12a7e1640 1 -- 172.21.15.112:0/1743747681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1241a6e30 con 0x7fc1241083a0 2024-09-21T03:28:19.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.728+0000 7fc121ffb640 1 -- 172.21.15.112:0/1743747681 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fc10c01a070 con 0x7fc1241083a0 2024-09-21T03:28:19.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.728+0000 7fc130d55740 1 Processor -- start 2024-09-21T03:28:19.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.728+0000 7fc130d55740 1 -- start start 2024-09-21T03:28:19.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc121ffb640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc1100b65f0 0x7fc1100b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc130d55740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786aca2f60 0x55786b048f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc130d55740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 0x55786b049440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc130d55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 0x55786b049980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc130d55740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55786afc0c10 con 0x55786ac82280 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc130d55740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55786afb0040 con 0x55786ac7a0a0 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc130d55740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55786afb3e50 con 0x55786aca2f60 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc128d7e640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc1100b65f0 0x7fc1100b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 0x55786b049440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc128d7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 0x55786b049980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc128d7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 0x55786b049980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40158/0 (socket says 172.21.15.112:40158) 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc128d7e640 1 -- 172.21.15.112:0/2243702253 learned_addr learned my addr 172.21.15.112:0/2243702253 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc121ffb640 1 -- 172.21.15.112:0/1743747681 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc10c0e69d0 con 0x7fc1241083a0 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc123fff640 1 --2- 172.21.15.112:0/2243702253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786aca2f60 0x55786b048f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc11bfff640 1 -- 172.21.15.112:0/2243702253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786aca2f60 msgr2=0x55786b048f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc11bfff640 1 --2- 172.21.15.112:0/2243702253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786aca2f60 0x55786b048f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc11bfff640 1 -- 172.21.15.112:0/2243702253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 msgr2=0x55786b049980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:19.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc11bfff640 1 --2- 172.21.15.112:0/2243702253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 0x55786b049980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc11bfff640 1 -- 172.21.15.112:0/2243702253 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55786ad0e540 con 0x55786ac7a0a0 2024-09-21T03:28:19.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc128d7e640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc1100b65f0 0x7fc1100b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc11c02f500 tx=0x7fc11c02f3f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:19.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.729+0000 7fc11bfff640 1 --2- 172.21.15.112:0/2243702253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 0x55786b049440 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0e42c0 tx=0x7fc10c00de80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab4d7aa3aa29daaa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:19.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.730+0000 7fc11a7fc640 1 -- 172.21.15.112:0/2243702253 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c021070 con 0x55786ac7a0a0 2024-09-21T03:28:19.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.730+0000 7fc11a7fc640 1 -- 172.21.15.112:0/2243702253 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc10c0ac450 con 0x55786ac7a0a0 2024-09-21T03:28:19.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.730+0000 7fc11a7fc640 1 -- 172.21.15.112:0/2243702253 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0c0b00 con 0x55786ac7a0a0 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.730+0000 7fc130d55740 1 -- 172.21.15.112:0/2243702253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 msgr2=0x55786b049440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.730+0000 7fc130d55740 1 --2- 172.21.15.112:0/2243702253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 0x55786b049440 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0e42c0 tx=0x7fc10c00de80 comp rx=0 tx=0).stop 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.731+0000 7fc130d55740 1 -- 172.21.15.112:0/2243702253 shutdown_connections 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.731+0000 7fc130d55740 1 --2- 172.21.15.112:0/2243702253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 0x55786b049980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.731+0000 7fc130d55740 1 --2- 172.21.15.112:0/2243702253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 0x55786b049440 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.731+0000 7fc130d55740 1 --2- 172.21.15.112:0/2243702253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786aca2f60 0x55786b048f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.731+0000 7fc130d55740 1 -- 172.21.15.112:0/2243702253 >> 172.21.15.112:0/2243702253 conn(0x55786ac97350 msgr2=0x55786b046df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.731+0000 7fc130d55740 1 -- 172.21.15.112:0/2243702253 shutdown_connections 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.731+0000 7fc130d55740 1 -- 172.21.15.112:0/2243702253 wait complete. 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.731+0000 7fc130d55740 1 -- start start 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.731+0000 7fc130d55740 1 Processor -- start 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc130d55740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 0x55786b058dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc130d55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 0x55786b059300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc130d55740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786aca2f60 0x55786b064cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc130d55740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55786ae115f0 con 0x55786ac82280 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc130d55740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55786af4b010 con 0x55786ac7a0a0 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc130d55740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55786b05b980 con 0x55786aca2f60 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc130d55740 10 client.0 _ll_register_callbacks cb 0x55786afaa340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 0x55786b059300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 0x55786b059300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40168/0 (socket says 172.21.15.112:40168) 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc11bfff640 1 -- 172.21.15.112:0/497644421 learned_addr learned my addr 172.21.15.112:0/497644421 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc128d7e640 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786aca2f60 0x55786b064cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc123fff640 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 0x55786b058dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc11bfff640 1 -- 172.21.15.112:0/497644421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786aca2f60 msgr2=0x55786b064cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc11bfff640 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786aca2f60 0x55786b064cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc11bfff640 1 -- 172.21.15.112:0/497644421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 msgr2=0x55786b058dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc11bfff640 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 0x55786b058dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc11bfff640 1 -- 172.21.15.112:0/497644421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55786af78a00 con 0x55786ac82280 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc123fff640 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 0x55786b058dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc11bfff640 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 0x55786b059300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0a80b0 tx=0x7fc10c0ad0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14faa12651111db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc1037fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc1037fe640 1 -- 172.21.15.112:0/497644421 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c021030 con 0x55786ac82280 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc1037fe640 1 -- 172.21.15.112:0/497644421 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc10c0abdb0 con 0x55786ac82280 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.732+0000 7fc130d55740 1 -- 172.21.15.112:0/497644421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc10c0ac450 con 0x55786ac82280 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc1037fe640 1 -- 172.21.15.112:0/497644421 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc10c009de0 con 0x55786ac82280 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc130d55740 10 client.15225 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc130d55740 1 -- 172.21.15.112:0/497644421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55786b0678c0 con 0x55786ac82280 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc1037fe640 1 -- 172.21.15.112:0/497644421 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc10c0ac450 con 0x55786ac82280 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc1037fe640 1 -- 172.21.15.112:0/497644421 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc10c0b25b0 con 0x55786ac82280 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc130d55740 10 client.15225 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc130d55740 10 client.15225 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc130d55740 20 client.15225 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc130d55740 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55786ac618e0 0x55786b06b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc130d55740 4 client.15225 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc130d55740 1 -- 172.21.15.112:0/497644421 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55786afc2b60 con 0x55786ac618e0 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.733+0000 7fc123fff640 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55786ac618e0 0x55786b06b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.734+0000 7fc1037fe640 10 client.15225 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.734+0000 7fc123fff640 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55786ac618e0 0x55786b06b3c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f215d10524871d server_cookie=33a395eb23014f32 in_seq=0 out_seq=0 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc1037fe640 1 -- 172.21.15.112:0/497644421 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55786afc2b60 con 0x55786ac618e0 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc1037fe640 10 client.15225 handle_command_reply: tid=0 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 1 client.15225 shutdown 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 1 -- 172.21.15.112:0/497644421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 msgr2=0x55786b059300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 0x55786b059300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0a80b0 tx=0x7fc10c0ad0b0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 1 -- 172.21.15.112:0/497644421 shutdown_connections 2024-09-21T03:28:19.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55786ac618e0 0x55786b06b3c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786aca2f60 0x55786b064cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc1037fe640 0 client.15225 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:28:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786ac82280 0x55786b059300 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 1 --2- 172.21.15.112:0/497644421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac7a0a0 0x55786b058dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 1 -- 172.21.15.112:0/497644421 >> 172.21.15.112:0/497644421 conn(0x55786ac97350 msgr2=0x55786b04a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 1 -- 172.21.15.112:0/497644421 shutdown_connections 2024-09-21T03:28:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 1 -- 172.21.15.112:0/497644421 wait complete. 2024-09-21T03:28:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.735+0000 7fc130d55740 20 client.15225 trim_cache size 0 max 16384 2024-09-21T03:28:19.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc130d55740 1 Processor -- start 2024-09-21T03:28:19.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc130d55740 1 -- start start 2024-09-21T03:28:19.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc130d55740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786ac7a0a0 0x55786b064c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc130d55740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac82280 0x55786b065160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc130d55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b053d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc130d55740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55786b069700 con 0x55786aca2f60 2024-09-21T03:28:19.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc130d55740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55786b069dd0 con 0x55786ac82280 2024-09-21T03:28:19.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc130d55740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55786b06a4a0 con 0x55786ac7a0a0 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc128d7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b053d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc128d7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b053d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40184/0 (socket says 172.21.15.112:40184) 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc128d7e640 1 -- 172.21.15.112:0/1051818703 learned_addr learned my addr 172.21.15.112:0/1051818703 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.753+0000 7fc11bfff640 1 --2- 172.21.15.112:0/1051818703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac82280 0x55786b065160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.754+0000 7fc128d7e640 1 -- 172.21.15.112:0/1051818703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786ac7a0a0 msgr2=0x55786b064c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.754+0000 7fc123fff640 1 --2- 172.21.15.112:0/1051818703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786ac7a0a0 0x55786b064c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.754+0000 7fc128d7e640 1 --2- 172.21.15.112:0/1051818703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786ac7a0a0 0x55786b064c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.754+0000 7fc128d7e640 1 -- 172.21.15.112:0/1051818703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac82280 msgr2=0x55786b065160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.754+0000 7fc128d7e640 1 --2- 172.21.15.112:0/1051818703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac82280 0x55786b065160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.754+0000 7fc128d7e640 1 -- 172.21.15.112:0/1051818703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55786b059e80 con 0x55786aca2f60 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.754+0000 7fc11bfff640 1 --2- 172.21.15.112:0/1051818703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac82280 0x55786b065160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.754+0000 7fc128d7e640 1 --2- 172.21.15.112:0/1051818703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b053d00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009a60 tx=0x7fc11c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330f6d34a7582659 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:19.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.754+0000 7fc119ffb640 1 -- 172.21.15.112:0/1051818703 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c031d20 con 0x55786aca2f60 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.754+0000 7fc119ffb640 1 -- 172.21.15.112:0/1051818703 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc11c051070 con 0x55786aca2f60 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.754+0000 7fc119ffb640 1 -- 172.21.15.112:0/1051818703 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c0047f0 con 0x55786aca2f60 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.755+0000 7fc130d55740 1 -- 172.21.15.112:0/1051818703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 msgr2=0x55786b053d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.755+0000 7fc130d55740 1 --2- 172.21.15.112:0/1051818703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b053d00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009a60 tx=0x7fc11c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.755+0000 7fc130d55740 1 -- 172.21.15.112:0/1051818703 shutdown_connections 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.755+0000 7fc130d55740 1 --2- 172.21.15.112:0/1051818703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b053d00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.755+0000 7fc130d55740 1 --2- 172.21.15.112:0/1051818703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac82280 0x55786b065160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.755+0000 7fc130d55740 1 --2- 172.21.15.112:0/1051818703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786ac7a0a0 0x55786b064c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.755+0000 7fc130d55740 1 -- 172.21.15.112:0/1051818703 >> 172.21.15.112:0/1051818703 conn(0x55786ac97350 msgr2=0x55786b058da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.755+0000 7fc130d55740 1 -- 172.21.15.112:0/1051818703 shutdown_connections 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.755+0000 7fc130d55740 1 -- 172.21.15.112:0/1051818703 wait complete. 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.755+0000 7fc130d55740 1 -- start start 2024-09-21T03:28:19.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.755+0000 7fc130d55740 1 Processor -- start 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc130d55740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786ac7a0a0 0x55786b073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc130d55740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac82280 0x55786b06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc130d55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc130d55740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55786b06b240 con 0x55786aca2f60 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc130d55740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55786b06ab70 con 0x55786ac82280 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc130d55740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55786b06b910 con 0x55786ac7a0a0 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc130d55740 10 client.0 _ll_register_callbacks cb 0x7fc11c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc128d7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc128d7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b06d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40196/0 (socket says 172.21.15.112:40196) 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc128d7e640 1 -- 172.21.15.112:0/3086889695 learned_addr learned my addr 172.21.15.112:0/3086889695 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc11bfff640 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac82280 0x55786b06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc123fff640 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786ac7a0a0 0x55786b073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc128d7e640 1 -- 172.21.15.112:0/3086889695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786ac7a0a0 msgr2=0x55786b073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc128d7e640 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786ac7a0a0 0x55786b073c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc128d7e640 1 -- 172.21.15.112:0/3086889695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac82280 msgr2=0x55786b06d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc128d7e640 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac82280 0x55786b06d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc128d7e640 1 -- 172.21.15.112:0/3086889695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55786b06dbf0 con 0x55786aca2f60 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc128d7e640 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b06d630 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc11c031a60 tx=0x7fc11c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa778aec72afde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.756+0000 7fc1037fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc1037fe640 1 -- 172.21.15.112:0/3086889695 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c003e40 con 0x55786aca2f60 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc1037fe640 1 -- 172.21.15.112:0/3086889695 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc11c02f660 con 0x55786aca2f60 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc130d55740 1 -- 172.21.15.112:0/3086889695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc11c051070 con 0x55786aca2f60 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc1037fe640 1 -- 172.21.15.112:0/3086889695 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc11c02f860 con 0x55786aca2f60 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc130d55740 10 client.15231 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:19.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc130d55740 1 -- 172.21.15.112:0/3086889695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55786b073520 con 0x55786aca2f60 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc1037fe640 1 -- 172.21.15.112:0/3086889695 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc11c051070 con 0x55786aca2f60 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc1037fe640 1 -- 172.21.15.112:0/3086889695 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc11c04a040 con 0x55786aca2f60 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc130d55740 10 client.15231 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc130d55740 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc130d55740 20 client.15231 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc130d55740 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55786ac36c90 0x55786b0d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc130d55740 4 client.15231 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.757+0000 7fc130d55740 1 -- 172.21.15.112:0/3086889695 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "status"}) -- 0x55786afbe610 con 0x55786ac36c90 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc123fff640 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55786ac36c90 0x55786b0d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc1037fe640 10 client.15231 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc123fff640 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55786ac36c90 0x55786b0d9930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa9afa5c9eec706d server_cookie=11d6f18e25be960 in_seq=0 out_seq=0 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc1037fe640 1 -- 172.21.15.112:0/3086889695 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55786afbe610 con 0x55786ac36c90 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc1037fe640 10 client.15231 handle_command_reply: tid=0 2024-09-21T03:28:19.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc130d55740 1 client.15231 shutdown 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc130d55740 1 -- 172.21.15.112:0/3086889695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 msgr2=0x55786b06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc130d55740 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b06d630 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc11c031a60 tx=0x7fc11c004860 comp rx=0 tx=0).stop 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc130d55740 1 -- 172.21.15.112:0/3086889695 shutdown_connections 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc130d55740 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55786ac36c90 0x55786b0d9930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc130d55740 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55786aca2f60 0x55786b06d630 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc1037fe640 0 client.15231 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc130d55740 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55786ac82280 0x55786b06d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc130d55740 1 --2- 172.21.15.112:0/3086889695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55786ac7a0a0 0x55786b073c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.758+0000 7fc130d55740 1 -- 172.21.15.112:0/3086889695 >> 172.21.15.112:0/3086889695 conn(0x55786ac97350 msgr2=0x55786b057400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc130d55740 1 -- 172.21.15.112:0/3086889695 shutdown_connections 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc130d55740 1 -- 172.21.15.112:0/3086889695 wait complete. 2024-09-21T03:28:19.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc130d55740 20 client.15231 trim_cache size 0 max 16384 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "063b32f6-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24674, 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 70.394638920999995, 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 76, 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T03:28:19.767 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 87.470266730000006 2024-09-21T03:28:19.768 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:28:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 -- 172.21.15.112:0/1743747681 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc1100b65f0 msgr2=0x7fc1100b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc1100b65f0 0x7fc1100b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc11c02f500 tx=0x7fc11c02f3f0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 -- 172.21.15.112:0/1743747681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1241083a0 msgr2=0x7fc12419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1241083a0 0x7fc12419aa10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00d770 tx=0x7fc10c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:28:19.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 -- 172.21.15.112:0/1743747681 shutdown_connections 2024-09-21T03:28:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc1100b65f0 0x7fc1100b8b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc124108cf0 0x7fc12419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1241083a0 0x7fc12419aa10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 --2- 172.21.15.112:0/1743747681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc124102a60 0x7fc12419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 -- 172.21.15.112:0/1743747681 >> 172.21.15.112:0/1743747681 conn(0x7fc124079730 msgr2=0x7fc124104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 -- 172.21.15.112:0/1743747681 shutdown_connections 2024-09-21T03:28:19.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:19.759+0000 7fc12a7e1640 1 -- 172.21.15.112:0/1743747681 wait complete. 2024-09-21T03:28:20.526 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:28:20.771 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:21.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.214+0000 7f975f4d1640 1 -- 172.21.15.112:0/3553572714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9758109df0 msgr2=0x7f9758110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.214+0000 7f975f4d1640 1 --2- 172.21.15.112:0/3553572714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9758109df0 0x7f9758110620 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9750009ca0 tx=0x7f975002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.214+0000 7f975f4d1640 1 -- 172.21.15.112:0/3553572714 shutdown_connections 2024-09-21T03:28:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.214+0000 7f975f4d1640 1 --2- 172.21.15.112:0/3553572714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9758109df0 0x7f9758110620 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.214+0000 7f975f4d1640 1 --2- 172.21.15.112:0/3553572714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97581094a0 0x7f97581098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.214+0000 7f975f4d1640 1 --2- 172.21.15.112:0/3553572714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9758103b60 0x7f9758103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.214+0000 7f975f4d1640 1 -- 172.21.15.112:0/3553572714 >> 172.21.15.112:0/3553572714 conn(0x7f975807b850 msgr2=0x7f975807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.214+0000 7f975f4d1640 1 -- 172.21.15.112:0/3553572714 shutdown_connections 2024-09-21T03:28:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.214+0000 7f975f4d1640 1 -- 172.21.15.112:0/3553572714 wait complete. 2024-09-21T03:28:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.215+0000 7f975f4d1640 1 Processor -- start 2024-09-21T03:28:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.215+0000 7f975f4d1640 1 -- start start 2024-09-21T03:28:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.215+0000 7f975f4d1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9758103b60 0x7f9758075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.215+0000 7f975f4d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97581094a0 0x7f9758075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.215+0000 7f975f4d1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9758109df0 0x7f9758076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.215+0000 7f975f4d1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9758113ab0 con 0x7f97581094a0 2024-09-21T03:28:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.215+0000 7f975f4d1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9758113930 con 0x7f9758109df0 2024-09-21T03:28:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.215+0000 7f975f4d1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9758113c30 con 0x7f9758103b60 2024-09-21T03:28:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975d26d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9758103b60 0x7f9758075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975ca6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97581094a0 0x7f9758075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975d26d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9758103b60 0x7f9758075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:57628/0 (socket says 172.21.15.112:57628) 2024-09-21T03:28:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975ca6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97581094a0 0x7f9758075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40204/0 (socket says 172.21.15.112:40204) 2024-09-21T03:28:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975d26d640 1 -- 172.21.15.112:0/2038811189 learned_addr learned my addr 172.21.15.112:0/2038811189 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975da6e640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9758109df0 0x7f9758076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975ca6c640 1 -- 172.21.15.112:0/2038811189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9758103b60 msgr2=0x7f9758075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975ca6c640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9758103b60 0x7f9758075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975ca6c640 1 -- 172.21.15.112:0/2038811189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9758109df0 msgr2=0x7f9758076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975ca6c640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9758109df0 0x7f9758076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975ca6c640 1 -- 172.21.15.112:0/2038811189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97581a86e0 con 0x7f97581094a0 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975ca6c640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97581094a0 0x7f9758075c50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f975806c9f0 tx=0x7f974c00db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f97567fc640 1 -- 172.21.15.112:0/2038811189 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c017070 con 0x7f97581094a0 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f97567fc640 1 -- 172.21.15.112:0/2038811189 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f974c009820 con 0x7f97581094a0 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975f4d1640 1 -- 172.21.15.112:0/2038811189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97581a89d0 con 0x7f97581094a0 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f97567fc640 1 -- 172.21.15.112:0/2038811189 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c004b90 con 0x7f97581094a0 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.216+0000 7f975f4d1640 1 -- 172.21.15.112:0/2038811189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97581a8f10 con 0x7f97581094a0 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.219+0000 7f97567fc640 1 -- 172.21.15.112:0/2038811189 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f974c004d30 con 0x7f97581094a0 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.219+0000 7f9765a45740 1 Processor -- start 2024-09-21T03:28:21.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.219+0000 7f9765a45740 1 -- start start 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.219+0000 7f9765a45740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96c8f60 0x5572b9a6ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.219+0000 7f9765a45740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a6f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.219+0000 7f9765a45740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a8280 0x5572b9a6f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.219+0000 7f9765a45740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572b99e6c10 con 0x5572b96a8280 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.219+0000 7f9765a45740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5572b99d6040 con 0x5572b96c8f60 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.219+0000 7f9765a45740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5572b99d9e50 con 0x5572b96a00a0 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975ca6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a8280 0x5572b9a6f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975ca6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a8280 0x5572b9a6f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40220/0 (socket says 172.21.15.112:40220) 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975ca6c640 1 -- 172.21.15.112:0/826304894 learned_addr learned my addr 172.21.15.112:0/826304894 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975d26d640 1 --2- 172.21.15.112:0/826304894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a6f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975da6e640 1 --2- 172.21.15.112:0/826304894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96c8f60 0x5572b9a6ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f97567fc640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97400b65f0 0x7f97400b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975ca6c640 1 -- 172.21.15.112:0/826304894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 msgr2=0x5572b9a6f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975ca6c640 1 --2- 172.21.15.112:0/826304894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a6f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975ca6c640 1 -- 172.21.15.112:0/826304894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96c8f60 msgr2=0x5572b9a6ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975ca6c640 1 --2- 172.21.15.112:0/826304894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96c8f60 0x5572b9a6ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975ca6c640 1 -- 172.21.15.112:0/826304894 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b9734540 con 0x5572b96a8280 2024-09-21T03:28:21.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975da6e640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97400b65f0 0x7f97400b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f97567fc640 1 -- 172.21.15.112:0/2038811189 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f974c0e6ea0 con 0x7f97581094a0 2024-09-21T03:28:21.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f975ca6c640 1 --2- 172.21.15.112:0/826304894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a8280 0x5572b9a6f980 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f974c072d70 tx=0x7f974c0713f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ab4833eb778af6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.220+0000 7f973f7fe640 1 -- 172.21.15.112:0/826304894 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c0acc30 con 0x5572b96a8280 2024-09-21T03:28:21.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.221+0000 7f973f7fe640 1 -- 172.21.15.112:0/826304894 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f974c072da0 con 0x5572b96a8280 2024-09-21T03:28:21.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.221+0000 7f973f7fe640 1 -- 172.21.15.112:0/826304894 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c0eaa40 con 0x5572b96a8280 2024-09-21T03:28:21.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.221+0000 7f975da6e640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97400b65f0 0x7f97400b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9750006010 tx=0x7f97500095c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.221+0000 7f9765a45740 1 -- 172.21.15.112:0/826304894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a8280 msgr2=0x5572b9a6f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.221+0000 7f9765a45740 1 --2- 172.21.15.112:0/826304894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a8280 0x5572b9a6f980 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f974c072d70 tx=0x7f974c0713f0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.221+0000 7f9765a45740 1 -- 172.21.15.112:0/826304894 shutdown_connections 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.221+0000 7f9765a45740 1 --2- 172.21.15.112:0/826304894 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a8280 0x5572b9a6f980 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.221+0000 7f9765a45740 1 --2- 172.21.15.112:0/826304894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a6f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.221+0000 7f9765a45740 1 --2- 172.21.15.112:0/826304894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96c8f60 0x5572b9a6ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.221+0000 7f9765a45740 1 -- 172.21.15.112:0/826304894 >> 172.21.15.112:0/826304894 conn(0x5572b96bd350 msgr2=0x5572b9a6cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.222+0000 7f9765a45740 1 -- 172.21.15.112:0/826304894 shutdown_connections 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.222+0000 7f9765a45740 1 -- 172.21.15.112:0/826304894 wait complete. 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.222+0000 7f9765a45740 1 -- start start 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.222+0000 7f9765a45740 1 Processor -- start 2024-09-21T03:28:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.222+0000 7f9765a45740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a7a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.222+0000 7f9765a45740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a7aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.222+0000 7f9765a45740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a869a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.222+0000 7f9765a45740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572b98375f0 con 0x5572b96c8f60 2024-09-21T03:28:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.222+0000 7f9765a45740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5572b9971010 con 0x5572b96a8280 2024-09-21T03:28:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.222+0000 7f9765a45740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5572b9a7d680 con 0x5572b96a00a0 2024-09-21T03:28:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f9765a45740 10 client.0 _ll_register_callbacks cb 0x5572b99d0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f975da6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a869a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f975da6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a869a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40234/0 (socket says 172.21.15.112:40234) 2024-09-21T03:28:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f975da6e640 1 -- 172.21.15.112:0/324253148 learned_addr learned my addr 172.21.15.112:0/324253148 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f975d26d640 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a7a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f975ca6c640 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a7aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f975da6e640 1 -- 172.21.15.112:0/324253148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 msgr2=0x5572b9a7a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f975da6e640 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a7a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f975da6e640 1 -- 172.21.15.112:0/324253148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 msgr2=0x5572b9a7aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f975da6e640 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a7aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f975da6e640 1 -- 172.21.15.112:0/324253148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b999ea00 con 0x5572b96c8f60 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f975da6e640 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a869a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9750031570 tx=0x7f975002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cec43a9a2eda797 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f973cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f973cff9640 1 -- 172.21.15.112:0/324253148 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750003c40 con 0x5572b96c8f60 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f9765a45740 1 -- 172.21.15.112:0/324253148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572b99e8b60 con 0x5572b96c8f60 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f973cff9640 1 -- 172.21.15.112:0/324253148 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f9750003de0 con 0x5572b96c8f60 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.223+0000 7f973cff9640 1 -- 172.21.15.112:0/324253148 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9750004040 con 0x5572b96c8f60 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.224+0000 7f9765a45740 10 client.15243 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:21.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.224+0000 7f9765a45740 1 -- 172.21.15.112:0/324253148 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5572b9a895f0 con 0x5572b96c8f60 2024-09-21T03:28:21.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.224+0000 7f973cff9640 1 -- 172.21.15.112:0/324253148 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5572b99e8b60 con 0x5572b96c8f60 2024-09-21T03:28:21.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.224+0000 7f973cff9640 1 -- 172.21.15.112:0/324253148 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f975002fc70 con 0x5572b96c8f60 2024-09-21T03:28:21.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.224+0000 7f9765a45740 10 client.15243 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:21.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.224+0000 7f9765a45740 10 client.15243 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:28:21.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.224+0000 7f9765a45740 20 client.15243 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:21.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.224+0000 7f9765a45740 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5572b96878e0 0x5572b9a8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.224+0000 7f9765a45740 4 client.15243 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:21.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.224+0000 7f9765a45740 1 -- 172.21.15.112:0/324253148 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572b99e4610 con 0x5572b96878e0 2024-09-21T03:28:21.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.224+0000 7f975d26d640 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5572b96878e0 0x5572b9a8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.225+0000 7f973cff9640 10 client.15243 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:21.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.225+0000 7f975d26d640 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5572b96878e0 0x5572b9a8d090 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff9165af473d969 server_cookie=ed3906a0c0e4d80 in_seq=0 out_seq=0 2024-09-21T03:28:21.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f973cff9640 1 -- 172.21.15.112:0/324253148 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572b99e4610 con 0x5572b96878e0 2024-09-21T03:28:21.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f973cff9640 10 client.15243 handle_command_reply: tid=0 2024-09-21T03:28:21.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 1 client.15243 shutdown 2024-09-21T03:28:21.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 1 -- 172.21.15.112:0/324253148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 msgr2=0x5572b9a869a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a869a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9750031570 tx=0x7f975002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 1 -- 172.21.15.112:0/324253148 shutdown_connections 2024-09-21T03:28:21.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5572b96878e0 0x5572b9a8d090 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a869a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f973cff9640 0 client.15243 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:21.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a7aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 1 --2- 172.21.15.112:0/324253148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a7a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 1 -- 172.21.15.112:0/324253148 >> 172.21.15.112:0/324253148 conn(0x5572b96bd350 msgr2=0x5572b9a72090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 1 -- 172.21.15.112:0/324253148 shutdown_connections 2024-09-21T03:28:21.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 1 -- 172.21.15.112:0/324253148 wait complete. 2024-09-21T03:28:21.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.226+0000 7f9765a45740 20 client.15243 trim_cache size 0 max 16384 2024-09-21T03:28:21.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.244+0000 7f9765a45740 1 Processor -- start 2024-09-21T03:28:21.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.244+0000 7f9765a45740 1 -- start start 2024-09-21T03:28:21.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.244+0000 7f9765a45740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a67ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.244+0000 7f9765a45740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a86910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.244+0000 7f9765a45740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a86e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.244+0000 7f9765a45740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572b9a8b3d0 con 0x5572b96c8f60 2024-09-21T03:28:21.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.244+0000 7f9765a45740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5572b9a8baa0 con 0x5572b96a8280 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.244+0000 7f9765a45740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5572b9a8c170 con 0x5572b96a00a0 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.244+0000 7f975da6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a86e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.244+0000 7f975da6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a86e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40248/0 (socket says 172.21.15.112:40248) 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.244+0000 7f975da6e640 1 -- 172.21.15.112:0/1742125877 learned_addr learned my addr 172.21.15.112:0/1742125877 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f975ca6c640 1 --2- 172.21.15.112:0/1742125877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a86910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f975d26d640 1 --2- 172.21.15.112:0/1742125877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a67ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f975da6e640 1 -- 172.21.15.112:0/1742125877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 msgr2=0x5572b9a67ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f975da6e640 1 --2- 172.21.15.112:0/1742125877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a67ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f975da6e640 1 -- 172.21.15.112:0/1742125877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 msgr2=0x5572b9a86910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f975da6e640 1 --2- 172.21.15.112:0/1742125877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a86910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f975da6e640 1 -- 172.21.15.112:0/1742125877 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b9a7ba60 con 0x5572b96c8f60 2024-09-21T03:28:21.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f975d26d640 1 --2- 172.21.15.112:0/1742125877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a67ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f975da6e640 1 --2- 172.21.15.112:0/1742125877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a86e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9750030780 tx=0x7f9750003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1f6be08b4cd3b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f973effd640 1 -- 172.21.15.112:0/1742125877 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750030a80 con 0x5572b96c8f60 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f973effd640 1 -- 172.21.15.112:0/1742125877 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9750036040 con 0x5572b96c8f60 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.245+0000 7f973effd640 1 -- 172.21.15.112:0/1742125877 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975002f8c0 con 0x5572b96c8f60 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.246+0000 7f9765a45740 1 -- 172.21.15.112:0/1742125877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 msgr2=0x5572b9a86e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.246+0000 7f9765a45740 1 --2- 172.21.15.112:0/1742125877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a86e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9750030780 tx=0x7f9750003e80 comp rx=0 tx=0).stop 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.246+0000 7f9765a45740 1 -- 172.21.15.112:0/1742125877 shutdown_connections 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.246+0000 7f9765a45740 1 --2- 172.21.15.112:0/1742125877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96c8f60 0x5572b9a86e50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.246+0000 7f9765a45740 1 --2- 172.21.15.112:0/1742125877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a86910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.246+0000 7f9765a45740 1 --2- 172.21.15.112:0/1742125877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96a00a0 0x5572b9a67ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.246+0000 7f9765a45740 1 -- 172.21.15.112:0/1742125877 >> 172.21.15.112:0/1742125877 conn(0x5572b96bd350 msgr2=0x5572b9a758e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.246+0000 7f9765a45740 1 -- 172.21.15.112:0/1742125877 shutdown_connections 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.246+0000 7f9765a45740 1 -- 172.21.15.112:0/1742125877 wait complete. 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.246+0000 7f9765a45740 1 -- start start 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f9765a45740 1 Processor -- start 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f9765a45740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a00a0 0x5572b9a95d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f9765a45740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a8f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f9765a45740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96c8f60 0x5572b9a8fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f9765a45740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572b9a8cf10 con 0x5572b96a00a0 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f9765a45740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5572b9a8c840 con 0x5572b96a8280 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f9765a45740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5572b9a8d5e0 con 0x5572b96c8f60 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f9765a45740 10 client.0 _ll_register_callbacks cb 0x7f9750031b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f975d26d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a00a0 0x5572b9a95d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f975d26d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a00a0 0x5572b9a95d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40260/0 (socket says 172.21.15.112:40260) 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f975d26d640 1 -- 172.21.15.112:0/560085197 learned_addr learned my addr 172.21.15.112:0/560085197 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f975ca6c640 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a8f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f975da6e640 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96c8f60 0x5572b9a8fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.247+0000 7f975d26d640 1 -- 172.21.15.112:0/560085197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96c8f60 msgr2=0x5572b9a8fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f975d26d640 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96c8f60 0x5572b9a8fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f975d26d640 1 -- 172.21.15.112:0/560085197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 msgr2=0x5572b9a8f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f975d26d640 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a8f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f975d26d640 1 -- 172.21.15.112:0/560085197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b9a94990 con 0x5572b96a00a0 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f975da6e640 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96c8f60 0x5572b9a8fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f975d26d640 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a00a0 0x5572b9a95d50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5572b9a8de30 tx=0x7f9748013980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3b66e61ad525b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f973cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f973cff9640 1 -- 172.21.15.112:0/560085197 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748005020 con 0x5572b96a00a0 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f973cff9640 1 -- 172.21.15.112:0/560085197 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f97480051c0 con 0x5572b96a00a0 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f9765a45740 1 -- 172.21.15.112:0/560085197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9750036040 con 0x5572b96a00a0 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f973cff9640 1 -- 172.21.15.112:0/560085197 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f97480053c0 con 0x5572b96a00a0 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f9765a45740 10 client.15249 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f9765a45740 1 -- 172.21.15.112:0/560085197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5572b9af86e0 con 0x5572b96a00a0 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.248+0000 7f973cff9640 1 -- 172.21.15.112:0/560085197 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9750036040 con 0x5572b96a00a0 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.249+0000 7f973cff9640 1 -- 172.21.15.112:0/560085197 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9748004910 con 0x5572b96a00a0 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.249+0000 7f9765a45740 10 client.15249 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.249+0000 7f9765a45740 10 client.15249 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.249+0000 7f9765a45740 20 client.15249 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.249+0000 7f9765a45740 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5572b965cc90 0x5572b9afbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.249+0000 7f9765a45740 4 client.15249 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.249+0000 7f9765a45740 1 -- 172.21.15.112:0/560085197 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "status"}) -- 0x5572b99e1a40 con 0x5572b965cc90 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.249+0000 7f975ca6c640 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5572b965cc90 0x5572b9afbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.249+0000 7f973cff9640 10 client.15249 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.249+0000 7f975ca6c640 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5572b965cc90 0x5572b9afbfe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=904418748c350ec server_cookie=2db26b13025e5eaa in_seq=0 out_seq=0 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f973cff9640 1 -- 172.21.15.112:0/560085197 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5572b99e1a40 con 0x5572b965cc90 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f973cff9640 10 client.15249 handle_command_reply: tid=0 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 1 client.15249 shutdown 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 1 -- 172.21.15.112:0/560085197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a00a0 msgr2=0x5572b9a95d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a00a0 0x5572b9a95d50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5572b9a8de30 tx=0x7f9748013980 comp rx=0 tx=0).stop 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 1 -- 172.21.15.112:0/560085197 shutdown_connections 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5572b965cc90 0x5572b9afbfe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5572b96c8f60 0x5572b9a8fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5572b96a8280 0x5572b9a8f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 1 --2- 172.21.15.112:0/560085197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572b96a00a0 0x5572b9a95d50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 1 -- 172.21.15.112:0/560085197 >> 172.21.15.112:0/560085197 conn(0x5572b96bd350 msgr2=0x5572b9a72000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f973cff9640 0 client.15249 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 1 -- 172.21.15.112:0/560085197 shutdown_connections 2024-09-21T03:28:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 1 -- 172.21.15.112:0/560085197 wait complete. 2024-09-21T03:28:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.250+0000 7f9765a45740 20 client.15249 trim_cache size 0 max 16384 2024-09-21T03:28:21.269 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:28:21.269 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "063b32f6-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T03:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24685, 2024-09-21T03:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T03:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T03:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T03:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 78.934164042999996, 2024-09-21T03:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-09-21T03:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 76, 2024-09-21T03:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T03:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 97.212309579000006 2024-09-21T03:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:28:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 -- 172.21.15.112:0/2038811189 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97400b65f0 msgr2=0x7f97400b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97400b65f0 0x7f97400b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9750006010 tx=0x7f97500095c0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 -- 172.21.15.112:0/2038811189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97581094a0 msgr2=0x7f9758075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97581094a0 0x7f9758075c50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f975806c9f0 tx=0x7f974c00db30 comp rx=0 tx=0).stop 2024-09-21T03:28:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 -- 172.21.15.112:0/2038811189 shutdown_connections 2024-09-21T03:28:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97400b65f0 0x7f97400b8b00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9758109df0 0x7f9758076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97581094a0 0x7f9758075c50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 --2- 172.21.15.112:0/2038811189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9758103b60 0x7f9758075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 -- 172.21.15.112:0/2038811189 >> 172.21.15.112:0/2038811189 conn(0x7f975807b850 msgr2=0x7f9758107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 -- 172.21.15.112:0/2038811189 shutdown_connections 2024-09-21T03:28:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:21.251+0000 7f975f4d1640 1 -- 172.21.15.112:0/2038811189 wait complete. 2024-09-21T03:28:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:21 smithi112 ceph-mon[31442]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:21.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:21 smithi203 ceph-mon[34723]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:21 smithi187 ceph-mon[34603]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:22.045 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T03:28:22.045 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:28:22.054 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T03:28:22.054 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T03:28:22.300 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:22.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.763+0000 7f5b77753640 1 -- 172.21.15.112:0/3448158466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b70103bb0 msgr2=0x7f5b70103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.763+0000 7f5b77753640 1 --2- 172.21.15.112:0/3448158466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b70103bb0 0x7f5b70103fc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009900 tx=0x7f5b6c02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.763+0000 7f5b77753640 1 -- 172.21.15.112:0/3448158466 shutdown_connections 2024-09-21T03:28:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.763+0000 7f5b77753640 1 --2- 172.21.15.112:0/3448158466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b70109e40 0x7f5b70110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.763+0000 7f5b77753640 1 --2- 172.21.15.112:0/3448158466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b701094f0 0x7f5b70109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.763+0000 7f5b77753640 1 --2- 172.21.15.112:0/3448158466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b70103bb0 0x7f5b70103fc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.763+0000 7f5b77753640 1 -- 172.21.15.112:0/3448158466 >> 172.21.15.112:0/3448158466 conn(0x7f5b700ff600 msgr2=0x7f5b70101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.763+0000 7f5b77753640 1 -- 172.21.15.112:0/3448158466 shutdown_connections 2024-09-21T03:28:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.763+0000 7f5b77753640 1 -- 172.21.15.112:0/3448158466 wait complete. 2024-09-21T03:28:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.764+0000 7f5b77753640 1 Processor -- start 2024-09-21T03:28:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.764+0000 7f5b77753640 1 -- start start 2024-09-21T03:28:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.764+0000 7f5b77753640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b70103bb0 0x7f5b7019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.764+0000 7f5b77753640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b701094f0 0x7f5b7019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.764+0000 7f5b77753640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b70109e40 0x7f5b7019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.764+0000 7f5b77753640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70113b60 con 0x7f5b70103bb0 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.764+0000 7f5b77753640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701139e0 con 0x7f5b701094f0 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.764+0000 7f5b77753640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70113ce0 con 0x7f5b70109e40 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.764+0000 7f5b754ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b70103bb0 0x7f5b7019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b754ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b70103bb0 0x7f5b7019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40276/0 (socket says 172.21.15.112:40276) 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b754ef640 1 -- 172.21.15.112:0/745996515 learned_addr learned my addr 172.21.15.112:0/745996515 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b75cf0640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b70109e40 0x7f5b7019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b74cee640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b701094f0 0x7f5b7019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b754ef640 1 -- 172.21.15.112:0/745996515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b70109e40 msgr2=0x7f5b7019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b754ef640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b70109e40 0x7f5b7019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b754ef640 1 -- 172.21.15.112:0/745996515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b701094f0 msgr2=0x7f5b7019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b754ef640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b701094f0 0x7f5b7019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:22.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b754ef640 1 -- 172.21.15.112:0/745996515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b701a03e0 con 0x7f5b70103bb0 2024-09-21T03:28:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b74cee640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b701094f0 0x7f5b7019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b754ef640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b70103bb0 0x7f5b7019a530 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c006010 tx=0x7f5b6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b667fc640 1 -- 172.21.15.112:0/745996515 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c03f030 con 0x7f5b70103bb0 2024-09-21T03:28:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b667fc640 1 -- 172.21.15.112:0/745996515 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6c003800 con 0x7f5b70103bb0 2024-09-21T03:28:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b77753640 1 -- 172.21.15.112:0/745996515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b701a0670 con 0x7f5b70103bb0 2024-09-21T03:28:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b667fc640 1 -- 172.21.15.112:0/745996515 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c035070 con 0x7f5b70103bb0 2024-09-21T03:28:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.765+0000 7f5b77753640 1 -- 172.21.15.112:0/745996515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b701a6ef0 con 0x7f5b70103bb0 2024-09-21T03:28:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.767+0000 7f5b77753640 1 -- 172.21.15.112:0/745996515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b70103fc0 con 0x7f5b70103bb0 2024-09-21T03:28:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.768+0000 7f5b667fc640 1 -- 172.21.15.112:0/745996515 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f5b6c003cb0 con 0x7f5b70103bb0 2024-09-21T03:28:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.769+0000 7f5b667fc640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5b5c0b66a0 0x7f5b5c0b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.769+0000 7f5b74cee640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5b5c0b66a0 0x7f5b5c0b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.769+0000 7f5b667fc640 1 -- 172.21.15.112:0/745996515 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f5b6c1080f0 con 0x7f5b70103bb0 2024-09-21T03:28:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.769+0000 7f5b74cee640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5b5c0b66a0 0x7f5b5c0b8bb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5b70108f90 tx=0x7f5b6000a5c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:22.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.772+0000 7f5b667fc640 1 -- 172.21.15.112:0/745996515 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b6c0ce760 con 0x7f5b70103bb0 2024-09-21T03:28:22.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.969+0000 7f5b77753640 1 -- 172.21.15.112:0/745996515 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f5b70063800 con 0x7f5b5c0b66a0 2024-09-21T03:28:22.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.970+0000 7f5b667fc640 1 -- 172.21.15.112:0/745996515 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f5b70063800 con 0x7f5b5c0b66a0 2024-09-21T03:28:22.972 INFO:teuthology.orchestra.run.smithi112.stdout:Backend: cephadm 2024-09-21T03:28:22.972 INFO:teuthology.orchestra.run.smithi112.stdout:Available: Yes 2024-09-21T03:28:22.972 INFO:teuthology.orchestra.run.smithi112.stdout:Paused: No 2024-09-21T03:28:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 -- 172.21.15.112:0/745996515 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5b5c0b66a0 msgr2=0x7f5b5c0b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5b5c0b66a0 0x7f5b5c0b8bb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5b70108f90 tx=0x7f5b6000a5c0 comp rx=0 tx=0).stop 2024-09-21T03:28:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 -- 172.21.15.112:0/745996515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b70103bb0 msgr2=0x7f5b7019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b70103bb0 0x7f5b7019a530 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c006010 tx=0x7f5b6c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:28:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 -- 172.21.15.112:0/745996515 shutdown_connections 2024-09-21T03:28:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5b5c0b66a0 0x7f5b5c0b8bb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b70109e40 0x7f5b7019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b701094f0 0x7f5b7019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 --2- 172.21.15.112:0/745996515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b70103bb0 0x7f5b7019a530 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 -- 172.21.15.112:0/745996515 >> 172.21.15.112:0/745996515 conn(0x7f5b700ff600 msgr2=0x7f5b70101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:22.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 -- 172.21.15.112:0/745996515 shutdown_connections 2024-09-21T03:28:22.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:22.973+0000 7f5b77753640 1 -- 172.21.15.112:0/745996515 wait complete. 2024-09-21T03:28:23.667 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:23 smithi112 ceph-mon[31442]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:23.669 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T03:28:23.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:23 smithi187 ceph-mon[34603]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:23.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:23 smithi203 ceph-mon[34723]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:24.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.397+0000 7f553d7c5640 1 -- 172.21.15.112:0/3284966407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55381094a0 msgr2=0x7f55381098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:24.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.397+0000 7f553d7c5640 1 --2- 172.21.15.112:0/3284966407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55381094a0 0x7f55381098b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5520009900 tx=0x7f552002a410 comp rx=0 tx=0).stop 2024-09-21T03:28:24.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.397+0000 7f553d7c5640 1 -- 172.21.15.112:0/3284966407 shutdown_connections 2024-09-21T03:28:24.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.397+0000 7f553d7c5640 1 --2- 172.21.15.112:0/3284966407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5538109df0 0x7f5538110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.397+0000 7f553d7c5640 1 --2- 172.21.15.112:0/3284966407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55381094a0 0x7f55381098b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.397+0000 7f553d7c5640 1 --2- 172.21.15.112:0/3284966407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5538103b60 0x7f5538103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.397+0000 7f553d7c5640 1 -- 172.21.15.112:0/3284966407 >> 172.21.15.112:0/3284966407 conn(0x7f553807b850 msgr2=0x7f553807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:24.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.397+0000 7f553d7c5640 1 -- 172.21.15.112:0/3284966407 shutdown_connections 2024-09-21T03:28:24.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.397+0000 7f553d7c5640 1 -- 172.21.15.112:0/3284966407 wait complete. 2024-09-21T03:28:24.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.398+0000 7f553d7c5640 1 Processor -- start 2024-09-21T03:28:24.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f553d7c5640 1 -- start start 2024-09-21T03:28:24.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f553d7c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5538103b60 0x7f553819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:24.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f553d7c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55381094a0 0x7f553819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:24.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f553d7c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5538109df0 0x7f553819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:24.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f553d7c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5538113ad0 con 0x7f5538103b60 2024-09-21T03:28:24.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f553d7c5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5538113950 con 0x7f55381094a0 2024-09-21T03:28:24.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f553d7c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5538113c50 con 0x7f5538109df0 2024-09-21T03:28:24.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f5536ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5538103b60 0x7f553819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:24.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f5536ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5538103b60 0x7f553819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40294/0 (socket says 172.21.15.112:40294) 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f5536ffd640 1 -- 172.21.15.112:0/1728815468 learned_addr learned my addr 172.21.15.112:0/1728815468 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f552e7fc640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55381094a0 0x7f553819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f55377fe640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5538109df0 0x7f553819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f5536ffd640 1 -- 172.21.15.112:0/1728815468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5538109df0 msgr2=0x7f553819fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f5536ffd640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5538109df0 0x7f553819fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f5536ffd640 1 -- 172.21.15.112:0/1728815468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55381094a0 msgr2=0x7f553819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f5536ffd640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55381094a0 0x7f553819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f5536ffd640 1 -- 172.21.15.112:0/1728815468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55381a04b0 con 0x7f5538103b60 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.399+0000 7f55377fe640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5538109df0 0x7f553819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.400+0000 7f5536ffd640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5538103b60 0x7f553819a540 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f552800b4a0 tx=0x7f552800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.400+0000 7f5534ff9640 1 -- 172.21.15.112:0/1728815468 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528014070 con 0x7f5538103b60 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.400+0000 7f553d7c5640 1 -- 172.21.15.112:0/1728815468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55381a07a0 con 0x7f5538103b60 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.400+0000 7f5534ff9640 1 -- 172.21.15.112:0/1728815468 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5528010ac0 con 0x7f5538103b60 2024-09-21T03:28:24.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.400+0000 7f5534ff9640 1 -- 172.21.15.112:0/1728815468 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528017c30 con 0x7f5538103b60 2024-09-21T03:28:24.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.400+0000 7f553d7c5640 1 -- 172.21.15.112:0/1728815468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55381a6f00 con 0x7f5538103b60 2024-09-21T03:28:24.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.401+0000 7f553d7c5640 1 -- 172.21.15.112:0/1728815468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5538103f70 con 0x7f5538103b60 2024-09-21T03:28:24.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.403+0000 7f5534ff9640 1 -- 172.21.15.112:0/1728815468 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f5528017070 con 0x7f5538103b60 2024-09-21T03:28:24.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.404+0000 7f5534ff9640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f55240b65f0 0x7f55240b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:24.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.404+0000 7f552e7fc640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f55240b65f0 0x7f55240b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:24.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.404+0000 7f5534ff9640 1 -- 172.21.15.112:0/1728815468 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f55280e6b60 con 0x7f5538103b60 2024-09-21T03:28:24.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.405+0000 7f552e7fc640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f55240b65f0 0x7f55240b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5520009230 tx=0x7f5520031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:24.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.407+0000 7f5534ff9640 1 -- 172.21.15.112:0/1728815468 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55280ad1d0 con 0x7f5538103b60 2024-09-21T03:28:24.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.596+0000 7f553d7c5640 1 -- 172.21.15.112:0/1728815468 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5538063800 con 0x7f55240b65f0 2024-09-21T03:28:24.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:24 smithi112 ceph-mon[31442]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:24.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.609+0000 7f5534ff9640 1 -- 172.21.15.112:0/1728815468 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f5538063800 con 0x7f55240b65f0 2024-09-21T03:28:24.611 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T03:28:24.611 INFO:teuthology.orchestra.run.smithi112.stdout:mds.a smithi112 running (95s) 17s ago 94s 83.3M - 19.3.0-5120-g18432218 ddce307e1f32 a02fbf2e7356 2024-09-21T03:28:24.611 INFO:teuthology.orchestra.run.smithi112.stdout:mds.b smithi187 running (92s) 25s ago 92s 85.2M - 19.3.0-5120-g18432218 ddce307e1f32 dc4f3bac727d 2024-09-21T03:28:24.611 INFO:teuthology.orchestra.run.smithi112.stdout:mds.c smithi203 running (105s) 25s ago 105s 85.5M - 19.3.0-5120-g18432218 ddce307e1f32 ad91d241c5b1 2024-09-21T03:28:24.611 INFO:teuthology.orchestra.run.smithi112.stdout:mds.d smithi112 running (113s) 17s ago 112s 83.3M - 19.3.0-5120-g18432218 ddce307e1f32 694f730c71ba 2024-09-21T03:28:24.611 INFO:teuthology.orchestra.run.smithi112.stdout:mds.e smithi187 running (110s) 25s ago 110s 84.8M - 19.3.0-5120-g18432218 ddce307e1f32 0e278118c1b8 2024-09-21T03:28:24.611 INFO:teuthology.orchestra.run.smithi112.stdout:mds.f smithi203 running (108s) 25s ago 107s 85.2M - 19.3.0-5120-g18432218 ddce307e1f32 7ca03a4afbb1 2024-09-21T03:28:24.611 INFO:teuthology.orchestra.run.smithi112.stdout:mds.g smithi112 running (98s) 17s ago 97s 82.1M - 19.3.0-5120-g18432218 ddce307e1f32 5d392b3753e0 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:mds.h smithi187 running (103s) 25s ago 102s 84.1M - 19.3.0-5120-g18432218 ddce307e1f32 0471c51f1b1f 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:mds.i smithi203 running (100s) 25s ago 100s 93.9M - 19.3.0-5120-g18432218 ddce307e1f32 2b2a8383862e 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:mds.j smithi112 running (116s) 17s ago 116s 82.5M - 19.3.0-5120-g18432218 ddce307e1f32 671905f460a2 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:mds.k smithi187 running (119s) 25s ago 119s 82.2M - 19.3.0-5120-g18432218 ddce307e1f32 3fbc25157879 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:mds.l smithi203 running (2m) 25s ago 2m 81.9M - 19.3.0-5120-g18432218 ddce307e1f32 5e23b10be529 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.x smithi112 *:9283,8765 running (9m) 17s ago 9m 584M - 19.3.0-5120-g18432218 ddce307e1f32 0d4f99a98dc8 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.y smithi187 *:8443,8765 running (7m) 25s ago 7m 524M - 19.3.0-5120-g18432218 ddce307e1f32 6187016459bf 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.z smithi203 *:8443,8765 running (7m) 25s ago 7m 527M - 19.3.0-5120-g18432218 ddce307e1f32 77c02ea8cb23 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:mon.a smithi112 running (9m) 17s ago 9m 120M 2048M 19.3.0-5120-g18432218 ddce307e1f32 aaae2dc90557 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:mon.b smithi187 running (8m) 25s ago 8m 110M 2048M 19.3.0-5120-g18432218 ddce307e1f32 8c10b0113f0a 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:mon.c smithi203 running (8m) 25s ago 8m 109M 2048M 19.3.0-5120-g18432218 ddce307e1f32 27cd630757a9 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (6m) 17s ago 6m 118M 4272M 19.3.0-5120-g18432218 ddce307e1f32 36ccb3881447 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi187 running (6m) 25s ago 6m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 1088e34bc0e2 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi203 running (6m) 25s ago 6m 125M 4272M 19.3.0-5120-g18432218 ddce307e1f32 cfb905ef1a41 2024-09-21T03:28:24.612 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi112 running (5m) 17s ago 5m 121M 4272M 19.3.0-5120-g18432218 ddce307e1f32 52a95e62ac65 2024-09-21T03:28:24.613 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi187 running (5m) 25s ago 5m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 478dd9d59f1d 2024-09-21T03:28:24.613 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi203 running (4m) 25s ago 4m 114M 4272M 19.3.0-5120-g18432218 ddce307e1f32 2bc8155096eb 2024-09-21T03:28:24.613 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 smithi112 running (4m) 17s ago 4m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 5b32ed713d87 2024-09-21T03:28:24.613 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 smithi187 running (3m) 25s ago 3m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 0a4f5915555a 2024-09-21T03:28:24.613 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 smithi203 running (3m) 25s ago 3m 114M 4272M 19.3.0-5120-g18432218 ddce307e1f32 25474c3d780f 2024-09-21T03:28:24.613 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 smithi112 running (3m) 17s ago 3m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 0092d6555270 2024-09-21T03:28:24.613 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 smithi187 running (2m) 25s ago 2m 116M 4272M 19.3.0-5120-g18432218 ddce307e1f32 bf24e307b9c5 2024-09-21T03:28:24.613 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 smithi203 running (2m) 25s ago 2m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 25840ab2e263 2024-09-21T03:28:24.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.611+0000 7f553d7c5640 1 -- 172.21.15.112:0/1728815468 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f55240b65f0 msgr2=0x7f55240b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:24.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.611+0000 7f553d7c5640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f55240b65f0 0x7f55240b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5520009230 tx=0x7f5520031400 comp rx=0 tx=0).stop 2024-09-21T03:28:24.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.611+0000 7f553d7c5640 1 -- 172.21.15.112:0/1728815468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5538103b60 msgr2=0x7f553819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:24.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.611+0000 7f553d7c5640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5538103b60 0x7f553819a540 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f552800b4a0 tx=0x7f552800b960 comp rx=0 tx=0).stop 2024-09-21T03:28:24.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.611+0000 7f553d7c5640 1 -- 172.21.15.112:0/1728815468 shutdown_connections 2024-09-21T03:28:24.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.611+0000 7f553d7c5640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f55240b65f0 0x7f55240b8b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.611+0000 7f553d7c5640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5538109df0 0x7f553819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.611+0000 7f553d7c5640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55381094a0 0x7f553819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.611+0000 7f553d7c5640 1 --2- 172.21.15.112:0/1728815468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5538103b60 0x7f553819a540 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:24.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.611+0000 7f553d7c5640 1 -- 172.21.15.112:0/1728815468 >> 172.21.15.112:0/1728815468 conn(0x7f553807b850 msgr2=0x7f5538107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:24.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.611+0000 7f553d7c5640 1 -- 172.21.15.112:0/1728815468 shutdown_connections 2024-09-21T03:28:24.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:24.612+0000 7f553d7c5640 1 -- 172.21.15.112:0/1728815468 wait complete. 2024-09-21T03:28:24.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:24 smithi187 ceph-mon[34603]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:24.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:24 smithi203 ceph-mon[34723]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:25.265 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T03:28:25.511 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:25.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:25 smithi112 ceph-mon[31442]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:25 smithi187 ceph-mon[34603]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:25.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:25 smithi203 ceph-mon[34723]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.004+0000 7fa2320a7640 1 -- 172.21.15.112:0/498032988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1049e0 msgr2=0x7fa22c104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.004+0000 7fa2320a7640 1 --2- 172.21.15.112:0/498032988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1049e0 0x7fa22c104df0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009900 tx=0x7fa21c02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.004+0000 7fa2320a7640 1 -- 172.21.15.112:0/498032988 shutdown_connections 2024-09-21T03:28:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.004+0000 7fa2320a7640 1 --2- 172.21.15.112:0/498032988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa22c10de40 0x7fa22c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.004+0000 7fa2320a7640 1 --2- 172.21.15.112:0/498032988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa22c105330 0x7fa22c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.004+0000 7fa2320a7640 1 --2- 172.21.15.112:0/498032988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1049e0 0x7fa22c104df0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:26.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.004+0000 7fa2320a7640 1 -- 172.21.15.112:0/498032988 >> 172.21.15.112:0/498032988 conn(0x7fa22c079750 msgr2=0x7fa22c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:26.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.004+0000 7fa2320a7640 1 -- 172.21.15.112:0/498032988 shutdown_connections 2024-09-21T03:28:26.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.004+0000 7fa2320a7640 1 -- 172.21.15.112:0/498032988 wait complete. 2024-09-21T03:28:26.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa2320a7640 1 Processor -- start 2024-09-21T03:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa2320a7640 1 -- start start 2024-09-21T03:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa2320a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1049e0 0x7fa22c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa2320a7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa22c105330 0x7fa22c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa2320a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa22c10de40 0x7fa22c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa2320a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c113b00 con 0x7fa22c1049e0 2024-09-21T03:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa2320a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c113980 con 0x7fa22c10de40 2024-09-21T03:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa2320a7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c113c80 con 0x7fa22c105330 2024-09-21T03:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1049e0 0x7fa22c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1049e0 0x7fa22c19a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59494/0 (socket says 172.21.15.112:59494) 2024-09-21T03:28:26.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa22b7fe640 1 -- 172.21.15.112:0/1729673313 learned_addr learned my addr 172.21.15.112:0/1729673313 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa22affd640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa22c105330 0x7fa22c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.005+0000 7fa22bfff640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa22c10de40 0x7fa22c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.006+0000 7fa22b7fe640 1 -- 172.21.15.112:0/1729673313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa22c105330 msgr2=0x7fa22c19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.006+0000 7fa22b7fe640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa22c105330 0x7fa22c19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.006+0000 7fa22b7fe640 1 -- 172.21.15.112:0/1729673313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa22c10de40 msgr2=0x7fa22c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.006+0000 7fa22b7fe640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa22c10de40 0x7fa22c19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.006+0000 7fa22b7fe640 1 -- 172.21.15.112:0/1729673313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c1a0420 con 0x7fa22c1049e0 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.006+0000 7fa22b7fe640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1049e0 0x7fa22c19a570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa21c031a90 tx=0x7fa21c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.006+0000 7fa228ff9640 1 -- 172.21.15.112:0/1729673313 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c03f030 con 0x7fa22c1049e0 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.006+0000 7fa2320a7640 1 -- 172.21.15.112:0/1729673313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c1a06b0 con 0x7fa22c1049e0 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.006+0000 7fa228ff9640 1 -- 172.21.15.112:0/1729673313 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa21c004490 con 0x7fa22c1049e0 2024-09-21T03:28:26.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.006+0000 7fa228ff9640 1 -- 172.21.15.112:0/1729673313 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c035070 con 0x7fa22c1049e0 2024-09-21T03:28:26.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.007+0000 7fa2320a7640 1 -- 172.21.15.112:0/1729673313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c1a6f30 con 0x7fa22c1049e0 2024-09-21T03:28:26.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.008+0000 7fa2320a7640 1 -- 172.21.15.112:0/1729673313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa22c06c280 con 0x7fa22c1049e0 2024-09-21T03:28:26.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.009+0000 7fa228ff9640 1 -- 172.21.15.112:0/1729673313 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fa21c042030 con 0x7fa22c1049e0 2024-09-21T03:28:26.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.009+0000 7fa228ff9640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2140b6620 0x7fa2140b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:26.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.010+0000 7fa22affd640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2140b6620 0x7fa2140b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:26.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.010+0000 7fa228ff9640 1 -- 172.21.15.112:0/1729673313 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa21c107ad0 con 0x7fa22c1049e0 2024-09-21T03:28:26.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.010+0000 7fa22affd640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2140b6620 0x7fa2140b8b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa22c102470 tx=0x7fa220009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:26.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.014+0000 7fa228ff9640 1 -- 172.21.15.112:0/1729673313 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa21c0ce140 con 0x7fa22c1049e0 2024-09-21T03:28:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.200+0000 7fa2320a7640 1 -- 172.21.15.112:0/1729673313 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fa22c063800 con 0x7fa2140b6620 2024-09-21T03:28:26.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.206+0000 7fa228ff9640 1 -- 172.21.15.112:0/1729673313 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fa22c063800 con 0x7fa2140b6620 2024-09-21T03:28:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T03:28:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:mds.all 12/12 27s ago 2m smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-21T03:28:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:mgr 3/3 27s ago 7m smithi112=x;smithi187=y;smithi203=z;count:3 2024-09-21T03:28:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:mon 3/3 27s ago 8m smithi112:172.21.15.112=a;smithi187:172.21.15.187=b;smithi203:172.21.15.203=c;count:3 2024-09-21T03:28:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:osd 12 27s ago - 2024-09-21T03:28:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 -- 172.21.15.112:0/1729673313 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2140b6620 msgr2=0x7fa2140b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2140b6620 0x7fa2140b8b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa22c102470 tx=0x7fa220009110 comp rx=0 tx=0).stop 2024-09-21T03:28:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 -- 172.21.15.112:0/1729673313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1049e0 msgr2=0x7fa22c19a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1049e0 0x7fa22c19a570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa21c031a90 tx=0x7fa21c004270 comp rx=0 tx=0).stop 2024-09-21T03:28:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 -- 172.21.15.112:0/1729673313 shutdown_connections 2024-09-21T03:28:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2140b6620 0x7fa2140b8b30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa22c10de40 0x7fa22c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa22c105330 0x7fa22c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 --2- 172.21.15.112:0/1729673313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1049e0 0x7fa22c19a570 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 -- 172.21.15.112:0/1729673313 >> 172.21.15.112:0/1729673313 conn(0x7fa22c079750 msgr2=0x7fa22c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 -- 172.21.15.112:0/1729673313 shutdown_connections 2024-09-21T03:28:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:26.208+0000 7fa2320a7640 1 -- 172.21.15.112:0/1729673313 wait complete. 2024-09-21T03:28:26.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:26 smithi112 ceph-mon[31442]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:26 smithi187 ceph-mon[34603]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:26.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T03:28:26.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:26 smithi203 ceph-mon[34723]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:27.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:27 smithi112 ceph-mon[31442]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:27.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:27 smithi112 ceph-mon[31442]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:28:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.656+0000 7f7a9fadb640 1 -- 172.21.15.112:0/3227969868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a98114430 msgr2=0x7f7a98116900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.656+0000 7f7a9fadb640 1 --2- 172.21.15.112:0/3227969868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a98114430 0x7f7a98116900 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009ca0 tx=0x7f7a9002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.656+0000 7f7a9fadb640 1 -- 172.21.15.112:0/3227969868 shutdown_connections 2024-09-21T03:28:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.656+0000 7f7a9fadb640 1 --2- 172.21.15.112:0/3227969868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a98114430 0x7f7a98116900 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.656+0000 7f7a9fadb640 1 --2- 172.21.15.112:0/3227969868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a98075dd0 0x7f7a980761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.656+0000 7f7a9fadb640 1 --2- 172.21.15.112:0/3227969868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a98077380 0x7f7a980757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.656+0000 7f7a9fadb640 1 -- 172.21.15.112:0/3227969868 >> 172.21.15.112:0/3227969868 conn(0x7f7a9807b850 msgr2=0x7f7a9807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:27.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.656+0000 7f7a9fadb640 1 -- 172.21.15.112:0/3227969868 shutdown_connections 2024-09-21T03:28:27.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.656+0000 7f7a9fadb640 1 -- 172.21.15.112:0/3227969868 wait complete. 2024-09-21T03:28:27.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.656+0000 7f7a9fadb640 1 Processor -- start 2024-09-21T03:28:27.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.656+0000 7f7a9fadb640 1 -- start start 2024-09-21T03:28:27.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9fadb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a98075dd0 0x7f7a981a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9fadb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a98077380 0x7f7a981a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9fadb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a98114430 0x7f7a981a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9fadb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9811a060 con 0x7f7a98114430 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9fadb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7a98119ee0 con 0x7f7a98077380 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9fadb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9811a1e0 con 0x7f7a98075dd0 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9e078640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a98114430 0x7f7a981a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9e078640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a98114430 0x7f7a981a6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59510/0 (socket says 172.21.15.112:59510) 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9e078640 1 -- 172.21.15.112:0/1888030929 learned_addr learned my addr 172.21.15.112:0/1888030929 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9d877640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a98075dd0 0x7f7a981a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9d076640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a98077380 0x7f7a981a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9e078640 1 -- 172.21.15.112:0/1888030929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a98075dd0 msgr2=0x7f7a981a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9e078640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a98075dd0 0x7f7a981a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9e078640 1 -- 172.21.15.112:0/1888030929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a98077380 msgr2=0x7f7a981a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9e078640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a98077380 0x7f7a981a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9e078640 1 -- 172.21.15.112:0/1888030929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a981a68a0 con 0x7f7a98114430 2024-09-21T03:28:27.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9d076640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a98077380 0x7f7a981a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:27.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a9e078640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a98114430 0x7f7a981a6230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7a90005cc0 tx=0x7f7a9002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:27.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.657+0000 7f7a8effd640 1 -- 172.21.15.112:0/1888030929 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90003cf0 con 0x7f7a98114430 2024-09-21T03:28:27.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.658+0000 7f7a9fadb640 1 -- 172.21.15.112:0/1888030929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a981a6b30 con 0x7f7a98114430 2024-09-21T03:28:27.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.658+0000 7f7a8effd640 1 -- 172.21.15.112:0/1888030929 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a900044f0 con 0x7f7a98114430 2024-09-21T03:28:27.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.658+0000 7f7a9fadb640 1 -- 172.21.15.112:0/1888030929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a981ad3b0 con 0x7f7a98114430 2024-09-21T03:28:27.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.658+0000 7f7a8effd640 1 -- 172.21.15.112:0/1888030929 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9002fd10 con 0x7f7a98114430 2024-09-21T03:28:27.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.659+0000 7f7a9fadb640 1 -- 172.21.15.112:0/1888030929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a980761e0 con 0x7f7a98114430 2024-09-21T03:28:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.660+0000 7f7a8effd640 1 -- 172.21.15.112:0/1888030929 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f7a90004040 con 0x7f7a98114430 2024-09-21T03:28:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.661+0000 7f7a8effd640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7a840b66a0 0x7f7a840b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.661+0000 7f7a9d877640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7a840b66a0 0x7f7a840b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.661+0000 7f7a8effd640 1 -- 172.21.15.112:0/1888030929 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f7a90108cd0 con 0x7f7a98114430 2024-09-21T03:28:27.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.662+0000 7f7a9d877640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7a840b66a0 0x7f7a840b8bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a940058c0 tx=0x7f7a9400a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:27.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.665+0000 7f7a8effd640 1 -- 172.21.15.112:0/1888030929 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7a900cf340 con 0x7f7a98114430 2024-09-21T03:28:27.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.849+0000 7f7a9fadb640 1 -- 172.21.15.112:0/1888030929 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7a98113710 con 0x7f7a840b66a0 2024-09-21T03:28:27.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.850+0000 7f7a8effd640 1 -- 172.21.15.112:0/1888030929 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7a98113710 con 0x7f7a840b66a0 2024-09-21T03:28:27.852 INFO:teuthology.orchestra.run.smithi112.stdout:HOST ADDR LABELS STATUS 2024-09-21T03:28:27.852 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 172.21.15.112 2024-09-21T03:28:27.852 INFO:teuthology.orchestra.run.smithi112.stdout:smithi187 172.21.15.187 2024-09-21T03:28:27.852 INFO:teuthology.orchestra.run.smithi112.stdout:smithi203 172.21.15.203 2024-09-21T03:28:27.852 INFO:teuthology.orchestra.run.smithi112.stdout:3 hosts in cluster 2024-09-21T03:28:27.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 -- 172.21.15.112:0/1888030929 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7a840b66a0 msgr2=0x7f7a840b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7a840b66a0 0x7f7a840b8bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a940058c0 tx=0x7f7a9400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 -- 172.21.15.112:0/1888030929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a98114430 msgr2=0x7f7a981a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:27.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a98114430 0x7f7a981a6230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7a90005cc0 tx=0x7f7a9002f470 comp rx=0 tx=0).stop 2024-09-21T03:28:27.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 -- 172.21.15.112:0/1888030929 shutdown_connections 2024-09-21T03:28:27.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7a840b66a0 0x7f7a840b8bb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a98114430 0x7f7a981a6230 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a98077380 0x7f7a981a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 --2- 172.21.15.112:0/1888030929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a98075dd0 0x7f7a981a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:27.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 -- 172.21.15.112:0/1888030929 >> 172.21.15.112:0/1888030929 conn(0x7f7a9807b850 msgr2=0x7f7a98113000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:27.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 -- 172.21.15.112:0/1888030929 shutdown_connections 2024-09-21T03:28:27.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:27.853+0000 7f7a9fadb640 1 -- 172.21.15.112:0/1888030929 wait complete. 2024-09-21T03:28:27.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:27 smithi203 ceph-mon[34723]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:27.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:27 smithi203 ceph-mon[34723]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:28:27.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:27 smithi187 ceph-mon[34603]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:27.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:27 smithi187 ceph-mon[34603]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:28:28.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T03:28:28.535 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:28 smithi112 ceph-mon[31442]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:28.535 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:28 smithi112 ceph-mon[31442]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:28:28.761 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:28.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:28 smithi203 ceph-mon[34723]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:28.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:28 smithi203 ceph-mon[34723]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:28:28.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:28 smithi187 ceph-mon[34603]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:28.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:28 smithi187 ceph-mon[34603]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:28:29.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.218+0000 7f89e8e55640 1 -- 172.21.15.112:0/2484092887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89e4108d10 msgr2=0x7f89e410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.218+0000 7f89e8e55640 1 --2- 172.21.15.112:0/2484092887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89e4108d10 0x7f89e410f540 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009ca0 tx=0x7f89dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.218+0000 7f89e8e55640 1 -- 172.21.15.112:0/2484092887 shutdown_connections 2024-09-21T03:28:29.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.218+0000 7f89e8e55640 1 --2- 172.21.15.112:0/2484092887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89e4108d10 0x7f89e410f540 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.218+0000 7f89e8e55640 1 --2- 172.21.15.112:0/2484092887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e41083c0 0x7f89e41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.218+0000 7f89e8e55640 1 --2- 172.21.15.112:0/2484092887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89e4102a80 0x7f89e4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.218+0000 7f89e8e55640 1 -- 172.21.15.112:0/2484092887 >> 172.21.15.112:0/2484092887 conn(0x7f89e4079740 msgr2=0x7f89e4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:29.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.218+0000 7f89e8e55640 1 -- 172.21.15.112:0/2484092887 shutdown_connections 2024-09-21T03:28:29.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.218+0000 7f89e8e55640 1 -- 172.21.15.112:0/2484092887 wait complete. 2024-09-21T03:28:29.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.219+0000 7f89e8e55640 1 Processor -- start 2024-09-21T03:28:29.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.219+0000 7f89e8e55640 1 -- start start 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e8e55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89e4102a80 0x7f89e4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e8e55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89e41083c0 0x7f89e4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e8e55640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e4108d10 0x7f89e419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e8e55640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89e4112a60 con 0x7f89e4102a80 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e8e55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f89e41128e0 con 0x7f89e41083c0 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e8e55640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f89e4112be0 con 0x7f89e4108d10 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89e4102a80 0x7f89e4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89e4102a80 0x7f89e4199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59534/0 (socket says 172.21.15.112:59534) 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e259c640 1 -- 172.21.15.112:0/3721191075 learned_addr learned my addr 172.21.15.112:0/3721191075 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e2d9d640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e4108d10 0x7f89e419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89d9d9b640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89e41083c0 0x7f89e4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e259c640 1 -- 172.21.15.112:0/3721191075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e4108d10 msgr2=0x7f89e419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e259c640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e4108d10 0x7f89e419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e259c640 1 -- 172.21.15.112:0/3721191075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89e41083c0 msgr2=0x7f89e4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e259c640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89e41083c0 0x7f89e4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e259c640 1 -- 172.21.15.112:0/3721191075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e419f480 con 0x7f89e4102a80 2024-09-21T03:28:29.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89d9d9b640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89e41083c0 0x7f89e4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89e259c640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89e4102a80 0x7f89e4199510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f89d400b4a0 tx=0x7f89d400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.220+0000 7f89dbfff640 1 -- 172.21.15.112:0/3721191075 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4014070 con 0x7f89e4102a80 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.221+0000 7f89e8e55640 1 -- 172.21.15.112:0/3721191075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e419f770 con 0x7f89e4102a80 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.221+0000 7f89dbfff640 1 -- 172.21.15.112:0/3721191075 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d4010ac0 con 0x7f89e4102a80 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.221+0000 7f89e8e55640 1 -- 172.21.15.112:0/3721191075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e41a5ed0 con 0x7f89e4102a80 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.221+0000 7f89dbfff640 1 -- 172.21.15.112:0/3721191075 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4017c30 con 0x7f89e4102a80 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.222+0000 7f89e8e55640 1 -- 172.21.15.112:0/3721191075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e4102f10 con 0x7f89e4102a80 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.223+0000 7f89dbfff640 1 -- 172.21.15.112:0/3721191075 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f89d4017070 con 0x7f89e4102a80 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.224+0000 7f89dbfff640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89d00b65f0 0x7f89d00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.224+0000 7f89d9d9b640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89d00b65f0 0x7f89d00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.224+0000 7f89dbfff640 1 -- 172.21.15.112:0/3721191075 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f89d40e6dc0 con 0x7f89e4102a80 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.224+0000 7f89d9d9b640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89d00b65f0 0x7f89d00b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f89cc006fd0 tx=0x7f89cc008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:29.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.227+0000 7f89dbfff640 1 -- 172.21.15.112:0/3721191075 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f89d40ad430 con 0x7f89e4102a80 2024-09-21T03:28:29.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.417+0000 7f89e8e55640 1 -- 172.21.15.112:0/3721191075 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f89e4063800 con 0x7f89d00b65f0 2024-09-21T03:28:29.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.418+0000 7f89dbfff640 1 -- 172.21.15.112:0/3721191075 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f89e4063800 con 0x7f89d00b65f0 2024-09-21T03:28:29.482 INFO:teuthology.orchestra.run.smithi112.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T03:28:29.482 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T03:28:29.482 INFO:teuthology.orchestra.run.smithi112.stdout:smithi187 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MW400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T03:28:29.482 INFO:teuthology.orchestra.run.smithi112.stdout:smithi203 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 -- 172.21.15.112:0/3721191075 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89d00b65f0 msgr2=0x7f89d00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89d00b65f0 0x7f89d00b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f89cc006fd0 tx=0x7f89cc008040 comp rx=0 tx=0).stop 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 -- 172.21.15.112:0/3721191075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89e4102a80 msgr2=0x7f89e4199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89e4102a80 0x7f89e4199510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f89d400b4a0 tx=0x7f89d400b960 comp rx=0 tx=0).stop 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 -- 172.21.15.112:0/3721191075 shutdown_connections 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89d00b65f0 0x7f89d00b8b00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e4108d10 0x7f89e419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89e41083c0 0x7f89e4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 --2- 172.21.15.112:0/3721191075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89e4102a80 0x7f89e4199510 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 -- 172.21.15.112:0/3721191075 >> 172.21.15.112:0/3721191075 conn(0x7f89e4079740 msgr2=0x7f89e4105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 -- 172.21.15.112:0/3721191075 shutdown_connections 2024-09-21T03:28:29.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:29.421+0000 7f89e8e55640 1 -- 172.21.15.112:0/3721191075 wait complete. 2024-09-21T03:28:29.717 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:29 smithi112 ceph-mon[31442]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:29.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:29 smithi203 ceph-mon[34723]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:29.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:29 smithi187 ceph-mon[34603]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:30.048 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T03:28:30.060 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T03:28:30.060 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T03:28:30.338 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:30.691 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:30 smithi112 ceph-mon[31442]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.775+0000 7fdd9e674640 1 -- 172.21.15.112:0/798594150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd98108d20 msgr2=0x7fdd9810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.775+0000 7fdd9e674640 1 --2- 172.21.15.112:0/798594150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd98108d20 0x7fdd9810f550 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009ca0 tx=0x7fdd9002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.775+0000 7fdd9e674640 1 -- 172.21.15.112:0/798594150 shutdown_connections 2024-09-21T03:28:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.775+0000 7fdd9e674640 1 --2- 172.21.15.112:0/798594150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd98108d20 0x7fdd9810f550 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.775+0000 7fdd9e674640 1 --2- 172.21.15.112:0/798594150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd981083d0 0x7fdd981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.775+0000 7fdd9e674640 1 --2- 172.21.15.112:0/798594150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd98102ae0 0x7fdd98102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:30.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.775+0000 7fdd9e674640 1 -- 172.21.15.112:0/798594150 >> 172.21.15.112:0/798594150 conn(0x7fdd98079760 msgr2=0x7fdd98079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.775+0000 7fdd9e674640 1 -- 172.21.15.112:0/798594150 shutdown_connections 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.775+0000 7fdd9e674640 1 -- 172.21.15.112:0/798594150 wait complete. 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd9e674640 1 Processor -- start 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd9e674640 1 -- start start 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd9e674640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd98102ae0 0x7fdd98199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd9e674640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd981083d0 0x7fdd98199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd9e674640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd98108d20 0x7fdd9819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd9e674640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98112a70 con 0x7fdd98102ae0 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd9e674640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981128f0 con 0x7fdd981083d0 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd9e674640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98112bf0 con 0x7fdd98108d20 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd97fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd98102ae0 0x7fdd98199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:30.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd97fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd98102ae0 0x7fdd98199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59550/0 (socket says 172.21.15.112:59550) 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd97fff640 1 -- 172.21.15.112:0/3703510446 learned_addr learned my addr 172.21.15.112:0/3703510446 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd977fe640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd981083d0 0x7fdd98199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd9cc11640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd98108d20 0x7fdd9819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd97fff640 1 -- 172.21.15.112:0/3703510446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd98108d20 msgr2=0x7fdd9819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd97fff640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd98108d20 0x7fdd9819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd97fff640 1 -- 172.21.15.112:0/3703510446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd981083d0 msgr2=0x7fdd98199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd97fff640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd981083d0 0x7fdd98199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.776+0000 7fdd97fff640 1 -- 172.21.15.112:0/3703510446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9819f3d0 con 0x7fdd98102ae0 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.777+0000 7fdd97fff640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd98102ae0 0x7fdd98199460 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800b4a0 tx=0x7fdd8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.777+0000 7fdd957fa640 1 -- 172.21.15.112:0/3703510446 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88014070 con 0x7fdd98102ae0 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.777+0000 7fdd9e674640 1 -- 172.21.15.112:0/3703510446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9819f6c0 con 0x7fdd98102ae0 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.777+0000 7fdd957fa640 1 -- 172.21.15.112:0/3703510446 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd88010ac0 con 0x7fdd98102ae0 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.777+0000 7fdd957fa640 1 -- 172.21.15.112:0/3703510446 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88017c30 con 0x7fdd98102ae0 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.777+0000 7fdd9e674640 1 -- 172.21.15.112:0/3703510446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd981a5e20 con 0x7fdd98102ae0 2024-09-21T03:28:30.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.778+0000 7fdd9e674640 1 -- 172.21.15.112:0/3703510446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd98103d10 con 0x7fdd98102ae0 2024-09-21T03:28:30.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.780+0000 7fdd957fa640 1 -- 172.21.15.112:0/3703510446 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fdd88017070 con 0x7fdd98102ae0 2024-09-21T03:28:30.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.780+0000 7fdd957fa640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdd800b65f0 0x7fdd800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:30.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.780+0000 7fdd977fe640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdd800b65f0 0x7fdd800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:30.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.780+0000 7fdd957fa640 1 -- 172.21.15.112:0/3703510446 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fdd880e6d10 con 0x7fdd98102ae0 2024-09-21T03:28:30.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.781+0000 7fdd977fe640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdd800b65f0 0x7fdd800b8b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd98107aa0 tx=0x7fdd8c008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:30.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:30.784+0000 7fdd957fa640 1 -- 172.21.15.112:0/3703510446 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdd880ad380 con 0x7fdd98102ae0 2024-09-21T03:28:30.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:30 smithi203 ceph-mon[34723]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:30.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:30 smithi187 ceph-mon[34603]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T03:28:31.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.001+0000 7fdd9e674640 1 -- 172.21.15.112:0/3703510446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fdd98102f70 con 0x7fdd98102ae0 2024-09-21T03:28:31.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.002+0000 7fdd957fa640 1 -- 172.21.15.112:0/3703510446 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3098 (secure 0 0 0) 0x7fdd880b1760 con 0x7fdd98102ae0 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:e24 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:btime 2024-09-21T03:27:13:388843+0000 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 24 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-21T03:27:01.886242+0000 2024-09-21T03:28:31.004 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-21T03:27:13.104308+0000 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 300 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 0 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 3 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1,2 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=24685,1=24674,2=24667} 2024-09-21T03:28:31.005 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data disabled 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 0 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 24685 members: 24667,24674,24685 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.i{0:24685} state up:active seq 6 addr [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.f{0:24658} state up:standby-replay seq 1 addr [v2:172.21.15.203:6834/3541929468,v1:172.21.15.203:6835/3541929468] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b{1:24674} state up:active seq 6 addr [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.006 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.h{1:24647} state up:standby-replay seq 1 addr [v2:172.21.15.187:6836/2217535204,v1:172.21.15.187:6837/2217535204] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c{2:24667} state up:active seq 9 addr [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.e{2:24626} state up:standby-replay seq 1 addr [v2:172.21.15.187:6834/2552593896,v1:172.21.15.187:6835/2552593896] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.j{-1:14772} state up:standby seq 1 addr [v2:172.21.15.112:6834/2583791645,v1:172.21.15.112:6835/2583791645] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d{-1:14784} state up:standby seq 1 addr [v2:172.21.15.112:6836/3042717510,v1:172.21.15.112:6837/3042717510] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.g{-1:14838} state up:standby seq 1 addr [v2:172.21.15.112:6838/1272407908,v1:172.21.15.112:6839/1272407908] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a{-1:14844} state up:standby seq 1 addr [v2:172.21.15.112:6840/395166022,v1:172.21.15.112:6841/395166022] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.k{-1:24599} state up:standby seq 1 addr [v2:172.21.15.187:6832/3187781490,v1:172.21.15.187:6833/3187781490] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.007 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.l{-1:24616} state up:standby seq 1 addr [v2:172.21.15.203:6832/4285434211,v1:172.21.15.203:6833/4285434211] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T03:28:31.008 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-09-21T03:28:31.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.004+0000 7fdd9e674640 1 -- 172.21.15.112:0/3703510446 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdd800b65f0 msgr2=0x7fdd800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:31.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.004+0000 7fdd9e674640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdd800b65f0 0x7fdd800b8b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd98107aa0 tx=0x7fdd8c008040 comp rx=0 tx=0).stop 2024-09-21T03:28:31.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.004+0000 7fdd9e674640 1 -- 172.21.15.112:0/3703510446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd98102ae0 msgr2=0x7fdd98199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:31.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.004+0000 7fdd9e674640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd98102ae0 0x7fdd98199460 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800b4a0 tx=0x7fdd8800b960 comp rx=0 tx=0).stop 2024-09-21T03:28:31.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.004+0000 7fdd9e674640 1 -- 172.21.15.112:0/3703510446 shutdown_connections 2024-09-21T03:28:31.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.004+0000 7fdd9e674640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fdd800b65f0 0x7fdd800b8b00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.004+0000 7fdd9e674640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd98108d20 0x7fdd9819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.004+0000 7fdd9e674640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd981083d0 0x7fdd98199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.004+0000 7fdd9e674640 1 --2- 172.21.15.112:0/3703510446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd98102ae0 0x7fdd98199460 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:31.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.004+0000 7fdd9e674640 1 -- 172.21.15.112:0/3703510446 >> 172.21.15.112:0/3703510446 conn(0x7fdd98079760 msgr2=0x7fdd98105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:31.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.005+0000 7fdd9e674640 1 -- 172.21.15.112:0/3703510446 shutdown_connections 2024-09-21T03:28:31.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:31.005+0000 7fdd9e674640 1 -- 172.21.15.112:0/3703510446 wait complete. 2024-09-21T03:28:31.559 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:31 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3703510446' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:28:31.560 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T03:28:31.805 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:31.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:31 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3703510446' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:28:31.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:31 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3703510446' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T03:28:32.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.284+0000 7f03366e4640 1 -- 172.21.15.112:0/280286789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03301083d0 msgr2=0x7f03301087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:32.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.284+0000 7f03366e4640 1 --2- 172.21.15.112:0/280286789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03301083d0 0x7f03301087e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0318009900 tx=0x7f031802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:32.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.284+0000 7f03366e4640 1 -- 172.21.15.112:0/280286789 shutdown_connections 2024-09-21T03:28:32.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.284+0000 7f03366e4640 1 --2- 172.21.15.112:0/280286789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330108d20 0x7f033010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.284+0000 7f03366e4640 1 --2- 172.21.15.112:0/280286789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03301083d0 0x7f03301087e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.284+0000 7f03366e4640 1 --2- 172.21.15.112:0/280286789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0330102a90 0x7f0330102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.284+0000 7f03366e4640 1 -- 172.21.15.112:0/280286789 >> 172.21.15.112:0/280286789 conn(0x7f0330079750 msgr2=0x7f0330079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:32.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.284+0000 7f03366e4640 1 -- 172.21.15.112:0/280286789 shutdown_connections 2024-09-21T03:28:32.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.284+0000 7f03366e4640 1 -- 172.21.15.112:0/280286789 wait complete. 2024-09-21T03:28:32.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f03366e4640 1 Processor -- start 2024-09-21T03:28:32.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f03366e4640 1 -- start start 2024-09-21T03:28:32.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f03366e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330102a90 0x7f033019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:32.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f03366e4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03301083d0 0x7f033019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:32.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f03366e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330108d20 0x7f033019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:32.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f03366e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03301129d0 con 0x7f0330108d20 2024-09-21T03:28:32.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f03366e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0330112850 con 0x7f0330102a90 2024-09-21T03:28:32.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f03366e4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0330112b50 con 0x7f03301083d0 2024-09-21T03:28:32.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f0334c81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330108d20 0x7f033019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:32.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f0334c81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330108d20 0x7f033019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59566/0 (socket says 172.21.15.112:59566) 2024-09-21T03:28:32.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f0334c81640 1 -- 172.21.15.112:0/3718002196 learned_addr learned my addr 172.21.15.112:0/3718002196 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:32.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f032ffff640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330102a90 0x7f033019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:32.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.285+0000 7f0327fff640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03301083d0 0x7f033019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:32.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f0334c81640 1 -- 172.21.15.112:0/3718002196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03301083d0 msgr2=0x7f033019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f0334c81640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03301083d0 0x7f033019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f0334c81640 1 -- 172.21.15.112:0/3718002196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330102a90 msgr2=0x7f033019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f0334c81640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330102a90 0x7f033019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f0334c81640 1 -- 172.21.15.112:0/3718002196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03301a0460 con 0x7f0330108d20 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f0327fff640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03301083d0 0x7f033019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f0334c81640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330108d20 0x7f033019fd30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f032800b970 tx=0x7f032800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f032dffb640 1 -- 172.21.15.112:0/3718002196 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328019070 con 0x7f0330108d20 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f03366e4640 1 -- 172.21.15.112:0/3718002196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03301a0750 con 0x7f0330108d20 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f032dffb640 1 -- 172.21.15.112:0/3718002196 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0328014440 con 0x7f0330108d20 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f032dffb640 1 -- 172.21.15.112:0/3718002196 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280075d0 con 0x7f0330108d20 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.286+0000 7f03366e4640 1 -- 172.21.15.112:0/3718002196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03301a6eb0 con 0x7f0330108d20 2024-09-21T03:28:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.287+0000 7f03366e4640 1 -- 172.21.15.112:0/3718002196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0330103d10 con 0x7f0330108d20 2024-09-21T03:28:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.289+0000 7f032dffb640 1 -- 172.21.15.112:0/3718002196 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f032801c070 con 0x7f0330108d20 2024-09-21T03:28:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.289+0000 7f032dffb640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f031c0b65f0 0x7f031c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.290+0000 7f032ffff640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f031c0b65f0 0x7f031c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.290+0000 7f032dffb640 1 -- 172.21.15.112:0/3718002196 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f03280e6f10 con 0x7f0330108d20 2024-09-21T03:28:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.290+0000 7f032ffff640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f031c0b65f0 0x7f031c0b8b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f032000bbe0 tx=0x7f032000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.293+0000 7f032dffb640 1 -- 172.21.15.112:0/3718002196 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f03280b1960 con 0x7f0330108d20 2024-09-21T03:28:32.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.476+0000 7f03366e4640 1 -- 172.21.15.112:0/3718002196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f0330102ea0 con 0x7f0330108d20 2024-09-21T03:28:32.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.477+0000 7f032dffb640 1 -- 172.21.15.112:0/3718002196 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v76) ==== 56+0+5686 (secure 0 0 0) 0x7f03280b0650 con 0x7f0330108d20 2024-09-21T03:28:32.479 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 76 2024-09-21T03:28:32.479 INFO:teuthology.orchestra.run.smithi112.stdout:fsid 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:28:32.479 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-09-21T03:18:41.875282+0000 2024-09-21T03:28:32.479 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-09-21T03:27:33.414969+0000 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:crush_version 26 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:full_ratio 0.95 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:backfillfull_ratio 0.9 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:nearfull_ratio 0.85 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:require_min_compat_client luminous 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:min_compat_client luminous 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:require_osd_release squid 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:stretch_mode_enabled false 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T03:28:32.480 INFO:teuthology.orchestra.run.smithi112.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:max_osd 12 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 up in weight 1 up_from 9 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6802/3601862796,v1:172.21.15.112:6803/3601862796] [v2:172.21.15.112:6804/3601862796,v1:172.21.15.112:6805/3601862796] exists,up ec165c44-c973-4f2b-b1ca-a88a9bf48968 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 up in weight 1 up_from 14 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6800/279076548,v1:172.21.15.187:6801/279076548] [v2:172.21.15.187:6802/279076548,v1:172.21.15.187:6803/279076548] exists,up 200471f7-a0d9-4b5e-8be4-0033c6a82688 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 up in weight 1 up_from 18 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6800/388961217,v1:172.21.15.203:6801/388961217] [v2:172.21.15.203:6802/388961217,v1:172.21.15.203:6803/388961217] exists,up 0afbab16-dfb7-45d0-895c-846a437a19b4 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 up in weight 1 up_from 25 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6810/2827467739,v1:172.21.15.112:6811/2827467739] [v2:172.21.15.112:6812/2827467739,v1:172.21.15.112:6813/2827467739] exists,up 495ece6a-e62f-4deb-8600-ece833469091 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 up in weight 1 up_from 30 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6808/3494239939,v1:172.21.15.187:6809/3494239939] [v2:172.21.15.187:6810/3494239939,v1:172.21.15.187:6811/3494239939] exists,up 0c46289b-1c19-4c58-bd00-5e96505df718 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 up in weight 1 up_from 35 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6808/3996799741,v1:172.21.15.203:6809/3996799741] [v2:172.21.15.203:6810/3996799741,v1:172.21.15.203:6811/3996799741] exists,up f4727682-1223-49ae-a893-2559f553b8b8 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 up in weight 1 up_from 41 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6818/92569356,v1:172.21.15.112:6819/92569356] [v2:172.21.15.112:6820/92569356,v1:172.21.15.112:6821/92569356] exists,up a3b2e075-188b-4640-827b-735e57c61d82 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 up in weight 1 up_from 47 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6816/4003842172,v1:172.21.15.187:6817/4003842172] [v2:172.21.15.187:6818/4003842172,v1:172.21.15.187:6819/4003842172] exists,up 530f9be1-24fa-4f16-8225-fddebd4aba99 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 up in weight 1 up_from 52 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6816/3883207460,v1:172.21.15.203:6817/3883207460] [v2:172.21.15.203:6818/3883207460,v1:172.21.15.203:6819/3883207460] exists,up 3c3869ee-6cae-409d-98bb-9ee286713295 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 up in weight 1 up_from 58 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6826/2553277143,v1:172.21.15.112:6827/2553277143] [v2:172.21.15.112:6828/2553277143,v1:172.21.15.112:6829/2553277143] exists,up c47f58d2-8b8a-4f98-b14d-879143b50781 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 up in weight 1 up_from 63 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6824/499555149,v1:172.21.15.187:6825/499555149] [v2:172.21.15.187:6826/499555149,v1:172.21.15.187:6827/499555149] exists,up 54812bd2-e71b-40eb-8411-d281bdbe0821 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 up in weight 1 up_from 68 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6824/520562655,v1:172.21.15.203:6825/520562655] [v2:172.21.15.203:6826/520562655,v1:172.21.15.203:6827/520562655] exists,up 4157bc4f-e275-4f4d-8d38-f934f9f61a95 2024-09-21T03:28:32.481 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.0 [7,11] 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.e [2,1] 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.10 [8,11] 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.1b [2,4] 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.3 [4,9] 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.13 [4,9] 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.28 [5,8] 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.2c [4,9] 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/4206336392 expires 2024-09-22T03:20:32.227264+0000 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/839399771 expires 2024-09-22T03:19:06.754886+0000 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1495858515 expires 2024-09-22T03:19:06.754886+0000 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/2996077964 expires 2024-09-22T03:19:31.820776+0000 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3545481828 expires 2024-09-22T03:19:06.754886+0000 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/3643082307 expires 2024-09-22T03:20:32.227264+0000 2024-09-21T03:28:32.482 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/839399771 expires 2024-09-22T03:19:06.754886+0000 2024-09-21T03:28:32.483 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1177659103 expires 2024-09-22T03:19:06.754886+0000 2024-09-21T03:28:32.483 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3522404182 expires 2024-09-22T03:19:31.820776+0000 2024-09-21T03:28:32.483 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/914655252 expires 2024-09-22T03:19:31.820776+0000 2024-09-21T03:28:32.483 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/3643082307 expires 2024-09-22T03:20:32.227264+0000 2024-09-21T03:28:32.483 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1049357498 expires 2024-09-22T03:20:32.227264+0000 2024-09-21T03:28:32.483 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/2996077964 expires 2024-09-22T03:19:31.820776+0000 2024-09-21T03:28:32.483 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/597633710 expires 2024-09-22T03:19:31.820776+0000 2024-09-21T03:28:32.483 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/2169183307 expires 2024-09-22T03:20:32.227264+0000 2024-09-21T03:28:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 -- 172.21.15.112:0/3718002196 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f031c0b65f0 msgr2=0x7f031c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f031c0b65f0 0x7f031c0b8b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f032000bbe0 tx=0x7f032000b390 comp rx=0 tx=0).stop 2024-09-21T03:28:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 -- 172.21.15.112:0/3718002196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330108d20 msgr2=0x7f033019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330108d20 0x7f033019fd30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f032800b970 tx=0x7f032800be30 comp rx=0 tx=0).stop 2024-09-21T03:28:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 -- 172.21.15.112:0/3718002196 shutdown_connections 2024-09-21T03:28:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f031c0b65f0 0x7f031c0b8b00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330108d20 0x7f033019fd30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03301083d0 0x7f033019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 --2- 172.21.15.112:0/3718002196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330102a90 0x7f033019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:32.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 -- 172.21.15.112:0/3718002196 >> 172.21.15.112:0/3718002196 conn(0x7f0330079750 msgr2=0x7f033006bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:32.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 -- 172.21.15.112:0/3718002196 shutdown_connections 2024-09-21T03:28:32.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:32.480+0000 7f03366e4640 1 -- 172.21.15.112:0/3718002196 wait complete. 2024-09-21T03:28:32.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:32 smithi112 ceph-mon[31442]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:32.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:32 smithi203 ceph-mon[34723]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:32.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:32 smithi187 ceph-mon[34603]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T03:28:33.084 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T03:28:33.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:33.339 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:33.712 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:33 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3718002196' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:28:33.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.799+0000 7f80d2185640 1 -- 172.21.15.112:0/1938959749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc102a60 msgr2=0x7f80cc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.799+0000 7f80d2185640 1 --2- 172.21.15.112:0/1938959749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc102a60 0x7f80cc102e70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009900 tx=0x7f80bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.799+0000 7f80d2185640 1 -- 172.21.15.112:0/1938959749 shutdown_connections 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.799+0000 7f80d2185640 1 --2- 172.21.15.112:0/1938959749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc108cf0 0x7f80cc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.799+0000 7f80d2185640 1 --2- 172.21.15.112:0/1938959749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80cc1083a0 0x7f80cc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.799+0000 7f80d2185640 1 --2- 172.21.15.112:0/1938959749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc102a60 0x7f80cc102e70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.799+0000 7f80d2185640 1 -- 172.21.15.112:0/1938959749 >> 172.21.15.112:0/1938959749 conn(0x7f80cc079760 msgr2=0x7f80cc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.799+0000 7f80d2185640 1 -- 172.21.15.112:0/1938959749 shutdown_connections 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.799+0000 7f80d2185640 1 -- 172.21.15.112:0/1938959749 wait complete. 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80d2185640 1 Processor -- start 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80d2185640 1 -- start start 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80d2185640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc102a60 0x7f80cc19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80d2185640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80cc1083a0 0x7f80cc19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80d2185640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc108cf0 0x7f80cc19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80d2185640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc112a10 con 0x7f80cc102a60 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80d2185640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc112890 con 0x7f80cc1083a0 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80d2185640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc112b90 con 0x7f80cc108cf0 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc102a60 0x7f80cc19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc102a60 0x7f80cc19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59592/0 (socket says 172.21.15.112:59592) 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80cb7fe640 1 -- 172.21.15.112:0/3708073777 learned_addr learned my addr 172.21.15.112:0/3708073777 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.800+0000 7f80caffd640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80cc1083a0 0x7f80cc19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80cb7fe640 1 -- 172.21.15.112:0/3708073777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc108cf0 msgr2=0x7f80cc19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80cbfff640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc108cf0 0x7f80cc19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80cb7fe640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc108cf0 0x7f80cc19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80cb7fe640 1 -- 172.21.15.112:0/3708073777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80cc1083a0 msgr2=0x7f80cc19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80cb7fe640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80cc1083a0 0x7f80cc19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:33.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80cb7fe640 1 -- 172.21.15.112:0/3708073777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc1a04c0 con 0x7f80cc102a60 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80cbfff640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc108cf0 0x7f80cc19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80cb7fe640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc102a60 0x7f80cc19a550 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f80bc005940 tx=0x7f80bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80c8ff9640 1 -- 172.21.15.112:0/3708073777 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc03f030 con 0x7f80cc102a60 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80d2185640 1 -- 172.21.15.112:0/3708073777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc1a0750 con 0x7f80cc102a60 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80c8ff9640 1 -- 172.21.15.112:0/3708073777 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80bc003cb0 con 0x7f80cc102a60 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80c8ff9640 1 -- 172.21.15.112:0/3708073777 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc035070 con 0x7f80cc102a60 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.801+0000 7f80d2185640 1 -- 172.21.15.112:0/3708073777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc1a6f10 con 0x7f80cc102a60 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.802+0000 7f80d2185640 1 -- 172.21.15.112:0/3708073777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc103ce0 con 0x7f80cc102a60 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.804+0000 7f80c8ff9640 1 -- 172.21.15.112:0/3708073777 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f80bc042030 con 0x7f80cc102a60 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.804+0000 7f80c8ff9640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f80b40b65f0 0x7f80b40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.804+0000 7f80caffd640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f80b40b65f0 0x7f80b40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.804+0000 7f80c8ff9640 1 -- 172.21.15.112:0/3708073777 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f80bc1080d0 con 0x7f80cc102a60 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.805+0000 7f80caffd640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f80b40b65f0 0x7f80b40b8b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f80cc079bb0 tx=0x7f80c000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:33.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:33.807+0000 7f80c8ff9640 1 -- 172.21.15.112:0/3708073777 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f80bc0ce740 con 0x7f80cc102a60 2024-09-21T03:28:33.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:33 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3718002196' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:28:33.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:33 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3718002196' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T03:28:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.021+0000 7f80d2185640 1 -- 172.21.15.112:0/3708073777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80cc10bb30 con 0x7f80cc102a60 2024-09-21T03:28:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.022+0000 7f80c8ff9640 1 -- 172.21.15.112:0/3708073777 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11929 (secure 0 0 0) 0x7f80bc0d2b20 con 0x7f80cc102a60 2024-09-21T03:28:34.024 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:34.024 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"btime":"2024-09-21T03:27:13:388843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:27:13.104308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:28:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-09-21T03:28:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 -- 172.21.15.112:0/3708073777 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f80b40b65f0 msgr2=0x7f80b40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f80b40b65f0 0x7f80b40b8b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f80cc079bb0 tx=0x7f80c000b390 comp rx=0 tx=0).stop 2024-09-21T03:28:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 -- 172.21.15.112:0/3708073777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc102a60 msgr2=0x7f80cc19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc102a60 0x7f80cc19a550 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f80bc005940 tx=0x7f80bc0042f0 comp rx=0 tx=0).stop 2024-09-21T03:28:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 -- 172.21.15.112:0/3708073777 shutdown_connections 2024-09-21T03:28:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f80b40b65f0 0x7f80b40b8b00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:34.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc108cf0 0x7f80cc19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80cc1083a0 0x7f80cc19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 --2- 172.21.15.112:0/3708073777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80cc102a60 0x7f80cc19a550 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 -- 172.21.15.112:0/3708073777 >> 172.21.15.112:0/3708073777 conn(0x7f80cc079760 msgr2=0x7f80cc06bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 -- 172.21.15.112:0/3708073777 shutdown_connections 2024-09-21T03:28:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:34.025+0000 7f80d2185640 1 -- 172.21.15.112:0/3708073777 wait complete. 2024-09-21T03:28:34.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:34 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3708073777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:34.582 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:34 smithi112 ceph-mon[31442]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T03:28:34.609 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:34.858 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:34 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3708073777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:34 smithi203 ceph-mon[34723]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T03:28:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:34 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3708073777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:34 smithi187 ceph-mon[34603]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-21T03:28:35.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.298+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1200615269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c01083a0 msgr2=0x7f32c01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:35.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.298+0000 7f32c6c2d640 1 --2- 172.21.15.112:0/1200615269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c01083a0 0x7f32c01087b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009ca0 tx=0x7f32b002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:35.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.298+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1200615269 shutdown_connections 2024-09-21T03:28:35.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.298+0000 7f32c6c2d640 1 --2- 172.21.15.112:0/1200615269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32c0108cf0 0x7f32c010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:35.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.298+0000 7f32c6c2d640 1 --2- 172.21.15.112:0/1200615269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c01083a0 0x7f32c01087b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:35.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.298+0000 7f32c6c2d640 1 --2- 172.21.15.112:0/1200615269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32c0102a60 0x7f32c0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:35.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.298+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1200615269 >> 172.21.15.112:0/1200615269 conn(0x7f32c0079730 msgr2=0x7f32c0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:35.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.298+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1200615269 shutdown_connections 2024-09-21T03:28:35.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.298+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1200615269 wait complete. 2024-09-21T03:28:35.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32c6c2d640 1 Processor -- start 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32c6c2d640 1 -- start start 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32c6c2d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32c0102a60 0x7f32c019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32c6c2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c01083a0 0x7f32c019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32c6c2d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32c0108cf0 0x7f32c019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32c6c2d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01129a0 con 0x7f32c01083a0 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32c6c2d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0112820 con 0x7f32c0108cf0 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32c6c2d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0112b20 con 0x7f32c0102a60 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c01083a0 0x7f32c019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c01083a0 0x7f32c019b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52112/0 (socket says 172.21.15.112:52112) 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32bffff640 1 -- 172.21.15.112:0/1259600052 learned_addr learned my addr 172.21.15.112:0/1259600052 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32c51ca640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32c0108cf0 0x7f32c019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32c49c9640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32c0102a60 0x7f32c019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32bffff640 1 -- 172.21.15.112:0/1259600052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32c0102a60 msgr2=0x7f32c019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:35.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32bffff640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32c0102a60 0x7f32c019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:35.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32bffff640 1 -- 172.21.15.112:0/1259600052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32c0108cf0 msgr2=0x7f32c019bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:35.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32bffff640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32c0108cf0 0x7f32c019bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:35.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32bffff640 1 -- 172.21.15.112:0/1259600052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c01a6580 con 0x7f32c01083a0 2024-09-21T03:28:35.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.299+0000 7f32c49c9640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32c0102a60 0x7f32c019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:35.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.300+0000 7f32bffff640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c01083a0 0x7f32c019b830 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f32b002ade0 tx=0x7f32b0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:35.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.300+0000 7f32bdffb640 1 -- 172.21.15.112:0/1259600052 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0003d60 con 0x7f32c01083a0 2024-09-21T03:28:35.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.300+0000 7f32bdffb640 1 -- 172.21.15.112:0/1259600052 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b00044f0 con 0x7f32c01083a0 2024-09-21T03:28:35.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.300+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1259600052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c01a6810 con 0x7f32c01083a0 2024-09-21T03:28:35.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.300+0000 7f32bdffb640 1 -- 172.21.15.112:0/1259600052 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b002fce0 con 0x7f32c01083a0 2024-09-21T03:28:35.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.300+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1259600052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c01a6e20 con 0x7f32c01083a0 2024-09-21T03:28:35.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.301+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1259600052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c0102e70 con 0x7f32c01083a0 2024-09-21T03:28:35.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.302+0000 7f32bdffb640 1 -- 172.21.15.112:0/1259600052 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f32b00046a0 con 0x7f32c01083a0 2024-09-21T03:28:35.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.303+0000 7f32bdffb640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32ac0b6610 0x7f32ac0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:35.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.303+0000 7f32c49c9640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32ac0b6610 0x7f32ac0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:35.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.303+0000 7f32bdffb640 1 -- 172.21.15.112:0/1259600052 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f32b0108a80 con 0x7f32c01083a0 2024-09-21T03:28:35.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.304+0000 7f32c49c9640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32ac0b6610 0x7f32ac0b8b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f32a8006fd0 tx=0x7f32a8008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:35.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.307+0000 7f32bdffb640 1 -- 172.21.15.112:0/1259600052 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f32b00cf250 con 0x7f32c01083a0 2024-09-21T03:28:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.525+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1259600052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32c006b270 con 0x7f32c01083a0 2024-09-21T03:28:35.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.526+0000 7f32bdffb640 1 -- 172.21.15.112:0/1259600052 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11929 (secure 0 0 0) 0x7f32c006b270 con 0x7f32c01083a0 2024-09-21T03:28:35.528 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:35.528 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":24,"btime":"2024-09-21T03:27:13:388843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:27:13.104308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:28:35.528 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 24 2024-09-21T03:28:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1259600052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32ac0b6610 msgr2=0x7f32ac0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32ac0b6610 0x7f32ac0b8b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f32a8006fd0 tx=0x7f32a8008040 comp rx=0 tx=0).stop 2024-09-21T03:28:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1259600052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c01083a0 msgr2=0x7f32c019b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c01083a0 0x7f32c019b830 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f32b002ade0 tx=0x7f32b0003970 comp rx=0 tx=0).stop 2024-09-21T03:28:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1259600052 shutdown_connections 2024-09-21T03:28:35.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32ac0b6610 0x7f32ac0b8b20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:35.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32c0108cf0 0x7f32c019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:35.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32c01083a0 0x7f32c019b830 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:35.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 --2- 172.21.15.112:0/1259600052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32c0102a60 0x7f32c019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:35.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1259600052 >> 172.21.15.112:0/1259600052 conn(0x7f32c0079730 msgr2=0x7f32c01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:35.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1259600052 shutdown_connections 2024-09-21T03:28:35.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:35.528+0000 7f32c6c2d640 1 -- 172.21.15.112:0/1259600052 wait complete. 2024-09-21T03:28:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:35 smithi112 ceph-mon[31442]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T03:28:35.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:35 smithi203 ceph-mon[34723]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T03:28:35.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:35 smithi187 ceph-mon[34603]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T03:28:36.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:28:36.386 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:36.777 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:36 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1259600052' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:36.777 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:36 smithi112 ceph-mon[31442]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T03:28:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.864+0000 7f62e3fff640 1 -- 172.21.15.112:0/1472693310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e41047d0 msgr2=0x7f62e4104be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.864+0000 7f62e3fff640 1 --2- 172.21.15.112:0/1472693310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e41047d0 0x7f62e4104be0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009900 tx=0x7f62d802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.864+0000 7f62e3fff640 1 -- 172.21.15.112:0/1472693310 shutdown_connections 2024-09-21T03:28:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.864+0000 7f62e3fff640 1 --2- 172.21.15.112:0/1472693310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e410dc30 0x7f62e4110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.864+0000 7f62e3fff640 1 --2- 172.21.15.112:0/1472693310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62e4105120 0x7f62e410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.864+0000 7f62e3fff640 1 --2- 172.21.15.112:0/1472693310 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e41047d0 0x7f62e4104be0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.864+0000 7f62e3fff640 1 -- 172.21.15.112:0/1472693310 >> 172.21.15.112:0/1472693310 conn(0x7f62e40fe2d0 msgr2=0x7f62e4100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.864+0000 7f62e3fff640 1 -- 172.21.15.112:0/1472693310 shutdown_connections 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.864+0000 7f62e3fff640 1 -- 172.21.15.112:0/1472693310 wait complete. 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.865+0000 7f62e3fff640 1 Processor -- start 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.865+0000 7f62e3fff640 1 -- start start 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e41047d0 0x7f62e4102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e4105120 0x7f62e4102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62e410dc30 0x7f62e4102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e3fff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41138b0 con 0x7f62e41047d0 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e3fff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4113730 con 0x7f62e4105120 2024-09-21T03:28:36.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e3fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4113a30 con 0x7f62e410dc30 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e41047d0 0x7f62e4102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e41047d0 0x7f62e4102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52132/0 (socket says 172.21.15.112:52132) 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e2ffd640 1 -- 172.21.15.112:0/1423653064 learned_addr learned my addr 172.21.15.112:0/1423653064 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e37fe640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62e410dc30 0x7f62e4102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e27fc640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e4105120 0x7f62e4102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e2ffd640 1 -- 172.21.15.112:0/1423653064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62e410dc30 msgr2=0x7f62e4102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e2ffd640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62e410dc30 0x7f62e4102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e2ffd640 1 -- 172.21.15.112:0/1423653064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e4105120 msgr2=0x7f62e4102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e2ffd640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e4105120 0x7f62e4102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e2ffd640 1 -- 172.21.15.112:0/1423653064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e41a85d0 con 0x7f62e41047d0 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e37fe640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62e410dc30 0x7f62e4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e27fc640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e4105120 0x7f62e4102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e2ffd640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e41047d0 0x7f62e4102510 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f62d8005940 tx=0x7f62d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:36.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62cbfff640 1 -- 172.21.15.112:0/1423653064 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d803f030 con 0x7f62e41047d0 2024-09-21T03:28:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62cbfff640 1 -- 172.21.15.112:0/1423653064 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d8003cb0 con 0x7f62e41047d0 2024-09-21T03:28:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.866+0000 7f62e3fff640 1 -- 172.21.15.112:0/1423653064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e41a8860 con 0x7f62e41047d0 2024-09-21T03:28:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.867+0000 7f62cbfff640 1 -- 172.21.15.112:0/1423653064 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8035070 con 0x7f62e41047d0 2024-09-21T03:28:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.867+0000 7f62e3fff640 1 -- 172.21.15.112:0/1423653064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e41a8d40 con 0x7f62e41047d0 2024-09-21T03:28:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.868+0000 7f62e3fff640 1 -- 172.21.15.112:0/1423653064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e4104be0 con 0x7f62e41047d0 2024-09-21T03:28:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.869+0000 7f62cbfff640 1 -- 172.21.15.112:0/1423653064 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f62d8042030 con 0x7f62e41047d0 2024-09-21T03:28:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.870+0000 7f62cbfff640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f62cc0b6560 0x7f62cc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.870+0000 7f62e27fc640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f62cc0b6560 0x7f62cc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.870+0000 7f62cbfff640 1 -- 172.21.15.112:0/1423653064 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f62d8107fd0 con 0x7f62e41047d0 2024-09-21T03:28:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.870+0000 7f62e27fc640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f62cc0b6560 0x7f62cc0b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62e41101b0 tx=0x7f62d4005e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:36.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:36.873+0000 7f62cbfff640 1 -- 172.21.15.112:0/1423653064 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f62d80ce7a0 con 0x7f62e41047d0 2024-09-21T03:28:36.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:36 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1259600052' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:36.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:36 smithi203 ceph-mon[34723]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T03:28:36.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:36 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1259600052' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:36.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:36 smithi187 ceph-mon[34603]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T03:28:37.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.056+0000 7f62e3fff640 1 -- 172.21.15.112:0/1423653064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62e41a9140 con 0x7f62e41047d0 2024-09-21T03:28:37.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.057+0000 7f62cbfff640 1 -- 172.21.15.112:0/1423653064 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23065 (secure 0 0 0) 0x7f62d80d2b80 con 0x7f62e41047d0 2024-09-21T03:28:37.060 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:37.060 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":76,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:27:33.414969+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:26:58.087349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:26:59.970444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:28:37.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.059+0000 7f62e3fff640 1 -- 172.21.15.112:0/1423653064 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f62cc0b6560 msgr2=0x7f62cc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:37.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.059+0000 7f62e3fff640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f62cc0b6560 0x7f62cc0b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62e41101b0 tx=0x7f62d4005e50 comp rx=0 tx=0).stop 2024-09-21T03:28:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.060+0000 7f62e3fff640 1 -- 172.21.15.112:0/1423653064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e41047d0 msgr2=0x7f62e4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.060+0000 7f62e3fff640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e41047d0 0x7f62e4102510 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f62d8005940 tx=0x7f62d80042f0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.060+0000 7f62e3fff640 1 -- 172.21.15.112:0/1423653064 shutdown_connections 2024-09-21T03:28:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.060+0000 7f62e3fff640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f62cc0b6560 0x7f62cc0b8a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.060+0000 7f62e3fff640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62e410dc30 0x7f62e4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.060+0000 7f62e3fff640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e4105120 0x7f62e4102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.060+0000 7f62e3fff640 1 --2- 172.21.15.112:0/1423653064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62e41047d0 0x7f62e4102510 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:37.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.060+0000 7f62e3fff640 1 -- 172.21.15.112:0/1423653064 >> 172.21.15.112:0/1423653064 conn(0x7f62e40fe2d0 msgr2=0x7f62e4107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:37.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.060+0000 7f62e3fff640 1 -- 172.21.15.112:0/1423653064 shutdown_connections 2024-09-21T03:28:37.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:37.060+0000 7f62e3fff640 1 -- 172.21.15.112:0/1423653064 wait complete. 2024-09-21T03:28:37.722 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:37 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1423653064' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:37.751 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T03:27:01.886242+0000', 'modified': '2024-09-21T03:27:13.104308+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24685, 'mds_1': 24674, 'mds_2': 24667}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24626': {'gid': 24626, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6835/2552593896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 2552593896}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 2552593896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24647': {'gid': 24647, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6837/2217535204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6836', 'nonce': 2217535204}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 2217535204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24658': {'gid': 24658, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/3541929468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 3541929468}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 3541929468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24667': {'gid': 24667, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.203:6837/4062574800', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 4062574800}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 4062574800}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24674': {'gid': 24674, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.187:6839/910214120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6838', 'nonce': 910214120}, {'type': 'v1', 'addr': '172.21.15.187:6839', 'nonce': 910214120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24685': {'gid': 24685, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.203:6839/482815172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 482815172}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 482815172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24685, 'qdb_cluster': [24667, 24674, 24685]} 2024-09-21T03:28:37.751 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T03:28:37.751 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T03:28:37.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:37 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1423653064' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:37.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:37 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1423653064' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:37.997 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:38.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.448+0000 7f9de41b6640 1 -- 172.21.15.112:0/661865557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc108cf0 msgr2=0x7f9ddc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.448+0000 7f9de41b6640 1 --2- 172.21.15.112:0/661865557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc108cf0 0x7f9ddc10f520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009ca0 tx=0x7f9dd402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.448+0000 7f9de41b6640 1 -- 172.21.15.112:0/661865557 shutdown_connections 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.448+0000 7f9de41b6640 1 --2- 172.21.15.112:0/661865557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc108cf0 0x7f9ddc10f520 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.448+0000 7f9de41b6640 1 --2- 172.21.15.112:0/661865557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.448+0000 7f9de41b6640 1 --2- 172.21.15.112:0/661865557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ddc102a60 0x7f9ddc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.448+0000 7f9de41b6640 1 -- 172.21.15.112:0/661865557 >> 172.21.15.112:0/661865557 conn(0x7f9ddc079730 msgr2=0x7f9ddc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.448+0000 7f9de41b6640 1 -- 172.21.15.112:0/661865557 shutdown_connections 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.448+0000 7f9de41b6640 1 -- 172.21.15.112:0/661865557 wait complete. 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de41b6640 1 Processor -- start 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de41b6640 1 -- start start 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de41b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a60 0x7f9ddc06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de41b6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de41b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ddc108cf0 0x7f9ddc0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de41b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc112960 con 0x7f9ddc102a60 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de41b6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc1127e0 con 0x7f9ddc108cf0 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de41b6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc112ae0 con 0x7f9ddc1083a0 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de1f52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a60 0x7f9ddc06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de1f52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a60 0x7f9ddc06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52146/0 (socket says 172.21.15.112:52146) 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de1f52640 1 -- 172.21.15.112:0/4156603825 learned_addr learned my addr 172.21.15.112:0/4156603825 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.449+0000 7f9de2753640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ddc108cf0 0x7f9ddc0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9de1751640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9de1f52640 1 -- 172.21.15.112:0/4156603825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ddc1083a0 msgr2=0x7f9ddc0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9de1f52640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9de1f52640 1 -- 172.21.15.112:0/4156603825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ddc108cf0 msgr2=0x7f9ddc0707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9de1f52640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ddc108cf0 0x7f9ddc0707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9de1f52640 1 -- 172.21.15.112:0/4156603825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc1aee70 con 0x7f9ddc102a60 2024-09-21T03:28:38.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9de1751640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9de1f52640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a60 0x7f9ddc06b9a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800d770 tx=0x7f9dd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9dd2ffd640 1 -- 172.21.15.112:0/4156603825 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8017070 con 0x7f9ddc102a60 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9dd2ffd640 1 -- 172.21.15.112:0/4156603825 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd80054d0 con 0x7f9ddc102a60 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9de41b6640 1 -- 172.21.15.112:0/4156603825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc1af040 con 0x7f9ddc102a60 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9dd2ffd640 1 -- 172.21.15.112:0/4156603825 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8004c30 con 0x7f9ddc102a60 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.450+0000 7f9de41b6640 1 -- 172.21.15.112:0/4156603825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc1af4a0 con 0x7f9ddc102a60 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9dd2ffd640 1 -- 172.21.15.112:0/4156603825 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f9dd8004e10 con 0x7f9ddc102a60 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9dea72a740 1 Processor -- start 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9dea72a740 1 -- start start 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d8325f60 0x55f9d86cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d82fd0a0 0x55f9d86cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9dea72a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8643c10 con 0x55f9d8305280 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9dea72a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8633040 con 0x55f9d8325f60 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9dea72a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f9d8636e50 con 0x55f9d82fd0a0 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9de1751640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9de1751640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52162/0 (socket says 172.21.15.112:52162) 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9de1751640 1 -- 172.21.15.112:0/2896121404 learned_addr learned my addr 172.21.15.112:0/2896121404 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9de2753640 1 --2- 172.21.15.112:0/2896121404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d8325f60 0x55f9d86cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.453+0000 7f9dd2ffd640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9dc80b6560 0x7f9dc80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9de1f52640 1 --2- 172.21.15.112:0/2896121404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d82fd0a0 0x55f9d86cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9de2753640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9dc80b6560 0x7f9dc80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9de1751640 1 -- 172.21.15.112:0/2896121404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d82fd0a0 msgr2=0x55f9d86cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9de1751640 1 --2- 172.21.15.112:0/2896121404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d82fd0a0 0x55f9d86cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9de1751640 1 -- 172.21.15.112:0/2896121404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d8325f60 msgr2=0x55f9d86cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9de1751640 1 --2- 172.21.15.112:0/2896121404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d8325f60 0x55f9d86cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9de1751640 1 -- 172.21.15.112:0/2896121404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d8391540 con 0x55f9d8305280 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9dd2ffd640 1 -- 172.21.15.112:0/4156603825 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f9dd80e6f40 con 0x7f9ddc102a60 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9de1751640 1 --2- 172.21.15.112:0/2896121404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86cc980 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009bf0 tx=0x7f9dcc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e85f812c0be777 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9dc7fff640 1 -- 172.21.15.112:0/2896121404 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc016070 con 0x55f9d8305280 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9de2753640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9dc80b6560 0x7f9dc80b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402f500 tx=0x7f9dd402a8c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9dc7fff640 1 -- 172.21.15.112:0/2896121404 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dcc0054d0 con 0x55f9d8305280 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.454+0000 7f9dc7fff640 1 -- 172.21.15.112:0/2896121404 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc011470 con 0x55f9d8305280 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.455+0000 7f9dea72a740 1 -- 172.21.15.112:0/2896121404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 msgr2=0x55f9d86cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.455+0000 7f9dea72a740 1 --2- 172.21.15.112:0/2896121404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86cc980 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009bf0 tx=0x7f9dcc00e6c0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.455+0000 7f9dea72a740 1 -- 172.21.15.112:0/2896121404 shutdown_connections 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.455+0000 7f9dea72a740 1 --2- 172.21.15.112:0/2896121404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86cc980 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.455+0000 7f9dea72a740 1 --2- 172.21.15.112:0/2896121404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d82fd0a0 0x55f9d86cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.455+0000 7f9dea72a740 1 --2- 172.21.15.112:0/2896121404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d8325f60 0x55f9d86cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.455+0000 7f9dea72a740 1 -- 172.21.15.112:0/2896121404 >> 172.21.15.112:0/2896121404 conn(0x55f9d831a350 msgr2=0x55f9d86c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.455+0000 7f9dea72a740 1 -- 172.21.15.112:0/2896121404 shutdown_connections 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.455+0000 7f9dea72a740 1 -- 172.21.15.112:0/2896121404 wait complete. 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9dea72a740 1 -- start start 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9dea72a740 1 Processor -- start 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86d7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d8305280 0x55f9d86d7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d8325f60 0x55f9d86e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9dea72a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9d84945f0 con 0x55f9d82fd0a0 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9dea72a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f9d85ce010 con 0x55f9d8325f60 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9dea72a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f9d86da6e0 con 0x55f9d8305280 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9dea72a740 10 client.0 _ll_register_callbacks cb 0x55f9d862d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9de1f52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86d7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9de1f52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86d7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52174/0 (socket says 172.21.15.112:52174) 2024-09-21T03:28:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9de1f52640 1 -- 172.21.15.112:0/2210402291 learned_addr learned my addr 172.21.15.112:0/2210402291 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.456+0000 7f9de1751640 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d8305280 0x55f9d86d7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9de1f52640 1 -- 172.21.15.112:0/2210402291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d8305280 msgr2=0x55f9d86d7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9de1f52640 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d8305280 0x55f9d86d7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9de1f52640 1 -- 172.21.15.112:0/2210402291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d8325f60 msgr2=0x55f9d86e39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9de1f52640 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d8325f60 0x55f9d86e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9de1f52640 1 -- 172.21.15.112:0/2210402291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d85fba00 con 0x55f9d82fd0a0 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9de1f52640 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86d7730 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80aa280 tx=0x7f9dd8079cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270e6085258ca0ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9dc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9dc57fa640 1 -- 172.21.15.112:0/2210402291 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8020050 con 0x55f9d82fd0a0 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9dc57fa640 1 -- 172.21.15.112:0/2210402291 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f9dd80ad270 con 0x55f9d82fd0a0 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9dea72a740 1 -- 172.21.15.112:0/2210402291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dcc0054d0 con 0x55f9d82fd0a0 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9dc57fa640 1 -- 172.21.15.112:0/2210402291 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9dd80ad710 con 0x55f9d82fd0a0 2024-09-21T03:28:38.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9dea72a740 10 client.15321 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9dea72a740 1 -- 172.21.15.112:0/2210402291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f9d86e65d0 con 0x55f9d82fd0a0 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9dc57fa640 1 -- 172.21.15.112:0/2210402291 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dcc0054d0 con 0x55f9d82fd0a0 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9dc57fa640 1 -- 172.21.15.112:0/2210402291 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9dd80b65a0 con 0x55f9d82fd0a0 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9dea72a740 10 client.15321 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.457+0000 7f9dea72a740 10 client.15321 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.458+0000 7f9dea72a740 20 client.15321 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.458+0000 7f9dea72a740 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f9d82e48e0 0x55f9d86ea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.458+0000 7f9dea72a740 4 client.15321 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.458+0000 7f9dea72a740 1 -- 172.21.15.112:0/2210402291 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9d8645b60 con 0x55f9d82e48e0 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.458+0000 7f9de1751640 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f9d82e48e0 0x55f9d86ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.458+0000 7f9dc57fa640 10 client.15321 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.458+0000 7f9de1751640 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f9d82e48e0 0x55f9d86ea010 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7cd56588c8a63ce5 server_cookie=84ca920c8b4cf0de in_seq=0 out_seq=0 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dc57fa640 1 -- 172.21.15.112:0/2210402291 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9d8645b60 con 0x55f9d82e48e0 2024-09-21T03:28:38.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dc57fa640 10 client.15321 handle_command_reply: tid=0 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dea72a740 1 client.15321 shutdown 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dea72a740 1 -- 172.21.15.112:0/2210402291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 msgr2=0x55f9d86d7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dea72a740 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86d7730 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80aa280 tx=0x7f9dd8079cc0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dea72a740 1 -- 172.21.15.112:0/2210402291 shutdown_connections 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dea72a740 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f9d82e48e0 0x55f9d86ea010 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dea72a740 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d8325f60 0x55f9d86e39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dea72a740 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d8305280 0x55f9d86d7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dc57fa640 0 client.15321 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dea72a740 1 --2- 172.21.15.112:0/2210402291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86d7730 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.459+0000 7f9dea72a740 1 -- 172.21.15.112:0/2210402291 >> 172.21.15.112:0/2210402291 conn(0x55f9d831a350 msgr2=0x55f9d86cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.460+0000 7f9dea72a740 1 -- 172.21.15.112:0/2210402291 shutdown_connections 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.460+0000 7f9dea72a740 1 -- 172.21.15.112:0/2210402291 wait complete. 2024-09-21T03:28:38.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.460+0000 7f9dea72a740 20 client.15321 trim_cache size 0 max 16384 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9dea72a740 1 Processor -- start 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9dea72a740 1 -- start start 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d82e48e0 0x55f9d86d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d82fd0a0 0x55f9d86e3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9dea72a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9d86e8350 con 0x55f9d8305280 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9dea72a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f9d86e8a20 con 0x55f9d82fd0a0 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9dea72a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f9d86e90f0 con 0x55f9d82e48e0 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de2753640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de2753640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86e3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52178/0 (socket says 172.21.15.112:52178) 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de2753640 1 -- 172.21.15.112:0/4160996021 learned_addr learned my addr 172.21.15.112:0/4160996021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de1f52640 1 --2- 172.21.15.112:0/4160996021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d82e48e0 0x55f9d86d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de1751640 1 --2- 172.21.15.112:0/4160996021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d82fd0a0 0x55f9d86e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de2753640 1 -- 172.21.15.112:0/4160996021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d82e48e0 msgr2=0x55f9d86d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de2753640 1 --2- 172.21.15.112:0/4160996021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d82e48e0 0x55f9d86d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de2753640 1 -- 172.21.15.112:0/4160996021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d82fd0a0 msgr2=0x55f9d86e3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de2753640 1 --2- 172.21.15.112:0/4160996021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d82fd0a0 0x55f9d86e3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de2753640 1 -- 172.21.15.112:0/4160996021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d86ce790 con 0x55f9d8305280 2024-09-21T03:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de1751640 1 --2- 172.21.15.112:0/4160996021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d82fd0a0 0x55f9d86e3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.479+0000 7f9de1f52640 1 --2- 172.21.15.112:0/4160996021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d82e48e0 0x55f9d86d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.480+0000 7f9de2753640 1 --2- 172.21.15.112:0/4160996021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86e3e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4035000 tx=0x7f9dd402f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e240387e437c88a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.480+0000 7f9dc77fe640 1 -- 172.21.15.112:0/4160996021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4051030 con 0x55f9d8305280 2024-09-21T03:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.480+0000 7f9dc77fe640 1 -- 172.21.15.112:0/4160996021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd4036050 con 0x55f9d8305280 2024-09-21T03:28:38.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.480+0000 7f9dc77fe640 1 -- 172.21.15.112:0/4160996021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd40045f0 con 0x55f9d8305280 2024-09-21T03:28:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.481+0000 7f9dea72a740 1 -- 172.21.15.112:0/4160996021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 msgr2=0x55f9d86e3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.481+0000 7f9dea72a740 1 --2- 172.21.15.112:0/4160996021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86e3e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4035000 tx=0x7f9dd402f310 comp rx=0 tx=0).stop 2024-09-21T03:28:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.481+0000 7f9dea72a740 1 -- 172.21.15.112:0/4160996021 shutdown_connections 2024-09-21T03:28:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.481+0000 7f9dea72a740 1 --2- 172.21.15.112:0/4160996021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d8305280 0x55f9d86e3e80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.481+0000 7f9dea72a740 1 --2- 172.21.15.112:0/4160996021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d82fd0a0 0x55f9d86e3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.481+0000 7f9dea72a740 1 --2- 172.21.15.112:0/4160996021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d82e48e0 0x55f9d86d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.481+0000 7f9dea72a740 1 -- 172.21.15.112:0/4160996021 >> 172.21.15.112:0/4160996021 conn(0x55f9d831a350 msgr2=0x55f9d86e7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.481+0000 7f9dea72a740 1 -- 172.21.15.112:0/4160996021 shutdown_connections 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.481+0000 7f9dea72a740 1 -- 172.21.15.112:0/4160996021 wait complete. 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.481+0000 7f9dea72a740 1 -- start start 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9dea72a740 1 Processor -- start 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d82e48e0 0x55f9d86f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9dea72a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d8305280 0x55f9d86ec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9dea72a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9d86e9e90 con 0x55f9d82fd0a0 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9dea72a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f9d86e97c0 con 0x55f9d82e48e0 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9dea72a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f9d86ea560 con 0x55f9d8305280 2024-09-21T03:28:38.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9dea72a740 10 client.0 _ll_register_callbacks cb 0x7f9dd80a8eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9de1751640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9de1751640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86ebed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52180/0 (socket says 172.21.15.112:52180) 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9de1751640 1 -- 172.21.15.112:0/3888489429 learned_addr learned my addr 172.21.15.112:0/3888489429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9de1f52640 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d82e48e0 0x55f9d86f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9de2753640 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d8305280 0x55f9d86ec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9de1751640 1 -- 172.21.15.112:0/3888489429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d8305280 msgr2=0x55f9d86ec410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9de1751640 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d8305280 0x55f9d86ec410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9de1751640 1 -- 172.21.15.112:0/3888489429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d82e48e0 msgr2=0x55f9d86f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9de1751640 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d82e48e0 0x55f9d86f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.482+0000 7f9de1751640 1 -- 172.21.15.112:0/3888489429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d86ecb20 con 0x55f9d82fd0a0 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.483+0000 7f9de1751640 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86ebed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009b10 tx=0x7f9dcc0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdca7d8a7ac9ffff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.483+0000 7f9dc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:38.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.483+0000 7f9dc57fa640 1 -- 172.21.15.112:0/3888489429 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc016040 con 0x55f9d82fd0a0 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.483+0000 7f9dc57fa640 1 -- 172.21.15.112:0/3888489429 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f9dcc004020 con 0x55f9d82fd0a0 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.483+0000 7f9dea72a740 1 -- 172.21.15.112:0/3888489429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd4036050 con 0x55f9d82fd0a0 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.483+0000 7f9dc57fa640 1 -- 172.21.15.112:0/3888489429 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9dcc004220 con 0x55f9d82fd0a0 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.483+0000 7f9dea72a740 10 client.15327 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.483+0000 7f9dea72a740 1 -- 172.21.15.112:0/3888489429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f9d87554e0 con 0x55f9d82fd0a0 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.483+0000 7f9dc57fa640 1 -- 172.21.15.112:0/3888489429 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd4036050 con 0x55f9d82fd0a0 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.483+0000 7f9dc57fa640 1 -- 172.21.15.112:0/3888489429 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9dcc00f040 con 0x55f9d82fd0a0 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.484+0000 7f9dea72a740 10 client.15327 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.484+0000 7f9dea72a740 10 client.15327 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.484+0000 7f9dea72a740 20 client.15327 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.484+0000 7f9dea72a740 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f9d8325f60 0x55f9d8758ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.484+0000 7f9dea72a740 4 client.15327 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:28:38.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.484+0000 7f9dea72a740 1 -- 172.21.15.112:0/3888489429 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "status"}) -- 0x55f9d8641610 con 0x55f9d8325f60 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.484+0000 7f9de1f52640 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f9d8325f60 0x55f9d8758ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.484+0000 7f9dc57fa640 10 client.15327 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.484+0000 7f9de1f52640 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f9d8325f60 0x55f9d8758ac0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=351ec13a17c5428d server_cookie=779f0a07761c7f9d in_seq=0 out_seq=0 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.484+0000 7f9dc57fa640 1 -- 172.21.15.112:0/3888489429 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f9d8641610 con 0x55f9d8325f60 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.484+0000 7f9dc57fa640 10 client.15327 handle_command_reply: tid=0 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 1 client.15327 shutdown 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 1 -- 172.21.15.112:0/3888489429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 msgr2=0x55f9d86ebed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86ebed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009b10 tx=0x7f9dcc0058e0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 1 -- 172.21.15.112:0/3888489429 shutdown_connections 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f9d8325f60 0x55f9d8758ac0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9d8305280 0x55f9d86ec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9d82fd0a0 0x55f9d86ebed0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dc57fa640 0 client.15327 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:28:38.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 1 --2- 172.21.15.112:0/3888489429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9d82e48e0 0x55f9d86f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 1 -- 172.21.15.112:0/3888489429 >> 172.21.15.112:0/3888489429 conn(0x55f9d831a350 msgr2=0x55f9d86e7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:38.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 1 -- 172.21.15.112:0/3888489429 shutdown_connections 2024-09-21T03:28:38.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 1 -- 172.21.15.112:0/3888489429 wait complete. 2024-09-21T03:28:38.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9dea72a740 20 client.15327 trim_cache size 0 max 16384 2024-09-21T03:28:38.501 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:28:38.501 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "063b32f6-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:28:38.501 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-09-21T03:28:38.501 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24667, 2024-09-21T03:28:38.501 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T03:28:38.502 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T03:28:38.502 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T03:28:38.502 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 88.106685236000004, 2024-09-21T03:28:38.502 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-09-21T03:28:38.502 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 76, 2024-09-21T03:28:38.502 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T03:28:38.502 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 119.46521414199999 2024-09-21T03:28:38.502 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:28:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.485+0000 7f9de41b6640 1 -- 172.21.15.112:0/4156603825 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9dc80b6560 msgr2=0x7f9dc80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.486+0000 7f9de41b6640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9dc80b6560 0x7f9dc80b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402f500 tx=0x7f9dd402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.486+0000 7f9de41b6640 1 -- 172.21.15.112:0/4156603825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a60 msgr2=0x7f9ddc06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.486+0000 7f9de41b6640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a60 0x7f9ddc06b9a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800d770 tx=0x7f9dd800dc30 comp rx=0 tx=0).stop 2024-09-21T03:28:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.486+0000 7f9de41b6640 1 -- 172.21.15.112:0/4156603825 shutdown_connections 2024-09-21T03:28:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.486+0000 7f9de41b6640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9dc80b6560 0x7f9dc80b8a70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.486+0000 7f9de41b6640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ddc108cf0 0x7f9ddc0707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.486+0000 7f9de41b6640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ddc1083a0 0x7f9ddc0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.486+0000 7f9de41b6640 1 --2- 172.21.15.112:0/4156603825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ddc102a60 0x7f9ddc06b9a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:38.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.486+0000 7f9de41b6640 1 -- 172.21.15.112:0/4156603825 >> 172.21.15.112:0/4156603825 conn(0x7f9ddc079730 msgr2=0x7f9ddc1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:38.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.486+0000 7f9de41b6640 1 -- 172.21.15.112:0/4156603825 shutdown_connections 2024-09-21T03:28:38.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:38.486+0000 7f9de41b6640 1 -- 172.21.15.112:0/4156603825 wait complete. 2024-09-21T03:28:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:38 smithi112 ceph-mon[31442]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T03:28:38.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:38 smithi203 ceph-mon[34723]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T03:28:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:38 smithi187 ceph-mon[34603]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T03:28:39.099 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T03:28:39.343 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:39.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.787+0000 7f771e180640 1 -- 172.21.15.112:0/2790204446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7718102a60 msgr2=0x7f7718102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.787+0000 7f771e180640 1 --2- 172.21.15.112:0/2790204446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7718102a60 0x7f7718102e70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7708009900 tx=0x7f770802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:39.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.787+0000 7f771e180640 1 -- 172.21.15.112:0/2790204446 shutdown_connections 2024-09-21T03:28:39.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.787+0000 7f771e180640 1 --2- 172.21.15.112:0/2790204446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7718108cf0 0x7f771810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.787+0000 7f771e180640 1 --2- 172.21.15.112:0/2790204446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77181083a0 0x7f77181087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.787+0000 7f771e180640 1 --2- 172.21.15.112:0/2790204446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7718102a60 0x7f7718102e70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.787+0000 7f771e180640 1 -- 172.21.15.112:0/2790204446 >> 172.21.15.112:0/2790204446 conn(0x7f7718079730 msgr2=0x7f7718079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.787+0000 7f771e180640 1 -- 172.21.15.112:0/2790204446 shutdown_connections 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.787+0000 7f771e180640 1 -- 172.21.15.112:0/2790204446 wait complete. 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.788+0000 7f771e180640 1 Processor -- start 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.788+0000 7f771e180640 1 -- start start 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.788+0000 7f771e180640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7718102a60 0x7f77181993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.788+0000 7f771e180640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77181083a0 0x7f7718199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f771e180640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7718108cf0 0x7f771819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f771e180640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7718112a10 con 0x7f7718108cf0 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f771e180640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7718112890 con 0x7f7718102a60 2024-09-21T03:28:39.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f771e180640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7718112b90 con 0x7f77181083a0 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7717fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7718108cf0 0x7f771819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7717fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7718108cf0 0x7f771819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52198/0 (socket says 172.21.15.112:52198) 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7717fff640 1 -- 172.21.15.112:0/4029135150 learned_addr learned my addr 172.21.15.112:0/4029135150 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f77177fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7718102a60 0x7f77181993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7716ffd640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77181083a0 0x7f7718199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7717fff640 1 -- 172.21.15.112:0/4029135150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77181083a0 msgr2=0x7f7718199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7717fff640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77181083a0 0x7f7718199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7717fff640 1 -- 172.21.15.112:0/4029135150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7718102a60 msgr2=0x7f77181993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7717fff640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7718102a60 0x7f77181993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7717fff640 1 -- 172.21.15.112:0/4029135150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771819f310 con 0x7f7718108cf0 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7717fff640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7718108cf0 0x7f771819ebe0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f771000b970 tx=0x7f771000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7714ff9640 1 -- 172.21.15.112:0/4029135150 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710019070 con 0x7f7718108cf0 2024-09-21T03:28:39.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7714ff9640 1 -- 172.21.15.112:0/4029135150 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7710014440 con 0x7f7718108cf0 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f771e180640 1 -- 172.21.15.112:0/4029135150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771819f600 con 0x7f7718108cf0 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f7714ff9640 1 -- 172.21.15.112:0/4029135150 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77100075d0 con 0x7f7718108cf0 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.789+0000 7f771e180640 1 -- 172.21.15.112:0/4029135150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77181a5d60 con 0x7f7718108cf0 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.792+0000 7f77246f4740 1 Processor -- start 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.792+0000 7f7714ff9640 1 -- 172.21.15.112:0/4029135150 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f771001c070 con 0x7f7718108cf0 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.792+0000 7f77246f4740 1 -- start start 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.792+0000 7f77246f4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a6092f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.792+0000 7f77246f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5cc40a0 0x5593a6093440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.792+0000 7f77246f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5ccc280 0x5593a6093980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.792+0000 7f77246f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5593a600ac10 con 0x5593a5cc40a0 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.792+0000 7f77246f4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5593a5ffa040 con 0x5593a5cecf60 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.792+0000 7f77246f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5593a5ffde50 con 0x5593a5ccc280 2024-09-21T03:28:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f77177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5cc40a0 0x5593a6093440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f77177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5cc40a0 0x5593a6093440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52202/0 (socket says 172.21.15.112:52202) 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f77177fe640 1 -- 172.21.15.112:0/442538553 learned_addr learned my addr 172.21.15.112:0/442538553 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f7716ffd640 1 --2- 172.21.15.112:0/442538553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5ccc280 0x5593a6093980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f7716ffd640 1 --2- 172.21.15.112:0/442538553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a6092f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f77177fe640 1 -- 172.21.15.112:0/442538553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5ccc280 msgr2=0x5593a6093980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f77177fe640 1 --2- 172.21.15.112:0/442538553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5ccc280 0x5593a6093980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f77177fe640 1 -- 172.21.15.112:0/442538553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 msgr2=0x5593a6092f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f77177fe640 1 --2- 172.21.15.112:0/442538553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a6092f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f77177fe640 1 -- 172.21.15.112:0/442538553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593a5d58540 con 0x5593a5cc40a0 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f7716ffd640 1 --2- 172.21.15.112:0/442538553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a6092f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f7714ff9640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77000b6560 0x7f77000b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f77177fe640 1 --2- 172.21.15.112:0/442538553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5cc40a0 0x5593a6093440 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7708009900 tx=0x7f7708004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365e73dd0c984ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f7716ffd640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77000b6560 0x7f77000b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f7714ff9640 1 -- 172.21.15.112:0/4029135150 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f77100e68c0 con 0x7f7718108cf0 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f76fdffb640 1 -- 172.21.15.112:0/442538553 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708035040 con 0x5593a5cc40a0 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f76fdffb640 1 -- 172.21.15.112:0/442538553 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7708004770 con 0x5593a5cc40a0 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.793+0000 7f76fdffb640 1 -- 172.21.15.112:0/442538553 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708003850 con 0x5593a5cc40a0 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.794+0000 7f77246f4740 1 -- 172.21.15.112:0/442538553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5cc40a0 msgr2=0x5593a6093440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.794+0000 7f7716ffd640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77000b6560 0x7f77000b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f770c005d00 tx=0x7f770c014040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.794+0000 7f77246f4740 1 --2- 172.21.15.112:0/442538553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5cc40a0 0x5593a6093440 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7708009900 tx=0x7f7708004390 comp rx=0 tx=0).stop 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.794+0000 7f77246f4740 1 -- 172.21.15.112:0/442538553 shutdown_connections 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.794+0000 7f77246f4740 1 --2- 172.21.15.112:0/442538553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5ccc280 0x5593a6093980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.794+0000 7f77246f4740 1 --2- 172.21.15.112:0/442538553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5cc40a0 0x5593a6093440 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.794+0000 7f77246f4740 1 --2- 172.21.15.112:0/442538553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a6092f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.794+0000 7f77246f4740 1 -- 172.21.15.112:0/442538553 >> 172.21.15.112:0/442538553 conn(0x5593a5ce1350 msgr2=0x5593a6090df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.794+0000 7f77246f4740 1 -- 172.21.15.112:0/442538553 shutdown_connections 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.794+0000 7f77246f4740 1 -- 172.21.15.112:0/442538553 wait complete. 2024-09-21T03:28:39.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f77246f4740 1 -- start start 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f77246f4740 1 Processor -- start 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f77246f4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cc40a0 0x5593a60a0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f77246f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f77246f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cecf60 0x5593a60acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f77246f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5593a5e5b5f0 con 0x5593a5ccc280 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f77246f4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5593a5f95010 con 0x5593a5cc40a0 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f77246f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5593a60a37e0 con 0x5593a5cecf60 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f77246f4740 10 client.0 _ll_register_callbacks cb 0x5593a5ff4340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f7716ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f7716ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52206/0 (socket says 172.21.15.112:52206) 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f7716ffd640 1 -- 172.21.15.112:0/740483035 learned_addr learned my addr 172.21.15.112:0/740483035 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f7717fff640 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cecf60 0x5593a60acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.795+0000 7f77177fe640 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cc40a0 0x5593a60a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f7716ffd640 1 -- 172.21.15.112:0/740483035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cecf60 msgr2=0x5593a60acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f7716ffd640 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cecf60 0x5593a60acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f7716ffd640 1 -- 172.21.15.112:0/740483035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cc40a0 msgr2=0x5593a60a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f7716ffd640 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cc40a0 0x5593a60a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f7716ffd640 1 -- 172.21.15.112:0/740483035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593a5fc2a00 con 0x5593a5ccc280 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f7716ffd640 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60a0f50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f770c0105e0 tx=0x7f770c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0a5edbb38d72d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f76f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f76f6ffd640 1 -- 172.21.15.112:0/740483035 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c004030 con 0x5593a5ccc280 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f77246f4740 1 -- 172.21.15.112:0/740483035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7708004770 con 0x5593a5ccc280 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f76f6ffd640 1 -- 172.21.15.112:0/740483035 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f770c0041d0 con 0x5593a5ccc280 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f76f6ffd640 1 -- 172.21.15.112:0/740483035 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f770c0043d0 con 0x5593a5ccc280 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f77246f4740 10 client.15339 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f77246f4740 1 -- 172.21.15.112:0/740483035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5593a60af740 con 0x5593a5ccc280 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f76f6ffd640 1 -- 172.21.15.112:0/740483035 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7708004770 con 0x5593a5ccc280 2024-09-21T03:28:39.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.796+0000 7f76f6ffd640 1 -- 172.21.15.112:0/740483035 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f770c0039e0 con 0x5593a5ccc280 2024-09-21T03:28:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.797+0000 7f77246f4740 10 client.15339 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.797+0000 7f77246f4740 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:28:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.797+0000 7f77246f4740 20 client.15339 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.797+0000 7f77246f4740 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5593a5cab8e0 0x5593a60b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.797+0000 7f77246f4740 4 client.15339 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.797+0000 7f77246f4740 1 -- 172.21.15.112:0/740483035 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593a600cb60 con 0x5593a5cab8e0 2024-09-21T03:28:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.797+0000 7f77177fe640 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5593a5cab8e0 0x5593a60b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.797+0000 7f76f6ffd640 10 client.15339 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:28:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.797+0000 7f77177fe640 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5593a5cab8e0 0x5593a60b31d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b391241ead72d1b2 server_cookie=c44ea5137d380bfd in_seq=0 out_seq=0 2024-09-21T03:28:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f76f6ffd640 1 -- 172.21.15.112:0/740483035 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5593a600cb60 con 0x5593a5cab8e0 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f76f6ffd640 10 client.15339 handle_command_reply: tid=0 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f77246f4740 1 client.15339 shutdown 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f77246f4740 1 -- 172.21.15.112:0/740483035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 msgr2=0x5593a60a0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f77246f4740 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60a0f50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f770c0105e0 tx=0x7f770c00b390 comp rx=0 tx=0).stop 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f77246f4740 1 -- 172.21.15.112:0/740483035 shutdown_connections 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f77246f4740 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5593a5cab8e0 0x5593a60b31d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f77246f4740 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cecf60 0x5593a60acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f77246f4740 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60a0f50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f76f6ffd640 0 client.15339 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f77246f4740 1 --2- 172.21.15.112:0/740483035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cc40a0 0x5593a60a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f77246f4740 1 -- 172.21.15.112:0/740483035 >> 172.21.15.112:0/740483035 conn(0x5593a5ce1350 msgr2=0x5593a60948f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f77246f4740 1 -- 172.21.15.112:0/740483035 shutdown_connections 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.798+0000 7f77246f4740 1 -- 172.21.15.112:0/740483035 wait complete. 2024-09-21T03:28:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.799+0000 7f77246f4740 20 client.15339 trim_cache size 0 max 16384 2024-09-21T03:28:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.816+0000 7f77246f4740 1 Processor -- start 2024-09-21T03:28:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f77246f4740 1 -- start start 2024-09-21T03:28:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f77246f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cc40a0 0x5593a608fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f77246f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a608bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f77246f4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a60acae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f77246f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5593a60b1510 con 0x5593a5ccc280 2024-09-21T03:28:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f77246f4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5593a60b1be0 con 0x5593a5cecf60 2024-09-21T03:28:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f77246f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5593a60b22b0 con 0x5593a5cc40a0 2024-09-21T03:28:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f7716ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a608bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f7716ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a608bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52216/0 (socket says 172.21.15.112:52216) 2024-09-21T03:28:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f7716ffd640 1 -- 172.21.15.112:0/3045694196 learned_addr learned my addr 172.21.15.112:0/3045694196 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f77177fe640 1 --2- 172.21.15.112:0/3045694196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cc40a0 0x5593a608fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f7716ffd640 1 -- 172.21.15.112:0/3045694196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cc40a0 msgr2=0x5593a608fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f7716ffd640 1 --2- 172.21.15.112:0/3045694196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cc40a0 0x5593a608fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f7716ffd640 1 -- 172.21.15.112:0/3045694196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 msgr2=0x5593a60acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f7716ffd640 1 --2- 172.21.15.112:0/3045694196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a60acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f7716ffd640 1 -- 172.21.15.112:0/3045694196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593a6098fe0 con 0x5593a5ccc280 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.817+0000 7f7716ffd640 1 --2- 172.21.15.112:0/3045694196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a608bad0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f770c00b4e0 tx=0x7f770c0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91d4d4ae108f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f76fd7fa640 1 -- 172.21.15.112:0/3045694196 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c03f070 con 0x5593a5ccc280 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f76fd7fa640 1 -- 172.21.15.112:0/3045694196 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f770c024070 con 0x5593a5ccc280 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f76fd7fa640 1 -- 172.21.15.112:0/3045694196 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c020cf0 con 0x5593a5ccc280 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f77246f4740 1 -- 172.21.15.112:0/3045694196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 msgr2=0x5593a608bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f77246f4740 1 --2- 172.21.15.112:0/3045694196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a608bad0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f770c00b4e0 tx=0x7f770c0045f0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f77246f4740 1 -- 172.21.15.112:0/3045694196 shutdown_connections 2024-09-21T03:28:39.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f77246f4740 1 --2- 172.21.15.112:0/3045694196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a60acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f77246f4740 1 --2- 172.21.15.112:0/3045694196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a608bad0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f77246f4740 1 --2- 172.21.15.112:0/3045694196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cc40a0 0x5593a608fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f77246f4740 1 -- 172.21.15.112:0/3045694196 >> 172.21.15.112:0/3045694196 conn(0x5593a5ce1350 msgr2=0x5593a609f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f77246f4740 1 -- 172.21.15.112:0/3045694196 shutdown_connections 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.818+0000 7f77246f4740 1 -- 172.21.15.112:0/3045694196 wait complete. 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f77246f4740 1 -- start start 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f77246f4740 1 Processor -- start 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f77246f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cc40a0 0x5593a60b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f77246f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f77246f4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a60b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f77246f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5593a60b3050 con 0x5593a5ccc280 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f77246f4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5593a60b2980 con 0x5593a5cecf60 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f77246f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5593a60b3720 con 0x5593a5cc40a0 2024-09-21T03:28:39.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f77246f4740 10 client.0 _ll_register_callbacks cb 0x7f770c00b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f7716ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f7716ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52228/0 (socket says 172.21.15.112:52228) 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.819+0000 7f7716ffd640 1 -- 172.21.15.112:0/2119320317 learned_addr learned my addr 172.21.15.112:0/2119320317 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f77177fe640 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cc40a0 0x5593a60b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f7717fff640 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a60b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f7716ffd640 1 -- 172.21.15.112:0/2119320317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cc40a0 msgr2=0x5593a60b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f7716ffd640 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cc40a0 0x5593a60b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f7716ffd640 1 -- 172.21.15.112:0/2119320317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 msgr2=0x5593a60b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f7716ffd640 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a60b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f7716ffd640 1 -- 172.21.15.112:0/2119320317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593a6095410 con 0x5593a5ccc280 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f7716ffd640 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60b5ec0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f770c004700 tx=0x7f770c005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a85771df68b32b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f76f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f76f6ffd640 1 -- 172.21.15.112:0/2119320317 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c03f070 con 0x5593a5ccc280 2024-09-21T03:28:39.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f76f6ffd640 1 -- 172.21.15.112:0/2119320317 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f770c020e00 con 0x5593a5ccc280 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f77246f4740 1 -- 172.21.15.112:0/2119320317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770c024070 con 0x5593a5ccc280 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f76f6ffd640 1 -- 172.21.15.112:0/2119320317 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f770c020700 con 0x5593a5ccc280 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f77246f4740 10 client.15345 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f77246f4740 1 -- 172.21.15.112:0/2119320317 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5593a6121060 con 0x5593a5ccc280 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.820+0000 7f76f6ffd640 1 -- 172.21.15.112:0/2119320317 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f770c024070 con 0x5593a5ccc280 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.821+0000 7f76f6ffd640 1 -- 172.21.15.112:0/2119320317 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f770c03a9d0 con 0x5593a5ccc280 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.821+0000 7f77246f4740 10 client.15345 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.821+0000 7f77246f4740 10 client.15345 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.821+0000 7f77246f4740 20 client.15345 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.821+0000 7f77246f4740 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5593a5c80c90 0x5593a61244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.821+0000 7f77246f4740 4 client.15345 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.821+0000 7f77246f4740 1 -- 172.21.15.112:0/2119320317 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "status"}) -- 0x5593a6008610 con 0x5593a5c80c90 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.821+0000 7f77177fe640 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5593a5c80c90 0x5593a61244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:39.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.821+0000 7f76f6ffd640 10 client.15345 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.821+0000 7f77177fe640 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5593a5c80c90 0x5593a61244c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a849f99000198d70 server_cookie=66e951df245bcec in_seq=0 out_seq=0 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f76f6ffd640 1 -- 172.21.15.112:0/2119320317 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5593a6008610 con 0x5593a5c80c90 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f76f6ffd640 10 client.15345 handle_command_reply: tid=0 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 1 client.15345 shutdown 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 1 -- 172.21.15.112:0/2119320317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 msgr2=0x5593a60b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60b5ec0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f770c004700 tx=0x7f770c005900 comp rx=0 tx=0).stop 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 1 -- 172.21.15.112:0/2119320317 shutdown_connections 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5593a5c80c90 0x5593a61244c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593a5cecf60 0x5593a60b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5593a5ccc280 0x5593a60b5ec0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 1 --2- 172.21.15.112:0/2119320317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5593a5cc40a0 0x5593a60b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f76f6ffd640 0 client.15345 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 1 -- 172.21.15.112:0/2119320317 >> 172.21.15.112:0/2119320317 conn(0x5593a5ce1350 msgr2=0x5593a60a0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 1 -- 172.21.15.112:0/2119320317 shutdown_connections 2024-09-21T03:28:39.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 1 -- 172.21.15.112:0/2119320317 wait complete. 2024-09-21T03:28:39.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.822+0000 7f77246f4740 20 client.15345 trim_cache size 0 max 16384 2024-09-21T03:28:39.837 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:28:39.837 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "063b32f6-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:28:39.837 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-09-21T03:28:39.837 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24674, 2024-09-21T03:28:39.837 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T03:28:39.837 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T03:28:39.837 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T03:28:39.838 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 90.459192329999993, 2024-09-21T03:28:39.838 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-09-21T03:28:39.838 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 76, 2024-09-21T03:28:39.838 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T03:28:39.838 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 107.53481992899999 2024-09-21T03:28:39.838 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:28:39.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 -- 172.21.15.112:0/4029135150 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77000b6560 msgr2=0x7f77000b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77000b6560 0x7f77000b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f770c005d00 tx=0x7f770c014040 comp rx=0 tx=0).stop 2024-09-21T03:28:39.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 -- 172.21.15.112:0/4029135150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7718108cf0 msgr2=0x7f771819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:39.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7718108cf0 0x7f771819ebe0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f771000b970 tx=0x7f771000be30 comp rx=0 tx=0).stop 2024-09-21T03:28:39.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 -- 172.21.15.112:0/4029135150 shutdown_connections 2024-09-21T03:28:39.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77000b6560 0x7f77000b8a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7718108cf0 0x7f771819ebe0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77181083a0 0x7f7718199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 --2- 172.21.15.112:0/4029135150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7718102a60 0x7f77181993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:39.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 -- 172.21.15.112:0/4029135150 >> 172.21.15.112:0/4029135150 conn(0x7f7718079730 msgr2=0x7f7718101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:39.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 -- 172.21.15.112:0/4029135150 shutdown_connections 2024-09-21T03:28:39.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:39.823+0000 7f771e180640 1 -- 172.21.15.112:0/4029135150 wait complete. 2024-09-21T03:28:40.692 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T03:28:40.936 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.396+0000 7f53a1ab2640 1 -- 172.21.15.112:0/3772164852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c102a60 msgr2=0x7f539c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:41.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.396+0000 7f53a1ab2640 1 --2- 172.21.15.112:0/3772164852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c102a60 0x7f539c102e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5384009900 tx=0x7f538402a410 comp rx=0 tx=0).stop 2024-09-21T03:28:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.396+0000 7f53a1ab2640 1 -- 172.21.15.112:0/3772164852 shutdown_connections 2024-09-21T03:28:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.396+0000 7f53a1ab2640 1 --2- 172.21.15.112:0/3772164852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f539c108cf0 0x7f539c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.396+0000 7f53a1ab2640 1 --2- 172.21.15.112:0/3772164852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539c1083a0 0x7f539c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.396+0000 7f53a1ab2640 1 --2- 172.21.15.112:0/3772164852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c102a60 0x7f539c102e70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.396+0000 7f53a1ab2640 1 -- 172.21.15.112:0/3772164852 >> 172.21.15.112:0/3772164852 conn(0x7f539c079730 msgr2=0x7f539c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.396+0000 7f53a1ab2640 1 -- 172.21.15.112:0/3772164852 shutdown_connections 2024-09-21T03:28:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.396+0000 7f53a1ab2640 1 -- 172.21.15.112:0/3772164852 wait complete. 2024-09-21T03:28:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f53a1ab2640 1 Processor -- start 2024-09-21T03:28:41.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f53a1ab2640 1 -- start start 2024-09-21T03:28:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f53a1ab2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f539c102a60 0x7f539c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f53a1ab2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539c1083a0 0x7f539c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f53a1ab2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c108cf0 0x7f539c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f53a1ab2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f539c112960 con 0x7f539c108cf0 2024-09-21T03:28:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f53a1ab2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1127e0 con 0x7f539c102a60 2024-09-21T03:28:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f53a1ab2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f539c112ae0 con 0x7f539c1083a0 2024-09-21T03:28:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f539bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c108cf0 0x7f539c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f539bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c108cf0 0x7f539c19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52250/0 (socket says 172.21.15.112:52250) 2024-09-21T03:28:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f539bfff640 1 -- 172.21.15.112:0/2317140388 learned_addr learned my addr 172.21.15.112:0/2317140388 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f539b7fe640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f539c102a60 0x7f539c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.397+0000 7f539affd640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539c1083a0 0x7f539c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f539bfff640 1 -- 172.21.15.112:0/2317140388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539c1083a0 msgr2=0x7f539c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f539bfff640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539c1083a0 0x7f539c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f539bfff640 1 -- 172.21.15.112:0/2317140388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f539c102a60 msgr2=0x7f539c1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f539bfff640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f539c102a60 0x7f539c1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f539bfff640 1 -- 172.21.15.112:0/2317140388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c19f260 con 0x7f539c108cf0 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f539affd640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539c1083a0 0x7f539c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f539bfff640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c108cf0 0x7f539c19ebf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f539400cad0 tx=0x7f539400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f5398ff9640 1 -- 172.21.15.112:0/2317140388 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539400ae20 con 0x7f539c108cf0 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f53a1ab2640 1 -- 172.21.15.112:0/2317140388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c19f550 con 0x7f539c108cf0 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f53a1ab2640 1 -- 172.21.15.112:0/2317140388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c1a5d70 con 0x7f539c108cf0 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f5398ff9640 1 -- 172.21.15.112:0/2317140388 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5394005020 con 0x7f539c108cf0 2024-09-21T03:28:41.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.398+0000 7f5398ff9640 1 -- 172.21.15.112:0/2317140388 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539400f6b0 con 0x7f539c108cf0 2024-09-21T03:28:41.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f53a8026740 1 Processor -- start 2024-09-21T03:28:41.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f5398ff9640 1 -- 172.21.15.112:0/2317140388 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f53940051d0 con 0x7f539c108cf0 2024-09-21T03:28:41.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f53a8026740 1 -- start start 2024-09-21T03:28:41.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f53a8026740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733f90f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f53a8026740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bc20a0 0x55b733f91440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f53a8026740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bca280 0x55b733f91980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f53a8026740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b733f08c10 con 0x55b733beaf60 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f53a8026740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b733ef8040 con 0x55b733bc20a0 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f53a8026740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b733efbe50 con 0x55b733bca280 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f539affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733f90f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f539affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733f90f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52260/0 (socket says 172.21.15.112:52260) 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.401+0000 7f539affd640 1 -- 172.21.15.112:0/2186515605 learned_addr learned my addr 172.21.15.112:0/2186515605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f539b7fe640 1 --2- 172.21.15.112:0/2186515605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bc20a0 0x55b733f91440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f539affd640 1 -- 172.21.15.112:0/2186515605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bca280 msgr2=0x55b733f91980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f539affd640 1 --2- 172.21.15.112:0/2186515605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bca280 0x55b733f91980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f539affd640 1 -- 172.21.15.112:0/2186515605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bc20a0 msgr2=0x55b733f91440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:41.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f539affd640 1 --2- 172.21.15.112:0/2186515605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bc20a0 0x55b733f91440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f539affd640 1 -- 172.21.15.112:0/2186515605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b733c56540 con 0x55b733beaf60 2024-09-21T03:28:41.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f5398ff9640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f53880b6590 0x7f53880b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f539b7fe640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f53880b6590 0x7f53880b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f539affd640 1 --2- 172.21.15.112:0/2186515605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733f90f00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f538c008d00 tx=0x7f538c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f6ebfc7c231ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:41.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f5398ff9640 1 -- 172.21.15.112:0/2317140388 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f53940e7400 con 0x7f539c108cf0 2024-09-21T03:28:41.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f5391ffb640 1 -- 172.21.15.112:0/2186515605 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c014070 con 0x55b733beaf60 2024-09-21T03:28:41.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f5391ffb640 1 -- 172.21.15.112:0/2186515605 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f538c0047a0 con 0x55b733beaf60 2024-09-21T03:28:41.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.402+0000 7f5391ffb640 1 -- 172.21.15.112:0/2186515605 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c010670 con 0x55b733beaf60 2024-09-21T03:28:41.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.403+0000 7f53a8026740 1 -- 172.21.15.112:0/2186515605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 msgr2=0x55b733f90f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:41.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.403+0000 7f53a8026740 1 --2- 172.21.15.112:0/2186515605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733f90f00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f538c008d00 tx=0x7f538c00e720 comp rx=0 tx=0).stop 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.403+0000 7f539b7fe640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f53880b6590 0x7f53880b8aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5384031440 tx=0x7f5384009650 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.403+0000 7f53a8026740 1 -- 172.21.15.112:0/2186515605 shutdown_connections 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.403+0000 7f53a8026740 1 --2- 172.21.15.112:0/2186515605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bca280 0x55b733f91980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.403+0000 7f53a8026740 1 --2- 172.21.15.112:0/2186515605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bc20a0 0x55b733f91440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.403+0000 7f53a8026740 1 --2- 172.21.15.112:0/2186515605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733f90f00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.403+0000 7f53a8026740 1 -- 172.21.15.112:0/2186515605 >> 172.21.15.112:0/2186515605 conn(0x55b733bdf350 msgr2=0x55b733f8edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.403+0000 7f53a8026740 1 -- 172.21.15.112:0/2186515605 shutdown_connections 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.403+0000 7f53a8026740 1 -- 172.21.15.112:0/2186515605 wait complete. 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.404+0000 7f53a8026740 1 -- start start 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.404+0000 7f53a8026740 1 Processor -- start 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.404+0000 7f53a8026740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bc20a0 0x55b733f9fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.404+0000 7f53a8026740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 0x55b733fa0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.404+0000 7f53a8026740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733fa49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.404+0000 7f53a8026740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b733d595f0 con 0x55b733beaf60 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.404+0000 7f53a8026740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b733e93010 con 0x55b733bca280 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.404+0000 7f53a8026740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b733f9f6e0 con 0x55b733bc20a0 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.404+0000 7f53a8026740 10 client.0 _ll_register_callbacks cb 0x55b733ef2340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.404+0000 7f539bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733fa49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733fa49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52264/0 (socket says 172.21.15.112:52264) 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539bfff640 1 -- 172.21.15.112:0/4152539833 learned_addr learned my addr 172.21.15.112:0/4152539833 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539affd640 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 0x55b733fa0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539b7fe640 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bc20a0 0x55b733f9fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539bfff640 1 -- 172.21.15.112:0/4152539833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bc20a0 msgr2=0x55b733f9fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539bfff640 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bc20a0 0x55b733f9fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539bfff640 1 -- 172.21.15.112:0/4152539833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 msgr2=0x55b733fa0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:41.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539bfff640 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 0x55b733fa0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539bfff640 1 -- 172.21.15.112:0/4152539833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b733ec0a00 con 0x55b733beaf60 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539affd640 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 0x55b733fa0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539b7fe640 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bc20a0 0x55b733f9fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f539bfff640 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733fa49c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53940a9aa0 tx=0x7f53940acc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48defc1fab7a50f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f537affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f537affd640 1 -- 172.21.15.112:0/4152539833 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539401d040 con 0x55b733beaf60 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f537affd640 1 -- 172.21.15.112:0/4152539833 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f53940a8e20 con 0x55b733beaf60 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f53a8026740 1 -- 172.21.15.112:0/4152539833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f538c0047a0 con 0x55b733beaf60 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f537affd640 1 -- 172.21.15.112:0/4152539833 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f53940b1fb0 con 0x55b733beaf60 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f53a8026740 10 client.15357 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.405+0000 7f53a8026740 1 -- 172.21.15.112:0/4152539833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b733fad910 con 0x55b733beaf60 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.406+0000 7f537affd640 1 -- 172.21.15.112:0/4152539833 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f538c0047a0 con 0x55b733beaf60 2024-09-21T03:28:41.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.406+0000 7f537affd640 1 -- 172.21.15.112:0/4152539833 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5394004a40 con 0x55b733beaf60 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.406+0000 7f53a8026740 10 client.15357 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.406+0000 7f53a8026740 10 client.15357 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.406+0000 7f53a8026740 20 client.15357 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.406+0000 7f53a8026740 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b733ba98e0 0x55b733fb13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.406+0000 7f53a8026740 4 client.15357 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.406+0000 7f53a8026740 1 -- 172.21.15.112:0/4152539833 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b733f0ab60 con 0x55b733ba98e0 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.406+0000 7f539b7fe640 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b733ba98e0 0x55b733fb13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.407+0000 7f537affd640 10 client.15357 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.407+0000 7f539b7fe640 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b733ba98e0 0x55b733fb13f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee40e571942f2dd5 server_cookie=e5afe6d867fd002c in_seq=0 out_seq=0 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.407+0000 7f537affd640 1 -- 172.21.15.112:0/4152539833 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b733f0ab60 con 0x55b733ba98e0 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.407+0000 7f537affd640 10 client.15357 handle_command_reply: tid=0 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.407+0000 7f53a8026740 1 client.15357 shutdown 2024-09-21T03:28:41.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f53a8026740 1 -- 172.21.15.112:0/4152539833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 msgr2=0x55b733fa49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:41.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f53a8026740 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733fa49c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53940a9aa0 tx=0x7f53940acc40 comp rx=0 tx=0).stop 2024-09-21T03:28:41.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f53a8026740 1 -- 172.21.15.112:0/4152539833 shutdown_connections 2024-09-21T03:28:41.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f53a8026740 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b733ba98e0 0x55b733fb13f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f53a8026740 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733beaf60 0x55b733fa49c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f53a8026740 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 0x55b733fa0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f53a8026740 1 --2- 172.21.15.112:0/4152539833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bc20a0 0x55b733f9fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f53a8026740 1 -- 172.21.15.112:0/4152539833 >> 172.21.15.112:0/4152539833 conn(0x55b733bdf350 msgr2=0x55b733f940e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:41.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f537affd640 0 client.15357 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:41.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f53a8026740 1 -- 172.21.15.112:0/4152539833 shutdown_connections 2024-09-21T03:28:41.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f53a8026740 1 -- 172.21.15.112:0/4152539833 wait complete. 2024-09-21T03:28:41.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.408+0000 7f53a8026740 20 client.15357 trim_cache size 0 max 16384 2024-09-21T03:28:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.426+0000 7f53a8026740 1 Processor -- start 2024-09-21T03:28:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.426+0000 7f53a8026740 1 -- start start 2024-09-21T03:28:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.426+0000 7f53a8026740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bc20a0 0x55b733f98400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.426+0000 7f53a8026740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 0x55b733f98940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.426+0000 7f53a8026740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733beaf60 0x55b733f9fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.426+0000 7f53a8026740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b733faf730 con 0x55b733bc20a0 2024-09-21T03:28:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.426+0000 7f53a8026740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b733fafe00 con 0x55b733bca280 2024-09-21T03:28:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f53a8026740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b733fb04d0 con 0x55b733beaf60 2024-09-21T03:28:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bc20a0 0x55b733f98400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bc20a0 0x55b733f98400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52270/0 (socket says 172.21.15.112:52270) 2024-09-21T03:28:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539b7fe640 1 -- 172.21.15.112:0/231057730 learned_addr learned my addr 172.21.15.112:0/231057730 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539affd640 1 --2- 172.21.15.112:0/231057730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 0x55b733f98940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539b7fe640 1 -- 172.21.15.112:0/231057730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733beaf60 msgr2=0x55b733f9fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539bfff640 1 --2- 172.21.15.112:0/231057730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733beaf60 0x55b733f9fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539b7fe640 1 --2- 172.21.15.112:0/231057730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733beaf60 0x55b733f9fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539b7fe640 1 -- 172.21.15.112:0/231057730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 msgr2=0x55b733f98940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539b7fe640 1 --2- 172.21.15.112:0/231057730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 0x55b733f98940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539b7fe640 1 -- 172.21.15.112:0/231057730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b733f89080 con 0x55b733bc20a0 2024-09-21T03:28:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539affd640 1 --2- 172.21.15.112:0/231057730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 0x55b733f98940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f539b7fe640 1 --2- 172.21.15.112:0/231057730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bc20a0 0x55b733f98400 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f538402a930 tx=0x7f538403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0f8f6830e5c579 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f53917fa640 1 -- 172.21.15.112:0/231057730 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384055030 con 0x55b733bc20a0 2024-09-21T03:28:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f53917fa640 1 -- 172.21.15.112:0/231057730 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5384004240 con 0x55b733bc20a0 2024-09-21T03:28:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.427+0000 7f53917fa640 1 -- 172.21.15.112:0/231057730 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384035040 con 0x55b733bc20a0 2024-09-21T03:28:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.428+0000 7f53a8026740 1 -- 172.21.15.112:0/231057730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bc20a0 msgr2=0x55b733f98400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.428+0000 7f53a8026740 1 --2- 172.21.15.112:0/231057730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bc20a0 0x55b733f98400 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f538402a930 tx=0x7f538403f000 comp rx=0 tx=0).stop 2024-09-21T03:28:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.428+0000 7f53a8026740 1 -- 172.21.15.112:0/231057730 shutdown_connections 2024-09-21T03:28:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.428+0000 7f53a8026740 1 --2- 172.21.15.112:0/231057730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733beaf60 0x55b733f9fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.428+0000 7f53a8026740 1 --2- 172.21.15.112:0/231057730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733bca280 0x55b733f98940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.428+0000 7f53a8026740 1 --2- 172.21.15.112:0/231057730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bc20a0 0x55b733f98400 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.428+0000 7f53a8026740 1 -- 172.21.15.112:0/231057730 >> 172.21.15.112:0/231057730 conn(0x55b733bdf350 msgr2=0x55b733f8e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.428+0000 7f53a8026740 1 -- 172.21.15.112:0/231057730 shutdown_connections 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.428+0000 7f53a8026740 1 -- 172.21.15.112:0/231057730 wait complete. 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f53a8026740 1 -- start start 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f53a8026740 1 Processor -- start 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f53a8026740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bc20a0 0x55b733fab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f53a8026740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bca280 0x55b733fab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f53a8026740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733beaf60 0x55b733fb95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f53a8026740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b733fb1270 con 0x55b733bca280 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f53a8026740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b733fb0ba0 con 0x55b733beaf60 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f53a8026740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b733fb1940 con 0x55b733bc20a0 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f53a8026740 10 client.0 _ll_register_callbacks cb 0x7f53940ace20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f539affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bca280 0x55b733fab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f539affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bca280 0x55b733fab5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52286/0 (socket says 172.21.15.112:52286) 2024-09-21T03:28:41.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f539affd640 1 -- 172.21.15.112:0/3745805013 learned_addr learned my addr 172.21.15.112:0/3745805013 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.429+0000 7f539bfff640 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733beaf60 0x55b733fb95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f539b7fe640 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bc20a0 0x55b733fab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f539affd640 1 -- 172.21.15.112:0/3745805013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bc20a0 msgr2=0x55b733fab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f539affd640 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bc20a0 0x55b733fab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f539affd640 1 -- 172.21.15.112:0/3745805013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733beaf60 msgr2=0x55b733fb95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f539affd640 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733beaf60 0x55b733fb95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f539affd640 1 -- 172.21.15.112:0/3745805013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b733fb9d40 con 0x55b733bca280 2024-09-21T03:28:41.436 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:41 smithi112 ceph-mon[31442]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f539affd640 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bca280 0x55b733fab5e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f538c008d00 tx=0x7f538c008910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8156ea2edd2d241 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f537affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f537affd640 1 -- 172.21.15.112:0/3745805013 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c014040 con 0x55b733bca280 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f537affd640 1 -- 172.21.15.112:0/3745805013 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f538c005470 con 0x55b733bca280 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f53a8026740 1 -- 172.21.15.112:0/3745805013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5384004240 con 0x55b733bca280 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f537affd640 1 -- 172.21.15.112:0/3745805013 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f538c033050 con 0x55b733bca280 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f53a8026740 10 client.15363 fetch_fsmap learned FSMap version 24 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.430+0000 7f53a8026740 1 -- 172.21.15.112:0/3745805013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b733faa6c0 con 0x55b733bca280 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.431+0000 7f537affd640 1 -- 172.21.15.112:0/3745805013 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5384004240 con 0x55b733bca280 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.431+0000 7f537affd640 1 -- 172.21.15.112:0/3745805013 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f538c010430 con 0x55b733bca280 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.431+0000 7f53a8026740 10 client.15363 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T03:28:41.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.431+0000 7f53a8026740 10 client.15363 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.431+0000 7f53a8026740 20 client.15363 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.431+0000 7f53a8026740 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b733b7ec90 0x55b73401f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.431+0000 7f53a8026740 4 client.15363 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.431+0000 7f53a8026740 1 -- 172.21.15.112:0/3745805013 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "status"}) -- 0x55b733f06610 con 0x55b733b7ec90 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.431+0000 7f539b7fe640 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b733b7ec90 0x55b73401f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.432+0000 7f537affd640 10 client.15363 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.432+0000 7f539b7fe640 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b733b7ec90 0x55b73401f870 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30ea287cf668609b server_cookie=271b888d64d68cd1 in_seq=0 out_seq=0 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.432+0000 7f537affd640 1 -- 172.21.15.112:0/3745805013 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b733f06610 con 0x55b733b7ec90 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.432+0000 7f537affd640 10 client.15363 handle_command_reply: tid=0 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.432+0000 7f53a8026740 1 client.15363 shutdown 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.432+0000 7f53a8026740 1 -- 172.21.15.112:0/3745805013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bca280 msgr2=0x55b733fab5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.432+0000 7f53a8026740 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bca280 0x55b733fab5e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f538c008d00 tx=0x7f538c008910 comp rx=0 tx=0).stop 2024-09-21T03:28:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a8026740 1 -- 172.21.15.112:0/3745805013 shutdown_connections 2024-09-21T03:28:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a8026740 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b733b7ec90 0x55b73401f870 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a8026740 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b733beaf60 0x55b733fb95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f537affd640 0 client.15363 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a8026740 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b733bca280 0x55b733fab5e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a8026740 1 --2- 172.21.15.112:0/3745805013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b733bc20a0 0x55b733fab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a8026740 1 -- 172.21.15.112:0/3745805013 >> 172.21.15.112:0/3745805013 conn(0x55b733bdf350 msgr2=0x55b733f9fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a8026740 1 -- 172.21.15.112:0/3745805013 shutdown_connections 2024-09-21T03:28:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a8026740 1 -- 172.21.15.112:0/3745805013 wait complete. 2024-09-21T03:28:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a8026740 20 client.15363 trim_cache size 0 max 16384 2024-09-21T03:28:41.442 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:28:41.442 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "063b32f6-77c8-11ef-baf5-efdc52797490", 2024-09-21T03:28:41.442 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-09-21T03:28:41.442 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24685, 2024-09-21T03:28:41.442 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-09-21T03:28:41.442 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-09-21T03:28:41.442 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-09-21T03:28:41.443 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 99.117672814000002, 2024-09-21T03:28:41.443 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 24, 2024-09-21T03:28:41.443 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 76, 2024-09-21T03:28:41.443 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T03:28:41.443 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 117.395818207 2024-09-21T03:28:41.443 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a1ab2640 1 -- 172.21.15.112:0/2317140388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f53880b6590 msgr2=0x7f53880b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a1ab2640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f53880b6590 0x7f53880b8aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5384031440 tx=0x7f5384009650 comp rx=0 tx=0).stop 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a1ab2640 1 -- 172.21.15.112:0/2317140388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c108cf0 msgr2=0x7f539c19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.433+0000 7f53a1ab2640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c108cf0 0x7f539c19ebf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f539400cad0 tx=0x7f539400cf90 comp rx=0 tx=0).stop 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.434+0000 7f53a1ab2640 1 -- 172.21.15.112:0/2317140388 shutdown_connections 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.434+0000 7f53a1ab2640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f53880b6590 0x7f53880b8aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.434+0000 7f53a1ab2640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c108cf0 0x7f539c19ebf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.434+0000 7f53a1ab2640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539c1083a0 0x7f539c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.434+0000 7f53a1ab2640 1 --2- 172.21.15.112:0/2317140388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f539c102a60 0x7f539c1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.434+0000 7f53a1ab2640 1 -- 172.21.15.112:0/2317140388 >> 172.21.15.112:0/2317140388 conn(0x7f539c079730 msgr2=0x7f539c101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.434+0000 7f53a1ab2640 1 -- 172.21.15.112:0/2317140388 shutdown_connections 2024-09-21T03:28:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:41.434+0000 7f53a1ab2640 1 -- 172.21.15.112:0/2317140388 wait complete. 2024-09-21T03:28:41.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:41 smithi187 ceph-mon[34603]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T03:28:41.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:41 smithi203 ceph-mon[34723]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T03:28:42.259 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T03:28:42.283 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T03:28:42.283 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T03:28:42.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:28:42.284 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T03:28:42.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:28:42.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-09-21T03:28:42.369 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-09-21T03:28:42.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:28:42.371 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-21T03:28:42.371 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:28:42.371 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:28:42.372 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T03:28:42.660 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.115+0000 7f9433c2e640 1 -- 172.21.15.112:0/637311937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f942c10de10 msgr2=0x7f942c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.115+0000 7f9433c2e640 1 --2- 172.21.15.112:0/637311937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f942c10de10 0x7f942c110300 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9424009ca0 tx=0x7f942402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.115+0000 7f9433c2e640 1 -- 172.21.15.112:0/637311937 shutdown_connections 2024-09-21T03:28:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.115+0000 7f9433c2e640 1 --2- 172.21.15.112:0/637311937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f942c10de10 0x7f942c110300 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.115+0000 7f9433c2e640 1 --2- 172.21.15.112:0/637311937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f942c105300 0x7f942c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.115+0000 7f9433c2e640 1 --2- 172.21.15.112:0/637311937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f942c1049b0 0x7f942c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.115+0000 7f9433c2e640 1 -- 172.21.15.112:0/637311937 >> 172.21.15.112:0/637311937 conn(0x7f942c079730 msgr2=0x7f942c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.115+0000 7f9433c2e640 1 -- 172.21.15.112:0/637311937 shutdown_connections 2024-09-21T03:28:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.115+0000 7f9433c2e640 1 -- 172.21.15.112:0/637311937 wait complete. 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f9433c2e640 1 Processor -- start 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f9433c2e640 1 -- start start 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f9433c2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f942c1049b0 0x7f942c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f9433c2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f942c105300 0x7f942c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f9433c2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f942c10de10 0x7f942c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f9433c2e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f942c113a70 con 0x7f942c1049b0 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f9433c2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f942c1138f0 con 0x7f942c10de10 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f9433c2e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f942c113bf0 con 0x7f942c105300 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f94319ca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f942c1049b0 0x7f942c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f94319ca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f942c1049b0 0x7f942c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52316/0 (socket says 172.21.15.112:52316) 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f94319ca640 1 -- 172.21.15.112:0/4117050881 learned_addr learned my addr 172.21.15.112:0/4117050881 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.116+0000 7f94321cb640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f942c10de10 0x7f942c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f94311c9640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f942c105300 0x7f942c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:43.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f94319ca640 1 -- 172.21.15.112:0/4117050881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f942c105300 msgr2=0x7f942c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f94319ca640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f942c105300 0x7f942c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f94319ca640 1 -- 172.21.15.112:0/4117050881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f942c10de10 msgr2=0x7f942c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f94319ca640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f942c10de10 0x7f942c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f94319ca640 1 -- 172.21.15.112:0/4117050881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c1a0390 con 0x7f942c1049b0 2024-09-21T03:28:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f94321cb640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f942c10de10 0x7f942c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:43.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f94319ca640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f942c1049b0 0x7f942c19a420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f942000d770 tx=0x7f942000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:43.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f941effd640 1 -- 172.21.15.112:0/4117050881 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420017070 con 0x7f942c1049b0 2024-09-21T03:28:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f9433c2e640 1 -- 172.21.15.112:0/4117050881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942c1a0680 con 0x7f942c1049b0 2024-09-21T03:28:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f941effd640 1 -- 172.21.15.112:0/4117050881 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9420009c30 con 0x7f942c1049b0 2024-09-21T03:28:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f941effd640 1 -- 172.21.15.112:0/4117050881 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420003cc0 con 0x7f942c1049b0 2024-09-21T03:28:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.117+0000 7f9433c2e640 1 -- 172.21.15.112:0/4117050881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c1a6de0 con 0x7f942c1049b0 2024-09-21T03:28:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.118+0000 7f9433c2e640 1 -- 172.21.15.112:0/4117050881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942c104dc0 con 0x7f942c1049b0 2024-09-21T03:28:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.120+0000 7f941effd640 1 -- 172.21.15.112:0/4117050881 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f9420004020 con 0x7f942c1049b0 2024-09-21T03:28:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.121+0000 7f941effd640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f94180b6560 0x7f94180b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.121+0000 7f94311c9640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f94180b6560 0x7f94180b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.121+0000 7f941effd640 1 -- 172.21.15.112:0/4117050881 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f94200e6e90 con 0x7f942c1049b0 2024-09-21T03:28:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.121+0000 7f94311c9640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f94180b6560 0x7f94180b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9428005fd0 tx=0x7f9428005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.129+0000 7f941effd640 1 -- 172.21.15.112:0/4117050881 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9420009de0 con 0x7f942c1049b0 2024-09-21T03:28:43.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.356+0000 7f9433c2e640 1 -- 172.21.15.112:0/4117050881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f942c1a7160 con 0x7f942c1049b0 2024-09-21T03:28:43.360 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:28:43.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.357+0000 7f941effd640 1 -- 172.21.15.112:0/4117050881 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f94200ad660 con 0x7f942c1049b0 2024-09-21T03:28:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 -- 172.21.15.112:0/4117050881 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f94180b6560 msgr2=0x7f94180b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f94180b6560 0x7f94180b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9428005fd0 tx=0x7f9428005850 comp rx=0 tx=0).stop 2024-09-21T03:28:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 -- 172.21.15.112:0/4117050881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f942c1049b0 msgr2=0x7f942c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f942c1049b0 0x7f942c19a420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f942000d770 tx=0x7f942000dc30 comp rx=0 tx=0).stop 2024-09-21T03:28:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 -- 172.21.15.112:0/4117050881 shutdown_connections 2024-09-21T03:28:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f94180b6560 0x7f94180b8a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f942c10de10 0x7f942c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f942c105300 0x7f942c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 --2- 172.21.15.112:0/4117050881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f942c1049b0 0x7f942c19a420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:43.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 -- 172.21.15.112:0/4117050881 >> 172.21.15.112:0/4117050881 conn(0x7f942c079730 msgr2=0x7f942c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:43.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 -- 172.21.15.112:0/4117050881 shutdown_connections 2024-09-21T03:28:43.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:43.360+0000 7f9433c2e640 1 -- 172.21.15.112:0/4117050881 wait complete. 2024-09-21T03:28:43.670 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:43 smithi112 ceph-mon[31442]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T03:28:43.671 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:43 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/4117050881' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:28:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:43 smithi187 ceph-mon[34603]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T03:28:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:43 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/4117050881' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:28:43.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:43 smithi203 ceph-mon[34723]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T03:28:43.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:43 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/4117050881' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:28:43.983 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T03:28:43.983 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:28:43.983 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T03:28:43.983 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:28:43.983 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:28:43.983 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T03:28:43.983 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:28:43.984 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:28:43.984 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:28:43.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:28:43.984 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T03:28:44.001 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:28:44.001 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:28:44.001 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:28:44.001 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:28:44.001 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T03:28:44.001 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:28:44.001 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:28:44.001 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T03:28:44.001 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T03:28:44.001 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:28:44.002 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T03:28:44.002 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T03:28:44.002 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:28:44.002 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:28:44.002 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:28:44.002 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:28:44.002 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T03:28:44.002 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42168sec preferred_lft 42168sec 2024-09-21T03:28:44.002 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T03:28:44.002 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:28:44.003 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:28:44.003 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:28:44.003 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T03:28:44.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:28:44.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-09-21T03:28:44.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-09-21T03:28:44.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:28:44.003 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T03:28:44.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:28:44.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:28:44.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:28:44.206 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:28:44.231 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T03:28:44.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:28:44.233 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T03:28:44.303 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T03:28:44.303 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:28:44.303 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T03:28:44.303 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T03:28:44.303 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:28:44.304 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:28:44.304 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T03:28:44.304 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T03:28:44.304 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T03:28:44.304 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T03:28:44.304 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T03:28:44.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:28:44.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:28:44.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:28:44.624 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T03:28:44.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:28:44.645 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-09-21T03:28:44.702 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:28:44.702 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T03:28:44.702 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:28:44.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T03:28:44.703 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T03:28:44.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:28:44.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:28:44.820 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T03:28:44.820 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:28:44.821 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T03:28:44.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T03:28:44.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:28:44.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T03:28:44.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T03:28:44.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T03:28:44.821 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T03:28:44.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:28:44.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:28:44.943 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:28:44.943 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T03:28:44.943 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T03:28:44.943 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:28:44.943 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T03:28:44.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:28:45.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:28:45.020 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:28:45.020 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:28:45.035 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:28:45.035 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:28:45.035 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:28:45.091 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T03:28:45.091 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T03:28:45.091 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/0439cab4-e277-4b08-8947-0a17e694fb85 2024-09-21T03:28:45.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:28:45.091 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/0439cab4-e277-4b08-8947-0a17e694fb85 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-21T03:28:45.199 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-21T03:28:45.199 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-21T03:28:45.340 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-21T03:28:45.340 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-21T03:28:45.340 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.187:3300,172.21.15.203:3300" 2024-09-21T03:28:45.340 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with new device syntax: 0@063b32f6-77c8-11ef-baf5-efdc52797490.cephfs=/volumes/qa/sv_1/0439cab4-e277-4b08-8947-0a17e694fb85 2024-09-21T03:28:45.340 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.112:3300/172.21.15.187:3300/172.21.15.203:3300" will pass to kernel 2024-09-21T03:28:45.341 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:28:45.341 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:28:45.342 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:28:45.342 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:28:45.342 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:28:45.344 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:28:45.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:28:45.344 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:28:45.373 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:28:45.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:28:45.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:28:45.470 INFO:teuthology.orchestra.run.smithi112.stdout:client25007 2024-09-21T03:28:45.470 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:28:45.470 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:28:45.487 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:28:45.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:28:45.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:28:45.584 INFO:teuthology.orchestra.run.smithi112.stdout:client25007 2024-09-21T03:28:45.584 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:28:45.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:28:45.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:28:45.639 INFO:teuthology.orchestra.run.smithi112.stdout:client25007 2024-09-21T03:28:45.640 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/063b32f6-77c8-11ef-baf5-efdc52797490.client25007/status') 2024-09-21T03:28:45.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:45 smithi187 ceph-mon[34603]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T03:28:45.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:28:45.695 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:45 smithi112 ceph-mon[31442]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T03:28:45.697 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.25007 (3)172.21.15.112:0/1402174862 2024-09-21T03:28:45.698 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-09-21T03:28:45.698 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-09-21T03:28:45.698 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-09-21T03:28:45.698 INFO:teuthology.orchestra.run.smithi112.stderr:72 bytes copied, 2.13e-05 s, 3.4 MB/s 2024-09-21T03:28:45.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:28:45.705 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T03:28:45.715 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:28:45.715 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T03:28:45.715 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-21T03:28:45.715 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:45 smithi203 ceph-mon[34723]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T03:28:45.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.824+0000 7f8f6e5da640 1 -- 172.21.15.112:0/3347189046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f68108300 msgr2=0x7f8f68108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:45.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.824+0000 7f8f6e5da640 1 --2- 172.21.15.112:0/3347189046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f68108300 0x7f8f68108710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009900 tx=0x7f8f5c02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:45.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.824+0000 7f8f6e5da640 1 -- 172.21.15.112:0/3347189046 shutdown_connections 2024-09-21T03:28:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.824+0000 7f8f6e5da640 1 --2- 172.21.15.112:0/3347189046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f68108c50 0x7f8f6810f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.824+0000 7f8f6e5da640 1 --2- 172.21.15.112:0/3347189046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f68108300 0x7f8f68108710 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.824+0000 7f8f6e5da640 1 --2- 172.21.15.112:0/3347189046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f681029d0 0x7f8f68102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.824+0000 7f8f6e5da640 1 -- 172.21.15.112:0/3347189046 >> 172.21.15.112:0/3347189046 conn(0x7f8f680fe3b0 msgr2=0x7f8f68100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.824+0000 7f8f6e5da640 1 -- 172.21.15.112:0/3347189046 shutdown_connections 2024-09-21T03:28:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.824+0000 7f8f6e5da640 1 -- 172.21.15.112:0/3347189046 wait complete. 2024-09-21T03:28:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f6e5da640 1 Processor -- start 2024-09-21T03:28:45.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f6e5da640 1 -- start start 2024-09-21T03:28:45.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f6e5da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f681029d0 0x7f8f6819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:45.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f6e5da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f68108300 0x7f8f6819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:45.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f6e5da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f68108c50 0x7f8f6819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:45.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f6e5da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f68113a30 con 0x7f8f68108300 2024-09-21T03:28:45.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f6e5da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f681138b0 con 0x7f8f68108c50 2024-09-21T03:28:45.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f6e5da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f68113bb0 con 0x7f8f681029d0 2024-09-21T03:28:45.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f68108300 0x7f8f6819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:45.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f68108300 0x7f8f6819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58284/0 (socket says 172.21.15.112:58284) 2024-09-21T03:28:45.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f677fe640 1 -- 172.21.15.112:0/2087442977 learned_addr learned my addr 172.21.15.112:0/2087442977 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:45.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f6cb77640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f68108c50 0x7f8f6819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:45.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.825+0000 7f8f67fff640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f681029d0 0x7f8f6819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:45.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.826+0000 7f8f677fe640 1 -- 172.21.15.112:0/2087442977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f681029d0 msgr2=0x7f8f6819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:45.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.826+0000 7f8f677fe640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f681029d0 0x7f8f6819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:45.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.826+0000 7f8f677fe640 1 -- 172.21.15.112:0/2087442977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f68108c50 msgr2=0x7f8f6819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:45.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.826+0000 7f8f677fe640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f68108c50 0x7f8f6819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:45.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.826+0000 7f8f677fe640 1 -- 172.21.15.112:0/2087442977 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f681a0380 con 0x7f8f68108300 2024-09-21T03:28:45.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.826+0000 7f8f677fe640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f68108300 0x7f8f6819a9f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c02a930 tx=0x7f8f5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:45.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.826+0000 7f8f657fa640 1 -- 172.21.15.112:0/2087442977 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c03f030 con 0x7f8f68108300 2024-09-21T03:28:45.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.826+0000 7f8f657fa640 1 -- 172.21.15.112:0/2087442977 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f5c003cb0 con 0x7f8f68108300 2024-09-21T03:28:45.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.826+0000 7f8f6e5da640 1 -- 172.21.15.112:0/2087442977 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f681a0610 con 0x7f8f68108300 2024-09-21T03:28:45.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.826+0000 7f8f657fa640 1 -- 172.21.15.112:0/2087442977 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c035070 con 0x7f8f68108300 2024-09-21T03:28:45.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.826+0000 7f8f6e5da640 1 -- 172.21.15.112:0/2087442977 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f681a6e00 con 0x7f8f68108300 2024-09-21T03:28:45.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.827+0000 7f8f6e5da640 1 -- 172.21.15.112:0/2087442977 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f68103c40 con 0x7f8f68108300 2024-09-21T03:28:45.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.829+0000 7f8f657fa640 1 -- 172.21.15.112:0/2087442977 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f8f5c042030 con 0x7f8f68108300 2024-09-21T03:28:45.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.830+0000 7f8f657fa640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8f500b6560 0x7f8f500b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:45.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.830+0000 7f8f67fff640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8f500b6560 0x7f8f500b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:45.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.830+0000 7f8f657fa640 1 -- 172.21.15.112:0/2087442977 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f8f5c107dd0 con 0x7f8f68108300 2024-09-21T03:28:45.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.830+0000 7f8f67fff640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8f500b6560 0x7f8f500b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f580058c0 tx=0x7f8f5800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:45.833+0000 7f8f657fa640 1 -- 172.21.15.112:0/2087442977 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8f5c0d2980 con 0x7f8f68108300 2024-09-21T03:28:46.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.047+0000 7f8f6e5da640 1 -- 172.21.15.112:0/2087442977 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f8f6810aa20 con 0x7f8f68108300 2024-09-21T03:28:46.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.427+0000 7f8f657fa640 1 -- 172.21.15.112:0/2087442977 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f8f5c0d78c0 con 0x7f8f68108300 2024-09-21T03:28:46.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.429+0000 7f8f6e5da640 1 -- 172.21.15.112:0/2087442977 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8f500b6560 msgr2=0x7f8f500b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.429+0000 7f8f6e5da640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8f500b6560 0x7f8f500b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f580058c0 tx=0x7f8f5800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.429+0000 7f8f6e5da640 1 -- 172.21.15.112:0/2087442977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f68108300 msgr2=0x7f8f6819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.429+0000 7f8f6e5da640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f68108300 0x7f8f6819a9f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c02a930 tx=0x7f8f5c004270 comp rx=0 tx=0).stop 2024-09-21T03:28:46.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.429+0000 7f8f6e5da640 1 -- 172.21.15.112:0/2087442977 shutdown_connections 2024-09-21T03:28:46.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.429+0000 7f8f6e5da640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8f500b6560 0x7f8f500b8a70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.429+0000 7f8f6e5da640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f68108c50 0x7f8f6819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.429+0000 7f8f6e5da640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f68108300 0x7f8f6819a9f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.429+0000 7f8f6e5da640 1 --2- 172.21.15.112:0/2087442977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f681029d0 0x7f8f6819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.429+0000 7f8f6e5da640 1 -- 172.21.15.112:0/2087442977 >> 172.21.15.112:0/2087442977 conn(0x7f8f680fe3b0 msgr2=0x7f8f681007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:46.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.429+0000 7f8f6e5da640 1 -- 172.21.15.112:0/2087442977 shutdown_connections 2024-09-21T03:28:46.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.430+0000 7f8f6e5da640 1 -- 172.21.15.112:0/2087442977 wait complete. 2024-09-21T03:28:46.443 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T03:28:46.454 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T03:28:46.495 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:28:46.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.603+0000 7f080707e640 1 -- 172.21.15.112:0/3073860321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800102980 msgr2=0x7f0800102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.603+0000 7f080707e640 1 --2- 172.21.15.112:0/3073860321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800102980 0x7f0800102d90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009900 tx=0x7f07f002a410 comp rx=0 tx=0).stop 2024-09-21T03:28:46.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.603+0000 7f080707e640 1 -- 172.21.15.112:0/3073860321 shutdown_connections 2024-09-21T03:28:46.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.603+0000 7f080707e640 1 --2- 172.21.15.112:0/3073860321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0800108c00 0x7f080010f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.603+0000 7f080707e640 1 --2- 172.21.15.112:0/3073860321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08001082b0 0x7f08001086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.603+0000 7f080707e640 1 --2- 172.21.15.112:0/3073860321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800102980 0x7f0800102d90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.603+0000 7f080707e640 1 -- 172.21.15.112:0/3073860321 >> 172.21.15.112:0/3073860321 conn(0x7f08000fe380 msgr2=0x7f08001007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:46.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.603+0000 7f080707e640 1 -- 172.21.15.112:0/3073860321 shutdown_connections 2024-09-21T03:28:46.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.603+0000 7f080707e640 1 -- 172.21.15.112:0/3073860321 wait complete. 2024-09-21T03:28:46.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080707e640 1 Processor -- start 2024-09-21T03:28:46.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080707e640 1 -- start start 2024-09-21T03:28:46.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080707e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0800102980 0x7f080019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080707e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08001082b0 0x7f080019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080707e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800108c00 0x7f080019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080707e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08001128c0 con 0x7f0800108c00 2024-09-21T03:28:46.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080707e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0800112740 con 0x7f08001082b0 2024-09-21T03:28:46.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080707e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0800112a40 con 0x7f0800102980 2024-09-21T03:28:46.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080561b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800108c00 0x7f080019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080561b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800108c00 0x7f080019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58298/0 (socket says 172.21.15.112:58298) 2024-09-21T03:28:46.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080561b640 1 -- 172.21.15.112:0/3698020960 learned_addr learned my addr 172.21.15.112:0/3698020960 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f07fffff640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08001082b0 0x7f080019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f0804e1a640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0800102980 0x7f080019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080561b640 1 -- 172.21.15.112:0/3698020960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0800102980 msgr2=0x7f080019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080561b640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0800102980 0x7f080019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080561b640 1 -- 172.21.15.112:0/3698020960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08001082b0 msgr2=0x7f080019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080561b640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08001082b0 0x7f080019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.604+0000 7f080561b640 1 -- 172.21.15.112:0/3698020960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08001a0300 con 0x7f0800108c00 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.605+0000 7f080561b640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800108c00 0x7f080019fc00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f07f800eaa0 tx=0x7f07f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.605+0000 7f07fdffb640 1 -- 172.21.15.112:0/3698020960 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f800cdb0 con 0x7f0800108c00 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.605+0000 7f07fdffb640 1 -- 172.21.15.112:0/3698020960 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f8005020 con 0x7f0800108c00 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.605+0000 7f080707e640 1 -- 172.21.15.112:0/3698020960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08001a05f0 con 0x7f0800108c00 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.605+0000 7f07fdffb640 1 -- 172.21.15.112:0/3698020960 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f80116d0 con 0x7f0800108c00 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.605+0000 7f080707e640 1 -- 172.21.15.112:0/3698020960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08001a6d80 con 0x7f0800108c00 2024-09-21T03:28:46.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.606+0000 7f080707e640 1 -- 172.21.15.112:0/3698020960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0800103bf0 con 0x7f0800108c00 2024-09-21T03:28:46.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.607+0000 7f07fdffb640 1 -- 172.21.15.112:0/3698020960 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f07f80051d0 con 0x7f0800108c00 2024-09-21T03:28:46.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.608+0000 7f07fdffb640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07ec0b6560 0x7f07ec0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.608+0000 7f07fdffb640 1 -- 172.21.15.112:0/3698020960 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f07f80e75c0 con 0x7f0800108c00 2024-09-21T03:28:46.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.608+0000 7f0804e1a640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07ec0b6560 0x7f07ec0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.608+0000 7f0804e1a640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07ec0b6560 0x7f07ec0b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f07f0031440 tx=0x7f07f0005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:46.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.612+0000 7f07fdffb640 1 -- 172.21.15.112:0/3698020960 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07f8016030 con 0x7f0800108c00 2024-09-21T03:28:46.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:46 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2087442977' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:28:46.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:46 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2087442977' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:28:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.791+0000 7f080707e640 1 -- 172.21.15.112:0/3698020960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08001a70c0 con 0x7f0800108c00 2024-09-21T03:28:46.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.792+0000 7f07fdffb640 1 -- 172.21.15.112:0/3698020960 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23065 (secure 0 0 0) 0x7f07f80add90 con 0x7f0800108c00 2024-09-21T03:28:46.794 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:46.794 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":76,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:27:33.414969+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:26:58.087349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:26:59.970444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:28:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.794+0000 7f080707e640 1 -- 172.21.15.112:0/3698020960 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07ec0b6560 msgr2=0x7f07ec0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.794+0000 7f080707e640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07ec0b6560 0x7f07ec0b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f07f0031440 tx=0x7f07f0005d50 comp rx=0 tx=0).stop 2024-09-21T03:28:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.795+0000 7f080707e640 1 -- 172.21.15.112:0/3698020960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800108c00 msgr2=0x7f080019fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.795+0000 7f080707e640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800108c00 0x7f080019fc00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f07f800eaa0 tx=0x7f07f800ef60 comp rx=0 tx=0).stop 2024-09-21T03:28:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.795+0000 7f080707e640 1 -- 172.21.15.112:0/3698020960 shutdown_connections 2024-09-21T03:28:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.795+0000 7f080707e640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07ec0b6560 0x7f07ec0b8a70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.795+0000 7f080707e640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800108c00 0x7f080019fc00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.795+0000 7f080707e640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08001082b0 0x7f080019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.795+0000 7f080707e640 1 --2- 172.21.15.112:0/3698020960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0800102980 0x7f080019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.795+0000 7f080707e640 1 -- 172.21.15.112:0/3698020960 >> 172.21.15.112:0/3698020960 conn(0x7f08000fe380 msgr2=0x7f08001078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.795+0000 7f080707e640 1 -- 172.21.15.112:0/3698020960 shutdown_connections 2024-09-21T03:28:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.795+0000 7f080707e640 1 -- 172.21.15.112:0/3698020960 wait complete. 2024-09-21T03:28:46.808 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:22:30.533177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T03:26:58.087349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T03:26:59.970444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:28:46.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:28:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:46 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2087442977' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T03:28:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.918+0000 7f81c68a4640 1 -- 172.21.15.112:0/1455933845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c0104950 msgr2=0x7f81c0104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.918+0000 7f81c68a4640 1 --2- 172.21.15.112:0/1455933845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c0104950 0x7f81c0104d60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009900 tx=0x7f81b002a410 comp rx=0 tx=0).stop 2024-09-21T03:28:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.919+0000 7f81c68a4640 1 -- 172.21.15.112:0/1455933845 shutdown_connections 2024-09-21T03:28:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.919+0000 7f81c68a4640 1 --2- 172.21.15.112:0/1455933845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c010dd70 0x7f81c0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.919+0000 7f81c68a4640 1 --2- 172.21.15.112:0/1455933845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81c01052a0 0x7f81c010d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.919+0000 7f81c68a4640 1 --2- 172.21.15.112:0/1455933845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c0104950 0x7f81c0104d60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.919+0000 7f81c68a4640 1 -- 172.21.15.112:0/1455933845 >> 172.21.15.112:0/1455933845 conn(0x7f81c00fe3c0 msgr2=0x7f81c0100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.919+0000 7f81c68a4640 1 -- 172.21.15.112:0/1455933845 shutdown_connections 2024-09-21T03:28:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.919+0000 7f81c68a4640 1 -- 172.21.15.112:0/1455933845 wait complete. 2024-09-21T03:28:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.919+0000 7f81c68a4640 1 Processor -- start 2024-09-21T03:28:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.919+0000 7f81c68a4640 1 -- start start 2024-09-21T03:28:46.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81c68a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c0104950 0x7f81c019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81c68a4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c01052a0 0x7f81c019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81c68a4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81c010dd70 0x7f81c019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81c68a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81c01139e0 con 0x7f81c0104950 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81c68a4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f81c0113860 con 0x7f81c01052a0 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81c68a4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81c0113b60 con 0x7f81c010dd70 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c0104950 0x7f81c019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c0104950 0x7f81c019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58312/0 (socket says 172.21.15.112:58312) 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81bffff640 1 -- 172.21.15.112:0/623357055 learned_addr learned my addr 172.21.15.112:0/623357055 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81c4e41640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81c010dd70 0x7f81c019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81bf7fe640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c01052a0 0x7f81c019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81bffff640 1 -- 172.21.15.112:0/623357055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81c010dd70 msgr2=0x7f81c019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81bffff640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81c010dd70 0x7f81c019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81bffff640 1 -- 172.21.15.112:0/623357055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c01052a0 msgr2=0x7f81c019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81bffff640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c01052a0 0x7f81c019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81bffff640 1 -- 172.21.15.112:0/623357055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c01a0390 con 0x7f81c0104950 2024-09-21T03:28:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81c4e41640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81c010dd70 0x7f81c019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81bffff640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c0104950 0x7f81c019a460 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f81b0005940 tx=0x7f81b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.920+0000 7f81bd7fa640 1 -- 172.21.15.112:0/623357055 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b003f030 con 0x7f81c0104950 2024-09-21T03:28:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.921+0000 7f81c68a4640 1 -- 172.21.15.112:0/623357055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c01a0620 con 0x7f81c0104950 2024-09-21T03:28:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.921+0000 7f81bd7fa640 1 -- 172.21.15.112:0/623357055 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b0003cb0 con 0x7f81c0104950 2024-09-21T03:28:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.921+0000 7f81bd7fa640 1 -- 172.21.15.112:0/623357055 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0035070 con 0x7f81c0104950 2024-09-21T03:28:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.921+0000 7f81c68a4640 1 -- 172.21.15.112:0/623357055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c01a6e10 con 0x7f81c0104950 2024-09-21T03:28:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.922+0000 7f81c68a4640 1 -- 172.21.15.112:0/623357055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c006bf40 con 0x7f81c0104950 2024-09-21T03:28:46.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.923+0000 7f81bd7fa640 1 -- 172.21.15.112:0/623357055 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f81b0042030 con 0x7f81c0104950 2024-09-21T03:28:46.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.924+0000 7f81bd7fa640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81a80b6560 0x7f81a80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:46.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.924+0000 7f81bf7fe640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81a80b6560 0x7f81a80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:46.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.924+0000 7f81bd7fa640 1 -- 172.21.15.112:0/623357055 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f81b0108030 con 0x7f81c0104950 2024-09-21T03:28:46.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.924+0000 7f81bf7fe640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81a80b6560 0x7f81a80b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f81b4005fd0 tx=0x7f81b4005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:46.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:46.927+0000 7f81bd7fa640 1 -- 172.21.15.112:0/623357055 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81b00ce800 con 0x7f81c0104950 2024-09-21T03:28:47.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.104+0000 7f81c68a4640 1 -- 172.21.15.112:0/623357055 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81c01a6fe0 con 0x7f81c0104950 2024-09-21T03:28:47.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.105+0000 7f81bd7fa640 1 -- 172.21.15.112:0/623357055 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f81b00d2be0 con 0x7f81c0104950 2024-09-21T03:28:47.107 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-09-21T03:28:47.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 -- 172.21.15.112:0/623357055 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81a80b6560 msgr2=0x7f81a80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81a80b6560 0x7f81a80b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f81b4005fd0 tx=0x7f81b4005850 comp rx=0 tx=0).stop 2024-09-21T03:28:47.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 -- 172.21.15.112:0/623357055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c0104950 msgr2=0x7f81c019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c0104950 0x7f81c019a460 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f81b0005940 tx=0x7f81b00042f0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 -- 172.21.15.112:0/623357055 shutdown_connections 2024-09-21T03:28:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81a80b6560 0x7f81a80b8a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81c010dd70 0x7f81c019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81c01052a0 0x7f81c019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 --2- 172.21.15.112:0/623357055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81c0104950 0x7f81c019a460 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 -- 172.21.15.112:0/623357055 >> 172.21.15.112:0/623357055 conn(0x7f81c00fe3c0 msgr2=0x7f81c01078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 -- 172.21.15.112:0/623357055 shutdown_connections 2024-09-21T03:28:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.107+0000 7f81c68a4640 1 -- 172.21.15.112:0/623357055 wait complete. 2024-09-21T03:28:47.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:28:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.230+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2786600197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc104900 msgr2=0x7f81dc104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.230+0000 7f81e3b2b640 1 --2- 172.21.15.112:0/2786600197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc104900 0x7f81dc104d10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009900 tx=0x7f81d802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.230+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2786600197 shutdown_connections 2024-09-21T03:28:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.230+0000 7f81e3b2b640 1 --2- 172.21.15.112:0/2786600197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81dc10dd60 0x7f81dc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.231+0000 7f81e3b2b640 1 --2- 172.21.15.112:0/2786600197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc105250 0x7f81dc10d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.231+0000 7f81e3b2b640 1 --2- 172.21.15.112:0/2786600197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc104900 0x7f81dc104d10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.231+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2786600197 >> 172.21.15.112:0/2786600197 conn(0x7f81dc0fe3d0 msgr2=0x7f81dc100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.231+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2786600197 shutdown_connections 2024-09-21T03:28:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.231+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2786600197 wait complete. 2024-09-21T03:28:47.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.231+0000 7f81e3b2b640 1 Processor -- start 2024-09-21T03:28:47.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.231+0000 7f81e3b2b640 1 -- start start 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.231+0000 7f81e3b2b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81dc104900 0x7f81dc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.231+0000 7f81e3b2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc105250 0x7f81dc19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.231+0000 7f81e3b2b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc10dd60 0x7f81dc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e3b2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1139e0 con 0x7f81dc105250 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e3b2b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc113860 con 0x7f81dc10dd60 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e3b2b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc113b60 con 0x7f81dc104900 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e10c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc105250 0x7f81dc19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e10c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc105250 0x7f81dc19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58322/0 (socket says 172.21.15.112:58322) 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e10c6640 1 -- 172.21.15.112:0/2944778875 learned_addr learned my addr 172.21.15.112:0/2944778875 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e18c7640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81dc104900 0x7f81dc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e10c6640 1 -- 172.21.15.112:0/2944778875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81dc104900 msgr2=0x7f81dc19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e10c6640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81dc104900 0x7f81dc19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e10c6640 1 -- 172.21.15.112:0/2944778875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc10dd60 msgr2=0x7f81dc19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e20c8640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc10dd60 0x7f81dc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e10c6640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc10dd60 0x7f81dc19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e10c6640 1 -- 172.21.15.112:0/2944778875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc1a03b0 con 0x7f81dc105250 2024-09-21T03:28:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e10c6640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc105250 0x7f81dc19aa20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00d740 tx=0x7f81cc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81d2ffd640 1 -- 172.21.15.112:0/2944778875 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc017070 con 0x7f81dc105250 2024-09-21T03:28:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81d2ffd640 1 -- 172.21.15.112:0/2944778875 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81cc00de50 con 0x7f81dc105250 2024-09-21T03:28:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81d2ffd640 1 -- 172.21.15.112:0/2944778875 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00f620 con 0x7f81dc105250 2024-09-21T03:28:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2944778875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc1a06a0 con 0x7f81dc105250 2024-09-21T03:28:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.232+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2944778875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc1a6e30 con 0x7f81dc105250 2024-09-21T03:28:47.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.233+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2944778875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc104d10 con 0x7f81dc105250 2024-09-21T03:28:47.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.235+0000 7f81d2ffd640 1 -- 172.21.15.112:0/2944778875 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f81cc01a070 con 0x7f81dc105250 2024-09-21T03:28:47.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.235+0000 7f81d2ffd640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81c80b6560 0x7f81c80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:47.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.236+0000 7f81e18c7640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81c80b6560 0x7f81c80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:47.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.236+0000 7f81d2ffd640 1 -- 172.21.15.112:0/2944778875 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f81cc0e6ee0 con 0x7f81dc105250 2024-09-21T03:28:47.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.236+0000 7f81e18c7640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81c80b6560 0x7f81c80b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f81d8031440 tx=0x7f81d8005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:47.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.240+0000 7f81d2ffd640 1 -- 172.21.15.112:0/2944778875 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81cc0ad6b0 con 0x7f81dc105250 2024-09-21T03:28:47.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.420+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2944778875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f81dc1a7120 con 0x7f81dc105250 2024-09-21T03:28:47.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.423+0000 7f81d2ffd640 1 -- 172.21.15.112:0/2944778875 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) ==== 104+0+11 (secure 0 0 0) 0x7f81cc0b1a90 con 0x7f81dc105250 2024-09-21T03:28:47.425 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-09-21T03:28:47.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2944778875 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81c80b6560 msgr2=0x7f81c80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81c80b6560 0x7f81c80b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f81d8031440 tx=0x7f81d8005d50 comp rx=0 tx=0).stop 2024-09-21T03:28:47.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2944778875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc105250 msgr2=0x7f81dc19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc105250 0x7f81dc19aa20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00d740 tx=0x7f81cc00dc00 comp rx=0 tx=0).stop 2024-09-21T03:28:47.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2944778875 shutdown_connections 2024-09-21T03:28:47.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f81c80b6560 0x7f81c80b8a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81dc10dd60 0x7f81dc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81dc105250 0x7f81dc19aa20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 --2- 172.21.15.112:0/2944778875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81dc104900 0x7f81dc19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2944778875 >> 172.21.15.112:0/2944778875 conn(0x7f81dc0fe3d0 msgr2=0x7f81dc100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:47.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2944778875 shutdown_connections 2024-09-21T03:28:47.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.425+0000 7f81e3b2b640 1 -- 172.21.15.112:0/2944778875 wait complete. 2024-09-21T03:28:47.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:28:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.549+0000 7fcd503c3640 1 -- 172.21.15.112:0/1900308301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd48104910 msgr2=0x7fcd48104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.549+0000 7fcd503c3640 1 --2- 172.21.15.112:0/1900308301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd48104910 0x7fcd48104d20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009900 tx=0x7fcd4402a410 comp rx=0 tx=0).stop 2024-09-21T03:28:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.550+0000 7fcd503c3640 1 -- 172.21.15.112:0/1900308301 shutdown_connections 2024-09-21T03:28:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.550+0000 7fcd503c3640 1 --2- 172.21.15.112:0/1900308301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd4810dd70 0x7fcd48110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.550+0000 7fcd503c3640 1 --2- 172.21.15.112:0/1900308301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd48105260 0x7fcd4810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.550+0000 7fcd503c3640 1 --2- 172.21.15.112:0/1900308301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd48104910 0x7fcd48104d20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.550+0000 7fcd503c3640 1 -- 172.21.15.112:0/1900308301 >> 172.21.15.112:0/1900308301 conn(0x7fcd480fe400 msgr2=0x7fcd48100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.550+0000 7fcd503c3640 1 -- 172.21.15.112:0/1900308301 shutdown_connections 2024-09-21T03:28:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.550+0000 7fcd503c3640 1 -- 172.21.15.112:0/1900308301 wait complete. 2024-09-21T03:28:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.550+0000 7fcd503c3640 1 Processor -- start 2024-09-21T03:28:47.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.550+0000 7fcd503c3640 1 -- start start 2024-09-21T03:28:47.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd503c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd48104910 0x7fcd48102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:47.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd503c3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd48105260 0x7fcd48102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:47.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd503c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4810dd70 0x7fcd481030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:47.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd503c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcd48113a50 con 0x7fcd48104910 2024-09-21T03:28:47.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd503c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcd481138d0 con 0x7fcd4810dd70 2024-09-21T03:28:47.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd503c3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd48113bd0 con 0x7fcd48105260 2024-09-21T03:28:47.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4e15f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd48104910 0x7fcd48102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:47.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4e15f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd48104910 0x7fcd48102640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58354/0 (socket says 172.21.15.112:58354) 2024-09-21T03:28:47.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4e15f640 1 -- 172.21.15.112:0/3822918651 learned_addr learned my addr 172.21.15.112:0/3822918651 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:47.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4d95e640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd48105260 0x7fcd48102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:47.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4e960640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4810dd70 0x7fcd481030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:47.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4e15f640 1 -- 172.21.15.112:0/3822918651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd48105260 msgr2=0x7fcd48102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4e15f640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd48105260 0x7fcd48102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4e15f640 1 -- 172.21.15.112:0/3822918651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4810dd70 msgr2=0x7fcd481030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4e15f640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4810dd70 0x7fcd481030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4e15f640 1 -- 172.21.15.112:0/3822918651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd481a8720 con 0x7fcd48104910 2024-09-21T03:28:47.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4e960640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4810dd70 0x7fcd481030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:47.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd4e15f640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd48104910 0x7fcd48102640 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcd44006010 tx=0x7fcd440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:47.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd3b7fe640 1 -- 172.21.15.112:0/3822918651 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4403f030 con 0x7fcd48104910 2024-09-21T03:28:47.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd3b7fe640 1 -- 172.21.15.112:0/3822918651 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd44003800 con 0x7fcd48104910 2024-09-21T03:28:47.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd503c3640 1 -- 172.21.15.112:0/3822918651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd481a89b0 con 0x7fcd48104910 2024-09-21T03:28:47.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd3b7fe640 1 -- 172.21.15.112:0/3822918651 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44035070 con 0x7fcd48104910 2024-09-21T03:28:47.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.551+0000 7fcd503c3640 1 -- 172.21.15.112:0/3822918651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd481a8f70 con 0x7fcd48104910 2024-09-21T03:28:47.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.553+0000 7fcd503c3640 1 -- 172.21.15.112:0/3822918651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd48104d20 con 0x7fcd48104910 2024-09-21T03:28:47.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.554+0000 7fcd3b7fe640 1 -- 172.21.15.112:0/3822918651 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7fcd44003cb0 con 0x7fcd48104910 2024-09-21T03:28:47.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.555+0000 7fcd3b7fe640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd340b6610 0x7fcd340b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:47.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.555+0000 7fcd4d95e640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd340b6610 0x7fcd340b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:47.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.555+0000 7fcd3b7fe640 1 -- 172.21.15.112:0/3822918651 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fcd44107f10 con 0x7fcd48104910 2024-09-21T03:28:47.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.555+0000 7fcd4d95e640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd340b6610 0x7fcd340b8b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcd48100b10 tx=0x7fcd3c00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:47.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.558+0000 7fcd3b7fe640 1 -- 172.21.15.112:0/3822918651 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcd440ce6e0 con 0x7fcd48104910 2024-09-21T03:28:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:47 smithi187 ceph-mon[34603]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:28:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:47 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2087442977' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T03:28:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:47 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:28:47.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:47 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3698020960' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:47.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:47 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/623357055' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:28:47.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.734+0000 7fcd503c3640 1 -- 172.21.15.112:0/3822918651 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcd481a9340 con 0x7fcd48104910 2024-09-21T03:28:47.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.735+0000 7fcd3b7fe640 1 -- 172.21.15.112:0/3822918651 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) ==== 100+0+11 (secure 0 0 0) 0x7fcd440d2ac0 con 0x7fcd48104910 2024-09-21T03:28:47.737 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-09-21T03:28:47.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 -- 172.21.15.112:0/3822918651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd340b6610 msgr2=0x7fcd340b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd340b6610 0x7fcd340b8b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcd48100b10 tx=0x7fcd3c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 -- 172.21.15.112:0/3822918651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd48104910 msgr2=0x7fcd48102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:47.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd48104910 0x7fcd48102640 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcd44006010 tx=0x7fcd440042f0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 -- 172.21.15.112:0/3822918651 shutdown_connections 2024-09-21T03:28:47.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcd340b6610 0x7fcd340b8b20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4810dd70 0x7fcd481030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd48105260 0x7fcd48102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 --2- 172.21.15.112:0/3822918651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd48104910 0x7fcd48102640 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:47.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 -- 172.21.15.112:0/3822918651 >> 172.21.15.112:0/3822918651 conn(0x7fcd480fe400 msgr2=0x7fcd48107810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:47.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 -- 172.21.15.112:0/3822918651 shutdown_connections 2024-09-21T03:28:47.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:47.737+0000 7fcd503c3640 1 -- 172.21.15.112:0/3822918651 wait complete. 2024-09-21T03:28:47.749 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:47 smithi112 ceph-mon[31442]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:28:47.749 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:47 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2087442977' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T03:28:47.750 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:47 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:28:47.750 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:47 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3698020960' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:47.750 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:47 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/623357055' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:28:47.751 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T03:28:47.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:47.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:47 smithi203 ceph-mon[34723]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T03:28:47.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:47 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2087442977' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T03:28:47.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:47 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:28:47.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:47 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3698020960' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:47.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:47 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/623357055' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:28:47.999 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.494+0000 7f17ad670640 1 -- 172.21.15.112:0/2787750247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17a8109e50 msgr2=0x7f17a8110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.494+0000 7f17ad670640 1 --2- 172.21.15.112:0/2787750247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17a8109e50 0x7f17a8110680 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009ca0 tx=0x7f17a002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.494+0000 7f17ad670640 1 -- 172.21.15.112:0/2787750247 shutdown_connections 2024-09-21T03:28:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.494+0000 7f17ad670640 1 --2- 172.21.15.112:0/2787750247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17a8109e50 0x7f17a8110680 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.494+0000 7f17ad670640 1 --2- 172.21.15.112:0/2787750247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a8109500 0x7f17a8109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.494+0000 7f17ad670640 1 --2- 172.21.15.112:0/2787750247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17a8101fc0 0x7f17a81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.494+0000 7f17ad670640 1 -- 172.21.15.112:0/2787750247 >> 172.21.15.112:0/2787750247 conn(0x7f17a8079f90 msgr2=0x7f17a807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:48.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.494+0000 7f17ad670640 1 -- 172.21.15.112:0/2787750247 shutdown_connections 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.494+0000 7f17ad670640 1 -- 172.21.15.112:0/2787750247 wait complete. 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17ad670640 1 Processor -- start 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17ad670640 1 -- start start 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17ad670640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a8101fc0 0x7f17a8102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17ad670640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17a8109500 0x7f17a81034a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17ad670640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17a8109e50 0x7f17a81039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17ad670640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8113b60 con 0x7f17a8109e50 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17ad670640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81139e0 con 0x7f17a8101fc0 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17ad670640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8113ce0 con 0x7f17a8109500 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17a8109e50 0x7f17a81039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17a8109e50 0x7f17a81039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58360/0 (socket says 172.21.15.112:58360) 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17a77fe640 1 -- 172.21.15.112:0/3645941536 learned_addr learned my addr 172.21.15.112:0/3645941536 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:48.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17a6ffd640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a8101fc0 0x7f17a8102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f179ffff640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17a8109500 0x7f17a81034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17a77fe640 1 -- 172.21.15.112:0/3645941536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17a8109500 msgr2=0x7f17a81034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17a77fe640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17a8109500 0x7f17a81034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17a77fe640 1 -- 172.21.15.112:0/3645941536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a8101fc0 msgr2=0x7f17a8102f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17a77fe640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a8101fc0 0x7f17a8102f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f17a77fe640 1 -- 172.21.15.112:0/3645941536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a81a86f0 con 0x7f17a8109e50 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.495+0000 7f179ffff640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17a8109500 0x7f17a81034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.496+0000 7f17a77fe640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17a8109e50 0x7f17a81039e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009430 tx=0x7f17a00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.496+0000 7f17a4ff9640 1 -- 172.21.15.112:0/3645941536 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0003c30 con 0x7f17a8109e50 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.496+0000 7f17a4ff9640 1 -- 172.21.15.112:0/3645941536 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a0003dd0 con 0x7f17a8109e50 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.496+0000 7f17ad670640 1 -- 172.21.15.112:0/3645941536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a81a8980 con 0x7f17a8109e50 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.496+0000 7f17a4ff9640 1 -- 172.21.15.112:0/3645941536 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a002faa0 con 0x7f17a8109e50 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.496+0000 7f17ad670640 1 -- 172.21.15.112:0/3645941536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a81a8f10 con 0x7f17a8109e50 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.497+0000 7f17ad670640 1 -- 172.21.15.112:0/3645941536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a806b910 con 0x7f17a8109e50 2024-09-21T03:28:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.498+0000 7f17a4ff9640 1 -- 172.21.15.112:0/3645941536 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f17a002fc40 con 0x7f17a8109e50 2024-09-21T03:28:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.499+0000 7f17a4ff9640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f17940b65e0 0x7f17940b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.500+0000 7f17a6ffd640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f17940b65e0 0x7f17940b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.500+0000 7f17a4ff9640 1 -- 172.21.15.112:0/3645941536 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f17a01088b0 con 0x7f17a8109e50 2024-09-21T03:28:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.500+0000 7f17a6ffd640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f17940b65e0 0x7f17940b8af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f17980058c0 tx=0x7f179800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.503+0000 7f17a4ff9640 1 -- 172.21.15.112:0/3645941536 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f17a00cf080 con 0x7f17a8109e50 2024-09-21T03:28:48.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:48 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2944778875' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:28:48.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:48 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3822918651' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:28:48.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.716+0000 7f17ad670640 1 -- 172.21.15.112:0/3645941536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17a8104260 con 0x7f17a8109e50 2024-09-21T03:28:48.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:48 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2944778875' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:28:48.719 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:48 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3822918651' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:28:48.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.717+0000 7f17a4ff9640 1 -- 172.21.15.112:0/3645941536 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11929 (secure 0 0 0) 0x7f17a8104260 con 0x7f17a8109e50 2024-09-21T03:28:48.720 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:48.720 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"btime":"2024-09-21T03:28:46:418112+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:28:46.048810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:28:48.720 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-09-21T03:28:48.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.719+0000 7f17ad670640 1 -- 172.21.15.112:0/3645941536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f17940b65e0 msgr2=0x7f17940b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:48.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.719+0000 7f17ad670640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f17940b65e0 0x7f17940b8af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f17980058c0 tx=0x7f179800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.719+0000 7f17ad670640 1 -- 172.21.15.112:0/3645941536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17a8109e50 msgr2=0x7f17a81039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:48.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.719+0000 7f17ad670640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17a8109e50 0x7f17a81039e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009430 tx=0x7f17a00319f0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.719+0000 7f17ad670640 1 -- 172.21.15.112:0/3645941536 shutdown_connections 2024-09-21T03:28:48.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.719+0000 7f17ad670640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f17940b65e0 0x7f17940b8af0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.719+0000 7f17ad670640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17a8109e50 0x7f17a81039e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.719+0000 7f17ad670640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17a8109500 0x7f17a81034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.719+0000 7f17ad670640 1 --2- 172.21.15.112:0/3645941536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a8101fc0 0x7f17a8102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:48.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.719+0000 7f17ad670640 1 -- 172.21.15.112:0/3645941536 >> 172.21.15.112:0/3645941536 conn(0x7f17a8079f90 msgr2=0x7f17a81074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:48.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.720+0000 7f17ad670640 1 -- 172.21.15.112:0/3645941536 shutdown_connections 2024-09-21T03:28:48.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:48.720+0000 7f17ad670640 1 -- 172.21.15.112:0/3645941536 wait complete. 2024-09-21T03:28:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:48 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2944778875' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:28:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:48 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3822918651' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:28:49.523 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T03:28:49.523 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T03:28:49.524 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:28:49.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.639+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3108578863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d81082e0 msgr2=0x7f25d81086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:49.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.639+0000 7f25dd0f9640 1 --2- 172.21.15.112:0/3108578863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d81082e0 0x7f25d81086f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009900 tx=0x7f25cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:49.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.639+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3108578863 shutdown_connections 2024-09-21T03:28:49.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.639+0000 7f25dd0f9640 1 --2- 172.21.15.112:0/3108578863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d8108c30 0x7f25d810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.639+0000 7f25dd0f9640 1 --2- 172.21.15.112:0/3108578863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d81082e0 0x7f25d81086f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.639+0000 7f25dd0f9640 1 --2- 172.21.15.112:0/3108578863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25d81029b0 0x7f25d8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.639+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3108578863 >> 172.21.15.112:0/3108578863 conn(0x7f25d807a670 msgr2=0x7f25d807aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:49.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.639+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3108578863 shutdown_connections 2024-09-21T03:28:49.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.639+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3108578863 wait complete. 2024-09-21T03:28:49.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.639+0000 7f25dd0f9640 1 Processor -- start 2024-09-21T03:28:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25dd0f9640 1 -- start start 2024-09-21T03:28:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25dd0f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d81029b0 0x7f25d819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25dd0f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25d81082e0 0x7f25d819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25dd0f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d8108c30 0x7f25d819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25dd0f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81128f0 con 0x7f25d8108c30 2024-09-21T03:28:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25dd0f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8112770 con 0x7f25d81029b0 2024-09-21T03:28:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25dd0f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8112a70 con 0x7f25d81082e0 2024-09-21T03:28:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25d759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d8108c30 0x7f25d819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25d759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d8108c30 0x7f25d819bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58378/0 (socket says 172.21.15.112:58378) 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25d759e640 1 -- 172.21.15.112:0/3891569207 learned_addr learned my addr 172.21.15.112:0/3891569207 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25d6d9d640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d81029b0 0x7f25d819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25d659c640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25d81082e0 0x7f25d819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25d759e640 1 -- 172.21.15.112:0/3891569207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25d81082e0 msgr2=0x7f25d819b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25d759e640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25d81082e0 0x7f25d819b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25d759e640 1 -- 172.21.15.112:0/3891569207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d81029b0 msgr2=0x7f25d819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25d759e640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d81029b0 0x7f25d819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.640+0000 7f25d759e640 1 -- 172.21.15.112:0/3891569207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d81a64c0 con 0x7f25d8108c30 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.641+0000 7f25d659c640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25d81082e0 0x7f25d819b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.641+0000 7f25d759e640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d8108c30 0x7f25d819bc80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f25d000c970 tx=0x7f25d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.641+0000 7f25bffff640 1 -- 172.21.15.112:0/3891569207 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0007bb0 con 0x7f25d8108c30 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.641+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3891569207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d81a67b0 con 0x7f25d8108c30 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.641+0000 7f25bffff640 1 -- 172.21.15.112:0/3891569207 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d0007d50 con 0x7f25d8108c30 2024-09-21T03:28:49.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.641+0000 7f25bffff640 1 -- 172.21.15.112:0/3891569207 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0004e60 con 0x7f25d8108c30 2024-09-21T03:28:49.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.641+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3891569207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d81a6cb0 con 0x7f25d8108c30 2024-09-21T03:28:49.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.642+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3891569207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d8102dc0 con 0x7f25d8108c30 2024-09-21T03:28:49.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.643+0000 7f25bffff640 1 -- 172.21.15.112:0/3891569207 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f25d0005000 con 0x7f25d8108c30 2024-09-21T03:28:49.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.644+0000 7f25bffff640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c00b6560 0x7f25c00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:49.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.644+0000 7f25d6d9d640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c00b6560 0x7f25c00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:49.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.644+0000 7f25bffff640 1 -- 172.21.15.112:0/3891569207 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f25d00e75b0 con 0x7f25d8108c30 2024-09-21T03:28:49.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.645+0000 7f25d6d9d640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c00b6560 0x7f25c00b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f25d806c470 tx=0x7f25c80094a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:49.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.647+0000 7f25bffff640 1 -- 172.21.15.112:0/3891569207 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f25d0014030 con 0x7f25d8108c30 2024-09-21T03:28:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:49 smithi112 ceph-mon[31442]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:28:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:49 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3645941536' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:49.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.825+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3891569207 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25d8063800 con 0x7f25c00b6560 2024-09-21T03:28:49.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.830+0000 7f25bffff640 1 -- 172.21.15.112:0/3891569207 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258811 (secure 0 0 0) 0x7f25d8063800 con 0x7f25c00b6560 2024-09-21T03:28:49.834 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:49.835 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T03:28:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3891569207 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c00b6560 msgr2=0x7f25c00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c00b6560 0x7f25c00b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f25d806c470 tx=0x7f25c80094a0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3891569207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d8108c30 msgr2=0x7f25d819bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d8108c30 0x7f25d819bc80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f25d000c970 tx=0x7f25d000ce30 comp rx=0 tx=0).stop 2024-09-21T03:28:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3891569207 shutdown_connections 2024-09-21T03:28:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c00b6560 0x7f25c00b8a70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25d8108c30 0x7f25d819bc80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25d81082e0 0x7f25d819b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 --2- 172.21.15.112:0/3891569207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25d81029b0 0x7f25d819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3891569207 >> 172.21.15.112:0/3891569207 conn(0x7f25d807a670 msgr2=0x7f25d8104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:49.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3891569207 shutdown_connections 2024-09-21T03:28:49.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.833+0000 7f25dd0f9640 1 -- 172.21.15.112:0/3891569207 wait complete. 2024-09-21T03:28:49.846 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-21T03:28:48.361585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659750,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":664,"num_read_kb":432,"num_write":297,"num_write_kb":2273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952574,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":739,"ondisk_log_size":739,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381976,"kb_used_data":11928,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559848,"statfs":{"total":1151940427776,"available":1151549284352,"internally_reserved":0,"allocated":12214272,"data_stored":6931434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003303"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.582283+0000","last_change":"2024-09-21T03:27:01.339980+0000","last_active":"2024-09-21T03:28:32.582283+0000","last_peered":"2024-09-21T03:28:32.582283+0000","last_clean":"2024-09-21T03:28:32.582283+0000","last_became_active":"2024-09-21T03:27:01.339552+0000","last_became_peered":"2024-09-21T03:27:01.339552+0000","last_unstale":"2024-09-21T03:28:32.582283+0000","last_undegraded":"2024-09-21T03:28:32.582283+0000","last_fullsized":"2024-09-21T03:28:32.582283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:04.240862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.577369+0000","last_change":"2024-09-21T03:27:01.334171+0000","last_active":"2024-09-21T03:28:32.577369+0000","last_peered":"2024-09-21T03:28:32.577369+0000","last_clean":"2024-09-21T03:28:32.577369+0000","last_became_active":"2024-09-21T03:27:01.333851+0000","last_became_peered":"2024-09-21T03:27:01.333851+0000","last_unstale":"2024-09-21T03:28:32.577369+0000","last_undegraded":"2024-09-21T03:28:32.577369+0000","last_fullsized":"2024-09-21T03:28:32.577369+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:49.163142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565746+0000","last_change":"2024-09-21T03:27:01.339388+0000","last_active":"2024-09-21T03:28:32.565746+0000","last_peered":"2024-09-21T03:28:32.565746+0000","last_clean":"2024-09-21T03:28:32.565746+0000","last_became_active":"2024-09-21T03:27:01.339015+0000","last_became_peered":"2024-09-21T03:27:01.339015+0000","last_unstale":"2024-09-21T03:28:32.565746+0000","last_undegraded":"2024-09-21T03:28:32.565746+0000","last_fullsized":"2024-09-21T03:28:32.565746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:06.026397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565177+0000","last_change":"2024-09-21T03:27:01.344700+0000","last_active":"2024-09-21T03:28:32.565177+0000","last_peered":"2024-09-21T03:28:32.565177+0000","last_clean":"2024-09-21T03:28:32.565177+0000","last_became_active":"2024-09-21T03:27:01.344349+0000","last_became_peered":"2024-09-21T03:27:01.344349+0000","last_unstale":"2024-09-21T03:28:32.565177+0000","last_undegraded":"2024-09-21T03:28:32.565177+0000","last_fullsized":"2024-09-21T03:28:32.565177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:00.271820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.558498+0000","last_change":"2024-09-21T03:27:01.334904+0000","last_active":"2024-09-21T03:28:32.558498+0000","last_peered":"2024-09-21T03:28:32.558498+0000","last_clean":"2024-09-21T03:28:32.558498+0000","last_became_active":"2024-09-21T03:27:01.331916+0000","last_became_peered":"2024-09-21T03:27:01.331916+0000","last_unstale":"2024-09-21T03:28:32.558498+0000","last_undegraded":"2024-09-21T03:28:32.558498+0000","last_fullsized":"2024-09-21T03:28:32.558498+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:49.376481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.559921+0000","last_change":"2024-09-21T03:27:01.343256+0000","last_active":"2024-09-21T03:28:32.559921+0000","last_peered":"2024-09-21T03:28:32.559921+0000","last_clean":"2024-09-21T03:28:32.559921+0000","last_became_active":"2024-09-21T03:27:01.342861+0000","last_became_peered":"2024-09-21T03:27:01.342861+0000","last_unstale":"2024-09-21T03:28:32.559921+0000","last_undegraded":"2024-09-21T03:28:32.559921+0000","last_fullsized":"2024-09-21T03:28:32.559921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:00.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.568926+0000","last_change":"2024-09-21T03:27:01.330618+0000","last_active":"2024-09-21T03:28:32.568926+0000","last_peered":"2024-09-21T03:28:32.568926+0000","last_clean":"2024-09-21T03:28:32.568926+0000","last_became_active":"2024-09-21T03:27:01.330018+0000","last_became_peered":"2024-09-21T03:27:01.330018+0000","last_unstale":"2024-09-21T03:28:32.568926+0000","last_undegraded":"2024-09-21T03:28:32.568926+0000","last_fullsized":"2024-09-21T03:28:32.568926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:03.291698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.569953+0000","last_change":"2024-09-21T03:27:01.332840+0000","last_active":"2024-09-21T03:28:32.569953+0000","last_peered":"2024-09-21T03:28:32.569953+0000","last_clean":"2024-09-21T03:28:32.569953+0000","last_became_active":"2024-09-21T03:27:01.332513+0000","last_became_peered":"2024-09-21T03:27:01.332513+0000","last_unstale":"2024-09-21T03:28:32.569953+0000","last_undegraded":"2024-09-21T03:28:32.569953+0000","last_fullsized":"2024-09-21T03:28:32.569953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:17.554934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.580544+0000","last_change":"2024-09-21T03:27:01.338147+0000","last_active":"2024-09-21T03:28:32.580544+0000","last_peered":"2024-09-21T03:28:32.580544+0000","last_clean":"2024-09-21T03:28:32.580544+0000","last_became_active":"2024-09-21T03:27:01.337759+0000","last_became_peered":"2024-09-21T03:27:01.337759+0000","last_unstale":"2024-09-21T03:28:32.580544+0000","last_undegraded":"2024-09-21T03:28:32.580544+0000","last_fullsized":"2024-09-21T03:28:32.580544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:38.999593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.570880+0000","last_change":"2024-09-21T03:27:01.343373+0000","last_active":"2024-09-21T03:28:32.570880+0000","last_peered":"2024-09-21T03:28:32.570880+0000","last_clean":"2024-09-21T03:28:32.570880+0000","last_became_active":"2024-09-21T03:27:01.342924+0000","last_became_peered":"2024-09-21T03:27:01.342924+0000","last_unstale":"2024-09-21T03:28:32.570880+0000","last_undegraded":"2024-09-21T03:28:32.570880+0000","last_fullsized":"2024-09-21T03:28:32.570880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:33.869584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574841+0000","last_change":"2024-09-21T03:27:01.340120+0000","last_active":"2024-09-21T03:28:32.574841+0000","last_peered":"2024-09-21T03:28:32.574841+0000","last_clean":"2024-09-21T03:28:32.574841+0000","last_became_active":"2024-09-21T03:27:01.339708+0000","last_became_peered":"2024-09-21T03:27:01.339708+0000","last_unstale":"2024-09-21T03:28:32.574841+0000","last_undegraded":"2024-09-21T03:28:32.574841+0000","last_fullsized":"2024-09-21T03:28:32.574841+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:56.193494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564924+0000","last_change":"2024-09-21T03:27:01.340205+0000","last_active":"2024-09-21T03:28:32.564924+0000","last_peered":"2024-09-21T03:28:32.564924+0000","last_clean":"2024-09-21T03:28:32.564924+0000","last_became_active":"2024-09-21T03:27:01.339839+0000","last_became_peered":"2024-09-21T03:27:01.339839+0000","last_unstale":"2024-09-21T03:28:32.564924+0000","last_undegraded":"2024-09-21T03:28:32.564924+0000","last_fullsized":"2024-09-21T03:28:32.564924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:05.248636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.568589+0000","last_change":"2024-09-21T03:27:01.334715+0000","last_active":"2024-09-21T03:28:32.568589+0000","last_peered":"2024-09-21T03:28:32.568589+0000","last_clean":"2024-09-21T03:28:32.568589+0000","last_became_active":"2024-09-21T03:27:01.334392+0000","last_became_peered":"2024-09-21T03:27:01.334392+0000","last_unstale":"2024-09-21T03:28:32.568589+0000","last_undegraded":"2024-09-21T03:28:32.568589+0000","last_fullsized":"2024-09-21T03:28:32.568589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:18.934063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.554357+0000","last_change":"2024-09-21T03:27:01.337167+0000","last_active":"2024-09-21T03:28:32.554357+0000","last_peered":"2024-09-21T03:28:32.554357+0000","last_clean":"2024-09-21T03:28:32.554357+0000","last_became_active":"2024-09-21T03:27:01.336748+0000","last_became_peered":"2024-09-21T03:27:01.336748+0000","last_unstale":"2024-09-21T03:28:32.554357+0000","last_undegraded":"2024-09-21T03:28:32.554357+0000","last_fullsized":"2024-09-21T03:28:32.554357+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:12.186727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564435+0000","last_change":"2024-09-21T03:27:01.336260+0000","last_active":"2024-09-21T03:28:32.564435+0000","last_peered":"2024-09-21T03:28:32.564435+0000","last_clean":"2024-09-21T03:28:32.564435+0000","last_became_active":"2024-09-21T03:27:01.335889+0000","last_became_peered":"2024-09-21T03:27:01.335889+0000","last_unstale":"2024-09-21T03:28:32.564435+0000","last_undegraded":"2024-09-21T03:28:32.564435+0000","last_fullsized":"2024-09-21T03:28:32.564435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:02.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.575723+0000","last_change":"2024-09-21T03:27:01.337181+0000","last_active":"2024-09-21T03:28:32.575723+0000","last_peered":"2024-09-21T03:28:32.575723+0000","last_clean":"2024-09-21T03:28:32.575723+0000","last_became_active":"2024-09-21T03:27:01.336813+0000","last_became_peered":"2024-09-21T03:27:01.336813+0000","last_unstale":"2024-09-21T03:28:32.575723+0000","last_undegraded":"2024-09-21T03:28:32.575723+0000","last_fullsized":"2024-09-21T03:28:32.575723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:04.052319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.582323+0000","last_change":"2024-09-21T03:27:01.342079+0000","last_active":"2024-09-21T03:28:32.582323+0000","last_peered":"2024-09-21T03:28:32.582323+0000","last_clean":"2024-09-21T03:28:32.582323+0000","last_became_active":"2024-09-21T03:27:01.341430+0000","last_became_peered":"2024-09-21T03:27:01.341430+0000","last_unstale":"2024-09-21T03:28:32.582323+0000","last_undegraded":"2024-09-21T03:28:32.582323+0000","last_fullsized":"2024-09-21T03:28:32.582323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:08.029978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.568191+0000","last_change":"2024-09-21T03:27:01.338694+0000","last_active":"2024-09-21T03:28:32.568191+0000","last_peered":"2024-09-21T03:28:32.568191+0000","last_clean":"2024-09-21T03:28:32.568191+0000","last_became_active":"2024-09-21T03:27:01.338328+0000","last_became_peered":"2024-09-21T03:27:01.338328+0000","last_unstale":"2024-09-21T03:28:32.568191+0000","last_undegraded":"2024-09-21T03:28:32.568191+0000","last_fullsized":"2024-09-21T03:28:32.568191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:50.292238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565193+0000","last_change":"2024-09-21T03:27:01.336974+0000","last_active":"2024-09-21T03:28:32.565193+0000","last_peered":"2024-09-21T03:28:32.565193+0000","last_clean":"2024-09-21T03:28:32.565193+0000","last_became_active":"2024-09-21T03:27:01.336592+0000","last_became_peered":"2024-09-21T03:27:01.336592+0000","last_unstale":"2024-09-21T03:28:32.565193+0000","last_undegraded":"2024-09-21T03:28:32.565193+0000","last_fullsized":"2024-09-21T03:28:32.565193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:09.399348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.579658+0000","last_change":"2024-09-21T03:27:33.431767+0000","last_active":"2024-09-21T03:28:32.579658+0000","last_peered":"2024-09-21T03:28:32.579658+0000","last_clean":"2024-09-21T03:28:32.579658+0000","last_became_active":"2024-09-21T03:27:33.431233+0000","last_became_peered":"2024-09-21T03:27:33.431233+0000","last_unstale":"2024-09-21T03:28:32.579658+0000","last_undegraded":"2024-09-21T03:28:32.579658+0000","last_fullsized":"2024-09-21T03:28:32.579658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:48.425375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574746+0000","last_change":"2024-09-21T03:27:01.331242+0000","last_active":"2024-09-21T03:28:32.574746+0000","last_peered":"2024-09-21T03:28:32.574746+0000","last_clean":"2024-09-21T03:28:32.574746+0000","last_became_active":"2024-09-21T03:27:01.330883+0000","last_became_peered":"2024-09-21T03:27:01.330883+0000","last_unstale":"2024-09-21T03:28:32.574746+0000","last_undegraded":"2024-09-21T03:28:32.574746+0000","last_fullsized":"2024-09-21T03:28:32.574746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:07.158448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.576098+0000","last_change":"2024-09-21T03:27:01.335522+0000","last_active":"2024-09-21T03:28:32.576098+0000","last_peered":"2024-09-21T03:28:32.576098+0000","last_clean":"2024-09-21T03:28:32.576098+0000","last_became_active":"2024-09-21T03:27:01.335114+0000","last_became_peered":"2024-09-21T03:27:01.335114+0000","last_unstale":"2024-09-21T03:28:32.576098+0000","last_undegraded":"2024-09-21T03:28:32.576098+0000","last_fullsized":"2024-09-21T03:28:32.576098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:28.383410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.567676+0000","last_change":"2024-09-21T03:27:01.329428+0000","last_active":"2024-09-21T03:28:32.567676+0000","last_peered":"2024-09-21T03:28:32.567676+0000","last_clean":"2024-09-21T03:28:32.567676+0000","last_became_active":"2024-09-21T03:27:01.328882+0000","last_became_peered":"2024-09-21T03:27:01.328882+0000","last_unstale":"2024-09-21T03:28:32.567676+0000","last_undegraded":"2024-09-21T03:28:32.567676+0000","last_fullsized":"2024-09-21T03:28:32.567676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:58.634995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.561654+0000","last_change":"2024-09-21T03:27:33.432442+0000","last_active":"2024-09-21T03:28:32.561654+0000","last_peered":"2024-09-21T03:28:32.561654+0000","last_clean":"2024-09-21T03:28:32.561654+0000","last_became_active":"2024-09-21T03:27:33.431881+0000","last_became_peered":"2024-09-21T03:27:33.431881+0000","last_unstale":"2024-09-21T03:28:32.561654+0000","last_undegraded":"2024-09-21T03:28:32.561654+0000","last_fullsized":"2024-09-21T03:28:32.561654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:46.356644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.562491+0000","last_change":"2024-09-21T03:27:01.330988+0000","last_active":"2024-09-21T03:28:32.562491+0000","last_peered":"2024-09-21T03:28:32.562491+0000","last_clean":"2024-09-21T03:28:32.562491+0000","last_became_active":"2024-09-21T03:27:01.330574+0000","last_became_peered":"2024-09-21T03:27:01.330574+0000","last_unstale":"2024-09-21T03:28:32.562491+0000","last_undegraded":"2024-09-21T03:28:32.562491+0000","last_fullsized":"2024-09-21T03:28:32.562491+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:45.277947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.573563+0000","last_change":"2024-09-21T03:27:01.339606+0000","last_active":"2024-09-21T03:28:32.573563+0000","last_peered":"2024-09-21T03:28:32.573563+0000","last_clean":"2024-09-21T03:28:32.573563+0000","last_became_active":"2024-09-21T03:27:01.339208+0000","last_became_peered":"2024-09-21T03:27:01.339208+0000","last_unstale":"2024-09-21T03:28:32.573563+0000","last_undegraded":"2024-09-21T03:28:32.573563+0000","last_fullsized":"2024-09-21T03:28:32.573563+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:31.593363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.578941+0000","last_change":"2024-09-21T03:27:01.343634+0000","last_active":"2024-09-21T03:28:32.578941+0000","last_peered":"2024-09-21T03:28:32.578941+0000","last_clean":"2024-09-21T03:28:32.578941+0000","last_became_active":"2024-09-21T03:27:01.342965+0000","last_became_peered":"2024-09-21T03:27:01.342965+0000","last_unstale":"2024-09-21T03:28:32.578941+0000","last_undegraded":"2024-09-21T03:28:32.578941+0000","last_fullsized":"2024-09-21T03:28:32.578941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:46.916198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564776+0000","last_change":"2024-09-21T03:27:01.324993+0000","last_active":"2024-09-21T03:28:32.564776+0000","last_peered":"2024-09-21T03:28:32.564776+0000","last_clean":"2024-09-21T03:28:32.564776+0000","last_became_active":"2024-09-21T03:27:01.324593+0000","last_became_peered":"2024-09-21T03:27:01.324593+0000","last_unstale":"2024-09-21T03:28:32.564776+0000","last_undegraded":"2024-09-21T03:28:32.564776+0000","last_fullsized":"2024-09-21T03:28:32.564776+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:13.211539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563667+0000","last_change":"2024-09-21T03:27:01.325232+0000","last_active":"2024-09-21T03:28:32.563667+0000","last_peered":"2024-09-21T03:28:32.563667+0000","last_clean":"2024-09-21T03:28:32.563667+0000","last_became_active":"2024-09-21T03:27:01.324570+0000","last_became_peered":"2024-09-21T03:27:01.324570+0000","last_unstale":"2024-09-21T03:28:32.563667+0000","last_undegraded":"2024-09-21T03:28:32.563667+0000","last_fullsized":"2024-09-21T03:28:32.563667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:30.843334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563783+0000","last_change":"2024-09-21T03:27:01.338789+0000","last_active":"2024-09-21T03:28:32.563783+0000","last_peered":"2024-09-21T03:28:32.563783+0000","last_clean":"2024-09-21T03:28:32.563783+0000","last_became_active":"2024-09-21T03:27:01.338282+0000","last_became_peered":"2024-09-21T03:27:01.338282+0000","last_unstale":"2024-09-21T03:28:32.563783+0000","last_undegraded":"2024-09-21T03:28:32.563783+0000","last_fullsized":"2024-09-21T03:28:32.563783+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:57.213343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.566643+0000","last_change":"2024-09-21T03:27:01.339521+0000","last_active":"2024-09-21T03:28:32.566643+0000","last_peered":"2024-09-21T03:28:32.566643+0000","last_clean":"2024-09-21T03:28:32.566643+0000","last_became_active":"2024-09-21T03:27:01.339175+0000","last_became_peered":"2024-09-21T03:27:01.339175+0000","last_unstale":"2024-09-21T03:28:32.566643+0000","last_undegraded":"2024-09-21T03:28:32.566643+0000","last_fullsized":"2024-09-21T03:28:32.566643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:46.163698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.581098+0000","last_change":"2024-09-21T03:27:01.338352+0000","last_active":"2024-09-21T03:28:32.581098+0000","last_peered":"2024-09-21T03:28:32.581098+0000","last_clean":"2024-09-21T03:28:32.581098+0000","last_became_active":"2024-09-21T03:27:01.337306+0000","last_became_peered":"2024-09-21T03:27:01.337306+0000","last_unstale":"2024-09-21T03:28:32.581098+0000","last_undegraded":"2024-09-21T03:28:32.581098+0000","last_fullsized":"2024-09-21T03:28:32.581098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.799039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.562245+0000","last_change":"2024-09-21T03:27:01.332461+0000","last_active":"2024-09-21T03:28:32.562245+0000","last_peered":"2024-09-21T03:28:32.562245+0000","last_clean":"2024-09-21T03:28:32.562245+0000","last_became_active":"2024-09-21T03:27:01.331912+0000","last_became_peered":"2024-09-21T03:27:01.331912+0000","last_unstale":"2024-09-21T03:28:32.562245+0000","last_undegraded":"2024-09-21T03:28:32.562245+0000","last_fullsized":"2024-09-21T03:28:32.562245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:34.771691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":136,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:43.588186+0000","last_change":"2024-09-21T03:27:33.482302+0000","last_active":"2024-09-21T03:28:43.588186+0000","last_peered":"2024-09-21T03:28:43.588186+0000","last_clean":"2024-09-21T03:28:43.588186+0000","last_became_active":"2024-09-21T03:27:33.478350+0000","last_became_peered":"2024-09-21T03:27:33.478350+0000","last_unstale":"2024-09-21T03:28:43.588186+0000","last_undegraded":"2024-09-21T03:28:43.588186+0000","last_fullsized":"2024-09-21T03:28:43.588186+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:46.916762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.566839+0000","last_change":"2024-09-21T03:27:01.324364+0000","last_active":"2024-09-21T03:28:32.566839+0000","last_peered":"2024-09-21T03:28:32.566839+0000","last_clean":"2024-09-21T03:28:32.566839+0000","last_became_active":"2024-09-21T03:27:01.324018+0000","last_became_peered":"2024-09-21T03:27:01.324018+0000","last_unstale":"2024-09-21T03:28:32.566839+0000","last_undegraded":"2024-09-21T03:28:32.566839+0000","last_fullsized":"2024-09-21T03:28:32.566839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:56.404893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.431029+0000","last_change":"2024-09-21T03:26:59.313289+0000","last_active":"2024-09-21T03:27:33.431029+0000","last_peered":"2024-09-21T03:27:33.431029+0000","last_clean":"2024-09-21T03:27:33.431029+0000","last_became_active":"2024-09-21T03:26:59.312877+0000","last_became_peered":"2024-09-21T03:26:59.312877+0000","last_unstale":"2024-09-21T03:27:33.431029+0000","last_undegraded":"2024-09-21T03:27:33.431029+0000","last_fullsized":"2024-09-21T03:27:33.431029+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:53.856541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427079+0000","last_change":"2024-09-21T03:26:59.307098+0000","last_active":"2024-09-21T03:27:33.427079+0000","last_peered":"2024-09-21T03:27:33.427079+0000","last_clean":"2024-09-21T03:27:33.427079+0000","last_became_active":"2024-09-21T03:26:59.306650+0000","last_became_peered":"2024-09-21T03:26:59.306650+0000","last_unstale":"2024-09-21T03:27:33.427079+0000","last_undegraded":"2024-09-21T03:27:33.427079+0000","last_fullsized":"2024-09-21T03:27:33.427079+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:44.033899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.570911+0000","last_change":"2024-09-21T03:27:01.328443+0000","last_active":"2024-09-21T03:28:32.570911+0000","last_peered":"2024-09-21T03:28:32.570911+0000","last_clean":"2024-09-21T03:28:32.570911+0000","last_became_active":"2024-09-21T03:27:01.328107+0000","last_became_peered":"2024-09-21T03:27:01.328107+0000","last_unstale":"2024-09-21T03:28:32.570911+0000","last_undegraded":"2024-09-21T03:28:32.570911+0000","last_fullsized":"2024-09-21T03:28:32.570911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:47.798308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428946+0000","last_change":"2024-09-21T03:26:59.317934+0000","last_active":"2024-09-21T03:27:33.428946+0000","last_peered":"2024-09-21T03:27:33.428946+0000","last_clean":"2024-09-21T03:27:33.428946+0000","last_became_active":"2024-09-21T03:26:59.317166+0000","last_became_peered":"2024-09-21T03:26:59.317166+0000","last_unstale":"2024-09-21T03:27:33.428946+0000","last_undegraded":"2024-09-21T03:27:33.428946+0000","last_fullsized":"2024-09-21T03:27:33.428946+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:52.746406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564062+0000","last_change":"2024-09-21T03:27:01.331226+0000","last_active":"2024-09-21T03:28:32.564062+0000","last_peered":"2024-09-21T03:28:32.564062+0000","last_clean":"2024-09-21T03:28:32.564062+0000","last_became_active":"2024-09-21T03:27:01.330889+0000","last_became_peered":"2024-09-21T03:27:01.330889+0000","last_unstale":"2024-09-21T03:28:32.564062+0000","last_undegraded":"2024-09-21T03:28:32.564062+0000","last_fullsized":"2024-09-21T03:28:32.564062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:51.181554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'3","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:41.584516+0000","last_change":"2024-09-21T03:26:59.311671+0000","last_active":"2024-09-21T03:28:41.584516+0000","last_peered":"2024-09-21T03:28:41.584516+0000","last_clean":"2024-09-21T03:28:41.584516+0000","last_became_active":"2024-09-21T03:26:59.311149+0000","last_became_peered":"2024-09-21T03:26:59.311149+0000","last_unstale":"2024-09-21T03:28:41.584516+0000","last_undegraded":"2024-09-21T03:28:41.584516+0000","last_fullsized":"2024-09-21T03:28:41.584516+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:47.878274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.570949+0000","last_change":"2024-09-21T03:27:01.336142+0000","last_active":"2024-09-21T03:28:32.570949+0000","last_peered":"2024-09-21T03:28:32.570949+0000","last_clean":"2024-09-21T03:28:32.570949+0000","last_became_active":"2024-09-21T03:27:01.335795+0000","last_became_peered":"2024-09-21T03:27:01.335795+0000","last_unstale":"2024-09-21T03:28:32.570949+0000","last_undegraded":"2024-09-21T03:28:32.570949+0000","last_fullsized":"2024-09-21T03:28:32.570949+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:29.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429365+0000","last_change":"2024-09-21T03:26:59.315077+0000","last_active":"2024-09-21T03:27:33.429365+0000","last_peered":"2024-09-21T03:27:33.429365+0000","last_clean":"2024-09-21T03:27:33.429365+0000","last_became_active":"2024-09-21T03:26:59.312347+0000","last_became_peered":"2024-09-21T03:26:59.312347+0000","last_unstale":"2024-09-21T03:27:33.429365+0000","last_undegraded":"2024-09-21T03:27:33.429365+0000","last_fullsized":"2024-09-21T03:27:33.429365+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:24.529066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565121+0000","last_change":"2024-09-21T03:27:01.324346+0000","last_active":"2024-09-21T03:28:32.565121+0000","last_peered":"2024-09-21T03:28:32.565121+0000","last_clean":"2024-09-21T03:28:32.565121+0000","last_became_active":"2024-09-21T03:27:01.323976+0000","last_became_peered":"2024-09-21T03:27:01.323976+0000","last_unstale":"2024-09-21T03:28:32.565121+0000","last_undegraded":"2024-09-21T03:28:32.565121+0000","last_fullsized":"2024-09-21T03:28:32.565121+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:04.591595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429516+0000","last_change":"2024-09-21T03:26:59.326206+0000","last_active":"2024-09-21T03:27:33.429516+0000","last_peered":"2024-09-21T03:27:33.429516+0000","last_clean":"2024-09-21T03:27:33.429516+0000","last_became_active":"2024-09-21T03:26:59.325755+0000","last_became_peered":"2024-09-21T03:26:59.325755+0000","last_unstale":"2024-09-21T03:27:33.429516+0000","last_undegraded":"2024-09-21T03:27:33.429516+0000","last_fullsized":"2024-09-21T03:27:33.429516+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:23.220795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574096+0000","last_change":"2024-09-21T03:27:01.341501+0000","last_active":"2024-09-21T03:28:32.574096+0000","last_peered":"2024-09-21T03:28:32.574096+0000","last_clean":"2024-09-21T03:28:32.574096+0000","last_became_active":"2024-09-21T03:27:01.341039+0000","last_became_peered":"2024-09-21T03:27:01.341039+0000","last_unstale":"2024-09-21T03:28:32.574096+0000","last_undegraded":"2024-09-21T03:28:32.574096+0000","last_fullsized":"2024-09-21T03:28:32.574096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:15.132180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'31","reported_seq":247,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:42.593727+0000","last_change":"2024-09-21T03:26:59.313489+0000","last_active":"2024-09-21T03:28:42.593727+0000","last_peered":"2024-09-21T03:28:42.593727+0000","last_clean":"2024-09-21T03:28:42.593727+0000","last_became_active":"2024-09-21T03:26:59.313022+0000","last_became_peered":"2024-09-21T03:26:59.313022+0000","last_unstale":"2024-09-21T03:28:42.593727+0000","last_undegraded":"2024-09-21T03:28:42.593727+0000","last_fullsized":"2024-09-21T03:28:42.593727+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:08.541315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65006,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":193,"num_read_kb":159,"num_write":32,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.578105+0000","last_change":"2024-09-21T03:27:01.339329+0000","last_active":"2024-09-21T03:28:32.578105+0000","last_peered":"2024-09-21T03:28:32.578105+0000","last_clean":"2024-09-21T03:28:32.578105+0000","last_became_active":"2024-09-21T03:27:01.338982+0000","last_became_peered":"2024-09-21T03:27:01.338982+0000","last_unstale":"2024-09-21T03:28:32.578105+0000","last_undegraded":"2024-09-21T03:28:32.578105+0000","last_fullsized":"2024-09-21T03:28:32.578105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:41.759450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'6","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430224+0000","last_change":"2024-09-21T03:26:59.313270+0000","last_active":"2024-09-21T03:27:33.430224+0000","last_peered":"2024-09-21T03:27:33.430224+0000","last_clean":"2024-09-21T03:27:33.430224+0000","last_became_active":"2024-09-21T03:26:59.312867+0000","last_became_peered":"2024-09-21T03:26:59.312867+0000","last_unstale":"2024-09-21T03:27:33.430224+0000","last_undegraded":"2024-09-21T03:27:33.430224+0000","last_fullsized":"2024-09-21T03:27:33.430224+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:53.798022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.572806+0000","last_change":"2024-09-21T03:27:01.331268+0000","last_active":"2024-09-21T03:28:32.572806+0000","last_peered":"2024-09-21T03:28:32.572806+0000","last_clean":"2024-09-21T03:28:32.572806+0000","last_became_active":"2024-09-21T03:27:01.330922+0000","last_became_peered":"2024-09-21T03:27:01.330922+0000","last_unstale":"2024-09-21T03:28:32.572806+0000","last_undegraded":"2024-09-21T03:28:32.572806+0000","last_fullsized":"2024-09-21T03:28:32.572806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428917+0000","last_change":"2024-09-21T03:26:59.318388+0000","last_active":"2024-09-21T03:27:33.428917+0000","last_peered":"2024-09-21T03:27:33.428917+0000","last_clean":"2024-09-21T03:27:33.428917+0000","last_became_active":"2024-09-21T03:26:59.317938+0000","last_became_peered":"2024-09-21T03:26:59.317938+0000","last_unstale":"2024-09-21T03:27:33.428917+0000","last_undegraded":"2024-09-21T03:27:33.428917+0000","last_fullsized":"2024-09-21T03:27:33.428917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.499699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.575303+0000","last_change":"2024-09-21T03:27:33.432191+0000","last_active":"2024-09-21T03:28:32.575303+0000","last_peered":"2024-09-21T03:28:32.575303+0000","last_clean":"2024-09-21T03:28:32.575303+0000","last_became_active":"2024-09-21T03:27:33.431667+0000","last_became_peered":"2024-09-21T03:27:33.431667+0000","last_unstale":"2024-09-21T03:28:32.575303+0000","last_undegraded":"2024-09-21T03:28:32.575303+0000","last_fullsized":"2024-09-21T03:28:32.575303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:13.293063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'5","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:43.550712+0000","last_change":"2024-09-21T03:26:59.311345+0000","last_active":"2024-09-21T03:28:43.550712+0000","last_peered":"2024-09-21T03:28:43.550712+0000","last_clean":"2024-09-21T03:28:43.550712+0000","last_became_active":"2024-09-21T03:26:59.310928+0000","last_became_peered":"2024-09-21T03:26:59.310928+0000","last_unstale":"2024-09-21T03:28:43.550712+0000","last_undegraded":"2024-09-21T03:28:43.550712+0000","last_fullsized":"2024-09-21T03:28:43.550712+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.614955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.571566+0000","last_change":"2024-09-21T03:27:01.335785+0000","last_active":"2024-09-21T03:28:32.571566+0000","last_peered":"2024-09-21T03:28:32.571566+0000","last_clean":"2024-09-21T03:28:32.571566+0000","last_became_active":"2024-09-21T03:27:01.335396+0000","last_became_peered":"2024-09-21T03:27:01.335396+0000","last_unstale":"2024-09-21T03:28:32.571566+0000","last_undegraded":"2024-09-21T03:28:32.571566+0000","last_fullsized":"2024-09-21T03:28:32.571566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:52.785641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'15","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.448967+0000","last_change":"2024-09-21T03:27:33.448404+0000","last_active":"2024-09-21T03:27:33.448967+0000","last_peered":"2024-09-21T03:27:33.448967+0000","last_clean":"2024-09-21T03:27:33.448967+0000","last_became_active":"2024-09-21T03:27:33.434405+0000","last_became_peered":"2024-09-21T03:27:33.434405+0000","last_unstale":"2024-09-21T03:27:33.448967+0000","last_undegraded":"2024-09-21T03:27:33.448967+0000","last_fullsized":"2024-09-21T03:27:33.448967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:05.107391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.573683+0000","last_change":"2024-09-21T03:27:01.342282+0000","last_active":"2024-09-21T03:28:32.573683+0000","last_peered":"2024-09-21T03:28:32.573683+0000","last_clean":"2024-09-21T03:28:32.573683+0000","last_became_active":"2024-09-21T03:27:01.341790+0000","last_became_peered":"2024-09-21T03:27:01.341790+0000","last_unstale":"2024-09-21T03:28:32.573683+0000","last_undegraded":"2024-09-21T03:28:32.573683+0000","last_fullsized":"2024-09-21T03:28:32.573683+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:05.432720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428084+0000","last_change":"2024-09-21T03:26:59.312442+0000","last_active":"2024-09-21T03:27:33.428084+0000","last_peered":"2024-09-21T03:27:33.428084+0000","last_clean":"2024-09-21T03:27:33.428084+0000","last_became_active":"2024-09-21T03:26:59.311601+0000","last_became_peered":"2024-09-21T03:26:59.311601+0000","last_unstale":"2024-09-21T03:27:33.428084+0000","last_undegraded":"2024-09-21T03:27:33.428084+0000","last_fullsized":"2024-09-21T03:27:33.428084+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:32.421228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.579550+0000","last_change":"2024-09-21T03:27:01.337917+0000","last_active":"2024-09-21T03:28:32.579550+0000","last_peered":"2024-09-21T03:28:32.579550+0000","last_clean":"2024-09-21T03:28:32.579550+0000","last_became_active":"2024-09-21T03:27:01.337457+0000","last_became_peered":"2024-09-21T03:27:01.337457+0000","last_unstale":"2024-09-21T03:28:32.579550+0000","last_undegraded":"2024-09-21T03:28:32.579550+0000","last_fullsized":"2024-09-21T03:28:32.579550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:49.011795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.597334+0000","last_change":"2024-09-21T03:27:33.597239+0000","last_active":"2024-09-21T03:27:33.597334+0000","last_peered":"2024-09-21T03:27:33.597334+0000","last_clean":"2024-09-21T03:27:33.597334+0000","last_became_active":"2024-09-21T03:27:33.593787+0000","last_became_peered":"2024-09-21T03:27:33.593787+0000","last_unstale":"2024-09-21T03:27:33.597334+0000","last_undegraded":"2024-09-21T03:27:33.597334+0000","last_fullsized":"2024-09-21T03:27:33.597334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:35.015446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.581336+0000","last_change":"2024-09-21T03:27:01.338489+0000","last_active":"2024-09-21T03:28:32.581336+0000","last_peered":"2024-09-21T03:28:32.581336+0000","last_clean":"2024-09-21T03:28:32.581336+0000","last_became_active":"2024-09-21T03:27:01.338123+0000","last_became_peered":"2024-09-21T03:27:01.338123+0000","last_unstale":"2024-09-21T03:28:32.581336+0000","last_undegraded":"2024-09-21T03:28:32.581336+0000","last_fullsized":"2024-09-21T03:28:32.581336+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:15.137546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429644+0000","last_change":"2024-09-21T03:26:59.315371+0000","last_active":"2024-09-21T03:27:33.429644+0000","last_peered":"2024-09-21T03:27:33.429644+0000","last_clean":"2024-09-21T03:27:33.429644+0000","last_became_active":"2024-09-21T03:26:59.314439+0000","last_became_peered":"2024-09-21T03:26:59.314439+0000","last_unstale":"2024-09-21T03:27:33.429644+0000","last_undegraded":"2024-09-21T03:27:33.429644+0000","last_fullsized":"2024-09-21T03:27:33.429644+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:13.656324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.567006+0000","last_change":"2024-09-21T03:27:01.331086+0000","last_active":"2024-09-21T03:28:32.567006+0000","last_peered":"2024-09-21T03:28:32.567006+0000","last_clean":"2024-09-21T03:28:32.567006+0000","last_became_active":"2024-09-21T03:27:01.329113+0000","last_became_peered":"2024-09-21T03:27:01.329113+0000","last_unstale":"2024-09-21T03:28:32.567006+0000","last_undegraded":"2024-09-21T03:28:32.567006+0000","last_fullsized":"2024-09-21T03:28:32.567006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:25.514730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.433098+0000","last_change":"2024-09-21T03:26:59.554087+0000","last_active":"2024-09-21T03:27:33.433098+0000","last_peered":"2024-09-21T03:27:33.433098+0000","last_clean":"2024-09-21T03:27:33.433098+0000","last_became_active":"2024-09-21T03:26:59.553610+0000","last_became_peered":"2024-09-21T03:26:59.553610+0000","last_unstale":"2024-09-21T03:27:33.433098+0000","last_undegraded":"2024-09-21T03:27:33.433098+0000","last_fullsized":"2024-09-21T03:27:33.433098+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:22.048057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.577544+0000","last_change":"2024-09-21T03:27:01.339987+0000","last_active":"2024-09-21T03:28:32.577544+0000","last_peered":"2024-09-21T03:28:32.577544+0000","last_clean":"2024-09-21T03:28:32.577544+0000","last_became_active":"2024-09-21T03:27:01.339060+0000","last_became_peered":"2024-09-21T03:27:01.339060+0000","last_unstale":"2024-09-21T03:28:32.577544+0000","last_undegraded":"2024-09-21T03:28:32.577544+0000","last_fullsized":"2024-09-21T03:28:32.577544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.901337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429743+0000","last_change":"2024-09-21T03:25:50.381920+0000","last_active":"2024-09-21T03:27:33.429743+0000","last_peered":"2024-09-21T03:27:33.429743+0000","last_clean":"2024-09-21T03:27:33.429743+0000","last_became_active":"2024-09-21T03:25:50.376170+0000","last_became_peered":"2024-09-21T03:25:50.376170+0000","last_unstale":"2024-09-21T03:27:33.429743+0000","last_undegraded":"2024-09-21T03:27:33.429743+0000","last_fullsized":"2024-09-21T03:27:33.429743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_clean_scrub_stamp":"2024-09-21T03:22:31.269883+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:57.281150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'3","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.477054+0000","last_change":"2024-09-21T03:26:59.313612+0000","last_active":"2024-09-21T03:27:33.477054+0000","last_peered":"2024-09-21T03:27:33.477054+0000","last_clean":"2024-09-21T03:27:33.477054+0000","last_became_active":"2024-09-21T03:26:59.313060+0000","last_became_peered":"2024-09-21T03:26:59.313060+0000","last_unstale":"2024-09-21T03:27:33.477054+0000","last_undegraded":"2024-09-21T03:27:33.477054+0000","last_fullsized":"2024-09-21T03:27:33.477054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:06.193526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.572676+0000","last_change":"2024-09-21T03:27:01.337784+0000","last_active":"2024-09-21T03:28:32.572676+0000","last_peered":"2024-09-21T03:28:32.572676+0000","last_clean":"2024-09-21T03:28:32.572676+0000","last_became_active":"2024-09-21T03:27:01.337415+0000","last_became_peered":"2024-09-21T03:27:01.337415+0000","last_unstale":"2024-09-21T03:28:32.572676+0000","last_undegraded":"2024-09-21T03:28:32.572676+0000","last_fullsized":"2024-09-21T03:28:32.572676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:40.482054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'3","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.436960+0000","last_change":"2024-09-21T03:27:33.436319+0000","last_active":"2024-09-21T03:27:33.436960+0000","last_peered":"2024-09-21T03:27:33.436960+0000","last_clean":"2024-09-21T03:27:33.436960+0000","last_became_active":"2024-09-21T03:27:33.431555+0000","last_became_peered":"2024-09-21T03:27:33.431555+0000","last_unstale":"2024-09-21T03:27:33.436960+0000","last_undegraded":"2024-09-21T03:27:33.436960+0000","last_fullsized":"2024-09-21T03:27:33.436960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:16.073988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563635+0000","last_change":"2024-09-21T03:27:01.333769+0000","last_active":"2024-09-21T03:28:32.563635+0000","last_peered":"2024-09-21T03:28:32.563635+0000","last_clean":"2024-09-21T03:28:32.563635+0000","last_became_active":"2024-09-21T03:27:01.333436+0000","last_became_peered":"2024-09-21T03:27:01.333436+0000","last_unstale":"2024-09-21T03:28:32.563635+0000","last_undegraded":"2024-09-21T03:28:32.563635+0000","last_fullsized":"2024-09-21T03:28:32.563635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:30.395916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:43.550125+0000","last_change":"2024-09-21T03:26:59.325314+0000","last_active":"2024-09-21T03:28:43.550125+0000","last_peered":"2024-09-21T03:28:43.550125+0000","last_clean":"2024-09-21T03:28:43.550125+0000","last_became_active":"2024-09-21T03:26:59.324752+0000","last_became_peered":"2024-09-21T03:26:59.324752+0000","last_unstale":"2024-09-21T03:28:43.550125+0000","last_undegraded":"2024-09-21T03:28:43.550125+0000","last_fullsized":"2024-09-21T03:28:43.550125+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.252596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.559825+0000","last_change":"2024-09-21T03:27:01.335154+0000","last_active":"2024-09-21T03:28:32.559825+0000","last_peered":"2024-09-21T03:28:32.559825+0000","last_clean":"2024-09-21T03:28:32.559825+0000","last_became_active":"2024-09-21T03:27:01.334633+0000","last_became_peered":"2024-09-21T03:27:01.334633+0000","last_unstale":"2024-09-21T03:28:32.559825+0000","last_undegraded":"2024-09-21T03:28:32.559825+0000","last_fullsized":"2024-09-21T03:28:32.559825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:27.992858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429757+0000","last_change":"2024-09-21T03:26:59.314019+0000","last_active":"2024-09-21T03:27:33.429757+0000","last_peered":"2024-09-21T03:27:33.429757+0000","last_clean":"2024-09-21T03:27:33.429757+0000","last_became_active":"2024-09-21T03:26:59.311915+0000","last_became_peered":"2024-09-21T03:26:59.311915+0000","last_unstale":"2024-09-21T03:27:33.429757+0000","last_undegraded":"2024-09-21T03:27:33.429757+0000","last_fullsized":"2024-09-21T03:27:33.429757+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:51.412303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.558537+0000","last_change":"2024-09-21T03:27:33.433193+0000","last_active":"2024-09-21T03:28:32.558537+0000","last_peered":"2024-09-21T03:28:32.558537+0000","last_clean":"2024-09-21T03:28:32.558537+0000","last_became_active":"2024-09-21T03:27:33.432729+0000","last_became_peered":"2024-09-21T03:27:33.432729+0000","last_unstale":"2024-09-21T03:28:32.558537+0000","last_undegraded":"2024-09-21T03:28:32.558537+0000","last_fullsized":"2024-09-21T03:28:32.558537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:37.575110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424782+0000","last_change":"2024-09-21T03:26:59.325943+0000","last_active":"2024-09-21T03:27:33.424782+0000","last_peered":"2024-09-21T03:27:33.424782+0000","last_clean":"2024-09-21T03:27:33.424782+0000","last_became_active":"2024-09-21T03:26:59.325524+0000","last_became_peered":"2024-09-21T03:26:59.325524+0000","last_unstale":"2024-09-21T03:27:33.424782+0000","last_undegraded":"2024-09-21T03:27:33.424782+0000","last_fullsized":"2024-09-21T03:27:33.424782+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:45.111480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563254+0000","last_change":"2024-09-21T03:27:01.335369+0000","last_active":"2024-09-21T03:28:32.563254+0000","last_peered":"2024-09-21T03:28:32.563254+0000","last_clean":"2024-09-21T03:28:32.563254+0000","last_became_active":"2024-09-21T03:27:01.334997+0000","last_became_peered":"2024-09-21T03:27:01.334997+0000","last_unstale":"2024-09-21T03:28:32.563254+0000","last_undegraded":"2024-09-21T03:28:32.563254+0000","last_fullsized":"2024-09-21T03:28:32.563254+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:10.257451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429553+0000","last_change":"2024-09-21T03:26:59.326204+0000","last_active":"2024-09-21T03:27:33.429553+0000","last_peered":"2024-09-21T03:27:33.429553+0000","last_clean":"2024-09-21T03:27:33.429553+0000","last_became_active":"2024-09-21T03:26:59.325431+0000","last_became_peered":"2024-09-21T03:26:59.325431+0000","last_unstale":"2024-09-21T03:27:33.429553+0000","last_undegraded":"2024-09-21T03:27:33.429553+0000","last_fullsized":"2024-09-21T03:27:33.429553+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:27.863217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.558220+0000","last_change":"2024-09-21T03:27:01.333861+0000","last_active":"2024-09-21T03:28:32.558220+0000","last_peered":"2024-09-21T03:28:32.558220+0000","last_clean":"2024-09-21T03:28:32.558220+0000","last_became_active":"2024-09-21T03:27:01.333485+0000","last_became_peered":"2024-09-21T03:27:01.333485+0000","last_unstale":"2024-09-21T03:28:32.558220+0000","last_undegraded":"2024-09-21T03:28:32.558220+0000","last_fullsized":"2024-09-21T03:28:32.558220+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:04.475811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'12","reported_seq":35,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424914+0000","last_change":"2024-09-21T03:26:59.309767+0000","last_active":"2024-09-21T03:27:33.424914+0000","last_peered":"2024-09-21T03:27:33.424914+0000","last_clean":"2024-09-21T03:27:33.424914+0000","last_became_active":"2024-09-21T03:26:59.309354+0000","last_became_peered":"2024-09-21T03:26:59.309354+0000","last_unstale":"2024-09-21T03:27:33.424914+0000","last_undegraded":"2024-09-21T03:27:33.424914+0000","last_fullsized":"2024-09-21T03:27:33.424914+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:26.728719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574479+0000","last_change":"2024-09-21T03:27:01.333097+0000","last_active":"2024-09-21T03:28:32.574479+0000","last_peered":"2024-09-21T03:28:32.574479+0000","last_clean":"2024-09-21T03:28:32.574479+0000","last_became_active":"2024-09-21T03:27:01.332743+0000","last_became_peered":"2024-09-21T03:27:01.332743+0000","last_unstale":"2024-09-21T03:28:32.574479+0000","last_undegraded":"2024-09-21T03:28:32.574479+0000","last_fullsized":"2024-09-21T03:28:32.574479+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:41.004213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.476588+0000","last_change":"2024-09-21T03:26:59.553974+0000","last_active":"2024-09-21T03:27:33.476588+0000","last_peered":"2024-09-21T03:27:33.476588+0000","last_clean":"2024-09-21T03:27:33.476588+0000","last_became_active":"2024-09-21T03:26:59.553647+0000","last_became_peered":"2024-09-21T03:26:59.553647+0000","last_unstale":"2024-09-21T03:27:33.476588+0000","last_undegraded":"2024-09-21T03:27:33.476588+0000","last_fullsized":"2024-09-21T03:27:33.476588+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:00.490951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563631+0000","last_change":"2024-09-21T03:27:01.325308+0000","last_active":"2024-09-21T03:28:32.563631+0000","last_peered":"2024-09-21T03:28:32.563631+0000","last_clean":"2024-09-21T03:28:32.563631+0000","last_became_active":"2024-09-21T03:27:01.324977+0000","last_became_peered":"2024-09-21T03:27:01.324977+0000","last_unstale":"2024-09-21T03:28:32.563631+0000","last_undegraded":"2024-09-21T03:28:32.563631+0000","last_fullsized":"2024-09-21T03:28:32.563631+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:10.093856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592537+0000","last_change":"2024-09-21T03:26:59.326017+0000","last_active":"2024-09-21T03:27:33.592537+0000","last_peered":"2024-09-21T03:27:33.592537+0000","last_clean":"2024-09-21T03:27:33.592537+0000","last_became_active":"2024-09-21T03:26:59.325588+0000","last_became_peered":"2024-09-21T03:26:59.325588+0000","last_unstale":"2024-09-21T03:27:33.592537+0000","last_undegraded":"2024-09-21T03:27:33.592537+0000","last_fullsized":"2024-09-21T03:27:33.592537+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:39.740449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.531947+0000","last_change":"2024-09-21T03:27:01.336871+0000","last_active":"2024-09-21T03:28:32.531947+0000","last_peered":"2024-09-21T03:28:32.531947+0000","last_clean":"2024-09-21T03:28:32.531947+0000","last_became_active":"2024-09-21T03:27:01.336507+0000","last_became_peered":"2024-09-21T03:27:01.336507+0000","last_unstale":"2024-09-21T03:28:32.531947+0000","last_undegraded":"2024-09-21T03:28:32.531947+0000","last_fullsized":"2024-09-21T03:28:32.531947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:58.674375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430477+0000","last_change":"2024-09-21T03:26:59.308997+0000","last_active":"2024-09-21T03:27:33.430477+0000","last_peered":"2024-09-21T03:27:33.430477+0000","last_clean":"2024-09-21T03:27:33.430477+0000","last_became_active":"2024-09-21T03:26:59.308338+0000","last_became_peered":"2024-09-21T03:26:59.308338+0000","last_unstale":"2024-09-21T03:27:33.430477+0000","last_undegraded":"2024-09-21T03:27:33.430477+0000","last_fullsized":"2024-09-21T03:27:33.430477+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:15.328468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.576878+0000","last_change":"2024-09-21T03:27:01.338808+0000","last_active":"2024-09-21T03:28:32.576878+0000","last_peered":"2024-09-21T03:28:32.576878+0000","last_clean":"2024-09-21T03:28:32.576878+0000","last_became_active":"2024-09-21T03:27:01.337987+0000","last_became_peered":"2024-09-21T03:27:01.337987+0000","last_unstale":"2024-09-21T03:28:32.576878+0000","last_undegraded":"2024-09-21T03:28:32.576878+0000","last_fullsized":"2024-09-21T03:28:32.576878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:56.265014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430766+0000","last_change":"2024-09-21T03:26:59.312684+0000","last_active":"2024-09-21T03:27:33.430766+0000","last_peered":"2024-09-21T03:27:33.430766+0000","last_clean":"2024-09-21T03:27:33.430766+0000","last_became_active":"2024-09-21T03:26:59.312038+0000","last_became_peered":"2024-09-21T03:26:59.312038+0000","last_unstale":"2024-09-21T03:27:33.430766+0000","last_undegraded":"2024-09-21T03:27:33.430766+0000","last_fullsized":"2024-09-21T03:27:33.430766+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:02.672255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.569931+0000","last_change":"2024-09-21T03:27:01.333080+0000","last_active":"2024-09-21T03:28:32.569931+0000","last_peered":"2024-09-21T03:28:32.569931+0000","last_clean":"2024-09-21T03:28:32.569931+0000","last_became_active":"2024-09-21T03:27:01.332663+0000","last_became_peered":"2024-09-21T03:27:01.332663+0000","last_unstale":"2024-09-21T03:28:32.569931+0000","last_undegraded":"2024-09-21T03:28:32.569931+0000","last_fullsized":"2024-09-21T03:28:32.569931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:34.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427673+0000","last_change":"2024-09-21T03:26:59.315490+0000","last_active":"2024-09-21T03:27:33.427673+0000","last_peered":"2024-09-21T03:27:33.427673+0000","last_clean":"2024-09-21T03:27:33.427673+0000","last_became_active":"2024-09-21T03:26:59.313789+0000","last_became_peered":"2024-09-21T03:26:59.313789+0000","last_unstale":"2024-09-21T03:27:33.427673+0000","last_undegraded":"2024-09-21T03:27:33.427673+0000","last_fullsized":"2024-09-21T03:27:33.427673+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:10.140711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574483+0000","last_change":"2024-09-21T03:27:01.341019+0000","last_active":"2024-09-21T03:28:32.574483+0000","last_peered":"2024-09-21T03:28:32.574483+0000","last_clean":"2024-09-21T03:28:32.574483+0000","last_became_active":"2024-09-21T03:27:01.340660+0000","last_became_peered":"2024-09-21T03:27:01.340660+0000","last_unstale":"2024-09-21T03:28:32.574483+0000","last_undegraded":"2024-09-21T03:28:32.574483+0000","last_fullsized":"2024-09-21T03:28:32.574483+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:16.736993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.577115+0000","last_change":"2024-09-21T03:27:01.341165+0000","last_active":"2024-09-21T03:28:32.577115+0000","last_peered":"2024-09-21T03:28:32.577115+0000","last_clean":"2024-09-21T03:28:32.577115+0000","last_became_active":"2024-09-21T03:27:01.340765+0000","last_became_peered":"2024-09-21T03:27:01.340765+0000","last_unstale":"2024-09-21T03:28:32.577115+0000","last_undegraded":"2024-09-21T03:28:32.577115+0000","last_fullsized":"2024-09-21T03:28:32.577115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:15.455944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429107+0000","last_change":"2024-09-21T03:26:59.313410+0000","last_active":"2024-09-21T03:27:33.429107+0000","last_peered":"2024-09-21T03:27:33.429107+0000","last_clean":"2024-09-21T03:27:33.429107+0000","last_became_active":"2024-09-21T03:26:59.312233+0000","last_became_peered":"2024-09-21T03:26:59.312233+0000","last_unstale":"2024-09-21T03:27:33.429107+0000","last_undegraded":"2024-09-21T03:27:33.429107+0000","last_fullsized":"2024-09-21T03:27:33.429107+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:26.089611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.572571+0000","last_change":"2024-09-21T03:27:01.335332+0000","last_active":"2024-09-21T03:28:32.572571+0000","last_peered":"2024-09-21T03:28:32.572571+0000","last_clean":"2024-09-21T03:28:32.572571+0000","last_became_active":"2024-09-21T03:27:01.334872+0000","last_became_peered":"2024-09-21T03:27:01.334872+0000","last_unstale":"2024-09-21T03:28:32.572571+0000","last_undegraded":"2024-09-21T03:28:32.572571+0000","last_fullsized":"2024-09-21T03:28:32.572571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:39.631246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429421+0000","last_change":"2024-09-21T03:26:59.314250+0000","last_active":"2024-09-21T03:27:33.429421+0000","last_peered":"2024-09-21T03:27:33.429421+0000","last_clean":"2024-09-21T03:27:33.429421+0000","last_became_active":"2024-09-21T03:26:59.313707+0000","last_became_peered":"2024-09-21T03:26:59.313707+0000","last_unstale":"2024-09-21T03:27:33.429421+0000","last_undegraded":"2024-09-21T03:27:33.429421+0000","last_fullsized":"2024-09-21T03:27:33.429421+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:22.935292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565595+0000","last_change":"2024-09-21T03:27:01.339230+0000","last_active":"2024-09-21T03:28:32.565595+0000","last_peered":"2024-09-21T03:28:32.565595+0000","last_clean":"2024-09-21T03:28:32.565595+0000","last_became_active":"2024-09-21T03:27:01.338881+0000","last_became_peered":"2024-09-21T03:27:01.338881+0000","last_unstale":"2024-09-21T03:28:32.565595+0000","last_undegraded":"2024-09-21T03:28:32.565595+0000","last_fullsized":"2024-09-21T03:28:32.565595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:49.189180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430888+0000","last_change":"2024-09-21T03:26:59.326549+0000","last_active":"2024-09-21T03:27:33.430888+0000","last_peered":"2024-09-21T03:27:33.430888+0000","last_clean":"2024-09-21T03:27:33.430888+0000","last_became_active":"2024-09-21T03:26:59.326091+0000","last_became_peered":"2024-09-21T03:26:59.326091+0000","last_unstale":"2024-09-21T03:27:33.430888+0000","last_undegraded":"2024-09-21T03:27:33.430888+0000","last_fullsized":"2024-09-21T03:27:33.430888+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:13.507716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.566939+0000","last_change":"2024-09-21T03:27:01.331367+0000","last_active":"2024-09-21T03:28:32.566939+0000","last_peered":"2024-09-21T03:28:32.566939+0000","last_clean":"2024-09-21T03:28:32.566939+0000","last_became_active":"2024-09-21T03:27:01.330604+0000","last_became_peered":"2024-09-21T03:27:01.330604+0000","last_unstale":"2024-09-21T03:28:32.566939+0000","last_undegraded":"2024-09-21T03:28:32.566939+0000","last_fullsized":"2024-09-21T03:28:32.566939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:32.207561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'9","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424843+0000","last_change":"2024-09-21T03:26:59.311912+0000","last_active":"2024-09-21T03:27:33.424843+0000","last_peered":"2024-09-21T03:27:33.424843+0000","last_clean":"2024-09-21T03:27:33.424843+0000","last_became_active":"2024-09-21T03:26:59.311571+0000","last_became_peered":"2024-09-21T03:26:59.311571+0000","last_unstale":"2024-09-21T03:27:33.424843+0000","last_undegraded":"2024-09-21T03:27:33.424843+0000","last_fullsized":"2024-09-21T03:27:33.424843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:55.946494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69148,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":583,"num_read_kb":363,"num_write":172,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":208676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776163,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":843776,"data_stored":413299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939686,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1536000,"data_stored":1070206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103212,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":413423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299441,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":413231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32596,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712556,"statfs":{"total":95995035648,"available":95961657344,"internally_reserved":0,"allocated":1445888,"data_stored":1005722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":412812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855424,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1458176,"data_stored":1004566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":851968,"data_stored":414403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":413123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":478647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542229,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":477919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":414083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:28:49.851 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:28:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:49 smithi203 ceph-mon[34723]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:28:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:49 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3645941536' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:49 smithi187 ceph-mon[34603]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:28:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:49 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3645941536' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.994+0000 7f852b39d640 1 -- 172.21.15.112:0/2333653344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8524109220 msgr2=0x7f8524109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.994+0000 7f852b39d640 1 --2- 172.21.15.112:0/2333653344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8524109220 0x7f8524109630 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8514009900 tx=0x7f851402a410 comp rx=0 tx=0).stop 2024-09-21T03:28:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.994+0000 7f852b39d640 1 -- 172.21.15.112:0/2333653344 shutdown_connections 2024-09-21T03:28:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.994+0000 7f852b39d640 1 --2- 172.21.15.112:0/2333653344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8524109b70 0x7f85241103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.994+0000 7f852b39d640 1 --2- 172.21.15.112:0/2333653344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8524109220 0x7f8524109630 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.994+0000 7f852b39d640 1 --2- 172.21.15.112:0/2333653344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85241038f0 0x7f8524103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.994+0000 7f852b39d640 1 -- 172.21.15.112:0/2333653344 >> 172.21.15.112:0/2333653344 conn(0x7f85240ff330 msgr2=0x7f8524101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.994+0000 7f852b39d640 1 -- 172.21.15.112:0/2333653344 shutdown_connections 2024-09-21T03:28:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.994+0000 7f852b39d640 1 -- 172.21.15.112:0/2333653344 wait complete. 2024-09-21T03:28:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f852b39d640 1 Processor -- start 2024-09-21T03:28:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f852b39d640 1 -- start start 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f852b39d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85241038f0 0x7f852419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f852b39d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8524109220 0x7f852419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f852b39d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8524109b70 0x7f852419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f852b39d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85241137f0 con 0x7f8524109220 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f852b39d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8524113670 con 0x7f85241038f0 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f852b39d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8524113970 con 0x7f8524109b70 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f8529b9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8524109220 0x7f852419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f8529b9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8524109220 0x7f852419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58396/0 (socket says 172.21.15.112:58396) 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f8529b9a640 1 -- 172.21.15.112:0/2853111328 learned_addr learned my addr 172.21.15.112:0/2853111328 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f852ab9c640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8524109b70 0x7f852419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f852a39b640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85241038f0 0x7f852419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f8529b9a640 1 -- 172.21.15.112:0/2853111328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8524109b70 msgr2=0x7f852419fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f8529b9a640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8524109b70 0x7f852419fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f8529b9a640 1 -- 172.21.15.112:0/2853111328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85241038f0 msgr2=0x7f852419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f8529b9a640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85241038f0 0x7f852419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f8529b9a640 1 -- 172.21.15.112:0/2853111328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85241a0160 con 0x7f8524109220 2024-09-21T03:28:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.995+0000 7f852a39b640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85241038f0 0x7f852419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.996+0000 7f8529b9a640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8524109220 0x7f852419a7d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f851402a930 tx=0x7f8514004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.996+0000 7f851b7fe640 1 -- 172.21.15.112:0/2853111328 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851403f030 con 0x7f8524109220 2024-09-21T03:28:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.996+0000 7f852b39d640 1 -- 172.21.15.112:0/2853111328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85241a03f0 con 0x7f8524109220 2024-09-21T03:28:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.996+0000 7f851b7fe640 1 -- 172.21.15.112:0/2853111328 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8514003aa0 con 0x7f8524109220 2024-09-21T03:28:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.996+0000 7f851b7fe640 1 -- 172.21.15.112:0/2853111328 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514035070 con 0x7f8524109220 2024-09-21T03:28:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.996+0000 7f852b39d640 1 -- 172.21.15.112:0/2853111328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85241a6be0 con 0x7f8524109220 2024-09-21T03:28:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.997+0000 7f852b39d640 1 -- 172.21.15.112:0/2853111328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8524104b60 con 0x7f8524109220 2024-09-21T03:28:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.998+0000 7f851b7fe640 1 -- 172.21.15.112:0/2853111328 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f8514042030 con 0x7f8524109220 2024-09-21T03:28:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.999+0000 7f851b7fe640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85100b6560 0x7f85100b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.999+0000 7f851b7fe640 1 -- 172.21.15.112:0/2853111328 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f8514107fb0 con 0x7f8524109220 2024-09-21T03:28:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:49.999+0000 7f852a39b640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85100b6560 0x7f85100b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.000+0000 7f852a39b640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85100b6560 0x7f85100b8a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8520009b40 tx=0x7f8520005e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:50.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.002+0000 7f851b7fe640 1 -- 172.21.15.112:0/2853111328 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f85140ce780 con 0x7f8524109220 2024-09-21T03:28:50.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.177+0000 7f852b39d640 1 -- 172.21.15.112:0/2853111328 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8524063800 con 0x7f85100b6560 2024-09-21T03:28:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.181+0000 7f851b7fe640 1 -- 172.21.15.112:0/2853111328 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258811 (secure 0 0 0) 0x7f8524063800 con 0x7f85100b6560 2024-09-21T03:28:50.185 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:50.187 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T03:28:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 -- 172.21.15.112:0/2853111328 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85100b6560 msgr2=0x7f85100b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85100b6560 0x7f85100b8a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8520009b40 tx=0x7f8520005e50 comp rx=0 tx=0).stop 2024-09-21T03:28:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 -- 172.21.15.112:0/2853111328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8524109220 msgr2=0x7f852419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8524109220 0x7f852419a7d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f851402a930 tx=0x7f8514004270 comp rx=0 tx=0).stop 2024-09-21T03:28:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 -- 172.21.15.112:0/2853111328 shutdown_connections 2024-09-21T03:28:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85100b6560 0x7f85100b8a70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8524109b70 0x7f852419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8524109220 0x7f852419a7d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 --2- 172.21.15.112:0/2853111328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85241038f0 0x7f852419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 -- 172.21.15.112:0/2853111328 >> 172.21.15.112:0/2853111328 conn(0x7f85240ff330 msgr2=0x7f8524101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 -- 172.21.15.112:0/2853111328 shutdown_connections 2024-09-21T03:28:50.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.184+0000 7f852b39d640 1 -- 172.21.15.112:0/2853111328 wait complete. 2024-09-21T03:28:50.197 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-21T03:28:48.361585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659750,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":664,"num_read_kb":432,"num_write":297,"num_write_kb":2273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952574,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":739,"ondisk_log_size":739,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381976,"kb_used_data":11928,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559848,"statfs":{"total":1151940427776,"available":1151549284352,"internally_reserved":0,"allocated":12214272,"data_stored":6931434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003303"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.582283+0000","last_change":"2024-09-21T03:27:01.339980+0000","last_active":"2024-09-21T03:28:32.582283+0000","last_peered":"2024-09-21T03:28:32.582283+0000","last_clean":"2024-09-21T03:28:32.582283+0000","last_became_active":"2024-09-21T03:27:01.339552+0000","last_became_peered":"2024-09-21T03:27:01.339552+0000","last_unstale":"2024-09-21T03:28:32.582283+0000","last_undegraded":"2024-09-21T03:28:32.582283+0000","last_fullsized":"2024-09-21T03:28:32.582283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:04.240862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.577369+0000","last_change":"2024-09-21T03:27:01.334171+0000","last_active":"2024-09-21T03:28:32.577369+0000","last_peered":"2024-09-21T03:28:32.577369+0000","last_clean":"2024-09-21T03:28:32.577369+0000","last_became_active":"2024-09-21T03:27:01.333851+0000","last_became_peered":"2024-09-21T03:27:01.333851+0000","last_unstale":"2024-09-21T03:28:32.577369+0000","last_undegraded":"2024-09-21T03:28:32.577369+0000","last_fullsized":"2024-09-21T03:28:32.577369+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:49.163142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565746+0000","last_change":"2024-09-21T03:27:01.339388+0000","last_active":"2024-09-21T03:28:32.565746+0000","last_peered":"2024-09-21T03:28:32.565746+0000","last_clean":"2024-09-21T03:28:32.565746+0000","last_became_active":"2024-09-21T03:27:01.339015+0000","last_became_peered":"2024-09-21T03:27:01.339015+0000","last_unstale":"2024-09-21T03:28:32.565746+0000","last_undegraded":"2024-09-21T03:28:32.565746+0000","last_fullsized":"2024-09-21T03:28:32.565746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:06.026397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565177+0000","last_change":"2024-09-21T03:27:01.344700+0000","last_active":"2024-09-21T03:28:32.565177+0000","last_peered":"2024-09-21T03:28:32.565177+0000","last_clean":"2024-09-21T03:28:32.565177+0000","last_became_active":"2024-09-21T03:27:01.344349+0000","last_became_peered":"2024-09-21T03:27:01.344349+0000","last_unstale":"2024-09-21T03:28:32.565177+0000","last_undegraded":"2024-09-21T03:28:32.565177+0000","last_fullsized":"2024-09-21T03:28:32.565177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:00.271820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.558498+0000","last_change":"2024-09-21T03:27:01.334904+0000","last_active":"2024-09-21T03:28:32.558498+0000","last_peered":"2024-09-21T03:28:32.558498+0000","last_clean":"2024-09-21T03:28:32.558498+0000","last_became_active":"2024-09-21T03:27:01.331916+0000","last_became_peered":"2024-09-21T03:27:01.331916+0000","last_unstale":"2024-09-21T03:28:32.558498+0000","last_undegraded":"2024-09-21T03:28:32.558498+0000","last_fullsized":"2024-09-21T03:28:32.558498+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:49.376481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.559921+0000","last_change":"2024-09-21T03:27:01.343256+0000","last_active":"2024-09-21T03:28:32.559921+0000","last_peered":"2024-09-21T03:28:32.559921+0000","last_clean":"2024-09-21T03:28:32.559921+0000","last_became_active":"2024-09-21T03:27:01.342861+0000","last_became_peered":"2024-09-21T03:27:01.342861+0000","last_unstale":"2024-09-21T03:28:32.559921+0000","last_undegraded":"2024-09-21T03:28:32.559921+0000","last_fullsized":"2024-09-21T03:28:32.559921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:00.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.568926+0000","last_change":"2024-09-21T03:27:01.330618+0000","last_active":"2024-09-21T03:28:32.568926+0000","last_peered":"2024-09-21T03:28:32.568926+0000","last_clean":"2024-09-21T03:28:32.568926+0000","last_became_active":"2024-09-21T03:27:01.330018+0000","last_became_peered":"2024-09-21T03:27:01.330018+0000","last_unstale":"2024-09-21T03:28:32.568926+0000","last_undegraded":"2024-09-21T03:28:32.568926+0000","last_fullsized":"2024-09-21T03:28:32.568926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:03.291698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.569953+0000","last_change":"2024-09-21T03:27:01.332840+0000","last_active":"2024-09-21T03:28:32.569953+0000","last_peered":"2024-09-21T03:28:32.569953+0000","last_clean":"2024-09-21T03:28:32.569953+0000","last_became_active":"2024-09-21T03:27:01.332513+0000","last_became_peered":"2024-09-21T03:27:01.332513+0000","last_unstale":"2024-09-21T03:28:32.569953+0000","last_undegraded":"2024-09-21T03:28:32.569953+0000","last_fullsized":"2024-09-21T03:28:32.569953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:17.554934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.580544+0000","last_change":"2024-09-21T03:27:01.338147+0000","last_active":"2024-09-21T03:28:32.580544+0000","last_peered":"2024-09-21T03:28:32.580544+0000","last_clean":"2024-09-21T03:28:32.580544+0000","last_became_active":"2024-09-21T03:27:01.337759+0000","last_became_peered":"2024-09-21T03:27:01.337759+0000","last_unstale":"2024-09-21T03:28:32.580544+0000","last_undegraded":"2024-09-21T03:28:32.580544+0000","last_fullsized":"2024-09-21T03:28:32.580544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:38.999593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.570880+0000","last_change":"2024-09-21T03:27:01.343373+0000","last_active":"2024-09-21T03:28:32.570880+0000","last_peered":"2024-09-21T03:28:32.570880+0000","last_clean":"2024-09-21T03:28:32.570880+0000","last_became_active":"2024-09-21T03:27:01.342924+0000","last_became_peered":"2024-09-21T03:27:01.342924+0000","last_unstale":"2024-09-21T03:28:32.570880+0000","last_undegraded":"2024-09-21T03:28:32.570880+0000","last_fullsized":"2024-09-21T03:28:32.570880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:33.869584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574841+0000","last_change":"2024-09-21T03:27:01.340120+0000","last_active":"2024-09-21T03:28:32.574841+0000","last_peered":"2024-09-21T03:28:32.574841+0000","last_clean":"2024-09-21T03:28:32.574841+0000","last_became_active":"2024-09-21T03:27:01.339708+0000","last_became_peered":"2024-09-21T03:27:01.339708+0000","last_unstale":"2024-09-21T03:28:32.574841+0000","last_undegraded":"2024-09-21T03:28:32.574841+0000","last_fullsized":"2024-09-21T03:28:32.574841+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:56.193494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564924+0000","last_change":"2024-09-21T03:27:01.340205+0000","last_active":"2024-09-21T03:28:32.564924+0000","last_peered":"2024-09-21T03:28:32.564924+0000","last_clean":"2024-09-21T03:28:32.564924+0000","last_became_active":"2024-09-21T03:27:01.339839+0000","last_became_peered":"2024-09-21T03:27:01.339839+0000","last_unstale":"2024-09-21T03:28:32.564924+0000","last_undegraded":"2024-09-21T03:28:32.564924+0000","last_fullsized":"2024-09-21T03:28:32.564924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:05.248636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.568589+0000","last_change":"2024-09-21T03:27:01.334715+0000","last_active":"2024-09-21T03:28:32.568589+0000","last_peered":"2024-09-21T03:28:32.568589+0000","last_clean":"2024-09-21T03:28:32.568589+0000","last_became_active":"2024-09-21T03:27:01.334392+0000","last_became_peered":"2024-09-21T03:27:01.334392+0000","last_unstale":"2024-09-21T03:28:32.568589+0000","last_undegraded":"2024-09-21T03:28:32.568589+0000","last_fullsized":"2024-09-21T03:28:32.568589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:18.934063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.554357+0000","last_change":"2024-09-21T03:27:01.337167+0000","last_active":"2024-09-21T03:28:32.554357+0000","last_peered":"2024-09-21T03:28:32.554357+0000","last_clean":"2024-09-21T03:28:32.554357+0000","last_became_active":"2024-09-21T03:27:01.336748+0000","last_became_peered":"2024-09-21T03:27:01.336748+0000","last_unstale":"2024-09-21T03:28:32.554357+0000","last_undegraded":"2024-09-21T03:28:32.554357+0000","last_fullsized":"2024-09-21T03:28:32.554357+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:12.186727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564435+0000","last_change":"2024-09-21T03:27:01.336260+0000","last_active":"2024-09-21T03:28:32.564435+0000","last_peered":"2024-09-21T03:28:32.564435+0000","last_clean":"2024-09-21T03:28:32.564435+0000","last_became_active":"2024-09-21T03:27:01.335889+0000","last_became_peered":"2024-09-21T03:27:01.335889+0000","last_unstale":"2024-09-21T03:28:32.564435+0000","last_undegraded":"2024-09-21T03:28:32.564435+0000","last_fullsized":"2024-09-21T03:28:32.564435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:02.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.575723+0000","last_change":"2024-09-21T03:27:01.337181+0000","last_active":"2024-09-21T03:28:32.575723+0000","last_peered":"2024-09-21T03:28:32.575723+0000","last_clean":"2024-09-21T03:28:32.575723+0000","last_became_active":"2024-09-21T03:27:01.336813+0000","last_became_peered":"2024-09-21T03:27:01.336813+0000","last_unstale":"2024-09-21T03:28:32.575723+0000","last_undegraded":"2024-09-21T03:28:32.575723+0000","last_fullsized":"2024-09-21T03:28:32.575723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:04.052319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.582323+0000","last_change":"2024-09-21T03:27:01.342079+0000","last_active":"2024-09-21T03:28:32.582323+0000","last_peered":"2024-09-21T03:28:32.582323+0000","last_clean":"2024-09-21T03:28:32.582323+0000","last_became_active":"2024-09-21T03:27:01.341430+0000","last_became_peered":"2024-09-21T03:27:01.341430+0000","last_unstale":"2024-09-21T03:28:32.582323+0000","last_undegraded":"2024-09-21T03:28:32.582323+0000","last_fullsized":"2024-09-21T03:28:32.582323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:08.029978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.568191+0000","last_change":"2024-09-21T03:27:01.338694+0000","last_active":"2024-09-21T03:28:32.568191+0000","last_peered":"2024-09-21T03:28:32.568191+0000","last_clean":"2024-09-21T03:28:32.568191+0000","last_became_active":"2024-09-21T03:27:01.338328+0000","last_became_peered":"2024-09-21T03:27:01.338328+0000","last_unstale":"2024-09-21T03:28:32.568191+0000","last_undegraded":"2024-09-21T03:28:32.568191+0000","last_fullsized":"2024-09-21T03:28:32.568191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:50.292238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565193+0000","last_change":"2024-09-21T03:27:01.336974+0000","last_active":"2024-09-21T03:28:32.565193+0000","last_peered":"2024-09-21T03:28:32.565193+0000","last_clean":"2024-09-21T03:28:32.565193+0000","last_became_active":"2024-09-21T03:27:01.336592+0000","last_became_peered":"2024-09-21T03:27:01.336592+0000","last_unstale":"2024-09-21T03:28:32.565193+0000","last_undegraded":"2024-09-21T03:28:32.565193+0000","last_fullsized":"2024-09-21T03:28:32.565193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:09.399348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.579658+0000","last_change":"2024-09-21T03:27:33.431767+0000","last_active":"2024-09-21T03:28:32.579658+0000","last_peered":"2024-09-21T03:28:32.579658+0000","last_clean":"2024-09-21T03:28:32.579658+0000","last_became_active":"2024-09-21T03:27:33.431233+0000","last_became_peered":"2024-09-21T03:27:33.431233+0000","last_unstale":"2024-09-21T03:28:32.579658+0000","last_undegraded":"2024-09-21T03:28:32.579658+0000","last_fullsized":"2024-09-21T03:28:32.579658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:48.425375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574746+0000","last_change":"2024-09-21T03:27:01.331242+0000","last_active":"2024-09-21T03:28:32.574746+0000","last_peered":"2024-09-21T03:28:32.574746+0000","last_clean":"2024-09-21T03:28:32.574746+0000","last_became_active":"2024-09-21T03:27:01.330883+0000","last_became_peered":"2024-09-21T03:27:01.330883+0000","last_unstale":"2024-09-21T03:28:32.574746+0000","last_undegraded":"2024-09-21T03:28:32.574746+0000","last_fullsized":"2024-09-21T03:28:32.574746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:07.158448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.576098+0000","last_change":"2024-09-21T03:27:01.335522+0000","last_active":"2024-09-21T03:28:32.576098+0000","last_peered":"2024-09-21T03:28:32.576098+0000","last_clean":"2024-09-21T03:28:32.576098+0000","last_became_active":"2024-09-21T03:27:01.335114+0000","last_became_peered":"2024-09-21T03:27:01.335114+0000","last_unstale":"2024-09-21T03:28:32.576098+0000","last_undegraded":"2024-09-21T03:28:32.576098+0000","last_fullsized":"2024-09-21T03:28:32.576098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:28.383410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.567676+0000","last_change":"2024-09-21T03:27:01.329428+0000","last_active":"2024-09-21T03:28:32.567676+0000","last_peered":"2024-09-21T03:28:32.567676+0000","last_clean":"2024-09-21T03:28:32.567676+0000","last_became_active":"2024-09-21T03:27:01.328882+0000","last_became_peered":"2024-09-21T03:27:01.328882+0000","last_unstale":"2024-09-21T03:28:32.567676+0000","last_undegraded":"2024-09-21T03:28:32.567676+0000","last_fullsized":"2024-09-21T03:28:32.567676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:58.634995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.561654+0000","last_change":"2024-09-21T03:27:33.432442+0000","last_active":"2024-09-21T03:28:32.561654+0000","last_peered":"2024-09-21T03:28:32.561654+0000","last_clean":"2024-09-21T03:28:32.561654+0000","last_became_active":"2024-09-21T03:27:33.431881+0000","last_became_peered":"2024-09-21T03:27:33.431881+0000","last_unstale":"2024-09-21T03:28:32.561654+0000","last_undegraded":"2024-09-21T03:28:32.561654+0000","last_fullsized":"2024-09-21T03:28:32.561654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:46.356644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.562491+0000","last_change":"2024-09-21T03:27:01.330988+0000","last_active":"2024-09-21T03:28:32.562491+0000","last_peered":"2024-09-21T03:28:32.562491+0000","last_clean":"2024-09-21T03:28:32.562491+0000","last_became_active":"2024-09-21T03:27:01.330574+0000","last_became_peered":"2024-09-21T03:27:01.330574+0000","last_unstale":"2024-09-21T03:28:32.562491+0000","last_undegraded":"2024-09-21T03:28:32.562491+0000","last_fullsized":"2024-09-21T03:28:32.562491+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:45.277947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.573563+0000","last_change":"2024-09-21T03:27:01.339606+0000","last_active":"2024-09-21T03:28:32.573563+0000","last_peered":"2024-09-21T03:28:32.573563+0000","last_clean":"2024-09-21T03:28:32.573563+0000","last_became_active":"2024-09-21T03:27:01.339208+0000","last_became_peered":"2024-09-21T03:27:01.339208+0000","last_unstale":"2024-09-21T03:28:32.573563+0000","last_undegraded":"2024-09-21T03:28:32.573563+0000","last_fullsized":"2024-09-21T03:28:32.573563+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:31.593363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.578941+0000","last_change":"2024-09-21T03:27:01.343634+0000","last_active":"2024-09-21T03:28:32.578941+0000","last_peered":"2024-09-21T03:28:32.578941+0000","last_clean":"2024-09-21T03:28:32.578941+0000","last_became_active":"2024-09-21T03:27:01.342965+0000","last_became_peered":"2024-09-21T03:27:01.342965+0000","last_unstale":"2024-09-21T03:28:32.578941+0000","last_undegraded":"2024-09-21T03:28:32.578941+0000","last_fullsized":"2024-09-21T03:28:32.578941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:46.916198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564776+0000","last_change":"2024-09-21T03:27:01.324993+0000","last_active":"2024-09-21T03:28:32.564776+0000","last_peered":"2024-09-21T03:28:32.564776+0000","last_clean":"2024-09-21T03:28:32.564776+0000","last_became_active":"2024-09-21T03:27:01.324593+0000","last_became_peered":"2024-09-21T03:27:01.324593+0000","last_unstale":"2024-09-21T03:28:32.564776+0000","last_undegraded":"2024-09-21T03:28:32.564776+0000","last_fullsized":"2024-09-21T03:28:32.564776+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:13.211539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563667+0000","last_change":"2024-09-21T03:27:01.325232+0000","last_active":"2024-09-21T03:28:32.563667+0000","last_peered":"2024-09-21T03:28:32.563667+0000","last_clean":"2024-09-21T03:28:32.563667+0000","last_became_active":"2024-09-21T03:27:01.324570+0000","last_became_peered":"2024-09-21T03:27:01.324570+0000","last_unstale":"2024-09-21T03:28:32.563667+0000","last_undegraded":"2024-09-21T03:28:32.563667+0000","last_fullsized":"2024-09-21T03:28:32.563667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:30.843334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563783+0000","last_change":"2024-09-21T03:27:01.338789+0000","last_active":"2024-09-21T03:28:32.563783+0000","last_peered":"2024-09-21T03:28:32.563783+0000","last_clean":"2024-09-21T03:28:32.563783+0000","last_became_active":"2024-09-21T03:27:01.338282+0000","last_became_peered":"2024-09-21T03:27:01.338282+0000","last_unstale":"2024-09-21T03:28:32.563783+0000","last_undegraded":"2024-09-21T03:28:32.563783+0000","last_fullsized":"2024-09-21T03:28:32.563783+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:57.213343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.566643+0000","last_change":"2024-09-21T03:27:01.339521+0000","last_active":"2024-09-21T03:28:32.566643+0000","last_peered":"2024-09-21T03:28:32.566643+0000","last_clean":"2024-09-21T03:28:32.566643+0000","last_became_active":"2024-09-21T03:27:01.339175+0000","last_became_peered":"2024-09-21T03:27:01.339175+0000","last_unstale":"2024-09-21T03:28:32.566643+0000","last_undegraded":"2024-09-21T03:28:32.566643+0000","last_fullsized":"2024-09-21T03:28:32.566643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:46.163698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.581098+0000","last_change":"2024-09-21T03:27:01.338352+0000","last_active":"2024-09-21T03:28:32.581098+0000","last_peered":"2024-09-21T03:28:32.581098+0000","last_clean":"2024-09-21T03:28:32.581098+0000","last_became_active":"2024-09-21T03:27:01.337306+0000","last_became_peered":"2024-09-21T03:27:01.337306+0000","last_unstale":"2024-09-21T03:28:32.581098+0000","last_undegraded":"2024-09-21T03:28:32.581098+0000","last_fullsized":"2024-09-21T03:28:32.581098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.799039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.562245+0000","last_change":"2024-09-21T03:27:01.332461+0000","last_active":"2024-09-21T03:28:32.562245+0000","last_peered":"2024-09-21T03:28:32.562245+0000","last_clean":"2024-09-21T03:28:32.562245+0000","last_became_active":"2024-09-21T03:27:01.331912+0000","last_became_peered":"2024-09-21T03:27:01.331912+0000","last_unstale":"2024-09-21T03:28:32.562245+0000","last_undegraded":"2024-09-21T03:28:32.562245+0000","last_fullsized":"2024-09-21T03:28:32.562245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:34.771691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'5","reported_seq":136,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:43.588186+0000","last_change":"2024-09-21T03:27:33.482302+0000","last_active":"2024-09-21T03:28:43.588186+0000","last_peered":"2024-09-21T03:28:43.588186+0000","last_clean":"2024-09-21T03:28:43.588186+0000","last_became_active":"2024-09-21T03:27:33.478350+0000","last_became_peered":"2024-09-21T03:27:33.478350+0000","last_unstale":"2024-09-21T03:28:43.588186+0000","last_undegraded":"2024-09-21T03:28:43.588186+0000","last_fullsized":"2024-09-21T03:28:43.588186+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:46.916762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.566839+0000","last_change":"2024-09-21T03:27:01.324364+0000","last_active":"2024-09-21T03:28:32.566839+0000","last_peered":"2024-09-21T03:28:32.566839+0000","last_clean":"2024-09-21T03:28:32.566839+0000","last_became_active":"2024-09-21T03:27:01.324018+0000","last_became_peered":"2024-09-21T03:27:01.324018+0000","last_unstale":"2024-09-21T03:28:32.566839+0000","last_undegraded":"2024-09-21T03:28:32.566839+0000","last_fullsized":"2024-09-21T03:28:32.566839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:56.404893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.431029+0000","last_change":"2024-09-21T03:26:59.313289+0000","last_active":"2024-09-21T03:27:33.431029+0000","last_peered":"2024-09-21T03:27:33.431029+0000","last_clean":"2024-09-21T03:27:33.431029+0000","last_became_active":"2024-09-21T03:26:59.312877+0000","last_became_peered":"2024-09-21T03:26:59.312877+0000","last_unstale":"2024-09-21T03:27:33.431029+0000","last_undegraded":"2024-09-21T03:27:33.431029+0000","last_fullsized":"2024-09-21T03:27:33.431029+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:53.856541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427079+0000","last_change":"2024-09-21T03:26:59.307098+0000","last_active":"2024-09-21T03:27:33.427079+0000","last_peered":"2024-09-21T03:27:33.427079+0000","last_clean":"2024-09-21T03:27:33.427079+0000","last_became_active":"2024-09-21T03:26:59.306650+0000","last_became_peered":"2024-09-21T03:26:59.306650+0000","last_unstale":"2024-09-21T03:27:33.427079+0000","last_undegraded":"2024-09-21T03:27:33.427079+0000","last_fullsized":"2024-09-21T03:27:33.427079+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:44.033899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.570911+0000","last_change":"2024-09-21T03:27:01.328443+0000","last_active":"2024-09-21T03:28:32.570911+0000","last_peered":"2024-09-21T03:28:32.570911+0000","last_clean":"2024-09-21T03:28:32.570911+0000","last_became_active":"2024-09-21T03:27:01.328107+0000","last_became_peered":"2024-09-21T03:27:01.328107+0000","last_unstale":"2024-09-21T03:28:32.570911+0000","last_undegraded":"2024-09-21T03:28:32.570911+0000","last_fullsized":"2024-09-21T03:28:32.570911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:47.798308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428946+0000","last_change":"2024-09-21T03:26:59.317934+0000","last_active":"2024-09-21T03:27:33.428946+0000","last_peered":"2024-09-21T03:27:33.428946+0000","last_clean":"2024-09-21T03:27:33.428946+0000","last_became_active":"2024-09-21T03:26:59.317166+0000","last_became_peered":"2024-09-21T03:26:59.317166+0000","last_unstale":"2024-09-21T03:27:33.428946+0000","last_undegraded":"2024-09-21T03:27:33.428946+0000","last_fullsized":"2024-09-21T03:27:33.428946+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:52.746406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564062+0000","last_change":"2024-09-21T03:27:01.331226+0000","last_active":"2024-09-21T03:28:32.564062+0000","last_peered":"2024-09-21T03:28:32.564062+0000","last_clean":"2024-09-21T03:28:32.564062+0000","last_became_active":"2024-09-21T03:27:01.330889+0000","last_became_peered":"2024-09-21T03:27:01.330889+0000","last_unstale":"2024-09-21T03:28:32.564062+0000","last_undegraded":"2024-09-21T03:28:32.564062+0000","last_fullsized":"2024-09-21T03:28:32.564062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:51.181554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'3","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:41.584516+0000","last_change":"2024-09-21T03:26:59.311671+0000","last_active":"2024-09-21T03:28:41.584516+0000","last_peered":"2024-09-21T03:28:41.584516+0000","last_clean":"2024-09-21T03:28:41.584516+0000","last_became_active":"2024-09-21T03:26:59.311149+0000","last_became_peered":"2024-09-21T03:26:59.311149+0000","last_unstale":"2024-09-21T03:28:41.584516+0000","last_undegraded":"2024-09-21T03:28:41.584516+0000","last_fullsized":"2024-09-21T03:28:41.584516+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:47.878274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.570949+0000","last_change":"2024-09-21T03:27:01.336142+0000","last_active":"2024-09-21T03:28:32.570949+0000","last_peered":"2024-09-21T03:28:32.570949+0000","last_clean":"2024-09-21T03:28:32.570949+0000","last_became_active":"2024-09-21T03:27:01.335795+0000","last_became_peered":"2024-09-21T03:27:01.335795+0000","last_unstale":"2024-09-21T03:28:32.570949+0000","last_undegraded":"2024-09-21T03:28:32.570949+0000","last_fullsized":"2024-09-21T03:28:32.570949+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:29.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429365+0000","last_change":"2024-09-21T03:26:59.315077+0000","last_active":"2024-09-21T03:27:33.429365+0000","last_peered":"2024-09-21T03:27:33.429365+0000","last_clean":"2024-09-21T03:27:33.429365+0000","last_became_active":"2024-09-21T03:26:59.312347+0000","last_became_peered":"2024-09-21T03:26:59.312347+0000","last_unstale":"2024-09-21T03:27:33.429365+0000","last_undegraded":"2024-09-21T03:27:33.429365+0000","last_fullsized":"2024-09-21T03:27:33.429365+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:24.529066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565121+0000","last_change":"2024-09-21T03:27:01.324346+0000","last_active":"2024-09-21T03:28:32.565121+0000","last_peered":"2024-09-21T03:28:32.565121+0000","last_clean":"2024-09-21T03:28:32.565121+0000","last_became_active":"2024-09-21T03:27:01.323976+0000","last_became_peered":"2024-09-21T03:27:01.323976+0000","last_unstale":"2024-09-21T03:28:32.565121+0000","last_undegraded":"2024-09-21T03:28:32.565121+0000","last_fullsized":"2024-09-21T03:28:32.565121+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:04.591595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429516+0000","last_change":"2024-09-21T03:26:59.326206+0000","last_active":"2024-09-21T03:27:33.429516+0000","last_peered":"2024-09-21T03:27:33.429516+0000","last_clean":"2024-09-21T03:27:33.429516+0000","last_became_active":"2024-09-21T03:26:59.325755+0000","last_became_peered":"2024-09-21T03:26:59.325755+0000","last_unstale":"2024-09-21T03:27:33.429516+0000","last_undegraded":"2024-09-21T03:27:33.429516+0000","last_fullsized":"2024-09-21T03:27:33.429516+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:23.220795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574096+0000","last_change":"2024-09-21T03:27:01.341501+0000","last_active":"2024-09-21T03:28:32.574096+0000","last_peered":"2024-09-21T03:28:32.574096+0000","last_clean":"2024-09-21T03:28:32.574096+0000","last_became_active":"2024-09-21T03:27:01.341039+0000","last_became_peered":"2024-09-21T03:27:01.341039+0000","last_unstale":"2024-09-21T03:28:32.574096+0000","last_undegraded":"2024-09-21T03:28:32.574096+0000","last_fullsized":"2024-09-21T03:28:32.574096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:15.132180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'31","reported_seq":247,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:42.593727+0000","last_change":"2024-09-21T03:26:59.313489+0000","last_active":"2024-09-21T03:28:42.593727+0000","last_peered":"2024-09-21T03:28:42.593727+0000","last_clean":"2024-09-21T03:28:42.593727+0000","last_became_active":"2024-09-21T03:26:59.313022+0000","last_became_peered":"2024-09-21T03:26:59.313022+0000","last_unstale":"2024-09-21T03:28:42.593727+0000","last_undegraded":"2024-09-21T03:28:42.593727+0000","last_fullsized":"2024-09-21T03:28:42.593727+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:08.541315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65006,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":193,"num_read_kb":159,"num_write":32,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.578105+0000","last_change":"2024-09-21T03:27:01.339329+0000","last_active":"2024-09-21T03:28:32.578105+0000","last_peered":"2024-09-21T03:28:32.578105+0000","last_clean":"2024-09-21T03:28:32.578105+0000","last_became_active":"2024-09-21T03:27:01.338982+0000","last_became_peered":"2024-09-21T03:27:01.338982+0000","last_unstale":"2024-09-21T03:28:32.578105+0000","last_undegraded":"2024-09-21T03:28:32.578105+0000","last_fullsized":"2024-09-21T03:28:32.578105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:41.759450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'6","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430224+0000","last_change":"2024-09-21T03:26:59.313270+0000","last_active":"2024-09-21T03:27:33.430224+0000","last_peered":"2024-09-21T03:27:33.430224+0000","last_clean":"2024-09-21T03:27:33.430224+0000","last_became_active":"2024-09-21T03:26:59.312867+0000","last_became_peered":"2024-09-21T03:26:59.312867+0000","last_unstale":"2024-09-21T03:27:33.430224+0000","last_undegraded":"2024-09-21T03:27:33.430224+0000","last_fullsized":"2024-09-21T03:27:33.430224+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:53.798022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.572806+0000","last_change":"2024-09-21T03:27:01.331268+0000","last_active":"2024-09-21T03:28:32.572806+0000","last_peered":"2024-09-21T03:28:32.572806+0000","last_clean":"2024-09-21T03:28:32.572806+0000","last_became_active":"2024-09-21T03:27:01.330922+0000","last_became_peered":"2024-09-21T03:27:01.330922+0000","last_unstale":"2024-09-21T03:28:32.572806+0000","last_undegraded":"2024-09-21T03:28:32.572806+0000","last_fullsized":"2024-09-21T03:28:32.572806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428917+0000","last_change":"2024-09-21T03:26:59.318388+0000","last_active":"2024-09-21T03:27:33.428917+0000","last_peered":"2024-09-21T03:27:33.428917+0000","last_clean":"2024-09-21T03:27:33.428917+0000","last_became_active":"2024-09-21T03:26:59.317938+0000","last_became_peered":"2024-09-21T03:26:59.317938+0000","last_unstale":"2024-09-21T03:27:33.428917+0000","last_undegraded":"2024-09-21T03:27:33.428917+0000","last_fullsized":"2024-09-21T03:27:33.428917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.499699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.575303+0000","last_change":"2024-09-21T03:27:33.432191+0000","last_active":"2024-09-21T03:28:32.575303+0000","last_peered":"2024-09-21T03:28:32.575303+0000","last_clean":"2024-09-21T03:28:32.575303+0000","last_became_active":"2024-09-21T03:27:33.431667+0000","last_became_peered":"2024-09-21T03:27:33.431667+0000","last_unstale":"2024-09-21T03:28:32.575303+0000","last_undegraded":"2024-09-21T03:28:32.575303+0000","last_fullsized":"2024-09-21T03:28:32.575303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:13.293063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'5","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:43.550712+0000","last_change":"2024-09-21T03:26:59.311345+0000","last_active":"2024-09-21T03:28:43.550712+0000","last_peered":"2024-09-21T03:28:43.550712+0000","last_clean":"2024-09-21T03:28:43.550712+0000","last_became_active":"2024-09-21T03:26:59.310928+0000","last_became_peered":"2024-09-21T03:26:59.310928+0000","last_unstale":"2024-09-21T03:28:43.550712+0000","last_undegraded":"2024-09-21T03:28:43.550712+0000","last_fullsized":"2024-09-21T03:28:43.550712+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.614955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.571566+0000","last_change":"2024-09-21T03:27:01.335785+0000","last_active":"2024-09-21T03:28:32.571566+0000","last_peered":"2024-09-21T03:28:32.571566+0000","last_clean":"2024-09-21T03:28:32.571566+0000","last_became_active":"2024-09-21T03:27:01.335396+0000","last_became_peered":"2024-09-21T03:27:01.335396+0000","last_unstale":"2024-09-21T03:28:32.571566+0000","last_undegraded":"2024-09-21T03:28:32.571566+0000","last_fullsized":"2024-09-21T03:28:32.571566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:52.785641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'15","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.448967+0000","last_change":"2024-09-21T03:27:33.448404+0000","last_active":"2024-09-21T03:27:33.448967+0000","last_peered":"2024-09-21T03:27:33.448967+0000","last_clean":"2024-09-21T03:27:33.448967+0000","last_became_active":"2024-09-21T03:27:33.434405+0000","last_became_peered":"2024-09-21T03:27:33.434405+0000","last_unstale":"2024-09-21T03:27:33.448967+0000","last_undegraded":"2024-09-21T03:27:33.448967+0000","last_fullsized":"2024-09-21T03:27:33.448967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:05.107391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.573683+0000","last_change":"2024-09-21T03:27:01.342282+0000","last_active":"2024-09-21T03:28:32.573683+0000","last_peered":"2024-09-21T03:28:32.573683+0000","last_clean":"2024-09-21T03:28:32.573683+0000","last_became_active":"2024-09-21T03:27:01.341790+0000","last_became_peered":"2024-09-21T03:27:01.341790+0000","last_unstale":"2024-09-21T03:28:32.573683+0000","last_undegraded":"2024-09-21T03:28:32.573683+0000","last_fullsized":"2024-09-21T03:28:32.573683+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:05.432720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428084+0000","last_change":"2024-09-21T03:26:59.312442+0000","last_active":"2024-09-21T03:27:33.428084+0000","last_peered":"2024-09-21T03:27:33.428084+0000","last_clean":"2024-09-21T03:27:33.428084+0000","last_became_active":"2024-09-21T03:26:59.311601+0000","last_became_peered":"2024-09-21T03:26:59.311601+0000","last_unstale":"2024-09-21T03:27:33.428084+0000","last_undegraded":"2024-09-21T03:27:33.428084+0000","last_fullsized":"2024-09-21T03:27:33.428084+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:32.421228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.579550+0000","last_change":"2024-09-21T03:27:01.337917+0000","last_active":"2024-09-21T03:28:32.579550+0000","last_peered":"2024-09-21T03:28:32.579550+0000","last_clean":"2024-09-21T03:28:32.579550+0000","last_became_active":"2024-09-21T03:27:01.337457+0000","last_became_peered":"2024-09-21T03:27:01.337457+0000","last_unstale":"2024-09-21T03:28:32.579550+0000","last_undegraded":"2024-09-21T03:28:32.579550+0000","last_fullsized":"2024-09-21T03:28:32.579550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:49.011795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.597334+0000","last_change":"2024-09-21T03:27:33.597239+0000","last_active":"2024-09-21T03:27:33.597334+0000","last_peered":"2024-09-21T03:27:33.597334+0000","last_clean":"2024-09-21T03:27:33.597334+0000","last_became_active":"2024-09-21T03:27:33.593787+0000","last_became_peered":"2024-09-21T03:27:33.593787+0000","last_unstale":"2024-09-21T03:27:33.597334+0000","last_undegraded":"2024-09-21T03:27:33.597334+0000","last_fullsized":"2024-09-21T03:27:33.597334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:35.015446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.581336+0000","last_change":"2024-09-21T03:27:01.338489+0000","last_active":"2024-09-21T03:28:32.581336+0000","last_peered":"2024-09-21T03:28:32.581336+0000","last_clean":"2024-09-21T03:28:32.581336+0000","last_became_active":"2024-09-21T03:27:01.338123+0000","last_became_peered":"2024-09-21T03:27:01.338123+0000","last_unstale":"2024-09-21T03:28:32.581336+0000","last_undegraded":"2024-09-21T03:28:32.581336+0000","last_fullsized":"2024-09-21T03:28:32.581336+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:15.137546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429644+0000","last_change":"2024-09-21T03:26:59.315371+0000","last_active":"2024-09-21T03:27:33.429644+0000","last_peered":"2024-09-21T03:27:33.429644+0000","last_clean":"2024-09-21T03:27:33.429644+0000","last_became_active":"2024-09-21T03:26:59.314439+0000","last_became_peered":"2024-09-21T03:26:59.314439+0000","last_unstale":"2024-09-21T03:27:33.429644+0000","last_undegraded":"2024-09-21T03:27:33.429644+0000","last_fullsized":"2024-09-21T03:27:33.429644+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:13.656324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.567006+0000","last_change":"2024-09-21T03:27:01.331086+0000","last_active":"2024-09-21T03:28:32.567006+0000","last_peered":"2024-09-21T03:28:32.567006+0000","last_clean":"2024-09-21T03:28:32.567006+0000","last_became_active":"2024-09-21T03:27:01.329113+0000","last_became_peered":"2024-09-21T03:27:01.329113+0000","last_unstale":"2024-09-21T03:28:32.567006+0000","last_undegraded":"2024-09-21T03:28:32.567006+0000","last_fullsized":"2024-09-21T03:28:32.567006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:25.514730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.433098+0000","last_change":"2024-09-21T03:26:59.554087+0000","last_active":"2024-09-21T03:27:33.433098+0000","last_peered":"2024-09-21T03:27:33.433098+0000","last_clean":"2024-09-21T03:27:33.433098+0000","last_became_active":"2024-09-21T03:26:59.553610+0000","last_became_peered":"2024-09-21T03:26:59.553610+0000","last_unstale":"2024-09-21T03:27:33.433098+0000","last_undegraded":"2024-09-21T03:27:33.433098+0000","last_fullsized":"2024-09-21T03:27:33.433098+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:22.048057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.577544+0000","last_change":"2024-09-21T03:27:01.339987+0000","last_active":"2024-09-21T03:28:32.577544+0000","last_peered":"2024-09-21T03:28:32.577544+0000","last_clean":"2024-09-21T03:28:32.577544+0000","last_became_active":"2024-09-21T03:27:01.339060+0000","last_became_peered":"2024-09-21T03:27:01.339060+0000","last_unstale":"2024-09-21T03:28:32.577544+0000","last_undegraded":"2024-09-21T03:28:32.577544+0000","last_fullsized":"2024-09-21T03:28:32.577544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.901337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429743+0000","last_change":"2024-09-21T03:25:50.381920+0000","last_active":"2024-09-21T03:27:33.429743+0000","last_peered":"2024-09-21T03:27:33.429743+0000","last_clean":"2024-09-21T03:27:33.429743+0000","last_became_active":"2024-09-21T03:25:50.376170+0000","last_became_peered":"2024-09-21T03:25:50.376170+0000","last_unstale":"2024-09-21T03:27:33.429743+0000","last_undegraded":"2024-09-21T03:27:33.429743+0000","last_fullsized":"2024-09-21T03:27:33.429743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_clean_scrub_stamp":"2024-09-21T03:22:31.269883+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:57.281150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'3","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.477054+0000","last_change":"2024-09-21T03:26:59.313612+0000","last_active":"2024-09-21T03:27:33.477054+0000","last_peered":"2024-09-21T03:27:33.477054+0000","last_clean":"2024-09-21T03:27:33.477054+0000","last_became_active":"2024-09-21T03:26:59.313060+0000","last_became_peered":"2024-09-21T03:26:59.313060+0000","last_unstale":"2024-09-21T03:27:33.477054+0000","last_undegraded":"2024-09-21T03:27:33.477054+0000","last_fullsized":"2024-09-21T03:27:33.477054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:06.193526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.572676+0000","last_change":"2024-09-21T03:27:01.337784+0000","last_active":"2024-09-21T03:28:32.572676+0000","last_peered":"2024-09-21T03:28:32.572676+0000","last_clean":"2024-09-21T03:28:32.572676+0000","last_became_active":"2024-09-21T03:27:01.337415+0000","last_became_peered":"2024-09-21T03:27:01.337415+0000","last_unstale":"2024-09-21T03:28:32.572676+0000","last_undegraded":"2024-09-21T03:28:32.572676+0000","last_fullsized":"2024-09-21T03:28:32.572676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:40.482054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'3","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.436960+0000","last_change":"2024-09-21T03:27:33.436319+0000","last_active":"2024-09-21T03:27:33.436960+0000","last_peered":"2024-09-21T03:27:33.436960+0000","last_clean":"2024-09-21T03:27:33.436960+0000","last_became_active":"2024-09-21T03:27:33.431555+0000","last_became_peered":"2024-09-21T03:27:33.431555+0000","last_unstale":"2024-09-21T03:27:33.436960+0000","last_undegraded":"2024-09-21T03:27:33.436960+0000","last_fullsized":"2024-09-21T03:27:33.436960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:16.073988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563635+0000","last_change":"2024-09-21T03:27:01.333769+0000","last_active":"2024-09-21T03:28:32.563635+0000","last_peered":"2024-09-21T03:28:32.563635+0000","last_clean":"2024-09-21T03:28:32.563635+0000","last_became_active":"2024-09-21T03:27:01.333436+0000","last_became_peered":"2024-09-21T03:27:01.333436+0000","last_unstale":"2024-09-21T03:28:32.563635+0000","last_undegraded":"2024-09-21T03:28:32.563635+0000","last_fullsized":"2024-09-21T03:28:32.563635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:30.395916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:43.550125+0000","last_change":"2024-09-21T03:26:59.325314+0000","last_active":"2024-09-21T03:28:43.550125+0000","last_peered":"2024-09-21T03:28:43.550125+0000","last_clean":"2024-09-21T03:28:43.550125+0000","last_became_active":"2024-09-21T03:26:59.324752+0000","last_became_peered":"2024-09-21T03:26:59.324752+0000","last_unstale":"2024-09-21T03:28:43.550125+0000","last_undegraded":"2024-09-21T03:28:43.550125+0000","last_fullsized":"2024-09-21T03:28:43.550125+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.252596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.559825+0000","last_change":"2024-09-21T03:27:01.335154+0000","last_active":"2024-09-21T03:28:32.559825+0000","last_peered":"2024-09-21T03:28:32.559825+0000","last_clean":"2024-09-21T03:28:32.559825+0000","last_became_active":"2024-09-21T03:27:01.334633+0000","last_became_peered":"2024-09-21T03:27:01.334633+0000","last_unstale":"2024-09-21T03:28:32.559825+0000","last_undegraded":"2024-09-21T03:28:32.559825+0000","last_fullsized":"2024-09-21T03:28:32.559825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:27.992858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429757+0000","last_change":"2024-09-21T03:26:59.314019+0000","last_active":"2024-09-21T03:27:33.429757+0000","last_peered":"2024-09-21T03:27:33.429757+0000","last_clean":"2024-09-21T03:27:33.429757+0000","last_became_active":"2024-09-21T03:26:59.311915+0000","last_became_peered":"2024-09-21T03:26:59.311915+0000","last_unstale":"2024-09-21T03:27:33.429757+0000","last_undegraded":"2024-09-21T03:27:33.429757+0000","last_fullsized":"2024-09-21T03:27:33.429757+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:51.412303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.558537+0000","last_change":"2024-09-21T03:27:33.433193+0000","last_active":"2024-09-21T03:28:32.558537+0000","last_peered":"2024-09-21T03:28:32.558537+0000","last_clean":"2024-09-21T03:28:32.558537+0000","last_became_active":"2024-09-21T03:27:33.432729+0000","last_became_peered":"2024-09-21T03:27:33.432729+0000","last_unstale":"2024-09-21T03:28:32.558537+0000","last_undegraded":"2024-09-21T03:28:32.558537+0000","last_fullsized":"2024-09-21T03:28:32.558537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:37.575110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424782+0000","last_change":"2024-09-21T03:26:59.325943+0000","last_active":"2024-09-21T03:27:33.424782+0000","last_peered":"2024-09-21T03:27:33.424782+0000","last_clean":"2024-09-21T03:27:33.424782+0000","last_became_active":"2024-09-21T03:26:59.325524+0000","last_became_peered":"2024-09-21T03:26:59.325524+0000","last_unstale":"2024-09-21T03:27:33.424782+0000","last_undegraded":"2024-09-21T03:27:33.424782+0000","last_fullsized":"2024-09-21T03:27:33.424782+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:45.111480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563254+0000","last_change":"2024-09-21T03:27:01.335369+0000","last_active":"2024-09-21T03:28:32.563254+0000","last_peered":"2024-09-21T03:28:32.563254+0000","last_clean":"2024-09-21T03:28:32.563254+0000","last_became_active":"2024-09-21T03:27:01.334997+0000","last_became_peered":"2024-09-21T03:27:01.334997+0000","last_unstale":"2024-09-21T03:28:32.563254+0000","last_undegraded":"2024-09-21T03:28:32.563254+0000","last_fullsized":"2024-09-21T03:28:32.563254+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:10.257451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429553+0000","last_change":"2024-09-21T03:26:59.326204+0000","last_active":"2024-09-21T03:27:33.429553+0000","last_peered":"2024-09-21T03:27:33.429553+0000","last_clean":"2024-09-21T03:27:33.429553+0000","last_became_active":"2024-09-21T03:26:59.325431+0000","last_became_peered":"2024-09-21T03:26:59.325431+0000","last_unstale":"2024-09-21T03:27:33.429553+0000","last_undegraded":"2024-09-21T03:27:33.429553+0000","last_fullsized":"2024-09-21T03:27:33.429553+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:27.863217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.558220+0000","last_change":"2024-09-21T03:27:01.333861+0000","last_active":"2024-09-21T03:28:32.558220+0000","last_peered":"2024-09-21T03:28:32.558220+0000","last_clean":"2024-09-21T03:28:32.558220+0000","last_became_active":"2024-09-21T03:27:01.333485+0000","last_became_peered":"2024-09-21T03:27:01.333485+0000","last_unstale":"2024-09-21T03:28:32.558220+0000","last_undegraded":"2024-09-21T03:28:32.558220+0000","last_fullsized":"2024-09-21T03:28:32.558220+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:04.475811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'12","reported_seq":35,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424914+0000","last_change":"2024-09-21T03:26:59.309767+0000","last_active":"2024-09-21T03:27:33.424914+0000","last_peered":"2024-09-21T03:27:33.424914+0000","last_clean":"2024-09-21T03:27:33.424914+0000","last_became_active":"2024-09-21T03:26:59.309354+0000","last_became_peered":"2024-09-21T03:26:59.309354+0000","last_unstale":"2024-09-21T03:27:33.424914+0000","last_undegraded":"2024-09-21T03:27:33.424914+0000","last_fullsized":"2024-09-21T03:27:33.424914+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:26.728719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574479+0000","last_change":"2024-09-21T03:27:01.333097+0000","last_active":"2024-09-21T03:28:32.574479+0000","last_peered":"2024-09-21T03:28:32.574479+0000","last_clean":"2024-09-21T03:28:32.574479+0000","last_became_active":"2024-09-21T03:27:01.332743+0000","last_became_peered":"2024-09-21T03:27:01.332743+0000","last_unstale":"2024-09-21T03:28:32.574479+0000","last_undegraded":"2024-09-21T03:28:32.574479+0000","last_fullsized":"2024-09-21T03:28:32.574479+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:41.004213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.476588+0000","last_change":"2024-09-21T03:26:59.553974+0000","last_active":"2024-09-21T03:27:33.476588+0000","last_peered":"2024-09-21T03:27:33.476588+0000","last_clean":"2024-09-21T03:27:33.476588+0000","last_became_active":"2024-09-21T03:26:59.553647+0000","last_became_peered":"2024-09-21T03:26:59.553647+0000","last_unstale":"2024-09-21T03:27:33.476588+0000","last_undegraded":"2024-09-21T03:27:33.476588+0000","last_fullsized":"2024-09-21T03:27:33.476588+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:00.490951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563631+0000","last_change":"2024-09-21T03:27:01.325308+0000","last_active":"2024-09-21T03:28:32.563631+0000","last_peered":"2024-09-21T03:28:32.563631+0000","last_clean":"2024-09-21T03:28:32.563631+0000","last_became_active":"2024-09-21T03:27:01.324977+0000","last_became_peered":"2024-09-21T03:27:01.324977+0000","last_unstale":"2024-09-21T03:28:32.563631+0000","last_undegraded":"2024-09-21T03:28:32.563631+0000","last_fullsized":"2024-09-21T03:28:32.563631+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:10.093856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592537+0000","last_change":"2024-09-21T03:26:59.326017+0000","last_active":"2024-09-21T03:27:33.592537+0000","last_peered":"2024-09-21T03:27:33.592537+0000","last_clean":"2024-09-21T03:27:33.592537+0000","last_became_active":"2024-09-21T03:26:59.325588+0000","last_became_peered":"2024-09-21T03:26:59.325588+0000","last_unstale":"2024-09-21T03:27:33.592537+0000","last_undegraded":"2024-09-21T03:27:33.592537+0000","last_fullsized":"2024-09-21T03:27:33.592537+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:39.740449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.531947+0000","last_change":"2024-09-21T03:27:01.336871+0000","last_active":"2024-09-21T03:28:32.531947+0000","last_peered":"2024-09-21T03:28:32.531947+0000","last_clean":"2024-09-21T03:28:32.531947+0000","last_became_active":"2024-09-21T03:27:01.336507+0000","last_became_peered":"2024-09-21T03:27:01.336507+0000","last_unstale":"2024-09-21T03:28:32.531947+0000","last_undegraded":"2024-09-21T03:28:32.531947+0000","last_fullsized":"2024-09-21T03:28:32.531947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:58.674375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430477+0000","last_change":"2024-09-21T03:26:59.308997+0000","last_active":"2024-09-21T03:27:33.430477+0000","last_peered":"2024-09-21T03:27:33.430477+0000","last_clean":"2024-09-21T03:27:33.430477+0000","last_became_active":"2024-09-21T03:26:59.308338+0000","last_became_peered":"2024-09-21T03:26:59.308338+0000","last_unstale":"2024-09-21T03:27:33.430477+0000","last_undegraded":"2024-09-21T03:27:33.430477+0000","last_fullsized":"2024-09-21T03:27:33.430477+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:15.328468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.576878+0000","last_change":"2024-09-21T03:27:01.338808+0000","last_active":"2024-09-21T03:28:32.576878+0000","last_peered":"2024-09-21T03:28:32.576878+0000","last_clean":"2024-09-21T03:28:32.576878+0000","last_became_active":"2024-09-21T03:27:01.337987+0000","last_became_peered":"2024-09-21T03:27:01.337987+0000","last_unstale":"2024-09-21T03:28:32.576878+0000","last_undegraded":"2024-09-21T03:28:32.576878+0000","last_fullsized":"2024-09-21T03:28:32.576878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:56.265014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430766+0000","last_change":"2024-09-21T03:26:59.312684+0000","last_active":"2024-09-21T03:27:33.430766+0000","last_peered":"2024-09-21T03:27:33.430766+0000","last_clean":"2024-09-21T03:27:33.430766+0000","last_became_active":"2024-09-21T03:26:59.312038+0000","last_became_peered":"2024-09-21T03:26:59.312038+0000","last_unstale":"2024-09-21T03:27:33.430766+0000","last_undegraded":"2024-09-21T03:27:33.430766+0000","last_fullsized":"2024-09-21T03:27:33.430766+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:02.672255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.569931+0000","last_change":"2024-09-21T03:27:01.333080+0000","last_active":"2024-09-21T03:28:32.569931+0000","last_peered":"2024-09-21T03:28:32.569931+0000","last_clean":"2024-09-21T03:28:32.569931+0000","last_became_active":"2024-09-21T03:27:01.332663+0000","last_became_peered":"2024-09-21T03:27:01.332663+0000","last_unstale":"2024-09-21T03:28:32.569931+0000","last_undegraded":"2024-09-21T03:28:32.569931+0000","last_fullsized":"2024-09-21T03:28:32.569931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:34.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427673+0000","last_change":"2024-09-21T03:26:59.315490+0000","last_active":"2024-09-21T03:27:33.427673+0000","last_peered":"2024-09-21T03:27:33.427673+0000","last_clean":"2024-09-21T03:27:33.427673+0000","last_became_active":"2024-09-21T03:26:59.313789+0000","last_became_peered":"2024-09-21T03:26:59.313789+0000","last_unstale":"2024-09-21T03:27:33.427673+0000","last_undegraded":"2024-09-21T03:27:33.427673+0000","last_fullsized":"2024-09-21T03:27:33.427673+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:10.140711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574483+0000","last_change":"2024-09-21T03:27:01.341019+0000","last_active":"2024-09-21T03:28:32.574483+0000","last_peered":"2024-09-21T03:28:32.574483+0000","last_clean":"2024-09-21T03:28:32.574483+0000","last_became_active":"2024-09-21T03:27:01.340660+0000","last_became_peered":"2024-09-21T03:27:01.340660+0000","last_unstale":"2024-09-21T03:28:32.574483+0000","last_undegraded":"2024-09-21T03:28:32.574483+0000","last_fullsized":"2024-09-21T03:28:32.574483+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:16.736993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.577115+0000","last_change":"2024-09-21T03:27:01.341165+0000","last_active":"2024-09-21T03:28:32.577115+0000","last_peered":"2024-09-21T03:28:32.577115+0000","last_clean":"2024-09-21T03:28:32.577115+0000","last_became_active":"2024-09-21T03:27:01.340765+0000","last_became_peered":"2024-09-21T03:27:01.340765+0000","last_unstale":"2024-09-21T03:28:32.577115+0000","last_undegraded":"2024-09-21T03:28:32.577115+0000","last_fullsized":"2024-09-21T03:28:32.577115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:15.455944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429107+0000","last_change":"2024-09-21T03:26:59.313410+0000","last_active":"2024-09-21T03:27:33.429107+0000","last_peered":"2024-09-21T03:27:33.429107+0000","last_clean":"2024-09-21T03:27:33.429107+0000","last_became_active":"2024-09-21T03:26:59.312233+0000","last_became_peered":"2024-09-21T03:26:59.312233+0000","last_unstale":"2024-09-21T03:27:33.429107+0000","last_undegraded":"2024-09-21T03:27:33.429107+0000","last_fullsized":"2024-09-21T03:27:33.429107+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:26.089611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.572571+0000","last_change":"2024-09-21T03:27:01.335332+0000","last_active":"2024-09-21T03:28:32.572571+0000","last_peered":"2024-09-21T03:28:32.572571+0000","last_clean":"2024-09-21T03:28:32.572571+0000","last_became_active":"2024-09-21T03:27:01.334872+0000","last_became_peered":"2024-09-21T03:27:01.334872+0000","last_unstale":"2024-09-21T03:28:32.572571+0000","last_undegraded":"2024-09-21T03:28:32.572571+0000","last_fullsized":"2024-09-21T03:28:32.572571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:39.631246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429421+0000","last_change":"2024-09-21T03:26:59.314250+0000","last_active":"2024-09-21T03:27:33.429421+0000","last_peered":"2024-09-21T03:27:33.429421+0000","last_clean":"2024-09-21T03:27:33.429421+0000","last_became_active":"2024-09-21T03:26:59.313707+0000","last_became_peered":"2024-09-21T03:26:59.313707+0000","last_unstale":"2024-09-21T03:27:33.429421+0000","last_undegraded":"2024-09-21T03:27:33.429421+0000","last_fullsized":"2024-09-21T03:27:33.429421+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:22.935292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565595+0000","last_change":"2024-09-21T03:27:01.339230+0000","last_active":"2024-09-21T03:28:32.565595+0000","last_peered":"2024-09-21T03:28:32.565595+0000","last_clean":"2024-09-21T03:28:32.565595+0000","last_became_active":"2024-09-21T03:27:01.338881+0000","last_became_peered":"2024-09-21T03:27:01.338881+0000","last_unstale":"2024-09-21T03:28:32.565595+0000","last_undegraded":"2024-09-21T03:28:32.565595+0000","last_fullsized":"2024-09-21T03:28:32.565595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:49.189180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430888+0000","last_change":"2024-09-21T03:26:59.326549+0000","last_active":"2024-09-21T03:27:33.430888+0000","last_peered":"2024-09-21T03:27:33.430888+0000","last_clean":"2024-09-21T03:27:33.430888+0000","last_became_active":"2024-09-21T03:26:59.326091+0000","last_became_peered":"2024-09-21T03:26:59.326091+0000","last_unstale":"2024-09-21T03:27:33.430888+0000","last_undegraded":"2024-09-21T03:27:33.430888+0000","last_fullsized":"2024-09-21T03:27:33.430888+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:13.507716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.566939+0000","last_change":"2024-09-21T03:27:01.331367+0000","last_active":"2024-09-21T03:28:32.566939+0000","last_peered":"2024-09-21T03:28:32.566939+0000","last_clean":"2024-09-21T03:28:32.566939+0000","last_became_active":"2024-09-21T03:27:01.330604+0000","last_became_peered":"2024-09-21T03:27:01.330604+0000","last_unstale":"2024-09-21T03:28:32.566939+0000","last_undegraded":"2024-09-21T03:28:32.566939+0000","last_fullsized":"2024-09-21T03:28:32.566939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:32.207561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'9","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424843+0000","last_change":"2024-09-21T03:26:59.311912+0000","last_active":"2024-09-21T03:27:33.424843+0000","last_peered":"2024-09-21T03:27:33.424843+0000","last_clean":"2024-09-21T03:27:33.424843+0000","last_became_active":"2024-09-21T03:26:59.311571+0000","last_became_peered":"2024-09-21T03:26:59.311571+0000","last_unstale":"2024-09-21T03:27:33.424843+0000","last_undegraded":"2024-09-21T03:27:33.424843+0000","last_fullsized":"2024-09-21T03:27:33.424843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:55.946494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69148,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":583,"num_read_kb":363,"num_write":172,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":208676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776163,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":843776,"data_stored":413299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939686,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1536000,"data_stored":1070206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103212,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":413423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299441,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":413231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32596,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712556,"statfs":{"total":95995035648,"available":95961657344,"internally_reserved":0,"allocated":1445888,"data_stored":1005722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":412812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855424,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1458176,"data_stored":1004566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":851968,"data_stored":414403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":413123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":478647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542229,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":477919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":414083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:28:50.201 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T03:28:50.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:28:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.345+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/3355092772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df81029b0 msgr2=0x7f8df8102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.345+0000 7f8dfcbf1640 1 --2- 172.21.15.112:0/3355092772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df81029b0 0x7f8df8102dc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009900 tx=0x7f8dec02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.345+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/3355092772 shutdown_connections 2024-09-21T03:28:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.345+0000 7f8dfcbf1640 1 --2- 172.21.15.112:0/3355092772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df8108c30 0x7f8df810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.345+0000 7f8dfcbf1640 1 --2- 172.21.15.112:0/3355092772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8df81082e0 0x7f8df81086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.345+0000 7f8dfcbf1640 1 --2- 172.21.15.112:0/3355092772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df81029b0 0x7f8df8102dc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.345+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/3355092772 >> 172.21.15.112:0/3355092772 conn(0x7f8df80fe3b0 msgr2=0x7f8df8100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.345+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/3355092772 shutdown_connections 2024-09-21T03:28:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.345+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/3355092772 wait complete. 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8dfcbf1640 1 Processor -- start 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8dfcbf1640 1 -- start start 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8dfcbf1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df81029b0 0x7f8df819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8dfcbf1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8df81082e0 0x7f8df819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8dfcbf1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df8108c30 0x7f8df819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8dfcbf1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81128f0 con 0x7f8df8108c30 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8dfcbf1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8df8112770 con 0x7f8df81082e0 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8dfcbf1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8df8112a70 con 0x7f8df81029b0 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8df6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df8108c30 0x7f8df819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8df6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df8108c30 0x7f8df819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58424/0 (socket says 172.21.15.112:58424) 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8df6d9d640 1 -- 172.21.15.112:0/1438016504 learned_addr learned my addr 172.21.15.112:0/1438016504 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8df5d9b640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8df81082e0 0x7f8df819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8df6d9d640 1 -- 172.21.15.112:0/1438016504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df81029b0 msgr2=0x7f8df819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8df6d9d640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df81029b0 0x7f8df819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8df6d9d640 1 -- 172.21.15.112:0/1438016504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8df81082e0 msgr2=0x7f8df819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.346+0000 7f8df6d9d640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8df81082e0 0x7f8df819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.347+0000 7f8df6d9d640 1 -- 172.21.15.112:0/1438016504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df81a03d0 con 0x7f8df8108c30 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.347+0000 7f8df6d9d640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df8108c30 0x7f8df819fcd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8df000cad0 tx=0x7f8df000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.347+0000 7f8de77fe640 1 -- 172.21.15.112:0/1438016504 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df000ae20 con 0x7f8df8108c30 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.347+0000 7f8de77fe640 1 -- 172.21.15.112:0/1438016504 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8df0005020 con 0x7f8df8108c30 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.347+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/1438016504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df81a06c0 con 0x7f8df8108c30 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.347+0000 7f8de77fe640 1 -- 172.21.15.112:0/1438016504 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df000f6b0 con 0x7f8df8108c30 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.347+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/1438016504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df81a6e50 con 0x7f8df8108c30 2024-09-21T03:28:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.348+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/1438016504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df8102dc0 con 0x7f8df8108c30 2024-09-21T03:28:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.350+0000 7f8de77fe640 1 -- 172.21.15.112:0/1438016504 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f8df00051d0 con 0x7f8df8108c30 2024-09-21T03:28:50.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.351+0000 7f8de77fe640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8de00b65e0 0x7f8de00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:50.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.351+0000 7f8df659c640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8de00b65e0 0x7f8de00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:50.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.351+0000 7f8de77fe640 1 -- 172.21.15.112:0/1438016504 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f8df00e7860 con 0x7f8df8108c30 2024-09-21T03:28:50.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.352+0000 7f8df659c640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8de00b65e0 0x7f8de00b8af0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8dec02a930 tx=0x7f8dec031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:50.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.354+0000 7f8de77fe640 1 -- 172.21.15.112:0/1438016504 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8df0014030 con 0x7f8df8108c30 2024-09-21T03:28:50.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.531+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/1438016504 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8df8063800 con 0x7f8de00b65e0 2024-09-21T03:28:50.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:50 smithi112 ceph-mon[31442]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:50.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:50 smithi112 ceph-mon[31442]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:50.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:50 smithi112 ceph-mon[31442]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T03:28:50.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.535+0000 7f8de77fe640 1 -- 172.21.15.112:0/1438016504 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258814 (secure 0 0 0) 0x7f8df8063800 con 0x7f8de00b65e0 2024-09-21T03:28:50.540 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:50.541 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-09-21T03:28:50.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/1438016504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8de00b65e0 msgr2=0x7f8de00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:50.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8de00b65e0 0x7f8de00b8af0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8dec02a930 tx=0x7f8dec031400 comp rx=0 tx=0).stop 2024-09-21T03:28:50.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/1438016504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df8108c30 msgr2=0x7f8df819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:50.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df8108c30 0x7f8df819fcd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8df000cad0 tx=0x7f8df000cf90 comp rx=0 tx=0).stop 2024-09-21T03:28:50.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/1438016504 shutdown_connections 2024-09-21T03:28:50.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8de00b65e0 0x7f8de00b8af0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8df8108c30 0x7f8df819fcd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8df81082e0 0x7f8df819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 --2- 172.21.15.112:0/1438016504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8df81029b0 0x7f8df819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:50.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/1438016504 >> 172.21.15.112:0/1438016504 conn(0x7f8df80fe3b0 msgr2=0x7f8df806bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:50.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/1438016504 shutdown_connections 2024-09-21T03:28:50.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:50.538+0000 7f8dfcbf1640 1 -- 172.21.15.112:0/1438016504 wait complete. 2024-09-21T03:28:50.552 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-21T03:28:50.362191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660366,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":686,"num_read_kb":444,"num_write":302,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952574,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381976,"kb_used_data":11928,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559848,"statfs":{"total":1151940427776,"available":1151549284352,"internally_reserved":0,"allocated":12214272,"data_stored":6932050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":43,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003426"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.582283+0000","last_change":"2024-09-21T03:27:01.339980+0000","last_active":"2024-09-21T03:28:32.582283+0000","last_peered":"2024-09-21T03:28:32.582283+0000","last_clean":"2024-09-21T03:28:32.582283+0000","last_became_active":"2024-09-21T03:27:01.339552+0000","last_became_peered":"2024-09-21T03:27:01.339552+0000","last_unstale":"2024-09-21T03:28:32.582283+0000","last_undegraded":"2024-09-21T03:28:32.582283+0000","last_fullsized":"2024-09-21T03:28:32.582283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:04.240862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.577369+0000","last_change":"2024-09-21T03:27:01.334171+0000","last_active":"2024-09-21T03:28:32.577369+0000","last_peered":"2024-09-21T03:28:32.577369+0000","last_clean":"2024-09-21T03:28:32.577369+0000","last_became_active":"2024-09-21T03:27:01.333851+0000","last_became_peered":"2024-09-21T03:27:01.333851+0000","last_unstale":"2024-09-21T03:28:32.577369+0000","last_undegraded":"2024-09-21T03:28:32.577369+0000","last_fullsized":"2024-09-21T03:28:32.577369+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:49.163142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565746+0000","last_change":"2024-09-21T03:27:01.339388+0000","last_active":"2024-09-21T03:28:32.565746+0000","last_peered":"2024-09-21T03:28:32.565746+0000","last_clean":"2024-09-21T03:28:32.565746+0000","last_became_active":"2024-09-21T03:27:01.339015+0000","last_became_peered":"2024-09-21T03:27:01.339015+0000","last_unstale":"2024-09-21T03:28:32.565746+0000","last_undegraded":"2024-09-21T03:28:32.565746+0000","last_fullsized":"2024-09-21T03:28:32.565746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:06.026397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565177+0000","last_change":"2024-09-21T03:27:01.344700+0000","last_active":"2024-09-21T03:28:32.565177+0000","last_peered":"2024-09-21T03:28:32.565177+0000","last_clean":"2024-09-21T03:28:32.565177+0000","last_became_active":"2024-09-21T03:27:01.344349+0000","last_became_peered":"2024-09-21T03:27:01.344349+0000","last_unstale":"2024-09-21T03:28:32.565177+0000","last_undegraded":"2024-09-21T03:28:32.565177+0000","last_fullsized":"2024-09-21T03:28:32.565177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:00.271820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.558498+0000","last_change":"2024-09-21T03:27:01.334904+0000","last_active":"2024-09-21T03:28:32.558498+0000","last_peered":"2024-09-21T03:28:32.558498+0000","last_clean":"2024-09-21T03:28:32.558498+0000","last_became_active":"2024-09-21T03:27:01.331916+0000","last_became_peered":"2024-09-21T03:27:01.331916+0000","last_unstale":"2024-09-21T03:28:32.558498+0000","last_undegraded":"2024-09-21T03:28:32.558498+0000","last_fullsized":"2024-09-21T03:28:32.558498+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:49.376481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.559921+0000","last_change":"2024-09-21T03:27:01.343256+0000","last_active":"2024-09-21T03:28:32.559921+0000","last_peered":"2024-09-21T03:28:32.559921+0000","last_clean":"2024-09-21T03:28:32.559921+0000","last_became_active":"2024-09-21T03:27:01.342861+0000","last_became_peered":"2024-09-21T03:27:01.342861+0000","last_unstale":"2024-09-21T03:28:32.559921+0000","last_undegraded":"2024-09-21T03:28:32.559921+0000","last_fullsized":"2024-09-21T03:28:32.559921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:00.661179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.568926+0000","last_change":"2024-09-21T03:27:01.330618+0000","last_active":"2024-09-21T03:28:32.568926+0000","last_peered":"2024-09-21T03:28:32.568926+0000","last_clean":"2024-09-21T03:28:32.568926+0000","last_became_active":"2024-09-21T03:27:01.330018+0000","last_became_peered":"2024-09-21T03:27:01.330018+0000","last_unstale":"2024-09-21T03:28:32.568926+0000","last_undegraded":"2024-09-21T03:28:32.568926+0000","last_fullsized":"2024-09-21T03:28:32.568926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:03.291698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.569953+0000","last_change":"2024-09-21T03:27:01.332840+0000","last_active":"2024-09-21T03:28:32.569953+0000","last_peered":"2024-09-21T03:28:32.569953+0000","last_clean":"2024-09-21T03:28:32.569953+0000","last_became_active":"2024-09-21T03:27:01.332513+0000","last_became_peered":"2024-09-21T03:27:01.332513+0000","last_unstale":"2024-09-21T03:28:32.569953+0000","last_undegraded":"2024-09-21T03:28:32.569953+0000","last_fullsized":"2024-09-21T03:28:32.569953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:17.554934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.580544+0000","last_change":"2024-09-21T03:27:01.338147+0000","last_active":"2024-09-21T03:28:32.580544+0000","last_peered":"2024-09-21T03:28:32.580544+0000","last_clean":"2024-09-21T03:28:32.580544+0000","last_became_active":"2024-09-21T03:27:01.337759+0000","last_became_peered":"2024-09-21T03:27:01.337759+0000","last_unstale":"2024-09-21T03:28:32.580544+0000","last_undegraded":"2024-09-21T03:28:32.580544+0000","last_fullsized":"2024-09-21T03:28:32.580544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:38.999593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.570880+0000","last_change":"2024-09-21T03:27:01.343373+0000","last_active":"2024-09-21T03:28:32.570880+0000","last_peered":"2024-09-21T03:28:32.570880+0000","last_clean":"2024-09-21T03:28:32.570880+0000","last_became_active":"2024-09-21T03:27:01.342924+0000","last_became_peered":"2024-09-21T03:27:01.342924+0000","last_unstale":"2024-09-21T03:28:32.570880+0000","last_undegraded":"2024-09-21T03:28:32.570880+0000","last_fullsized":"2024-09-21T03:28:32.570880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:33.869584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574841+0000","last_change":"2024-09-21T03:27:01.340120+0000","last_active":"2024-09-21T03:28:32.574841+0000","last_peered":"2024-09-21T03:28:32.574841+0000","last_clean":"2024-09-21T03:28:32.574841+0000","last_became_active":"2024-09-21T03:27:01.339708+0000","last_became_peered":"2024-09-21T03:27:01.339708+0000","last_unstale":"2024-09-21T03:28:32.574841+0000","last_undegraded":"2024-09-21T03:28:32.574841+0000","last_fullsized":"2024-09-21T03:28:32.574841+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:56.193494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564924+0000","last_change":"2024-09-21T03:27:01.340205+0000","last_active":"2024-09-21T03:28:32.564924+0000","last_peered":"2024-09-21T03:28:32.564924+0000","last_clean":"2024-09-21T03:28:32.564924+0000","last_became_active":"2024-09-21T03:27:01.339839+0000","last_became_peered":"2024-09-21T03:27:01.339839+0000","last_unstale":"2024-09-21T03:28:32.564924+0000","last_undegraded":"2024-09-21T03:28:32.564924+0000","last_fullsized":"2024-09-21T03:28:32.564924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:05.248636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.568589+0000","last_change":"2024-09-21T03:27:01.334715+0000","last_active":"2024-09-21T03:28:32.568589+0000","last_peered":"2024-09-21T03:28:32.568589+0000","last_clean":"2024-09-21T03:28:32.568589+0000","last_became_active":"2024-09-21T03:27:01.334392+0000","last_became_peered":"2024-09-21T03:27:01.334392+0000","last_unstale":"2024-09-21T03:28:32.568589+0000","last_undegraded":"2024-09-21T03:28:32.568589+0000","last_fullsized":"2024-09-21T03:28:32.568589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:18.934063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.554357+0000","last_change":"2024-09-21T03:27:01.337167+0000","last_active":"2024-09-21T03:28:32.554357+0000","last_peered":"2024-09-21T03:28:32.554357+0000","last_clean":"2024-09-21T03:28:32.554357+0000","last_became_active":"2024-09-21T03:27:01.336748+0000","last_became_peered":"2024-09-21T03:27:01.336748+0000","last_unstale":"2024-09-21T03:28:32.554357+0000","last_undegraded":"2024-09-21T03:28:32.554357+0000","last_fullsized":"2024-09-21T03:28:32.554357+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:12.186727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564435+0000","last_change":"2024-09-21T03:27:01.336260+0000","last_active":"2024-09-21T03:28:32.564435+0000","last_peered":"2024-09-21T03:28:32.564435+0000","last_clean":"2024-09-21T03:28:32.564435+0000","last_became_active":"2024-09-21T03:27:01.335889+0000","last_became_peered":"2024-09-21T03:27:01.335889+0000","last_unstale":"2024-09-21T03:28:32.564435+0000","last_undegraded":"2024-09-21T03:28:32.564435+0000","last_fullsized":"2024-09-21T03:28:32.564435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:02.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.575723+0000","last_change":"2024-09-21T03:27:01.337181+0000","last_active":"2024-09-21T03:28:32.575723+0000","last_peered":"2024-09-21T03:28:32.575723+0000","last_clean":"2024-09-21T03:28:32.575723+0000","last_became_active":"2024-09-21T03:27:01.336813+0000","last_became_peered":"2024-09-21T03:27:01.336813+0000","last_unstale":"2024-09-21T03:28:32.575723+0000","last_undegraded":"2024-09-21T03:28:32.575723+0000","last_fullsized":"2024-09-21T03:28:32.575723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:04.052319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.582323+0000","last_change":"2024-09-21T03:27:01.342079+0000","last_active":"2024-09-21T03:28:32.582323+0000","last_peered":"2024-09-21T03:28:32.582323+0000","last_clean":"2024-09-21T03:28:32.582323+0000","last_became_active":"2024-09-21T03:27:01.341430+0000","last_became_peered":"2024-09-21T03:27:01.341430+0000","last_unstale":"2024-09-21T03:28:32.582323+0000","last_undegraded":"2024-09-21T03:28:32.582323+0000","last_fullsized":"2024-09-21T03:28:32.582323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:08.029978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.568191+0000","last_change":"2024-09-21T03:27:01.338694+0000","last_active":"2024-09-21T03:28:32.568191+0000","last_peered":"2024-09-21T03:28:32.568191+0000","last_clean":"2024-09-21T03:28:32.568191+0000","last_became_active":"2024-09-21T03:27:01.338328+0000","last_became_peered":"2024-09-21T03:27:01.338328+0000","last_unstale":"2024-09-21T03:28:32.568191+0000","last_undegraded":"2024-09-21T03:28:32.568191+0000","last_fullsized":"2024-09-21T03:28:32.568191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:50.292238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565193+0000","last_change":"2024-09-21T03:27:01.336974+0000","last_active":"2024-09-21T03:28:32.565193+0000","last_peered":"2024-09-21T03:28:32.565193+0000","last_clean":"2024-09-21T03:28:32.565193+0000","last_became_active":"2024-09-21T03:27:01.336592+0000","last_became_peered":"2024-09-21T03:27:01.336592+0000","last_unstale":"2024-09-21T03:28:32.565193+0000","last_undegraded":"2024-09-21T03:28:32.565193+0000","last_fullsized":"2024-09-21T03:28:32.565193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:09.399348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.579658+0000","last_change":"2024-09-21T03:27:33.431767+0000","last_active":"2024-09-21T03:28:32.579658+0000","last_peered":"2024-09-21T03:28:32.579658+0000","last_clean":"2024-09-21T03:28:32.579658+0000","last_became_active":"2024-09-21T03:27:33.431233+0000","last_became_peered":"2024-09-21T03:27:33.431233+0000","last_unstale":"2024-09-21T03:28:32.579658+0000","last_undegraded":"2024-09-21T03:28:32.579658+0000","last_fullsized":"2024-09-21T03:28:32.579658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:48.425375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574746+0000","last_change":"2024-09-21T03:27:01.331242+0000","last_active":"2024-09-21T03:28:32.574746+0000","last_peered":"2024-09-21T03:28:32.574746+0000","last_clean":"2024-09-21T03:28:32.574746+0000","last_became_active":"2024-09-21T03:27:01.330883+0000","last_became_peered":"2024-09-21T03:27:01.330883+0000","last_unstale":"2024-09-21T03:28:32.574746+0000","last_undegraded":"2024-09-21T03:28:32.574746+0000","last_fullsized":"2024-09-21T03:28:32.574746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:07.158448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.576098+0000","last_change":"2024-09-21T03:27:01.335522+0000","last_active":"2024-09-21T03:28:32.576098+0000","last_peered":"2024-09-21T03:28:32.576098+0000","last_clean":"2024-09-21T03:28:32.576098+0000","last_became_active":"2024-09-21T03:27:01.335114+0000","last_became_peered":"2024-09-21T03:27:01.335114+0000","last_unstale":"2024-09-21T03:28:32.576098+0000","last_undegraded":"2024-09-21T03:28:32.576098+0000","last_fullsized":"2024-09-21T03:28:32.576098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:28.383410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.567676+0000","last_change":"2024-09-21T03:27:01.329428+0000","last_active":"2024-09-21T03:28:32.567676+0000","last_peered":"2024-09-21T03:28:32.567676+0000","last_clean":"2024-09-21T03:28:32.567676+0000","last_became_active":"2024-09-21T03:27:01.328882+0000","last_became_peered":"2024-09-21T03:27:01.328882+0000","last_unstale":"2024-09-21T03:28:32.567676+0000","last_undegraded":"2024-09-21T03:28:32.567676+0000","last_fullsized":"2024-09-21T03:28:32.567676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:58.634995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.561654+0000","last_change":"2024-09-21T03:27:33.432442+0000","last_active":"2024-09-21T03:28:32.561654+0000","last_peered":"2024-09-21T03:28:32.561654+0000","last_clean":"2024-09-21T03:28:32.561654+0000","last_became_active":"2024-09-21T03:27:33.431881+0000","last_became_peered":"2024-09-21T03:27:33.431881+0000","last_unstale":"2024-09-21T03:28:32.561654+0000","last_undegraded":"2024-09-21T03:28:32.561654+0000","last_fullsized":"2024-09-21T03:28:32.561654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:46.356644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.562491+0000","last_change":"2024-09-21T03:27:01.330988+0000","last_active":"2024-09-21T03:28:32.562491+0000","last_peered":"2024-09-21T03:28:32.562491+0000","last_clean":"2024-09-21T03:28:32.562491+0000","last_became_active":"2024-09-21T03:27:01.330574+0000","last_became_peered":"2024-09-21T03:27:01.330574+0000","last_unstale":"2024-09-21T03:28:32.562491+0000","last_undegraded":"2024-09-21T03:28:32.562491+0000","last_fullsized":"2024-09-21T03:28:32.562491+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:45.277947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.573563+0000","last_change":"2024-09-21T03:27:01.339606+0000","last_active":"2024-09-21T03:28:32.573563+0000","last_peered":"2024-09-21T03:28:32.573563+0000","last_clean":"2024-09-21T03:28:32.573563+0000","last_became_active":"2024-09-21T03:27:01.339208+0000","last_became_peered":"2024-09-21T03:27:01.339208+0000","last_unstale":"2024-09-21T03:28:32.573563+0000","last_undegraded":"2024-09-21T03:28:32.573563+0000","last_fullsized":"2024-09-21T03:28:32.573563+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:31.593363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.578941+0000","last_change":"2024-09-21T03:27:01.343634+0000","last_active":"2024-09-21T03:28:32.578941+0000","last_peered":"2024-09-21T03:28:32.578941+0000","last_clean":"2024-09-21T03:28:32.578941+0000","last_became_active":"2024-09-21T03:27:01.342965+0000","last_became_peered":"2024-09-21T03:27:01.342965+0000","last_unstale":"2024-09-21T03:28:32.578941+0000","last_undegraded":"2024-09-21T03:28:32.578941+0000","last_fullsized":"2024-09-21T03:28:32.578941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:46.916198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564776+0000","last_change":"2024-09-21T03:27:01.324993+0000","last_active":"2024-09-21T03:28:32.564776+0000","last_peered":"2024-09-21T03:28:32.564776+0000","last_clean":"2024-09-21T03:28:32.564776+0000","last_became_active":"2024-09-21T03:27:01.324593+0000","last_became_peered":"2024-09-21T03:27:01.324593+0000","last_unstale":"2024-09-21T03:28:32.564776+0000","last_undegraded":"2024-09-21T03:28:32.564776+0000","last_fullsized":"2024-09-21T03:28:32.564776+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:13.211539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563667+0000","last_change":"2024-09-21T03:27:01.325232+0000","last_active":"2024-09-21T03:28:32.563667+0000","last_peered":"2024-09-21T03:28:32.563667+0000","last_clean":"2024-09-21T03:28:32.563667+0000","last_became_active":"2024-09-21T03:27:01.324570+0000","last_became_peered":"2024-09-21T03:27:01.324570+0000","last_unstale":"2024-09-21T03:28:32.563667+0000","last_undegraded":"2024-09-21T03:28:32.563667+0000","last_fullsized":"2024-09-21T03:28:32.563667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:30.843334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563783+0000","last_change":"2024-09-21T03:27:01.338789+0000","last_active":"2024-09-21T03:28:32.563783+0000","last_peered":"2024-09-21T03:28:32.563783+0000","last_clean":"2024-09-21T03:28:32.563783+0000","last_became_active":"2024-09-21T03:27:01.338282+0000","last_became_peered":"2024-09-21T03:27:01.338282+0000","last_unstale":"2024-09-21T03:28:32.563783+0000","last_undegraded":"2024-09-21T03:28:32.563783+0000","last_fullsized":"2024-09-21T03:28:32.563783+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:57.213343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.566643+0000","last_change":"2024-09-21T03:27:01.339521+0000","last_active":"2024-09-21T03:28:32.566643+0000","last_peered":"2024-09-21T03:28:32.566643+0000","last_clean":"2024-09-21T03:28:32.566643+0000","last_became_active":"2024-09-21T03:27:01.339175+0000","last_became_peered":"2024-09-21T03:27:01.339175+0000","last_unstale":"2024-09-21T03:28:32.566643+0000","last_undegraded":"2024-09-21T03:28:32.566643+0000","last_fullsized":"2024-09-21T03:28:32.566643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:46.163698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.581098+0000","last_change":"2024-09-21T03:27:01.338352+0000","last_active":"2024-09-21T03:28:32.581098+0000","last_peered":"2024-09-21T03:28:32.581098+0000","last_clean":"2024-09-21T03:28:32.581098+0000","last_became_active":"2024-09-21T03:27:01.337306+0000","last_became_peered":"2024-09-21T03:27:01.337306+0000","last_unstale":"2024-09-21T03:28:32.581098+0000","last_undegraded":"2024-09-21T03:28:32.581098+0000","last_fullsized":"2024-09-21T03:28:32.581098+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:19.799039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.562245+0000","last_change":"2024-09-21T03:27:01.332461+0000","last_active":"2024-09-21T03:28:32.562245+0000","last_peered":"2024-09-21T03:28:32.562245+0000","last_clean":"2024-09-21T03:28:32.562245+0000","last_became_active":"2024-09-21T03:27:01.331912+0000","last_became_peered":"2024-09-21T03:27:01.331912+0000","last_unstale":"2024-09-21T03:28:32.562245+0000","last_undegraded":"2024-09-21T03:28:32.562245+0000","last_fullsized":"2024-09-21T03:28:32.562245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:34.771691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"76'6","reported_seq":143,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:48.596982+0000","last_change":"2024-09-21T03:27:33.482302+0000","last_active":"2024-09-21T03:28:48.596982+0000","last_peered":"2024-09-21T03:28:48.596982+0000","last_clean":"2024-09-21T03:28:48.596982+0000","last_became_active":"2024-09-21T03:27:33.478350+0000","last_became_peered":"2024-09-21T03:27:33.478350+0000","last_unstale":"2024-09-21T03:28:48.596982+0000","last_undegraded":"2024-09-21T03:28:48.596982+0000","last_fullsized":"2024-09-21T03:28:48.596982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:46.916762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.566839+0000","last_change":"2024-09-21T03:27:01.324364+0000","last_active":"2024-09-21T03:28:32.566839+0000","last_peered":"2024-09-21T03:28:32.566839+0000","last_clean":"2024-09-21T03:28:32.566839+0000","last_became_active":"2024-09-21T03:27:01.324018+0000","last_became_peered":"2024-09-21T03:27:01.324018+0000","last_unstale":"2024-09-21T03:28:32.566839+0000","last_undegraded":"2024-09-21T03:28:32.566839+0000","last_fullsized":"2024-09-21T03:28:32.566839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:56.404893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.431029+0000","last_change":"2024-09-21T03:26:59.313289+0000","last_active":"2024-09-21T03:27:33.431029+0000","last_peered":"2024-09-21T03:27:33.431029+0000","last_clean":"2024-09-21T03:27:33.431029+0000","last_became_active":"2024-09-21T03:26:59.312877+0000","last_became_peered":"2024-09-21T03:26:59.312877+0000","last_unstale":"2024-09-21T03:27:33.431029+0000","last_undegraded":"2024-09-21T03:27:33.431029+0000","last_fullsized":"2024-09-21T03:27:33.431029+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:53.856541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427079+0000","last_change":"2024-09-21T03:26:59.307098+0000","last_active":"2024-09-21T03:27:33.427079+0000","last_peered":"2024-09-21T03:27:33.427079+0000","last_clean":"2024-09-21T03:27:33.427079+0000","last_became_active":"2024-09-21T03:26:59.306650+0000","last_became_peered":"2024-09-21T03:26:59.306650+0000","last_unstale":"2024-09-21T03:27:33.427079+0000","last_undegraded":"2024-09-21T03:27:33.427079+0000","last_fullsized":"2024-09-21T03:27:33.427079+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:44.033899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.570911+0000","last_change":"2024-09-21T03:27:01.328443+0000","last_active":"2024-09-21T03:28:32.570911+0000","last_peered":"2024-09-21T03:28:32.570911+0000","last_clean":"2024-09-21T03:28:32.570911+0000","last_became_active":"2024-09-21T03:27:01.328107+0000","last_became_peered":"2024-09-21T03:27:01.328107+0000","last_unstale":"2024-09-21T03:28:32.570911+0000","last_undegraded":"2024-09-21T03:28:32.570911+0000","last_fullsized":"2024-09-21T03:28:32.570911+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:47.798308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428946+0000","last_change":"2024-09-21T03:26:59.317934+0000","last_active":"2024-09-21T03:27:33.428946+0000","last_peered":"2024-09-21T03:27:33.428946+0000","last_clean":"2024-09-21T03:27:33.428946+0000","last_became_active":"2024-09-21T03:26:59.317166+0000","last_became_peered":"2024-09-21T03:26:59.317166+0000","last_unstale":"2024-09-21T03:27:33.428946+0000","last_undegraded":"2024-09-21T03:27:33.428946+0000","last_fullsized":"2024-09-21T03:27:33.428946+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:52.746406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.564062+0000","last_change":"2024-09-21T03:27:01.331226+0000","last_active":"2024-09-21T03:28:32.564062+0000","last_peered":"2024-09-21T03:28:32.564062+0000","last_clean":"2024-09-21T03:28:32.564062+0000","last_became_active":"2024-09-21T03:27:01.330889+0000","last_became_peered":"2024-09-21T03:27:01.330889+0000","last_unstale":"2024-09-21T03:28:32.564062+0000","last_undegraded":"2024-09-21T03:28:32.564062+0000","last_fullsized":"2024-09-21T03:28:32.564062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:51.181554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'4","reported_seq":123,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:46.592737+0000","last_change":"2024-09-21T03:26:59.311671+0000","last_active":"2024-09-21T03:28:46.592737+0000","last_peered":"2024-09-21T03:28:46.592737+0000","last_clean":"2024-09-21T03:28:46.592737+0000","last_became_active":"2024-09-21T03:26:59.311149+0000","last_became_peered":"2024-09-21T03:26:59.311149+0000","last_unstale":"2024-09-21T03:28:46.592737+0000","last_undegraded":"2024-09-21T03:28:46.592737+0000","last_fullsized":"2024-09-21T03:28:46.592737+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:47.878274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.570949+0000","last_change":"2024-09-21T03:27:01.336142+0000","last_active":"2024-09-21T03:28:32.570949+0000","last_peered":"2024-09-21T03:28:32.570949+0000","last_clean":"2024-09-21T03:28:32.570949+0000","last_became_active":"2024-09-21T03:27:01.335795+0000","last_became_peered":"2024-09-21T03:27:01.335795+0000","last_unstale":"2024-09-21T03:28:32.570949+0000","last_undegraded":"2024-09-21T03:28:32.570949+0000","last_fullsized":"2024-09-21T03:28:32.570949+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:29.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429365+0000","last_change":"2024-09-21T03:26:59.315077+0000","last_active":"2024-09-21T03:27:33.429365+0000","last_peered":"2024-09-21T03:27:33.429365+0000","last_clean":"2024-09-21T03:27:33.429365+0000","last_became_active":"2024-09-21T03:26:59.312347+0000","last_became_peered":"2024-09-21T03:26:59.312347+0000","last_unstale":"2024-09-21T03:27:33.429365+0000","last_undegraded":"2024-09-21T03:27:33.429365+0000","last_fullsized":"2024-09-21T03:27:33.429365+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:24.529066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565121+0000","last_change":"2024-09-21T03:27:01.324346+0000","last_active":"2024-09-21T03:28:32.565121+0000","last_peered":"2024-09-21T03:28:32.565121+0000","last_clean":"2024-09-21T03:28:32.565121+0000","last_became_active":"2024-09-21T03:27:01.323976+0000","last_became_peered":"2024-09-21T03:27:01.323976+0000","last_unstale":"2024-09-21T03:28:32.565121+0000","last_undegraded":"2024-09-21T03:28:32.565121+0000","last_fullsized":"2024-09-21T03:28:32.565121+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:04.591595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429516+0000","last_change":"2024-09-21T03:26:59.326206+0000","last_active":"2024-09-21T03:27:33.429516+0000","last_peered":"2024-09-21T03:27:33.429516+0000","last_clean":"2024-09-21T03:27:33.429516+0000","last_became_active":"2024-09-21T03:26:59.325755+0000","last_became_peered":"2024-09-21T03:26:59.325755+0000","last_unstale":"2024-09-21T03:27:33.429516+0000","last_undegraded":"2024-09-21T03:27:33.429516+0000","last_fullsized":"2024-09-21T03:27:33.429516+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:23.220795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574096+0000","last_change":"2024-09-21T03:27:01.341501+0000","last_active":"2024-09-21T03:28:32.574096+0000","last_peered":"2024-09-21T03:28:32.574096+0000","last_clean":"2024-09-21T03:28:32.574096+0000","last_became_active":"2024-09-21T03:27:01.341039+0000","last_became_peered":"2024-09-21T03:27:01.341039+0000","last_unstale":"2024-09-21T03:28:32.574096+0000","last_undegraded":"2024-09-21T03:28:32.574096+0000","last_fullsized":"2024-09-21T03:28:32.574096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:15.132180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'32","reported_seq":259,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:47.602172+0000","last_change":"2024-09-21T03:26:59.313489+0000","last_active":"2024-09-21T03:28:47.602172+0000","last_peered":"2024-09-21T03:28:47.602172+0000","last_clean":"2024-09-21T03:28:47.602172+0000","last_became_active":"2024-09-21T03:26:59.313022+0000","last_became_peered":"2024-09-21T03:26:59.313022+0000","last_unstale":"2024-09-21T03:28:47.602172+0000","last_undegraded":"2024-09-21T03:28:47.602172+0000","last_fullsized":"2024-09-21T03:28:47.602172+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:08.541315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65314,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":204,"num_read_kb":165,"num_write":33,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.578105+0000","last_change":"2024-09-21T03:27:01.339329+0000","last_active":"2024-09-21T03:28:32.578105+0000","last_peered":"2024-09-21T03:28:32.578105+0000","last_clean":"2024-09-21T03:28:32.578105+0000","last_became_active":"2024-09-21T03:27:01.338982+0000","last_became_peered":"2024-09-21T03:27:01.338982+0000","last_unstale":"2024-09-21T03:28:32.578105+0000","last_undegraded":"2024-09-21T03:28:32.578105+0000","last_fullsized":"2024-09-21T03:28:32.578105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:41.759450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'6","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430224+0000","last_change":"2024-09-21T03:26:59.313270+0000","last_active":"2024-09-21T03:27:33.430224+0000","last_peered":"2024-09-21T03:27:33.430224+0000","last_clean":"2024-09-21T03:27:33.430224+0000","last_became_active":"2024-09-21T03:26:59.312867+0000","last_became_peered":"2024-09-21T03:26:59.312867+0000","last_unstale":"2024-09-21T03:27:33.430224+0000","last_undegraded":"2024-09-21T03:27:33.430224+0000","last_fullsized":"2024-09-21T03:27:33.430224+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:13:53.798022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.572806+0000","last_change":"2024-09-21T03:27:01.331268+0000","last_active":"2024-09-21T03:28:32.572806+0000","last_peered":"2024-09-21T03:28:32.572806+0000","last_clean":"2024-09-21T03:28:32.572806+0000","last_became_active":"2024-09-21T03:27:01.330922+0000","last_became_peered":"2024-09-21T03:27:01.330922+0000","last_unstale":"2024-09-21T03:28:32.572806+0000","last_undegraded":"2024-09-21T03:28:32.572806+0000","last_fullsized":"2024-09-21T03:28:32.572806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:17.667522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428917+0000","last_change":"2024-09-21T03:26:59.318388+0000","last_active":"2024-09-21T03:27:33.428917+0000","last_peered":"2024-09-21T03:27:33.428917+0000","last_clean":"2024-09-21T03:27:33.428917+0000","last_became_active":"2024-09-21T03:26:59.317938+0000","last_became_peered":"2024-09-21T03:26:59.317938+0000","last_unstale":"2024-09-21T03:27:33.428917+0000","last_undegraded":"2024-09-21T03:27:33.428917+0000","last_fullsized":"2024-09-21T03:27:33.428917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.499699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.575303+0000","last_change":"2024-09-21T03:27:33.432191+0000","last_active":"2024-09-21T03:28:32.575303+0000","last_peered":"2024-09-21T03:28:32.575303+0000","last_clean":"2024-09-21T03:28:32.575303+0000","last_became_active":"2024-09-21T03:27:33.431667+0000","last_became_peered":"2024-09-21T03:27:33.431667+0000","last_unstale":"2024-09-21T03:28:32.575303+0000","last_undegraded":"2024-09-21T03:28:32.575303+0000","last_fullsized":"2024-09-21T03:28:32.575303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:13.293063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'5","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:43.550712+0000","last_change":"2024-09-21T03:26:59.311345+0000","last_active":"2024-09-21T03:28:43.550712+0000","last_peered":"2024-09-21T03:28:43.550712+0000","last_clean":"2024-09-21T03:28:43.550712+0000","last_became_active":"2024-09-21T03:26:59.310928+0000","last_became_peered":"2024-09-21T03:26:59.310928+0000","last_unstale":"2024-09-21T03:28:43.550712+0000","last_undegraded":"2024-09-21T03:28:43.550712+0000","last_fullsized":"2024-09-21T03:28:43.550712+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:38.614955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.571566+0000","last_change":"2024-09-21T03:27:01.335785+0000","last_active":"2024-09-21T03:28:32.571566+0000","last_peered":"2024-09-21T03:28:32.571566+0000","last_clean":"2024-09-21T03:28:32.571566+0000","last_became_active":"2024-09-21T03:27:01.335396+0000","last_became_peered":"2024-09-21T03:27:01.335396+0000","last_unstale":"2024-09-21T03:28:32.571566+0000","last_undegraded":"2024-09-21T03:28:32.571566+0000","last_fullsized":"2024-09-21T03:28:32.571566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:52.785641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'15","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.448967+0000","last_change":"2024-09-21T03:27:33.448404+0000","last_active":"2024-09-21T03:27:33.448967+0000","last_peered":"2024-09-21T03:27:33.448967+0000","last_clean":"2024-09-21T03:27:33.448967+0000","last_became_active":"2024-09-21T03:27:33.434405+0000","last_became_peered":"2024-09-21T03:27:33.434405+0000","last_unstale":"2024-09-21T03:27:33.448967+0000","last_undegraded":"2024-09-21T03:27:33.448967+0000","last_fullsized":"2024-09-21T03:27:33.448967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:05.107391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.573683+0000","last_change":"2024-09-21T03:27:01.342282+0000","last_active":"2024-09-21T03:28:32.573683+0000","last_peered":"2024-09-21T03:28:32.573683+0000","last_clean":"2024-09-21T03:28:32.573683+0000","last_became_active":"2024-09-21T03:27:01.341790+0000","last_became_peered":"2024-09-21T03:27:01.341790+0000","last_unstale":"2024-09-21T03:28:32.573683+0000","last_undegraded":"2024-09-21T03:28:32.573683+0000","last_fullsized":"2024-09-21T03:28:32.573683+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:05.432720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.428084+0000","last_change":"2024-09-21T03:26:59.312442+0000","last_active":"2024-09-21T03:27:33.428084+0000","last_peered":"2024-09-21T03:27:33.428084+0000","last_clean":"2024-09-21T03:27:33.428084+0000","last_became_active":"2024-09-21T03:26:59.311601+0000","last_became_peered":"2024-09-21T03:26:59.311601+0000","last_unstale":"2024-09-21T03:27:33.428084+0000","last_undegraded":"2024-09-21T03:27:33.428084+0000","last_fullsized":"2024-09-21T03:27:33.428084+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:32.421228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.579550+0000","last_change":"2024-09-21T03:27:01.337917+0000","last_active":"2024-09-21T03:28:32.579550+0000","last_peered":"2024-09-21T03:28:32.579550+0000","last_clean":"2024-09-21T03:28:32.579550+0000","last_became_active":"2024-09-21T03:27:01.337457+0000","last_became_peered":"2024-09-21T03:27:01.337457+0000","last_unstale":"2024-09-21T03:28:32.579550+0000","last_undegraded":"2024-09-21T03:28:32.579550+0000","last_fullsized":"2024-09-21T03:28:32.579550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:49.011795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.597334+0000","last_change":"2024-09-21T03:27:33.597239+0000","last_active":"2024-09-21T03:27:33.597334+0000","last_peered":"2024-09-21T03:27:33.597334+0000","last_clean":"2024-09-21T03:27:33.597334+0000","last_became_active":"2024-09-21T03:27:33.593787+0000","last_became_peered":"2024-09-21T03:27:33.593787+0000","last_unstale":"2024-09-21T03:27:33.597334+0000","last_undegraded":"2024-09-21T03:27:33.597334+0000","last_fullsized":"2024-09-21T03:27:33.597334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:35.015446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.581336+0000","last_change":"2024-09-21T03:27:01.338489+0000","last_active":"2024-09-21T03:28:32.581336+0000","last_peered":"2024-09-21T03:28:32.581336+0000","last_clean":"2024-09-21T03:28:32.581336+0000","last_became_active":"2024-09-21T03:27:01.338123+0000","last_became_peered":"2024-09-21T03:27:01.338123+0000","last_unstale":"2024-09-21T03:28:32.581336+0000","last_undegraded":"2024-09-21T03:28:32.581336+0000","last_fullsized":"2024-09-21T03:28:32.581336+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:15.137546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429644+0000","last_change":"2024-09-21T03:26:59.315371+0000","last_active":"2024-09-21T03:27:33.429644+0000","last_peered":"2024-09-21T03:27:33.429644+0000","last_clean":"2024-09-21T03:27:33.429644+0000","last_became_active":"2024-09-21T03:26:59.314439+0000","last_became_peered":"2024-09-21T03:26:59.314439+0000","last_unstale":"2024-09-21T03:27:33.429644+0000","last_undegraded":"2024-09-21T03:27:33.429644+0000","last_fullsized":"2024-09-21T03:27:33.429644+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:54:13.656324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.567006+0000","last_change":"2024-09-21T03:27:01.331086+0000","last_active":"2024-09-21T03:28:32.567006+0000","last_peered":"2024-09-21T03:28:32.567006+0000","last_clean":"2024-09-21T03:28:32.567006+0000","last_became_active":"2024-09-21T03:27:01.329113+0000","last_became_peered":"2024-09-21T03:27:01.329113+0000","last_unstale":"2024-09-21T03:28:32.567006+0000","last_undegraded":"2024-09-21T03:28:32.567006+0000","last_fullsized":"2024-09-21T03:28:32.567006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:25.514730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.433098+0000","last_change":"2024-09-21T03:26:59.554087+0000","last_active":"2024-09-21T03:27:33.433098+0000","last_peered":"2024-09-21T03:27:33.433098+0000","last_clean":"2024-09-21T03:27:33.433098+0000","last_became_active":"2024-09-21T03:26:59.553610+0000","last_became_peered":"2024-09-21T03:26:59.553610+0000","last_unstale":"2024-09-21T03:27:33.433098+0000","last_undegraded":"2024-09-21T03:27:33.433098+0000","last_fullsized":"2024-09-21T03:27:33.433098+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:22.048057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.577544+0000","last_change":"2024-09-21T03:27:01.339987+0000","last_active":"2024-09-21T03:28:32.577544+0000","last_peered":"2024-09-21T03:28:32.577544+0000","last_clean":"2024-09-21T03:28:32.577544+0000","last_became_active":"2024-09-21T03:27:01.339060+0000","last_became_peered":"2024-09-21T03:27:01.339060+0000","last_unstale":"2024-09-21T03:28:32.577544+0000","last_undegraded":"2024-09-21T03:28:32.577544+0000","last_fullsized":"2024-09-21T03:28:32.577544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.901337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429743+0000","last_change":"2024-09-21T03:25:50.381920+0000","last_active":"2024-09-21T03:27:33.429743+0000","last_peered":"2024-09-21T03:27:33.429743+0000","last_clean":"2024-09-21T03:27:33.429743+0000","last_became_active":"2024-09-21T03:25:50.376170+0000","last_became_peered":"2024-09-21T03:25:50.376170+0000","last_unstale":"2024-09-21T03:27:33.429743+0000","last_undegraded":"2024-09-21T03:27:33.429743+0000","last_fullsized":"2024-09-21T03:27:33.429743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:22:31.269883+0000","last_clean_scrub_stamp":"2024-09-21T03:22:31.269883+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:57.281150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'3","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.477054+0000","last_change":"2024-09-21T03:26:59.313612+0000","last_active":"2024-09-21T03:27:33.477054+0000","last_peered":"2024-09-21T03:27:33.477054+0000","last_clean":"2024-09-21T03:27:33.477054+0000","last_became_active":"2024-09-21T03:26:59.313060+0000","last_became_peered":"2024-09-21T03:26:59.313060+0000","last_unstale":"2024-09-21T03:27:33.477054+0000","last_undegraded":"2024-09-21T03:27:33.477054+0000","last_fullsized":"2024-09-21T03:27:33.477054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:06.193526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.572676+0000","last_change":"2024-09-21T03:27:01.337784+0000","last_active":"2024-09-21T03:28:32.572676+0000","last_peered":"2024-09-21T03:28:32.572676+0000","last_clean":"2024-09-21T03:28:32.572676+0000","last_became_active":"2024-09-21T03:27:01.337415+0000","last_became_peered":"2024-09-21T03:27:01.337415+0000","last_unstale":"2024-09-21T03:28:32.572676+0000","last_undegraded":"2024-09-21T03:28:32.572676+0000","last_fullsized":"2024-09-21T03:28:32.572676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:40.482054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'3","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.436960+0000","last_change":"2024-09-21T03:27:33.436319+0000","last_active":"2024-09-21T03:27:33.436960+0000","last_peered":"2024-09-21T03:27:33.436960+0000","last_clean":"2024-09-21T03:27:33.436960+0000","last_became_active":"2024-09-21T03:27:33.431555+0000","last_became_peered":"2024-09-21T03:27:33.431555+0000","last_unstale":"2024-09-21T03:27:33.436960+0000","last_undegraded":"2024-09-21T03:27:33.436960+0000","last_fullsized":"2024-09-21T03:27:33.436960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:24:16.073988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563635+0000","last_change":"2024-09-21T03:27:01.333769+0000","last_active":"2024-09-21T03:28:32.563635+0000","last_peered":"2024-09-21T03:28:32.563635+0000","last_clean":"2024-09-21T03:28:32.563635+0000","last_became_active":"2024-09-21T03:27:01.333436+0000","last_became_peered":"2024-09-21T03:27:01.333436+0000","last_unstale":"2024-09-21T03:28:32.563635+0000","last_undegraded":"2024-09-21T03:28:32.563635+0000","last_fullsized":"2024-09-21T03:28:32.563635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:30.395916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:43.550125+0000","last_change":"2024-09-21T03:26:59.325314+0000","last_active":"2024-09-21T03:28:43.550125+0000","last_peered":"2024-09-21T03:28:43.550125+0000","last_clean":"2024-09-21T03:28:43.550125+0000","last_became_active":"2024-09-21T03:26:59.324752+0000","last_became_peered":"2024-09-21T03:26:59.324752+0000","last_unstale":"2024-09-21T03:28:43.550125+0000","last_undegraded":"2024-09-21T03:28:43.550125+0000","last_fullsized":"2024-09-21T03:28:43.550125+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.252596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.559825+0000","last_change":"2024-09-21T03:27:01.335154+0000","last_active":"2024-09-21T03:28:32.559825+0000","last_peered":"2024-09-21T03:28:32.559825+0000","last_clean":"2024-09-21T03:28:32.559825+0000","last_became_active":"2024-09-21T03:27:01.334633+0000","last_became_peered":"2024-09-21T03:27:01.334633+0000","last_unstale":"2024-09-21T03:28:32.559825+0000","last_undegraded":"2024-09-21T03:28:32.559825+0000","last_fullsized":"2024-09-21T03:28:32.559825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:27.992858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429757+0000","last_change":"2024-09-21T03:26:59.314019+0000","last_active":"2024-09-21T03:27:33.429757+0000","last_peered":"2024-09-21T03:27:33.429757+0000","last_clean":"2024-09-21T03:27:33.429757+0000","last_became_active":"2024-09-21T03:26:59.311915+0000","last_became_peered":"2024-09-21T03:26:59.311915+0000","last_unstale":"2024-09-21T03:27:33.429757+0000","last_undegraded":"2024-09-21T03:27:33.429757+0000","last_fullsized":"2024-09-21T03:27:33.429757+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:51.412303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.558537+0000","last_change":"2024-09-21T03:27:33.433193+0000","last_active":"2024-09-21T03:28:32.558537+0000","last_peered":"2024-09-21T03:28:32.558537+0000","last_clean":"2024-09-21T03:28:32.558537+0000","last_became_active":"2024-09-21T03:27:33.432729+0000","last_became_peered":"2024-09-21T03:27:33.432729+0000","last_unstale":"2024-09-21T03:28:32.558537+0000","last_undegraded":"2024-09-21T03:28:32.558537+0000","last_fullsized":"2024-09-21T03:28:32.558537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:37.575110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424782+0000","last_change":"2024-09-21T03:26:59.325943+0000","last_active":"2024-09-21T03:27:33.424782+0000","last_peered":"2024-09-21T03:27:33.424782+0000","last_clean":"2024-09-21T03:27:33.424782+0000","last_became_active":"2024-09-21T03:26:59.325524+0000","last_became_peered":"2024-09-21T03:26:59.325524+0000","last_unstale":"2024-09-21T03:27:33.424782+0000","last_undegraded":"2024-09-21T03:27:33.424782+0000","last_fullsized":"2024-09-21T03:27:33.424782+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:45.111480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563254+0000","last_change":"2024-09-21T03:27:01.335369+0000","last_active":"2024-09-21T03:28:32.563254+0000","last_peered":"2024-09-21T03:28:32.563254+0000","last_clean":"2024-09-21T03:28:32.563254+0000","last_became_active":"2024-09-21T03:27:01.334997+0000","last_became_peered":"2024-09-21T03:27:01.334997+0000","last_unstale":"2024-09-21T03:28:32.563254+0000","last_undegraded":"2024-09-21T03:28:32.563254+0000","last_fullsized":"2024-09-21T03:28:32.563254+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:10.257451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'1","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429553+0000","last_change":"2024-09-21T03:26:59.326204+0000","last_active":"2024-09-21T03:27:33.429553+0000","last_peered":"2024-09-21T03:27:33.429553+0000","last_clean":"2024-09-21T03:27:33.429553+0000","last_became_active":"2024-09-21T03:26:59.325431+0000","last_became_peered":"2024-09-21T03:26:59.325431+0000","last_unstale":"2024-09-21T03:27:33.429553+0000","last_undegraded":"2024-09-21T03:27:33.429553+0000","last_fullsized":"2024-09-21T03:27:33.429553+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:27.863217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.558220+0000","last_change":"2024-09-21T03:27:01.333861+0000","last_active":"2024-09-21T03:28:32.558220+0000","last_peered":"2024-09-21T03:28:32.558220+0000","last_clean":"2024-09-21T03:28:32.558220+0000","last_became_active":"2024-09-21T03:27:01.333485+0000","last_became_peered":"2024-09-21T03:27:01.333485+0000","last_unstale":"2024-09-21T03:28:32.558220+0000","last_undegraded":"2024-09-21T03:28:32.558220+0000","last_fullsized":"2024-09-21T03:28:32.558220+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:04.475811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'13","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:33.336636+0000","last_change":"2024-09-21T03:26:59.309767+0000","last_active":"2024-09-21T03:28:33.336636+0000","last_peered":"2024-09-21T03:28:33.336636+0000","last_clean":"2024-09-21T03:28:33.336636+0000","last_became_active":"2024-09-21T03:26:59.309354+0000","last_became_peered":"2024-09-21T03:26:59.309354+0000","last_unstale":"2024-09-21T03:28:33.336636+0000","last_undegraded":"2024-09-21T03:28:33.336636+0000","last_fullsized":"2024-09-21T03:28:33.336636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:26.728719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574479+0000","last_change":"2024-09-21T03:27:01.333097+0000","last_active":"2024-09-21T03:28:32.574479+0000","last_peered":"2024-09-21T03:28:32.574479+0000","last_clean":"2024-09-21T03:28:32.574479+0000","last_became_active":"2024-09-21T03:27:01.332743+0000","last_became_peered":"2024-09-21T03:27:01.332743+0000","last_unstale":"2024-09-21T03:28:32.574479+0000","last_undegraded":"2024-09-21T03:28:32.574479+0000","last_fullsized":"2024-09-21T03:28:32.574479+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:41.004213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.476588+0000","last_change":"2024-09-21T03:26:59.553974+0000","last_active":"2024-09-21T03:27:33.476588+0000","last_peered":"2024-09-21T03:27:33.476588+0000","last_clean":"2024-09-21T03:27:33.476588+0000","last_became_active":"2024-09-21T03:26:59.553647+0000","last_became_peered":"2024-09-21T03:26:59.553647+0000","last_unstale":"2024-09-21T03:27:33.476588+0000","last_undegraded":"2024-09-21T03:27:33.476588+0000","last_fullsized":"2024-09-21T03:27:33.476588+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:00.490951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.563631+0000","last_change":"2024-09-21T03:27:01.325308+0000","last_active":"2024-09-21T03:28:32.563631+0000","last_peered":"2024-09-21T03:28:32.563631+0000","last_clean":"2024-09-21T03:28:32.563631+0000","last_became_active":"2024-09-21T03:27:01.324977+0000","last_became_peered":"2024-09-21T03:27:01.324977+0000","last_unstale":"2024-09-21T03:28:32.563631+0000","last_undegraded":"2024-09-21T03:28:32.563631+0000","last_fullsized":"2024-09-21T03:28:32.563631+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:10.093856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.592537+0000","last_change":"2024-09-21T03:26:59.326017+0000","last_active":"2024-09-21T03:27:33.592537+0000","last_peered":"2024-09-21T03:27:33.592537+0000","last_clean":"2024-09-21T03:27:33.592537+0000","last_became_active":"2024-09-21T03:26:59.325588+0000","last_became_peered":"2024-09-21T03:26:59.325588+0000","last_unstale":"2024-09-21T03:27:33.592537+0000","last_undegraded":"2024-09-21T03:27:33.592537+0000","last_fullsized":"2024-09-21T03:27:33.592537+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:39.740449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.531947+0000","last_change":"2024-09-21T03:27:01.336871+0000","last_active":"2024-09-21T03:28:32.531947+0000","last_peered":"2024-09-21T03:28:32.531947+0000","last_clean":"2024-09-21T03:28:32.531947+0000","last_became_active":"2024-09-21T03:27:01.336507+0000","last_became_peered":"2024-09-21T03:27:01.336507+0000","last_unstale":"2024-09-21T03:28:32.531947+0000","last_undegraded":"2024-09-21T03:28:32.531947+0000","last_fullsized":"2024-09-21T03:28:32.531947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:58.674375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430477+0000","last_change":"2024-09-21T03:26:59.308997+0000","last_active":"2024-09-21T03:27:33.430477+0000","last_peered":"2024-09-21T03:27:33.430477+0000","last_clean":"2024-09-21T03:27:33.430477+0000","last_became_active":"2024-09-21T03:26:59.308338+0000","last_became_peered":"2024-09-21T03:26:59.308338+0000","last_unstale":"2024-09-21T03:27:33.430477+0000","last_undegraded":"2024-09-21T03:27:33.430477+0000","last_fullsized":"2024-09-21T03:27:33.430477+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:15.328468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.576878+0000","last_change":"2024-09-21T03:27:01.338808+0000","last_active":"2024-09-21T03:28:32.576878+0000","last_peered":"2024-09-21T03:28:32.576878+0000","last_clean":"2024-09-21T03:28:32.576878+0000","last_became_active":"2024-09-21T03:27:01.337987+0000","last_became_peered":"2024-09-21T03:27:01.337987+0000","last_unstale":"2024-09-21T03:28:32.576878+0000","last_undegraded":"2024-09-21T03:28:32.576878+0000","last_fullsized":"2024-09-21T03:28:32.576878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:56.265014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430766+0000","last_change":"2024-09-21T03:26:59.312684+0000","last_active":"2024-09-21T03:27:33.430766+0000","last_peered":"2024-09-21T03:27:33.430766+0000","last_clean":"2024-09-21T03:27:33.430766+0000","last_became_active":"2024-09-21T03:26:59.312038+0000","last_became_peered":"2024-09-21T03:26:59.312038+0000","last_unstale":"2024-09-21T03:27:33.430766+0000","last_undegraded":"2024-09-21T03:27:33.430766+0000","last_fullsized":"2024-09-21T03:27:33.430766+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:02.672255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.569931+0000","last_change":"2024-09-21T03:27:01.333080+0000","last_active":"2024-09-21T03:28:32.569931+0000","last_peered":"2024-09-21T03:28:32.569931+0000","last_clean":"2024-09-21T03:28:32.569931+0000","last_became_active":"2024-09-21T03:27:01.332663+0000","last_became_peered":"2024-09-21T03:27:01.332663+0000","last_unstale":"2024-09-21T03:28:32.569931+0000","last_undegraded":"2024-09-21T03:28:32.569931+0000","last_fullsized":"2024-09-21T03:28:32.569931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:34.569542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.427673+0000","last_change":"2024-09-21T03:26:59.315490+0000","last_active":"2024-09-21T03:27:33.427673+0000","last_peered":"2024-09-21T03:27:33.427673+0000","last_clean":"2024-09-21T03:27:33.427673+0000","last_became_active":"2024-09-21T03:26:59.313789+0000","last_became_peered":"2024-09-21T03:26:59.313789+0000","last_unstale":"2024-09-21T03:27:33.427673+0000","last_undegraded":"2024-09-21T03:27:33.427673+0000","last_fullsized":"2024-09-21T03:27:33.427673+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:10.140711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.574483+0000","last_change":"2024-09-21T03:27:01.341019+0000","last_active":"2024-09-21T03:28:32.574483+0000","last_peered":"2024-09-21T03:28:32.574483+0000","last_clean":"2024-09-21T03:28:32.574483+0000","last_became_active":"2024-09-21T03:27:01.340660+0000","last_became_peered":"2024-09-21T03:27:01.340660+0000","last_unstale":"2024-09-21T03:28:32.574483+0000","last_undegraded":"2024-09-21T03:28:32.574483+0000","last_fullsized":"2024-09-21T03:28:32.574483+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:16.736993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.577115+0000","last_change":"2024-09-21T03:27:01.341165+0000","last_active":"2024-09-21T03:28:32.577115+0000","last_peered":"2024-09-21T03:28:32.577115+0000","last_clean":"2024-09-21T03:28:32.577115+0000","last_became_active":"2024-09-21T03:27:01.340765+0000","last_became_peered":"2024-09-21T03:27:01.340765+0000","last_unstale":"2024-09-21T03:28:32.577115+0000","last_undegraded":"2024-09-21T03:28:32.577115+0000","last_fullsized":"2024-09-21T03:28:32.577115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:15.455944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429107+0000","last_change":"2024-09-21T03:26:59.313410+0000","last_active":"2024-09-21T03:27:33.429107+0000","last_peered":"2024-09-21T03:27:33.429107+0000","last_clean":"2024-09-21T03:27:33.429107+0000","last_became_active":"2024-09-21T03:26:59.312233+0000","last_became_peered":"2024-09-21T03:26:59.312233+0000","last_unstale":"2024-09-21T03:27:33.429107+0000","last_undegraded":"2024-09-21T03:27:33.429107+0000","last_fullsized":"2024-09-21T03:27:33.429107+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:26.089611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.572571+0000","last_change":"2024-09-21T03:27:01.335332+0000","last_active":"2024-09-21T03:28:32.572571+0000","last_peered":"2024-09-21T03:28:32.572571+0000","last_clean":"2024-09-21T03:28:32.572571+0000","last_became_active":"2024-09-21T03:27:01.334872+0000","last_became_peered":"2024-09-21T03:27:01.334872+0000","last_unstale":"2024-09-21T03:28:32.572571+0000","last_undegraded":"2024-09-21T03:28:32.572571+0000","last_fullsized":"2024-09-21T03:28:32.572571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:39.631246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.429421+0000","last_change":"2024-09-21T03:26:59.314250+0000","last_active":"2024-09-21T03:27:33.429421+0000","last_peered":"2024-09-21T03:27:33.429421+0000","last_clean":"2024-09-21T03:27:33.429421+0000","last_became_active":"2024-09-21T03:26:59.313707+0000","last_became_peered":"2024-09-21T03:26:59.313707+0000","last_unstale":"2024-09-21T03:27:33.429421+0000","last_undegraded":"2024-09-21T03:27:33.429421+0000","last_fullsized":"2024-09-21T03:27:33.429421+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:22.935292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.565595+0000","last_change":"2024-09-21T03:27:01.339230+0000","last_active":"2024-09-21T03:28:32.565595+0000","last_peered":"2024-09-21T03:28:32.565595+0000","last_clean":"2024-09-21T03:28:32.565595+0000","last_became_active":"2024-09-21T03:27:01.338881+0000","last_became_peered":"2024-09-21T03:27:01.338881+0000","last_unstale":"2024-09-21T03:28:32.565595+0000","last_undegraded":"2024-09-21T03:28:32.565595+0000","last_fullsized":"2024-09-21T03:28:32.565595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:49.189180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.430888+0000","last_change":"2024-09-21T03:26:59.326549+0000","last_active":"2024-09-21T03:27:33.430888+0000","last_peered":"2024-09-21T03:27:33.430888+0000","last_clean":"2024-09-21T03:27:33.430888+0000","last_became_active":"2024-09-21T03:26:59.326091+0000","last_became_peered":"2024-09-21T03:26:59.326091+0000","last_unstale":"2024-09-21T03:27:33.430888+0000","last_undegraded":"2024-09-21T03:27:33.430888+0000","last_fullsized":"2024-09-21T03:27:33.430888+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:13.507716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:28:32.566939+0000","last_change":"2024-09-21T03:27:01.331367+0000","last_active":"2024-09-21T03:28:32.566939+0000","last_peered":"2024-09-21T03:28:32.566939+0000","last_clean":"2024-09-21T03:28:32.566939+0000","last_became_active":"2024-09-21T03:27:01.330604+0000","last_became_peered":"2024-09-21T03:27:01.330604+0000","last_unstale":"2024-09-21T03:28:32.566939+0000","last_undegraded":"2024-09-21T03:28:32.566939+0000","last_fullsized":"2024-09-21T03:28:32.566939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:27:00.299543+0000","last_clean_scrub_stamp":"2024-09-21T03:27:00.299543+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:32.207561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'9","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T03:27:33.424843+0000","last_change":"2024-09-21T03:26:59.311912+0000","last_active":"2024-09-21T03:27:33.424843+0000","last_peered":"2024-09-21T03:27:33.424843+0000","last_clean":"2024-09-21T03:27:33.424843+0000","last_became_active":"2024-09-21T03:26:59.311571+0000","last_became_peered":"2024-09-21T03:26:59.311571+0000","last_unstale":"2024-09-21T03:27:33.424843+0000","last_undegraded":"2024-09-21T03:27:33.424843+0000","last_fullsized":"2024-09-21T03:27:33.424843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:26:58.289748+0000","last_clean_scrub_stamp":"2024-09-21T03:26:58.289748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:55.946494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69764,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":605,"num_read_kb":375,"num_write":177,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":209292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776164,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":843776,"data_stored":413299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939687,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1536000,"data_stored":1070514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103213,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":413423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299441,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":413231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32596,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712556,"statfs":{"total":95995035648,"available":95961657344,"internally_reserved":0,"allocated":1445888,"data_stored":1005722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":412812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855425,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1458176,"data_stored":1004874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":851968,"data_stored":414403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":413123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":478647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542229,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":477919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705753,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":414083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:28:50.556 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:28:50.835 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:50.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:50 smithi203 ceph-mon[34723]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:50.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:50 smithi203 ceph-mon[34723]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:50.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:50 smithi203 ceph-mon[34723]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T03:28:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:50 smithi187 ceph-mon[34603]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:50 smithi187 ceph-mon[34603]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:50 smithi187 ceph-mon[34603]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-21T03:28:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.310+0000 7fc52dfec640 1 -- 172.21.15.112:0/569592193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc528105330 msgr2=0x7fc52810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.310+0000 7fc52dfec640 1 --2- 172.21.15.112:0/569592193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc528105330 0x7fc52810d880 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009900 tx=0x7fc51c02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.310+0000 7fc52dfec640 1 -- 172.21.15.112:0/569592193 shutdown_connections 2024-09-21T03:28:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.310+0000 7fc52dfec640 1 --2- 172.21.15.112:0/569592193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc52810de40 0x7fc528110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.310+0000 7fc52dfec640 1 --2- 172.21.15.112:0/569592193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc528105330 0x7fc52810d880 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.310+0000 7fc52dfec640 1 --2- 172.21.15.112:0/569592193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5281049e0 0x7fc528104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.310+0000 7fc52dfec640 1 -- 172.21.15.112:0/569592193 >> 172.21.15.112:0/569592193 conn(0x7fc528079750 msgr2=0x7fc528079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.310+0000 7fc52dfec640 1 -- 172.21.15.112:0/569592193 shutdown_connections 2024-09-21T03:28:51.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.310+0000 7fc52dfec640 1 -- 172.21.15.112:0/569592193 wait complete. 2024-09-21T03:28:51.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.310+0000 7fc52dfec640 1 Processor -- start 2024-09-21T03:28:51.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc52dfec640 1 -- start start 2024-09-21T03:28:51.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc52dfec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5281049e0 0x7fc52819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:51.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc52dfec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc528105330 0x7fc52819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:51.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc52dfec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc52810de40 0x7fc52819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:51.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc52dfec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc528113aa0 con 0x7fc5281049e0 2024-09-21T03:28:51.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc52dfec640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc528113920 con 0x7fc52810de40 2024-09-21T03:28:51.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc52dfec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc528113c20 con 0x7fc528105330 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc5277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5281049e0 0x7fc52819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc5277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5281049e0 0x7fc52819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58454/0 (socket says 172.21.15.112:58454) 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc5277fe640 1 -- 172.21.15.112:0/2003744062 learned_addr learned my addr 172.21.15.112:0/2003744062 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc527fff640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc52810de40 0x7fc52819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc526ffd640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc528105330 0x7fc52819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc5277fe640 1 -- 172.21.15.112:0/2003744062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc528105330 msgr2=0x7fc52819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.311+0000 7fc5277fe640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc528105330 0x7fc52819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.312+0000 7fc5277fe640 1 -- 172.21.15.112:0/2003744062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc52810de40 msgr2=0x7fc52819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.312+0000 7fc5277fe640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc52810de40 0x7fc52819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.312+0000 7fc5277fe640 1 -- 172.21.15.112:0/2003744062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5281a0400 con 0x7fc5281049e0 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.312+0000 7fc5277fe640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5281049e0 0x7fc52819a490 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc51800d770 tx=0x7fc51800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.312+0000 7fc524ff9640 1 -- 172.21.15.112:0/2003744062 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518017070 con 0x7fc5281049e0 2024-09-21T03:28:51.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.312+0000 7fc52dfec640 1 -- 172.21.15.112:0/2003744062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5281a06f0 con 0x7fc5281049e0 2024-09-21T03:28:51.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.312+0000 7fc524ff9640 1 -- 172.21.15.112:0/2003744062 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc518009c30 con 0x7fc5281049e0 2024-09-21T03:28:51.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.312+0000 7fc524ff9640 1 -- 172.21.15.112:0/2003744062 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518003cc0 con 0x7fc5281049e0 2024-09-21T03:28:51.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.312+0000 7fc52dfec640 1 -- 172.21.15.112:0/2003744062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5281a6e50 con 0x7fc5281049e0 2024-09-21T03:28:51.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.313+0000 7fc52dfec640 1 -- 172.21.15.112:0/2003744062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52806c280 con 0x7fc5281049e0 2024-09-21T03:28:51.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.315+0000 7fc524ff9640 1 -- 172.21.15.112:0/2003744062 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7fc518003e60 con 0x7fc5281049e0 2024-09-21T03:28:51.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.315+0000 7fc524ff9640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc5100b6560 0x7fc5100b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:51.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.316+0000 7fc526ffd640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc5100b6560 0x7fc5100b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.316+0000 7fc524ff9640 1 -- 172.21.15.112:0/2003744062 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc5180e7080 con 0x7fc5281049e0 2024-09-21T03:28:51.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.316+0000 7fc526ffd640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc5100b6560 0x7fc5100b8a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009230 tx=0x7fc51c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:51.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.319+0000 7fc524ff9640 1 -- 172.21.15.112:0/2003744062 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc518009de0 con 0x7fc5281049e0 2024-09-21T03:28:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.533+0000 7fc52dfec640 1 -- 172.21.15.112:0/2003744062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc52819b5c0 con 0x7fc5281049e0 2024-09-21T03:28:51.535 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:51 smithi112 ceph-mon[31442]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.534+0000 7fc524ff9640 1 -- 172.21.15.112:0/2003744062 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11929 (secure 0 0 0) 0x7fc5180b1cd0 con 0x7fc5281049e0 2024-09-21T03:28:51.536 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:51.536 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"btime":"2024-09-21T03:28:46:418112+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:28:46.048810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:28:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-09-21T03:28:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 -- 172.21.15.112:0/2003744062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc5100b6560 msgr2=0x7fc5100b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc5100b6560 0x7fc5100b8a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009230 tx=0x7fc51c031400 comp rx=0 tx=0).stop 2024-09-21T03:28:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 -- 172.21.15.112:0/2003744062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5281049e0 msgr2=0x7fc52819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5281049e0 0x7fc52819a490 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc51800d770 tx=0x7fc51800dc30 comp rx=0 tx=0).stop 2024-09-21T03:28:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 -- 172.21.15.112:0/2003744062 shutdown_connections 2024-09-21T03:28:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc5100b6560 0x7fc5100b8a70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc52810de40 0x7fc52819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc528105330 0x7fc52819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 --2- 172.21.15.112:0/2003744062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5281049e0 0x7fc52819a490 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:51.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 -- 172.21.15.112:0/2003744062 >> 172.21.15.112:0/2003744062 conn(0x7fc528079750 msgr2=0x7fc528106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:51.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 -- 172.21.15.112:0/2003744062 shutdown_connections 2024-09-21T03:28:51.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:51.536+0000 7fc52dfec640 1 -- 172.21.15.112:0/2003744062 wait complete. 2024-09-21T03:28:51.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:51 smithi203 ceph-mon[34723]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:51.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:51 smithi187 ceph-mon[34603]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:28:52.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:28:52.488 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:52.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:52 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2003744062' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:52.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:52 smithi112 ceph-mon[31442]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T03:28:52.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:52 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2003744062' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:52.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:52 smithi203 ceph-mon[34723]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T03:28:52.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:52 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2003744062' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:28:52.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:52 smithi187 ceph-mon[34603]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-21T03:28:52.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.974+0000 7f881771e640 1 -- 172.21.15.112:0/2185864052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8810109df0 msgr2=0x7f8810110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:52.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.974+0000 7f881771e640 1 --2- 172.21.15.112:0/2185864052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8810109df0 0x7f8810110620 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8808009900 tx=0x7f880802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:52.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.974+0000 7f881771e640 1 -- 172.21.15.112:0/2185864052 shutdown_connections 2024-09-21T03:28:52.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.974+0000 7f881771e640 1 --2- 172.21.15.112:0/2185864052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8810109df0 0x7f8810110620 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.974+0000 7f881771e640 1 --2- 172.21.15.112:0/2185864052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88101094a0 0x7f88101098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.974+0000 7f881771e640 1 --2- 172.21.15.112:0/2185864052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8810103b60 0x7f8810103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.974+0000 7f881771e640 1 -- 172.21.15.112:0/2185864052 >> 172.21.15.112:0/2185864052 conn(0x7f881007b850 msgr2=0x7f881007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.974+0000 7f881771e640 1 -- 172.21.15.112:0/2185864052 shutdown_connections 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.974+0000 7f881771e640 1 -- 172.21.15.112:0/2185864052 wait complete. 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.974+0000 7f881771e640 1 Processor -- start 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f881771e640 1 -- start start 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f881771e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8810103b60 0x7f8810075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f881771e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88101094a0 0x7f8810075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f881771e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8810109df0 0x7f8810076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f881771e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8810113b10 con 0x7f8810103b60 2024-09-21T03:28:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f881771e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8810113990 con 0x7f88101094a0 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f881771e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8810113c90 con 0x7f8810109df0 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f88154ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8810103b60 0x7f8810075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f88154ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8810103b60 0x7f8810075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58468/0 (socket says 172.21.15.112:58468) 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f88154ba640 1 -- 172.21.15.112:0/2188682911 learned_addr learned my addr 172.21.15.112:0/2188682911 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f8814cb9640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88101094a0 0x7f8810075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f8815cbb640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8810109df0 0x7f8810076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f88154ba640 1 -- 172.21.15.112:0/2188682911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8810109df0 msgr2=0x7f8810076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f88154ba640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8810109df0 0x7f8810076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f88154ba640 1 -- 172.21.15.112:0/2188682911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88101094a0 msgr2=0x7f8810075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f88154ba640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88101094a0 0x7f8810075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.975+0000 7f88154ba640 1 -- 172.21.15.112:0/2188682911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88101a87a0 con 0x7f8810103b60 2024-09-21T03:28:52.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.976+0000 7f88154ba640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8810103b60 0x7f8810075710 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f880000ba50 tx=0x7f880000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:52.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.976+0000 7f880e7fc640 1 -- 172.21.15.112:0/2188682911 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800019070 con 0x7f8810103b60 2024-09-21T03:28:52.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.976+0000 7f881771e640 1 -- 172.21.15.112:0/2188682911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88101a8a90 con 0x7f8810103b60 2024-09-21T03:28:52.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.976+0000 7f880e7fc640 1 -- 172.21.15.112:0/2188682911 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f880000cbe0 con 0x7f8810103b60 2024-09-21T03:28:52.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.976+0000 7f880e7fc640 1 -- 172.21.15.112:0/2188682911 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88000075d0 con 0x7f8810103b60 2024-09-21T03:28:52.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.976+0000 7f881771e640 1 -- 172.21.15.112:0/2188682911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88101a8fd0 con 0x7f8810103b60 2024-09-21T03:28:52.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.977+0000 7f881771e640 1 -- 172.21.15.112:0/2188682911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8810103f70 con 0x7f8810103b60 2024-09-21T03:28:52.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.979+0000 7f880e7fc640 1 -- 172.21.15.112:0/2188682911 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f880001c070 con 0x7f8810103b60 2024-09-21T03:28:52.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.979+0000 7f880e7fc640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87f80b6560 0x7f87f80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:52.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.979+0000 7f8814cb9640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87f80b6560 0x7f87f80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:52.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.979+0000 7f880e7fc640 1 -- 172.21.15.112:0/2188682911 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f88000e6e20 con 0x7f8810103b60 2024-09-21T03:28:52.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.980+0000 7f8814cb9640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87f80b6560 0x7f87f80b8a70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8810108fc0 tx=0x7f880400a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:52.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:52.982+0000 7f880e7fc640 1 -- 172.21.15.112:0/2188682911 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88000b19d0 con 0x7f8810103b60 2024-09-21T03:28:53.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.160+0000 7f881771e640 1 -- 172.21.15.112:0/2188682911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f881002e640 con 0x7f8810103b60 2024-09-21T03:28:53.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.162+0000 7f880e7fc640 1 -- 172.21.15.112:0/2188682911 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23065 (secure 0 0 0) 0x7f88000b6910 con 0x7f8810103b60 2024-09-21T03:28:53.164 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:28:53.164 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":76,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:27:33.414969+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:26:58.087349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:26:59.970444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:28:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 -- 172.21.15.112:0/2188682911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87f80b6560 msgr2=0x7f87f80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87f80b6560 0x7f87f80b8a70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8810108fc0 tx=0x7f880400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:28:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 -- 172.21.15.112:0/2188682911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8810103b60 msgr2=0x7f8810075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8810103b60 0x7f8810075710 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f880000ba50 tx=0x7f880000bf10 comp rx=0 tx=0).stop 2024-09-21T03:28:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 -- 172.21.15.112:0/2188682911 shutdown_connections 2024-09-21T03:28:53.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f87f80b6560 0x7f87f80b8a70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8810109df0 0x7f8810076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88101094a0 0x7f8810075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 --2- 172.21.15.112:0/2188682911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8810103b60 0x7f8810075710 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 -- 172.21.15.112:0/2188682911 >> 172.21.15.112:0/2188682911 conn(0x7f881007b850 msgr2=0x7f88101074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 -- 172.21.15.112:0/2188682911 shutdown_connections 2024-09-21T03:28:53.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:53.164+0000 7f881771e640 1 -- 172.21.15.112:0/2188682911 wait complete. 2024-09-21T03:28:53.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:53 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2188682911' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:53.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:53 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2188682911' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:53.918 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T03:28:53.919 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T03:28:53.931 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:28:53.931 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T03:28:53.931 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T03:28:53.931 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T03:28:53.932 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:28:53.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:28:53.933 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:53 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2188682911' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:28:54.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.044+0000 7fc6ddfba640 1 -- 172.21.15.112:0/4020226164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d81030e0 msgr2=0x7fc6d81034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.044+0000 7fc6ddfba640 1 --2- 172.21.15.112:0/4020226164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d81030e0 0x7fc6d81034f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009900 tx=0x7fc6cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:54.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.044+0000 7fc6ddfba640 1 -- 172.21.15.112:0/4020226164 shutdown_connections 2024-09-21T03:28:54.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.044+0000 7fc6ddfba640 1 --2- 172.21.15.112:0/4020226164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d8108c90 0x7fc6d810f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.044+0000 7fc6ddfba640 1 --2- 172.21.15.112:0/4020226164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d8108340 0x7fc6d8108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.044+0000 7fc6ddfba640 1 --2- 172.21.15.112:0/4020226164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d81030e0 0x7fc6d81034f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.044+0000 7fc6ddfba640 1 -- 172.21.15.112:0/4020226164 >> 172.21.15.112:0/4020226164 conn(0x7fc6d8079d00 msgr2=0x7fc6d807a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.044+0000 7fc6ddfba640 1 -- 172.21.15.112:0/4020226164 shutdown_connections 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.044+0000 7fc6ddfba640 1 -- 172.21.15.112:0/4020226164 wait complete. 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.045+0000 7fc6ddfba640 1 Processor -- start 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.045+0000 7fc6ddfba640 1 -- start start 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.045+0000 7fc6ddfba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d81030e0 0x7fc6d80fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.045+0000 7fc6ddfba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d8108340 0x7fc6d80ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.045+0000 7fc6ddfba640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d8108c90 0x7fc6d80ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.045+0000 7fc6ddfba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81129b0 con 0x7fc6d8108340 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.045+0000 7fc6ddfba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d8112830 con 0x7fc6d81030e0 2024-09-21T03:28:54.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.045+0000 7fc6ddfba640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d8112b30 con 0x7fc6d8108c90 2024-09-21T03:28:54.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.045+0000 7fc6d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d8108340 0x7fc6d80ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.045+0000 7fc6d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d8108340 0x7fc6d80ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58486/0 (socket says 172.21.15.112:58486) 2024-09-21T03:28:54.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.045+0000 7fc6d6ffd640 1 -- 172.21.15.112:0/1458799654 learned_addr learned my addr 172.21.15.112:0/1458799654 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:54.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6d77fe640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d81030e0 0x7fc6d80fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6d7fff640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d8108c90 0x7fc6d80ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6d6ffd640 1 -- 172.21.15.112:0/1458799654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d8108c90 msgr2=0x7fc6d80ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6d6ffd640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d8108c90 0x7fc6d80ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6d6ffd640 1 -- 172.21.15.112:0/1458799654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d81030e0 msgr2=0x7fc6d80fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6d6ffd640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d81030e0 0x7fc6d80fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6d6ffd640 1 -- 172.21.15.112:0/1458799654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d81a87b0 con 0x7fc6d8108340 2024-09-21T03:28:54.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6d6ffd640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d8108340 0x7fc6d80ff270 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800d740 tx=0x7fc6c800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6d4ff9640 1 -- 172.21.15.112:0/1458799654 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8017070 con 0x7fc6d8108340 2024-09-21T03:28:54.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6ddfba640 1 -- 172.21.15.112:0/1458799654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d81a8aa0 con 0x7fc6d8108340 2024-09-21T03:28:54.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6d4ff9640 1 -- 172.21.15.112:0/1458799654 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6c800de50 con 0x7fc6d8108340 2024-09-21T03:28:54.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6d4ff9640 1 -- 172.21.15.112:0/1458799654 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c800f620 con 0x7fc6d8108340 2024-09-21T03:28:54.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.046+0000 7fc6ddfba640 1 -- 172.21.15.112:0/1458799654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d81a9040 con 0x7fc6d8108340 2024-09-21T03:28:54.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.047+0000 7fc6ddfba640 1 -- 172.21.15.112:0/1458799654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d81034f0 con 0x7fc6d8108340 2024-09-21T03:28:54.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.050+0000 7fc6d4ff9640 1 -- 172.21.15.112:0/1458799654 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7fc6c801a070 con 0x7fc6d8108340 2024-09-21T03:28:54.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.050+0000 7fc6d4ff9640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6c00b6590 0x7fc6c00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.050+0000 7fc6d4ff9640 1 -- 172.21.15.112:0/1458799654 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc6c80e6d50 con 0x7fc6d8108340 2024-09-21T03:28:54.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.051+0000 7fc6d77fe640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6c00b6590 0x7fc6c00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.051+0000 7fc6d77fe640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6c00b6590 0x7fc6c00b8aa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc000c00 tx=0x7fc6cc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.055+0000 7fc6d4ff9640 1 -- 172.21.15.112:0/1458799654 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc6c80ea030 con 0x7fc6d8108340 2024-09-21T03:28:54.189 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:54.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.271+0000 7fc6ddfba640 1 -- 172.21.15.112:0/1458799654 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc6d8063800 con 0x7fc6c00b6590 2024-09-21T03:28:54.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.438+0000 7fc6d4ff9640 1 -- 172.21.15.112:0/1458799654 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc6d8063800 con 0x7fc6c00b6590 2024-09-21T03:28:54.440 INFO:teuthology.orchestra.run.smithi112.stdout:Schedule set for path / 2024-09-21T03:28:54.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 -- 172.21.15.112:0/1458799654 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6c00b6590 msgr2=0x7fc6c00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6c00b6590 0x7fc6c00b8aa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc000c00 tx=0x7fc6cc031400 comp rx=0 tx=0).stop 2024-09-21T03:28:54.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 -- 172.21.15.112:0/1458799654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d8108340 msgr2=0x7fc6d80ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d8108340 0x7fc6d80ff270 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800d740 tx=0x7fc6c800dc00 comp rx=0 tx=0).stop 2024-09-21T03:28:54.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 -- 172.21.15.112:0/1458799654 shutdown_connections 2024-09-21T03:28:54.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc6c00b6590 0x7fc6c00b8aa0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6d8108c90 0x7fc6d80ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6d8108340 0x7fc6d80ff270 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 --2- 172.21.15.112:0/1458799654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6d81030e0 0x7fc6d80fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 -- 172.21.15.112:0/1458799654 >> 172.21.15.112:0/1458799654 conn(0x7fc6d8079d00 msgr2=0x7fc6d8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 -- 172.21.15.112:0/1458799654 shutdown_connections 2024-09-21T03:28:54.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.440+0000 7fc6ddfba640 1 -- 172.21.15.112:0/1458799654 wait complete. 2024-09-21T03:28:54.453 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T03:28:54.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:54 smithi112 ceph-mon[31442]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:54.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:54 smithi112 ceph-mon[31442]: pgmap v315: 97 pgs: 97 active+clean; 649 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T03:28:54.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.573+0000 7f1faf60e640 1 -- 172.21.15.112:0/1398981795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81029b0 msgr2=0x7f1fa8102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.573+0000 7f1faf60e640 1 --2- 172.21.15.112:0/1398981795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81029b0 0x7f1fa8102dc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009900 tx=0x7f1fa402a410 comp rx=0 tx=0).stop 2024-09-21T03:28:54.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.573+0000 7f1faf60e640 1 -- 172.21.15.112:0/1398981795 shutdown_connections 2024-09-21T03:28:54.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.573+0000 7f1faf60e640 1 --2- 172.21.15.112:0/1398981795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fa8108c30 0x7f1fa810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.573+0000 7f1faf60e640 1 --2- 172.21.15.112:0/1398981795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa81082e0 0x7f1fa81086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.573+0000 7f1faf60e640 1 --2- 172.21.15.112:0/1398981795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa81029b0 0x7f1fa8102dc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.573+0000 7f1faf60e640 1 -- 172.21.15.112:0/1398981795 >> 172.21.15.112:0/1398981795 conn(0x7f1fa807a740 msgr2=0x7f1fa807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.573+0000 7f1faf60e640 1 -- 172.21.15.112:0/1398981795 shutdown_connections 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.573+0000 7f1faf60e640 1 -- 172.21.15.112:0/1398981795 wait complete. 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.574+0000 7f1faf60e640 1 Processor -- start 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.574+0000 7f1faf60e640 1 -- start start 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.574+0000 7f1faf60e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa81029b0 0x7f1fa819b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.574+0000 7f1faf60e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fa81082e0 0x7f1fa819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.574+0000 7f1faf60e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa8108c30 0x7f1fa819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.574+0000 7f1faf60e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa81128f0 con 0x7f1fa8108c30 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.574+0000 7f1faf60e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa8112770 con 0x7f1fa81082e0 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.574+0000 7f1faf60e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa8112a70 con 0x7f1fa81029b0 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1fadbab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa8108c30 0x7f1fa819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1fadbab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa8108c30 0x7f1fa819bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58514/0 (socket says 172.21.15.112:58514) 2024-09-21T03:28:54.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1fadbab640 1 -- 172.21.15.112:0/3006278912 learned_addr learned my addr 172.21.15.112:0/3006278912 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:54.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1fad3aa640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa81029b0 0x7f1fa819b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1facba9640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fa81082e0 0x7f1fa819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1fadbab640 1 -- 172.21.15.112:0/3006278912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa81029b0 msgr2=0x7f1fa819b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1fadbab640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa81029b0 0x7f1fa819b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1fadbab640 1 -- 172.21.15.112:0/3006278912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fa81082e0 msgr2=0x7f1fa819b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1fadbab640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fa81082e0 0x7f1fa819b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1fadbab640 1 -- 172.21.15.112:0/3006278912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa81a64f0 con 0x7f1fa8108c30 2024-09-21T03:28:54.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1fad3aa640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa81029b0 0x7f1fa819b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:54.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1facba9640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fa81082e0 0x7f1fa819b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:54.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1fadbab640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa8108c30 0x7f1fa819bc60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000eaa0 tx=0x7f1fa000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1f9e7fc640 1 -- 172.21.15.112:0/3006278912 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa000cdb0 con 0x7f1fa8108c30 2024-09-21T03:28:54.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1f9e7fc640 1 -- 172.21.15.112:0/3006278912 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fa0005020 con 0x7f1fa8108c30 2024-09-21T03:28:54.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1faf60e640 1 -- 172.21.15.112:0/3006278912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa81a67e0 con 0x7f1fa8108c30 2024-09-21T03:28:54.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1f9e7fc640 1 -- 172.21.15.112:0/3006278912 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00116d0 con 0x7f1fa8108c30 2024-09-21T03:28:54.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.575+0000 7f1faf60e640 1 -- 172.21.15.112:0/3006278912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa81a6ce0 con 0x7f1fa8108c30 2024-09-21T03:28:54.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.576+0000 7f1faf60e640 1 -- 172.21.15.112:0/3006278912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa8102dc0 con 0x7f1fa8108c30 2024-09-21T03:28:54.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.578+0000 7f1f9e7fc640 1 -- 172.21.15.112:0/3006278912 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f1fa00051d0 con 0x7f1fa8108c30 2024-09-21T03:28:54.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.578+0000 7f1f9e7fc640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1f940b65f0 0x7f1f940b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.579+0000 7f1fad3aa640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1f940b65f0 0x7f1f940b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.579+0000 7f1f9e7fc640 1 -- 172.21.15.112:0/3006278912 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f1fa00e7c30 con 0x7f1fa8108c30 2024-09-21T03:28:54.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.579+0000 7f1fad3aa640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1f940b65f0 0x7f1f940b8b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4000c00 tx=0x7f1fa4009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.582+0000 7f1f9e7fc640 1 -- 172.21.15.112:0/3006278912 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1fa0016030 con 0x7f1fa8108c30 2024-09-21T03:28:54.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.723+0000 7f3dcccbe640 1 -- 172.21.15.112:0/891823163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8105380 msgr2=0x7f3dc810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.723+0000 7f3dcccbe640 1 --2- 172.21.15.112:0/891823163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8105380 0x7f3dc810d8d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009900 tx=0x7f3db002a410 comp rx=0 tx=0).stop 2024-09-21T03:28:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.723+0000 7f3dcccbe640 1 -- 172.21.15.112:0/891823163 shutdown_connections 2024-09-21T03:28:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.723+0000 7f3dcccbe640 1 --2- 172.21.15.112:0/891823163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc810de60 0x7f3dc8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.723+0000 7f3dcccbe640 1 --2- 172.21.15.112:0/891823163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8105380 0x7f3dc810d8d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.723+0000 7f3dcccbe640 1 --2- 172.21.15.112:0/891823163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dc8104a30 0x7f3dc8104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.723+0000 7f3dcccbe640 1 -- 172.21.15.112:0/891823163 >> 172.21.15.112:0/891823163 conn(0x7f3dc80797b0 msgr2=0x7f3dc8079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.723+0000 7f3dcccbe640 1 -- 172.21.15.112:0/891823163 shutdown_connections 2024-09-21T03:28:54.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.723+0000 7f3dcccbe640 1 -- 172.21.15.112:0/891823163 wait complete. 2024-09-21T03:28:54.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.724+0000 7f3dcccbe640 1 Processor -- start 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.724+0000 7f3dcccbe640 1 -- start start 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dcccbe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dc8104a30 0x7f3dc819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dcccbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8105380 0x7f3dc819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dcccbe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc810de60 0x7f3dc81a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dcccbe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8113a60 con 0x7f3dc8105380 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dcccbe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81138e0 con 0x7f3dc8104a30 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dcccbe640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8113be0 con 0x7f3dc810de60 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dc5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8105380 0x7f3dc819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dc5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8105380 0x7f3dc819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58524/0 (socket says 172.21.15.112:58524) 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dc5d9b640 1 -- 172.21.15.112:0/3573419912 learned_addr learned my addr 172.21.15.112:0/3573419912 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dc6d9d640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc810de60 0x7f3dc81a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dc659c640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dc8104a30 0x7f3dc819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dc5d9b640 1 -- 172.21.15.112:0/3573419912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc810de60 msgr2=0x7f3dc81a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dc5d9b640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc810de60 0x7f3dc81a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dc5d9b640 1 -- 172.21.15.112:0/3573419912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dc8104a30 msgr2=0x7f3dc819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dc5d9b640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dc8104a30 0x7f3dc819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dc5d9b640 1 -- 172.21.15.112:0/3573419912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc81a2640 con 0x7f3dc8105380 2024-09-21T03:28:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dc5d9b640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8105380 0x7f3dc819cc70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3db0006010 tx=0x7f3db0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.725+0000 7f3dbf7fe640 1 -- 172.21.15.112:0/3573419912 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db003f030 con 0x7f3dc8105380 2024-09-21T03:28:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.726+0000 7f3dcccbe640 1 -- 172.21.15.112:0/3573419912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc81a28d0 con 0x7f3dc8105380 2024-09-21T03:28:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.726+0000 7f3dbf7fe640 1 -- 172.21.15.112:0/3573419912 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3db0003aa0 con 0x7f3dc8105380 2024-09-21T03:28:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.726+0000 7f3dbf7fe640 1 -- 172.21.15.112:0/3573419912 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0035070 con 0x7f3dc8105380 2024-09-21T03:28:54.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.726+0000 7f3dcccbe640 1 -- 172.21.15.112:0/3573419912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc81a9090 con 0x7f3dc8105380 2024-09-21T03:28:54.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.728+0000 7f3dd3232740 1 Processor -- start 2024-09-21T03:28:54.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dd3232740 1 -- start start 2024-09-21T03:28:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 0x5588fd995430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5c5490 0x5588fd995970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5cd670 0x5588fd995eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dd3232740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5588fd90be60 con 0x5588fd5cd670 2024-09-21T03:28:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dd3232740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5588fd8fb290 con 0x5588fd5ee350 2024-09-21T03:28:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dd3232740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5588fd8ff0a0 con 0x5588fd5c5490 2024-09-21T03:28:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dbf7fe640 1 -- 172.21.15.112:0/3573419912 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f3db0042030 con 0x7f3dc8105380 2024-09-21T03:28:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dc659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5c5490 0x5588fd995970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dc6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 0x5588fd995430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dc659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5c5490 0x5588fd995970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46000/0 (socket says 172.21.15.112:46000) 2024-09-21T03:28:54.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.729+0000 7f3dc659c640 1 -- 172.21.15.112:0/2002704401 learned_addr learned my addr 172.21.15.112:0/2002704401 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dc5d9b640 1 --2- 172.21.15.112:0/2002704401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5cd670 0x5588fd995eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dc659c640 1 -- 172.21.15.112:0/2002704401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 msgr2=0x5588fd995430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dc659c640 1 --2- 172.21.15.112:0/2002704401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 0x5588fd995430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dc659c640 1 -- 172.21.15.112:0/2002704401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5cd670 msgr2=0x5588fd995eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dc659c640 1 --2- 172.21.15.112:0/2002704401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5cd670 0x5588fd995eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dc659c640 1 -- 172.21.15.112:0/2002704401 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588fd659930 con 0x5588fd5c5490 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dc6d9d640 1 --2- 172.21.15.112:0/2002704401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 0x5588fd995430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dc5d9b640 1 --2- 172.21.15.112:0/2002704401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5cd670 0x5588fd995eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dc659c640 1 --2- 172.21.15.112:0/2002704401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5c5490 0x5588fd995970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009c80 tx=0x7f3db80117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a2db62a156fdb4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dbcff9640 1 -- 172.21.15.112:0/2002704401 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8012450 con 0x5588fd5c5490 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dbf7fe640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3db40b65f0 0x7f3db40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dbcff9640 1 -- 172.21.15.112:0/2002704401 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3db8012aa0 con 0x5588fd5c5490 2024-09-21T03:28:54.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.730+0000 7f3dc6d9d640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3db40b65f0 0x7f3db40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.731+0000 7f3dbf7fe640 1 -- 172.21.15.112:0/3573419912 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3db0107a50 con 0x7f3dc8105380 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.731+0000 7f3dbcff9640 1 -- 172.21.15.112:0/2002704401 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db800fae0 con 0x5588fd5c5490 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.731+0000 7f3dd3232740 1 -- 172.21.15.112:0/2002704401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5c5490 msgr2=0x5588fd995970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.731+0000 7f3dd3232740 1 --2- 172.21.15.112:0/2002704401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5c5490 0x5588fd995970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009c80 tx=0x7f3db80117b0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.731+0000 7f3dc6d9d640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3db40b65f0 0x7f3db40b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0005fd0 tx=0x7f3dc00096a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.731+0000 7f3dd3232740 1 -- 172.21.15.112:0/2002704401 shutdown_connections 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.731+0000 7f3dd3232740 1 --2- 172.21.15.112:0/2002704401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5cd670 0x5588fd995eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.731+0000 7f3dd3232740 1 --2- 172.21.15.112:0/2002704401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5c5490 0x5588fd995970 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.731+0000 7f3dd3232740 1 --2- 172.21.15.112:0/2002704401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 0x5588fd995430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.731+0000 7f3dd3232740 1 -- 172.21.15.112:0/2002704401 >> 172.21.15.112:0/2002704401 conn(0x5588fd5e2740 msgr2=0x5588fd993320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dd3232740 1 -- 172.21.15.112:0/2002704401 shutdown_connections 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dd3232740 1 -- 172.21.15.112:0/2002704401 wait complete. 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dd3232740 1 -- start start 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dd3232740 1 Processor -- start 2024-09-21T03:28:54.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 0x5588fd9a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5cd670 0x5588fd9a1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5ee350 0x5588fd9acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dd3232740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5588fd75c890 con 0x5588fd5c5490 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dd3232740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5588fd895d40 con 0x5588fd5cd670 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dd3232740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5588fd9a3c60 con 0x5588fd5ee350 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dd3232740 10 client.0 _ll_register_callbacks cb 0x5588fd8f9c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.732+0000 7f3dc659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 0x5588fd9a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dc659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 0x5588fd9a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58546/0 (socket says 172.21.15.112:58546) 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dc659c640 1 -- 172.21.15.112:0/1021475163 learned_addr learned my addr 172.21.15.112:0/1021475163 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dc5d9b640 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5cd670 0x5588fd9a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dc6d9d640 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5ee350 0x5588fd9acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dc5d9b640 1 -- 172.21.15.112:0/1021475163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5ee350 msgr2=0x5588fd9acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:54.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dc5d9b640 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5ee350 0x5588fd9acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dc5d9b640 1 -- 172.21.15.112:0/1021475163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 msgr2=0x5588fd9a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dc5d9b640 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 0x5588fd9a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dc5d9b640 1 -- 172.21.15.112:0/1021475163 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588fd9ad510 con 0x5588fd5cd670 2024-09-21T03:28:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dc6d9d640 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5ee350 0x5588fd9acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dc5d9b640 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5cd670 0x5588fd9a1270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3db0105340 tx=0x7f3db003b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf13e7a50b624d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dadffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:28:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dadffb640 1 -- 172.21.15.112:0/1021475163 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0049070 con 0x5588fd5cd670 2024-09-21T03:28:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dadffb640 1 -- 172.21.15.112:0/1021475163 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3db00d7150 con 0x5588fd5cd670 2024-09-21T03:28:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.733+0000 7f3dd3232740 1 -- 172.21.15.112:0/1021475163 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db8012aa0 con 0x5588fd5cd670 2024-09-21T03:28:54.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dadffb640 1 -- 172.21.15.112:0/1021475163 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3db00cd790 con 0x5588fd5cd670 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dd3232740 10 client.25070 fetch_fsmap learned FSMap version 25 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dd3232740 1 -- 172.21.15.112:0/1021475163 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5588fd9afd70 con 0x5588fd5cd670 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dadffb640 1 -- 172.21.15.112:0/1021475163 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3db8012aa0 con 0x5588fd5cd670 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dadffb640 1 -- 172.21.15.112:0/1021475163 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3db00d20d0 con 0x5588fd5cd670 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dd3232740 10 client.25070 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dd3232740 10 client.25070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dd3232740 20 client.25070 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dd3232740 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5588fd5accd0 0x5588fd9b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dd3232740 4 client.25070 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dd3232740 1 -- 172.21.15.112:0/1021475163 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588fd90ddb0 con 0x5588fd5accd0 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.734+0000 7f3dc659c640 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5588fd5accd0 0x5588fd9b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.735+0000 7f3dadffb640 10 client.25070 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.735+0000 7f3dc659c640 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5588fd5accd0 0x5588fd9b31e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b22e73d346badc4 server_cookie=dcd947a9867de685 in_seq=0 out_seq=0 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.735+0000 7f3dadffb640 1 -- 172.21.15.112:0/1021475163 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588fd90ddb0 con 0x5588fd5accd0 2024-09-21T03:28:54.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.735+0000 7f3dadffb640 10 client.25070 handle_command_reply: tid=0 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.735+0000 7f3dd3232740 1 client.25070 shutdown 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dd3232740 1 -- 172.21.15.112:0/1021475163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5cd670 msgr2=0x5588fd9a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dd3232740 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5cd670 0x5588fd9a1270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3db0105340 tx=0x7f3db003b5d0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dd3232740 1 -- 172.21.15.112:0/1021475163 shutdown_connections 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dd3232740 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5588fd5accd0 0x5588fd9b31e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dd3232740 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5ee350 0x5588fd9acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dadffb640 0 client.25070 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dd3232740 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5cd670 0x5588fd9a1270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dd3232740 1 --2- 172.21.15.112:0/1021475163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 0x5588fd9a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dd3232740 1 -- 172.21.15.112:0/1021475163 >> 172.21.15.112:0/1021475163 conn(0x5588fd5e2740 msgr2=0x5588fd9985c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dd3232740 1 -- 172.21.15.112:0/1021475163 shutdown_connections 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dd3232740 1 -- 172.21.15.112:0/1021475163 wait complete. 2024-09-21T03:28:54.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.736+0000 7f3dd3232740 20 client.25070 trim_cache size 0 max 16384 2024-09-21T03:28:54.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.753+0000 7f3dd3232740 1 Processor -- start 2024-09-21T03:28:54.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.753+0000 7f3dd3232740 1 -- start start 2024-09-21T03:28:54.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 0x5588fd9a1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5cd670 0x5588fd9aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 0x5588fd992c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dd3232740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81b4120 con 0x5588fd5c5490 2024-09-21T03:28:54.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dd3232740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81b19f0 con 0x5588fd5ee350 2024-09-21T03:28:54.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dd3232740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81af840 con 0x5588fd5cd670 2024-09-21T03:28:54.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 0x5588fd9a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 0x5588fd9a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58560/0 (socket says 172.21.15.112:58560) 2024-09-21T03:28:54.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc659c640 1 -- 172.21.15.112:0/255995416 learned_addr learned my addr 172.21.15.112:0/255995416 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:54.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc5d9b640 1 --2- 172.21.15.112:0/255995416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5cd670 0x5588fd9aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc6d9d640 1 --2- 172.21.15.112:0/255995416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 0x5588fd992c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc659c640 1 -- 172.21.15.112:0/255995416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5cd670 msgr2=0x5588fd9aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc659c640 1 --2- 172.21.15.112:0/255995416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5cd670 0x5588fd9aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc659c640 1 -- 172.21.15.112:0/255995416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 msgr2=0x5588fd992c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc659c640 1 --2- 172.21.15.112:0/255995416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 0x5588fd992c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc659c640 1 -- 172.21.15.112:0/255995416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588fd99f190 con 0x5588fd5c5490 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc6d9d640 1 --2- 172.21.15.112:0/255995416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 0x5588fd992c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.754+0000 7f3dc659c640 1 --2- 172.21.15.112:0/255995416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 0x5588fd9a1310 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009eb0 tx=0x7f3db8005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25948b0960167572 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.755+0000 7f3daffff640 1 -- 172.21.15.112:0/255995416 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8032070 con 0x5588fd5c5490 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.755+0000 7f3daffff640 1 -- 172.21.15.112:0/255995416 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3db802a070 con 0x5588fd5c5490 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.755+0000 7f3daffff640 1 -- 172.21.15.112:0/255995416 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db800fb30 con 0x5588fd5c5490 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.755+0000 7f3dd3232740 1 -- 172.21.15.112:0/255995416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 msgr2=0x5588fd9a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.755+0000 7f3dd3232740 1 --2- 172.21.15.112:0/255995416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 0x5588fd9a1310 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009eb0 tx=0x7f3db8005870 comp rx=0 tx=0).stop 2024-09-21T03:28:54.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.755+0000 7f3dd3232740 1 -- 172.21.15.112:0/255995416 shutdown_connections 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.755+0000 7f3dd3232740 1 --2- 172.21.15.112:0/255995416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5ee350 0x5588fd992c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.755+0000 7f3dd3232740 1 --2- 172.21.15.112:0/255995416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5cd670 0x5588fd9aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.755+0000 7f3dd3232740 1 --2- 172.21.15.112:0/255995416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5c5490 0x5588fd9a1310 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.755+0000 7f3dd3232740 1 -- 172.21.15.112:0/255995416 >> 172.21.15.112:0/255995416 conn(0x5588fd5e2740 msgr2=0x5588fd9a0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dd3232740 1 -- 172.21.15.112:0/255995416 shutdown_connections 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dd3232740 1 -- 172.21.15.112:0/255995416 wait complete. 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dd3232740 1 -- start start 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dd3232740 1 Processor -- start 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5c5490 0x5588fd9a5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5cd670 0x5588fd9a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dd3232740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5ee350 0x5588fda1cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dd3232740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81afeb0 con 0x5588fd5ee350 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dd3232740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81b1320 con 0x5588fd5c5490 2024-09-21T03:28:54.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dd3232740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81b0580 con 0x5588fd5cd670 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dd3232740 10 client.0 _ll_register_callbacks cb 0x7f3db0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dc6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5ee350 0x5588fda1cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dc6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5ee350 0x5588fda1cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58576/0 (socket says 172.21.15.112:58576) 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.756+0000 7f3dc6d9d640 1 -- 172.21.15.112:0/496236968 learned_addr learned my addr 172.21.15.112:0/496236968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dc659c640 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5c5490 0x5588fd9a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dc5d9b640 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5cd670 0x5588fd9a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dc6d9d640 1 -- 172.21.15.112:0/496236968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5cd670 msgr2=0x5588fd9a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dc6d9d640 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5cd670 0x5588fd9a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dc6d9d640 1 -- 172.21.15.112:0/496236968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5c5490 msgr2=0x5588fd9a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dc6d9d640 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5c5490 0x5588fd9a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dc6d9d640 1 -- 172.21.15.112:0/496236968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588fda1d0e0 con 0x5588fd5ee350 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dc659c640 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5c5490 0x5588fd9a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:54.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dc6d9d640 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5ee350 0x5588fda1cba0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3dc001e930 tx=0x7f3dc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edca3ead3a9043d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dadffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dadffb640 1 -- 172.21.15.112:0/496236968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc002d060 con 0x5588fd5ee350 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dadffb640 1 -- 172.21.15.112:0/496236968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3dc001d7a0 con 0x5588fd5ee350 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dd3232740 1 -- 172.21.15.112:0/496236968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db802a070 con 0x5588fd5ee350 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dadffb640 1 -- 172.21.15.112:0/496236968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3dc001d9e0 con 0x5588fd5ee350 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dd3232740 10 client.15471 fetch_fsmap learned FSMap version 25 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.757+0000 7f3dd3232740 1 -- 172.21.15.112:0/496236968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5588fda1e330 con 0x5588fd5ee350 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.758+0000 7f3dadffb640 1 -- 172.21.15.112:0/496236968 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3db802a070 con 0x5588fd5ee350 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.758+0000 7f3dadffb640 1 -- 172.21.15.112:0/496236968 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3dc0028c00 con 0x5588fd5ee350 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.758+0000 7f3dd3232740 10 client.15471 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.758+0000 7f3dd3232740 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.758+0000 7f3dd3232740 20 client.15471 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:54.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.758+0000 7f3dd3232740 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5588fd582080 0x5588fda20b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.758+0000 7f3dd3232740 4 client.15471 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:28:54.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.758+0000 7f3dd3232740 1 -- 172.21.15.112:0/496236968 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5588fd909860 con 0x5588fd582080 2024-09-21T03:28:54.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.758+0000 7f3dc659c640 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5588fd582080 0x5588fda20b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.759+0000 7f3dc659c640 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5588fd582080 0x5588fda20b50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce9a566ab3053be server_cookie=44f45f8afb890c82 in_seq=0 out_seq=0 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.759+0000 7f3dadffb640 10 client.15471 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dadffb640 1 -- 172.21.15.112:0/496236968 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5588fd909860 con 0x5588fd582080 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dadffb640 10 client.15471 handle_command_reply: tid=0 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dd3232740 1 client.15471 shutdown 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dd3232740 1 -- 172.21.15.112:0/496236968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5ee350 msgr2=0x5588fda1cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dd3232740 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5ee350 0x5588fda1cba0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3dc001e930 tx=0x7f3dc0004a40 comp rx=0 tx=0).stop 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dd3232740 1 -- 172.21.15.112:0/496236968 shutdown_connections 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dd3232740 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5588fd582080 0x5588fda20b50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dd3232740 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5588fd5ee350 0x5588fda1cba0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dd3232740 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588fd5cd670 0x5588fd9a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dadffb640 0 client.15471 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:54.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dd3232740 1 --2- 172.21.15.112:0/496236968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588fd5c5490 0x5588fd9a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dd3232740 1 -- 172.21.15.112:0/496236968 >> 172.21.15.112:0/496236968 conn(0x5588fd5e2740 msgr2=0x5588fd992bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dd3232740 1 -- 172.21.15.112:0/496236968 shutdown_connections 2024-09-21T03:28:54.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.760+0000 7f3dd3232740 1 -- 172.21.15.112:0/496236968 wait complete. 2024-09-21T03:28:54.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dd3232740 20 client.15471 trim_cache size 0 max 16384 2024-09-21T03:28:54.766 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:28:54.766 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:28:54.766 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "790a90df-67c8-4c06-8295-aa8220726acb", 2024-09-21T03:28:54.766 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:28:54.766 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:28:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 -- 172.21.15.112:0/3573419912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3db40b65f0 msgr2=0x7f3db40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3db40b65f0 0x7f3db40b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0005fd0 tx=0x7f3dc00096a0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 -- 172.21.15.112:0/3573419912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8105380 msgr2=0x7f3dc819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8105380 0x7f3dc819cc70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3db0006010 tx=0x7f3db0004060 comp rx=0 tx=0).stop 2024-09-21T03:28:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 -- 172.21.15.112:0/3573419912 shutdown_connections 2024-09-21T03:28:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3db40b65f0 0x7f3db40b8b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dc810de60 0x7f3dc81a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dc8105380 0x7f3dc819cc70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 --2- 172.21.15.112:0/3573419912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dc8104a30 0x7f3dc819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 -- 172.21.15.112:0/3573419912 >> 172.21.15.112:0/3573419912 conn(0x7f3dc80797b0 msgr2=0x7f3dc80ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 -- 172.21.15.112:0/3573419912 shutdown_connections 2024-09-21T03:28:54.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.761+0000 7f3dcccbe640 1 -- 172.21.15.112:0/3573419912 wait complete. 2024-09-21T03:28:54.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.800+0000 7f1faf60e640 1 -- 172.21.15.112:0/3006278912 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f1fa8063800 con 0x7f1f940b65f0 2024-09-21T03:28:54.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.863+0000 7f1f9e7fc640 1 -- 172.21.15.112:0/3006278912 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f1fa8063800 con 0x7f1f940b65f0 2024-09-21T03:28:54.865 INFO:teuthology.orchestra.run.smithi112.stdout:Retention added to path / 2024-09-21T03:28:54.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 -- 172.21.15.112:0/3006278912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1f940b65f0 msgr2=0x7f1f940b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1f940b65f0 0x7f1f940b8b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4000c00 tx=0x7f1fa4009680 comp rx=0 tx=0).stop 2024-09-21T03:28:54.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 -- 172.21.15.112:0/3006278912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa8108c30 msgr2=0x7f1fa819bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa8108c30 0x7f1fa819bc60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000eaa0 tx=0x7f1fa000ef60 comp rx=0 tx=0).stop 2024-09-21T03:28:54.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 -- 172.21.15.112:0/3006278912 shutdown_connections 2024-09-21T03:28:54.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1f940b65f0 0x7f1f940b8b00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa8108c30 0x7f1fa819bc60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fa81082e0 0x7f1fa819b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 --2- 172.21.15.112:0/3006278912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa81029b0 0x7f1fa819b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 -- 172.21.15.112:0/3006278912 >> 172.21.15.112:0/3006278912 conn(0x7f1fa807a740 msgr2=0x7f1fa81063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 -- 172.21.15.112:0/3006278912 shutdown_connections 2024-09-21T03:28:54.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.865+0000 7f1faf60e640 1 -- 172.21.15.112:0/3006278912 wait complete. 2024-09-21T03:28:54.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T03:28:54.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:54 smithi203 ceph-mon[34723]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:54.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:54 smithi203 ceph-mon[34723]: pgmap v315: 97 pgs: 97 active+clean; 649 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T03:28:54.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:54 smithi187 ceph-mon[34603]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:54.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:54 smithi187 ceph-mon[34603]: pgmap v315: 97 pgs: 97 active+clean; 649 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T03:28:54.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.989+0000 7f39af6ec640 1 -- 172.21.15.112:0/499439045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39a8104930 msgr2=0x7f39a8104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.989+0000 7f39af6ec640 1 --2- 172.21.15.112:0/499439045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39a8104930 0x7f39a8104d40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009900 tx=0x7f39a402a410 comp rx=0 tx=0).stop 2024-09-21T03:28:54.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.989+0000 7f39af6ec640 1 -- 172.21.15.112:0/499439045 shutdown_connections 2024-09-21T03:28:54.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.989+0000 7f39af6ec640 1 --2- 172.21.15.112:0/499439045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a810dd90 0x7f39a8110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.989+0000 7f39af6ec640 1 --2- 172.21.15.112:0/499439045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39a8105280 0x7f39a810d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.989+0000 7f39af6ec640 1 --2- 172.21.15.112:0/499439045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39a8104930 0x7f39a8104d40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.989+0000 7f39af6ec640 1 -- 172.21.15.112:0/499439045 >> 172.21.15.112:0/499439045 conn(0x7f39a80fe3c0 msgr2=0x7f39a8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:54.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.989+0000 7f39af6ec640 1 -- 172.21.15.112:0/499439045 shutdown_connections 2024-09-21T03:28:54.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.989+0000 7f39af6ec640 1 -- 172.21.15.112:0/499439045 wait complete. 2024-09-21T03:28:54.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.990+0000 7f39af6ec640 1 Processor -- start 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.990+0000 7f39af6ec640 1 -- start start 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.990+0000 7f39af6ec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39a8104930 0x7f39a819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.990+0000 7f39af6ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a8105280 0x7f39a819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.990+0000 7f39af6ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39a810dd90 0x7f39a819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.990+0000 7f39af6ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8113a10 con 0x7f39a810dd90 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.990+0000 7f39af6ec640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8113890 con 0x7f39a8104930 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.990+0000 7f39af6ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8113b90 con 0x7f39a8105280 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39adc89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39a810dd90 0x7f39a819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39adc89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39a810dd90 0x7f39a819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58588/0 (socket says 172.21.15.112:58588) 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39adc89640 1 -- 172.21.15.112:0/1628650266 learned_addr learned my addr 172.21.15.112:0/1628650266 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39acc87640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a8105280 0x7f39a819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39ad488640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39a8104930 0x7f39a819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39adc89640 1 -- 172.21.15.112:0/1628650266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a8105280 msgr2=0x7f39a819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39adc89640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a8105280 0x7f39a819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39adc89640 1 -- 172.21.15.112:0/1628650266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39a8104930 msgr2=0x7f39a819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39adc89640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39a8104930 0x7f39a819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39adc89640 1 -- 172.21.15.112:0/1628650266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a81a0400 con 0x7f39a810dd90 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39adc89640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39a810dd90 0x7f39a819fd00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f39a000eaa0 tx=0x7f39a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f399e7fc640 1 -- 172.21.15.112:0/1628650266 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a000cdb0 con 0x7f39a810dd90 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f399e7fc640 1 -- 172.21.15.112:0/1628650266 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a0005020 con 0x7f39a810dd90 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f399e7fc640 1 -- 172.21.15.112:0/1628650266 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a00116d0 con 0x7f39a810dd90 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39af6ec640 1 -- 172.21.15.112:0/1628650266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a81a06f0 con 0x7f39a810dd90 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.991+0000 7f39af6ec640 1 -- 172.21.15.112:0/1628650266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a81a6e80 con 0x7f39a810dd90 2024-09-21T03:28:54.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.992+0000 7f39af6ec640 1 -- 172.21.15.112:0/1628650266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a8104d40 con 0x7f39a810dd90 2024-09-21T03:28:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.994+0000 7f399e7fc640 1 -- 172.21.15.112:0/1628650266 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f39a00051d0 con 0x7f39a810dd90 2024-09-21T03:28:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.995+0000 7f399e7fc640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39940b6670 0x7f39940b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.995+0000 7f39ad488640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39940b6670 0x7f39940b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.995+0000 7f399e7fc640 1 -- 172.21.15.112:0/1628650266 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f39a00e7a30 con 0x7f39a810dd90 2024-09-21T03:28:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.995+0000 7f39ad488640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39940b6670 0x7f39940b8b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f39a4031440 tx=0x7f39a4005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:54.998+0000 7f399e7fc640 1 -- 172.21.15.112:0/1628650266 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39a0016030 con 0x7f39a810dd90 2024-09-21T03:28:55.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.199+0000 7f39af6ec640 1 -- 172.21.15.112:0/1628650266 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f39a8107fb0 con 0x7f39940b6670 2024-09-21T03:28:55.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.206+0000 7f399e7fc640 1 -- 172.21.15.112:0/1628650266 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f39a8107fb0 con 0x7f39940b6670 2024-09-21T03:28:55.208 INFO:teuthology.orchestra.run.smithi112.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T03:28:54", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T03:28:55.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.208+0000 7f39af6ec640 1 -- 172.21.15.112:0/1628650266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39940b6670 msgr2=0x7f39940b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:55.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.208+0000 7f39af6ec640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39940b6670 0x7f39940b8b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f39a4031440 tx=0x7f39a4005d50 comp rx=0 tx=0).stop 2024-09-21T03:28:55.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.208+0000 7f39af6ec640 1 -- 172.21.15.112:0/1628650266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39a810dd90 msgr2=0x7f39a819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:55.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.208+0000 7f39af6ec640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39a810dd90 0x7f39a819fd00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f39a000eaa0 tx=0x7f39a000ef60 comp rx=0 tx=0).stop 2024-09-21T03:28:55.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.209+0000 7f39af6ec640 1 -- 172.21.15.112:0/1628650266 shutdown_connections 2024-09-21T03:28:55.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.209+0000 7f39af6ec640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39940b6670 0x7f39940b8b80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.209+0000 7f39af6ec640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39a810dd90 0x7f39a819fd00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.209+0000 7f39af6ec640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39a8105280 0x7f39a819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.209+0000 7f39af6ec640 1 --2- 172.21.15.112:0/1628650266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39a8104930 0x7f39a819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.209+0000 7f39af6ec640 1 -- 172.21.15.112:0/1628650266 >> 172.21.15.112:0/1628650266 conn(0x7f39a80fe3c0 msgr2=0x7f39a81078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:55.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.209+0000 7f39af6ec640 1 -- 172.21.15.112:0/1628650266 shutdown_connections 2024-09-21T03:28:55.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.209+0000 7f39af6ec640 1 -- 172.21.15.112:0/1628650266 wait complete. 2024-09-21T03:28:55.223 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T03:28:55.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.333+0000 7fefad58a640 1 -- 172.21.15.112:0/1036014088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa81082e0 msgr2=0x7fefa81086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:55.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.333+0000 7fefad58a640 1 --2- 172.21.15.112:0/1036014088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa81082e0 0x7fefa81086f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009900 tx=0x7fef9c02a410 comp rx=0 tx=0).stop 2024-09-21T03:28:55.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.333+0000 7fefad58a640 1 -- 172.21.15.112:0/1036014088 shutdown_connections 2024-09-21T03:28:55.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.333+0000 7fefad58a640 1 --2- 172.21.15.112:0/1036014088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefa8108c30 0x7fefa810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.333+0000 7fefad58a640 1 --2- 172.21.15.112:0/1036014088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa81082e0 0x7fefa81086f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.333+0000 7fefad58a640 1 --2- 172.21.15.112:0/1036014088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefa81029b0 0x7fefa8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.333+0000 7fefad58a640 1 -- 172.21.15.112:0/1036014088 >> 172.21.15.112:0/1036014088 conn(0x7fefa807a740 msgr2=0x7fefa807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:55.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.333+0000 7fefad58a640 1 -- 172.21.15.112:0/1036014088 shutdown_connections 2024-09-21T03:28:55.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.333+0000 7fefad58a640 1 -- 172.21.15.112:0/1036014088 wait complete. 2024-09-21T03:28:55.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.333+0000 7fefad58a640 1 Processor -- start 2024-09-21T03:28:55.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.333+0000 7fefad58a640 1 -- start start 2024-09-21T03:28:55.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefad58a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefa81029b0 0x7fefa819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:55.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefad58a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa81082e0 0x7fefa819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:55.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefad58a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefa8108c30 0x7fefa819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:55.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefad58a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8113a10 con 0x7fefa81082e0 2024-09-21T03:28:55.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefad58a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8113890 con 0x7fefa8108c30 2024-09-21T03:28:55.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefad58a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8113b90 con 0x7fefa81029b0 2024-09-21T03:28:55.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefa67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa81082e0 0x7fefa819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:55.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefa67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa81082e0 0x7fefa819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40484/0 (socket says 172.21.15.112:40484) 2024-09-21T03:28:55.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefa67fc640 1 -- 172.21.15.112:0/2870340449 learned_addr learned my addr 172.21.15.112:0/2870340449 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:55.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefa6ffd640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefa81029b0 0x7fefa819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:55.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefa77fe640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefa8108c30 0x7fefa819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:55.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefa67fc640 1 -- 172.21.15.112:0/2870340449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefa81029b0 msgr2=0x7fefa819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:55.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefa67fc640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefa81029b0 0x7fefa819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefa67fc640 1 -- 172.21.15.112:0/2870340449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefa8108c30 msgr2=0x7fefa819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:55.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefa67fc640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefa8108c30 0x7fefa819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefa67fc640 1 -- 172.21.15.112:0/2870340449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa81a0340 con 0x7fefa81082e0 2024-09-21T03:28:55.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.334+0000 7fefa67fc640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa81082e0 0x7fefa819a9b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fef9c02a930 tx=0x7fef9c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:55.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.335+0000 7fef8ffff640 1 -- 172.21.15.112:0/2870340449 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c03f030 con 0x7fefa81082e0 2024-09-21T03:28:55.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.335+0000 7fefad58a640 1 -- 172.21.15.112:0/2870340449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa81a05d0 con 0x7fefa81082e0 2024-09-21T03:28:55.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.335+0000 7fef8ffff640 1 -- 172.21.15.112:0/2870340449 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9c003aa0 con 0x7fefa81082e0 2024-09-21T03:28:55.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.335+0000 7fef8ffff640 1 -- 172.21.15.112:0/2870340449 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c035070 con 0x7fefa81082e0 2024-09-21T03:28:55.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.335+0000 7fefad58a640 1 -- 172.21.15.112:0/2870340449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa81a6dc0 con 0x7fefa81082e0 2024-09-21T03:28:55.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.336+0000 7fefad58a640 1 -- 172.21.15.112:0/2870340449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa8102dc0 con 0x7fefa81082e0 2024-09-21T03:28:55.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.337+0000 7fef8ffff640 1 -- 172.21.15.112:0/2870340449 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7fef9c042030 con 0x7fefa81082e0 2024-09-21T03:28:55.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.338+0000 7fef8ffff640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fef900b65f0 0x7fef900b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:55.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.338+0000 7fefa6ffd640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fef900b65f0 0x7fef900b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:55.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.338+0000 7fef8ffff640 1 -- 172.21.15.112:0/2870340449 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fef9c107720 con 0x7fefa81082e0 2024-09-21T03:28:55.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.338+0000 7fefa6ffd640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fef900b65f0 0x7fef900b8b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef9800bbe0 tx=0x7fef9800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:55.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.342+0000 7fef8ffff640 1 -- 172.21.15.112:0/2870340449 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fef9c0cdd90 con 0x7fefa81082e0 2024-09-21T03:28:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:55 smithi112 ceph-mon[31442]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:28:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:55 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:28:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:55 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:55 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:28:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:55 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:55 smithi112 ceph-mon[31442]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:55 smithi112 ceph-mon[31442]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:55.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.547+0000 7fefad58a640 1 -- 172.21.15.112:0/2870340449 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fefa8063800 con 0x7fef900b65f0 2024-09-21T03:28:55.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.554+0000 7fef8ffff640 1 -- 172.21.15.112:0/2870340449 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fefa8063800 con 0x7fef900b65f0 2024-09-21T03:28:55.556 INFO:teuthology.orchestra.run.smithi112.stdout:/ 1m 6m3h 2024-09-21T03:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 -- 172.21.15.112:0/2870340449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fef900b65f0 msgr2=0x7fef900b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fef900b65f0 0x7fef900b8b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef9800bbe0 tx=0x7fef9800b390 comp rx=0 tx=0).stop 2024-09-21T03:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 -- 172.21.15.112:0/2870340449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa81082e0 msgr2=0x7fefa819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa81082e0 0x7fefa819a9b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fef9c02a930 tx=0x7fef9c004270 comp rx=0 tx=0).stop 2024-09-21T03:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 -- 172.21.15.112:0/2870340449 shutdown_connections 2024-09-21T03:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fef900b65f0 0x7fef900b8b00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefa8108c30 0x7fefa819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa81082e0 0x7fefa819a9b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 --2- 172.21.15.112:0/2870340449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefa81029b0 0x7fefa819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 -- 172.21.15.112:0/2870340449 >> 172.21.15.112:0/2870340449 conn(0x7fefa807a740 msgr2=0x7fefa81063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:55.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 -- 172.21.15.112:0/2870340449 shutdown_connections 2024-09-21T03:28:55.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:55.557+0000 7fefad58a640 1 -- 172.21.15.112:0/2870340449 wait complete. 2024-09-21T03:28:55.573 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T03:28:55.595 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:28:55.599 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T03:28:55.611 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T03:28:55.647 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:28:55.647 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T03:28:55.647 INFO:tasks.workunit:timeout=3h 2024-09-21T03:28:55.647 INFO:tasks.workunit:cleanup=True 2024-09-21T03:28:55.647 DEBUG:teuthology.orchestra.run.smithi112:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T03:28:55.663 INFO:teuthology.orchestra.run.smithi112.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T03:28:55.663 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T03:28:55.663 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-21T03:28:55.663 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:28:55.663 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T03:28:55.663 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 03:27:19.754998123 +0000 2024-09-21T03:28:55.663 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 03:27:19.754998123 +0000 2024-09-21T03:28:55.663 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 03:28:45.365811307 +0000 2024-09-21T03:28:55.663 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 03:27:19.754998123 +0000 2024-09-21T03:28:55.664 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T03:28:55.664 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T03:28:55.738 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:28:55.794 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T03:28:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:28:55.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:55 smithi203 ceph-mon[34723]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:28:55.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:55 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:28:55.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:55 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:55.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:55 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:28:55.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:55 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:55.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:55 smithi203 ceph-mon[34723]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:55.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:55 smithi203 ceph-mon[34723]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:55.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:55 smithi187 ceph-mon[34603]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:28:55.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:55 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:28:55.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:55 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:55.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:55 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:28:55.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:55 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:28:55.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:55 smithi187 ceph-mon[34603]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:55.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:55 smithi187 ceph-mon[34603]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:56.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.401+0000 7f3b13f2d640 1 -- 172.21.15.112:0/2625625058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c103b90 msgr2=0x7f3b0c103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:56.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.401+0000 7f3b13f2d640 1 --2- 172.21.15.112:0/2625625058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c103b90 0x7f3b0c103fa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009900 tx=0x7f3b0802a410 comp rx=0 tx=0).stop 2024-09-21T03:28:56.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.401+0000 7f3b13f2d640 1 -- 172.21.15.112:0/2625625058 shutdown_connections 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.401+0000 7f3b13f2d640 1 --2- 172.21.15.112:0/2625625058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c109e20 0x7f3b0c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.401+0000 7f3b13f2d640 1 --2- 172.21.15.112:0/2625625058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b0c1094d0 0x7f3b0c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.401+0000 7f3b13f2d640 1 --2- 172.21.15.112:0/2625625058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c103b90 0x7f3b0c103fa0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.401+0000 7f3b13f2d640 1 -- 172.21.15.112:0/2625625058 >> 172.21.15.112:0/2625625058 conn(0x7f3b0c0ff5e0 msgr2=0x7f3b0c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.401+0000 7f3b13f2d640 1 -- 172.21.15.112:0/2625625058 shutdown_connections 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.401+0000 7f3b13f2d640 1 -- 172.21.15.112:0/2625625058 wait complete. 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b13f2d640 1 Processor -- start 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b13f2d640 1 -- start start 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b13f2d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c103b90 0x7f3b0c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b13f2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1094d0 0x7f3b0c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b13f2d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b0c109e20 0x7f3b0c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b13f2d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c113ae0 con 0x7f3b0c1094d0 2024-09-21T03:28:56.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b13f2d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c113960 con 0x7f3b0c103b90 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b13f2d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c113c60 con 0x7f3b0c109e20 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b114c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1094d0 0x7f3b0c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b114c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1094d0 0x7f3b0c19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40492/0 (socket says 172.21.15.112:40492) 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b114c8640 1 -- 172.21.15.112:0/3916489735 learned_addr learned my addr 172.21.15.112:0/3916489735 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.402+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c103b90 0x7f3b0c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b124ca640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b0c109e20 0x7f3b0c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b114c8640 1 -- 172.21.15.112:0/3916489735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b0c109e20 msgr2=0x7f3b0c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b114c8640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b0c109e20 0x7f3b0c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b114c8640 1 -- 172.21.15.112:0/3916489735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c103b90 msgr2=0x7f3b0c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b114c8640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c103b90 0x7f3b0c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b114c8640 1 -- 172.21.15.112:0/3916489735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c1a0440 con 0x7f3b0c1094d0 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b124ca640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b0c109e20 0x7f3b0c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:28:56.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b114c8640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1094d0 0x7f3b0c19aa70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c970 tx=0x7f3afc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3916489735 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc004220 con 0x7f3b0c1094d0 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3916489735 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3afc0043c0 con 0x7f3b0c1094d0 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b13f2d640 1 -- 172.21.15.112:0/3916489735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0c1a0730 con 0x7f3b0c1094d0 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3916489735 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc004a00 con 0x7f3b0c1094d0 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.403+0000 7f3b13f2d640 1 -- 172.21.15.112:0/3916489735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0c1a6e90 con 0x7f3b0c1094d0 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b1a4a1740 1 Processor -- start 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b1a4a1740 1 -- start start 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 0x55e54a571650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 0x55e54a571b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a5720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e54a4e7e50 con 0x55e54a1a9390 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e54a4d7280 con 0x55e54a1a11b0 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e54a4db090 con 0x55e54a1ca070 2024-09-21T03:28:56.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3916489735 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f3afc004ba0 con 0x7f3b0c1094d0 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b124ca640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 0x55e54a571650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b114c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a5720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b124ca640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 0x55e54a571650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:36902/0 (socket says 172.21.15.112:36902) 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b124ca640 1 -- 172.21.15.112:0/2309691935 learned_addr learned my addr 172.21.15.112:0/2309691935 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.406+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/2309691935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 0x55e54a571b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3b124ca640 1 -- 172.21.15.112:0/2309691935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 msgr2=0x55e54a571b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3b124ca640 1 --2- 172.21.15.112:0/2309691935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 0x55e54a571b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3b124ca640 1 -- 172.21.15.112:0/2309691935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 msgr2=0x55e54a5720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3b124ca640 1 --2- 172.21.15.112:0/2309691935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a5720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3b124ca640 1 -- 172.21.15.112:0/2309691935 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e54a235650 con 0x55e54a1ca070 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/2309691935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 0x55e54a571b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3b124ca640 1 --2- 172.21.15.112:0/2309691935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 0x55e54a571650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009f90 tx=0x7f3b0400f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=599ad0ad1f8a7d2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3b02ffd640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3af80b65f0 0x7f3af80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3af80b65f0 0x7f3af80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3af7fff640 1 -- 172.21.15.112:0/2309691935 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04003b70 con 0x55e54a1ca070 2024-09-21T03:28:56.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3b02ffd640 1 -- 172.21.15.112:0/3916489735 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3afc0e7390 con 0x7f3b0c1094d0 2024-09-21T03:28:56.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3af7fff640 1 -- 172.21.15.112:0/2309691935 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b04003d10 con 0x55e54a1ca070 2024-09-21T03:28:56.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.407+0000 7f3af7fff640 1 -- 172.21.15.112:0/2309691935 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04011900 con 0x55e54a1ca070 2024-09-21T03:28:56.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/2309691935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 msgr2=0x55e54a571650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:56.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3af80b65f0 0x7f3af80b8b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3b08031440 tx=0x7f3b08009640 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:56.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/2309691935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 0x55e54a571650 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009f90 tx=0x7f3b0400f7f0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/2309691935 shutdown_connections 2024-09-21T03:28:56.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/2309691935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a5720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/2309691935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 0x55e54a571b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/2309691935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 0x55e54a571650 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/2309691935 >> 172.21.15.112:0/2309691935 conn(0x55e54a1be460 msgr2=0x55e54a56f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/2309691935 shutdown_connections 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/2309691935 wait complete. 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b1a4a1740 1 -- start start 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.409+0000 7f3b1a4a1740 1 Processor -- start 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 0x55e54a57cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a57d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 0x55e54a589120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e54a338700 con 0x55e54a1a9390 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e54a472110 con 0x55e54a1a11b0 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e54a57fe30 con 0x55e54a1ca070 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b114c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a57d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b114c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a57d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40510/0 (socket says 172.21.15.112:40510) 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b11cc9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 0x55e54a57cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b114c8640 1 -- 172.21.15.112:0/918494843 learned_addr learned my addr 172.21.15.112:0/918494843 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:56.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b1a4a1740 10 client.0 _ll_register_callbacks cb 0x55e54a48b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.410+0000 7f3b124ca640 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 0x55e54a589120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3b114c8640 1 -- 172.21.15.112:0/918494843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 msgr2=0x55e54a589120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3b114c8640 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 0x55e54a589120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3b114c8640 1 -- 172.21.15.112:0/918494843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 msgr2=0x55e54a57cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3b114c8640 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 0x55e54a57cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3b114c8640 1 -- 172.21.15.112:0/918494843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e54a49fc40 con 0x55e54a1a9390 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 0x55e54a57cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3b124ca640 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 0x55e54a589120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3b114c8640 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a57d2b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c940 tx=0x7f3afc005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b0d73bcd42a6be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3af57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3af57fa640 1 -- 172.21.15.112:0/918494843 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc01d070 con 0x55e54a1a9390 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3af57fa640 1 -- 172.21.15.112:0/918494843 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3afc0a8c50 con 0x55e54a1a9390 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/918494843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b04003d10 con 0x55e54a1a9390 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3af57fa640 1 -- 172.21.15.112:0/918494843 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3afc0a91a0 con 0x55e54a1a9390 2024-09-21T03:28:56.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3b1a4a1740 10 client.15495 fetch_fsmap learned FSMap version 25 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.411+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/918494843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e54a58bbe0 con 0x55e54a1a9390 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.412+0000 7f3af57fa640 1 -- 172.21.15.112:0/918494843 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b04003d10 con 0x55e54a1a9390 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.413+0000 7f3af57fa640 1 -- 172.21.15.112:0/918494843 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3afc0a8e50 con 0x55e54a1a9390 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.413+0000 7f3b1a4a1740 10 client.15495 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.413+0000 7f3b1a4a1740 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.413+0000 7f3b1a4a1740 20 client.15495 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.413+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e54a1889f0 0x55e54a58f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.413+0000 7f3b1a4a1740 4 client.15495 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.413+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/918494843 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e54a4e9da0 con 0x55e54a1889f0 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.413+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e54a1889f0 0x55e54a58f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.413+0000 7f3af57fa640 10 client.15495 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.413+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e54a1889f0 0x55e54a58f0b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d82c16250c2a414 server_cookie=53cf4acb4d34663 in_seq=0 out_seq=0 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.414+0000 7f3af57fa640 1 -- 172.21.15.112:0/918494843 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e54a4e9da0 con 0x55e54a1889f0 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.414+0000 7f3af57fa640 10 client.15495 handle_command_reply: tid=0 2024-09-21T03:28:56.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.414+0000 7f3b1a4a1740 1 client.15495 shutdown 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.414+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/918494843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 msgr2=0x55e54a57d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.414+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a57d2b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c940 tx=0x7f3afc005260 comp rx=0 tx=0).stop 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.415+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/918494843 shutdown_connections 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.415+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e54a1889f0 0x55e54a58f0b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.415+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1ca070 0x55e54a589120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.415+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a57d2b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.415+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/918494843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1a11b0 0x55e54a57cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.415+0000 7f3af57fa640 0 client.15495 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.415+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/918494843 >> 172.21.15.112:0/918494843 conn(0x55e54a1be460 msgr2=0x55e54a5747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.415+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/918494843 shutdown_connections 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.415+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/918494843 wait complete. 2024-09-21T03:28:56.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.415+0000 7f3b1a4a1740 20 client.15495 trim_cache size 0 max 16384 2024-09-21T03:28:56.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b1a4a1740 1 Processor -- start 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b1a4a1740 1 -- start start 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1a11b0 0x55e54a57ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a572d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1ca070 0x55e54a56eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1afc80 con 0x55e54a1a9390 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1ad550 con 0x55e54a1ca070 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1ab3a0 con 0x55e54a1a11b0 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b114c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a572d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b114c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a572d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40514/0 (socket says 172.21.15.112:40514) 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b114c8640 1 -- 172.21.15.112:0/3768322110 learned_addr learned my addr 172.21.15.112:0/3768322110 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/3768322110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1a11b0 0x55e54a57ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.451+0000 7f3b124ca640 1 --2- 172.21.15.112:0/3768322110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1ca070 0x55e54a56eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.452+0000 7f3b114c8640 1 -- 172.21.15.112:0/3768322110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1a11b0 msgr2=0x55e54a57ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.452+0000 7f3b114c8640 1 --2- 172.21.15.112:0/3768322110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1a11b0 0x55e54a57ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.452+0000 7f3b114c8640 1 -- 172.21.15.112:0/3768322110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1ca070 msgr2=0x55e54a56eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.452+0000 7f3b114c8640 1 --2- 172.21.15.112:0/3768322110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1ca070 0x55e54a56eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.452+0000 7f3b114c8640 1 -- 172.21.15.112:0/3768322110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e54a572460 con 0x55e54a1a9390 2024-09-21T03:28:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.452+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/3768322110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1a11b0 0x55e54a57ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:28:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.452+0000 7f3b114c8640 1 --2- 172.21.15.112:0/3768322110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a572d90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0aa570 tx=0x7f3afc005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b860c454d2fde2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.452+0000 7f3af77fe640 1 -- 172.21.15.112:0/3768322110 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0d4030 con 0x55e54a1a9390 2024-09-21T03:28:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.452+0000 7f3af77fe640 1 -- 172.21.15.112:0/3768322110 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3afc0ace10 con 0x55e54a1a9390 2024-09-21T03:28:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.452+0000 7f3af77fe640 1 -- 172.21.15.112:0/3768322110 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc01d070 con 0x55e54a1a9390 2024-09-21T03:28:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.453+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/3768322110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 msgr2=0x55e54a572d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.453+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/3768322110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a572d90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0aa570 tx=0x7f3afc005350 comp rx=0 tx=0).stop 2024-09-21T03:28:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.453+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/3768322110 shutdown_connections 2024-09-21T03:28:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.453+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/3768322110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1ca070 0x55e54a56eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.453+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/3768322110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a572d90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.453+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/3768322110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1a11b0 0x55e54a57ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.453+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/3768322110 >> 172.21.15.112:0/3768322110 conn(0x55e54a1be460 msgr2=0x55e54a572680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.453+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/3768322110 shutdown_connections 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.453+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/3768322110 wait complete. 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.453+0000 7f3b1a4a1740 1 -- start start 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b1a4a1740 1 Processor -- start 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1a11b0 0x55e54a585920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a580dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b1a4a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1ca070 0x55e54a5813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1aba10 con 0x55e54a1a9390 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1ace80 con 0x55e54a1ca070 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b1a4a1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c1ac0e0 con 0x55e54a1a11b0 2024-09-21T03:28:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b1a4a1740 10 client.0 _ll_register_callbacks cb 0x7f3afc00a450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b114c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a580dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b114c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a580dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40518/0 (socket says 172.21.15.112:40518) 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b114c8640 1 -- 172.21.15.112:0/1609665490 learned_addr learned my addr 172.21.15.112:0/1609665490 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1a11b0 0x55e54a585920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.454+0000 7f3b124ca640 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1ca070 0x55e54a5813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.455+0000 7f3b114c8640 1 -- 172.21.15.112:0/1609665490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1a11b0 msgr2=0x55e54a585920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.455+0000 7f3b114c8640 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1a11b0 0x55e54a585920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.455+0000 7f3b114c8640 1 -- 172.21.15.112:0/1609665490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1ca070 msgr2=0x55e54a5813d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.455+0000 7f3b114c8640 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1ca070 0x55e54a5813d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.455+0000 7f3b114c8640 1 -- 172.21.15.112:0/1609665490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e54a581bb0 con 0x55e54a1a9390 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.455+0000 7f3b114c8640 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a580dd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0b32c0 tx=0x7f3afc0b33c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991b0c67992cf502 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.455+0000 7f3af57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.455+0000 7f3af57fa640 1 -- 172.21.15.112:0/1609665490 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0c8060 con 0x55e54a1a9390 2024-09-21T03:28:56.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.455+0000 7f3af57fa640 1 -- 172.21.15.112:0/1609665490 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3afc0ac1d0 con 0x55e54a1a9390 2024-09-21T03:28:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.455+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/1609665490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3afc0ace10 con 0x55e54a1a9390 2024-09-21T03:28:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.458+0000 7f3af57fa640 1 -- 172.21.15.112:0/1609665490 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3afc0cf050 con 0x55e54a1a9390 2024-09-21T03:28:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.458+0000 7f3b1a4a1740 10 client.15501 fetch_fsmap learned FSMap version 25 2024-09-21T03:28:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.458+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/1609665490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e54a5f9a70 con 0x55e54a1a9390 2024-09-21T03:28:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.458+0000 7f3af57fa640 1 -- 172.21.15.112:0/1609665490 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3afc0ace10 con 0x55e54a1a9390 2024-09-21T03:28:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.459+0000 7f3af57fa640 1 -- 172.21.15.112:0/1609665490 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3afc007c10 con 0x55e54a1a9390 2024-09-21T03:28:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.459+0000 7f3b1a4a1740 10 client.15501 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:28:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.459+0000 7f3b1a4a1740 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:28:56.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.459+0000 7f3b1a4a1740 20 client.15501 populate_metadata read hostname 'smithi112' 2024-09-21T03:28:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.459+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e54a15dda0 0x55e54a5fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:28:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.459+0000 7f3b1a4a1740 4 client.15501 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:28:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.459+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/1609665490 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e54a4e5850 con 0x55e54a15dda0 2024-09-21T03:28:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.459+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e54a15dda0 0x55e54a5fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:28:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.459+0000 7f3af57fa640 10 client.15501 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.459+0000 7f3b11cc9640 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e54a15dda0 0x55e54a5fbfa0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b405f85417cf8a21 server_cookie=70afde5c11a1166d in_seq=0 out_seq=0 2024-09-21T03:28:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3af57fa640 1 -- 172.21.15.112:0/1609665490 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x55e54a4e5850 con 0x55e54a15dda0 2024-09-21T03:28:56.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3af57fa640 10 client.15501 handle_command_reply: tid=0 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 1 client.15501 shutdown 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/1609665490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 msgr2=0x55e54a580dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a580dd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0b32c0 tx=0x7f3afc0b33c0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/1609665490 shutdown_connections 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e54a15dda0 0x55e54a5fbfa0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e54a1ca070 0x55e54a5813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e54a1a9390 0x55e54a580dd0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3af57fa640 0 client.15501 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 1 --2- 172.21.15.112:0/1609665490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e54a1a11b0 0x55e54a585920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/1609665490 >> 172.21.15.112:0/1609665490 conn(0x55e54a1be460 msgr2=0x55e54a57e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/1609665490 shutdown_connections 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 1 -- 172.21.15.112:0/1609665490 wait complete. 2024-09-21T03:28:56.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.460+0000 7f3b1a4a1740 20 client.15501 trim_cache size 0 max 16384 2024-09-21T03:28:56.466 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:28:56.466 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:28:56.466 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:28:56.466 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:28:56.466 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:28:56.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "790a90df-67c8-4c06-8295-aa8220726acb", 2024-09-21T03:28:56.466 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:28:56.466 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:28:56.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:28:56.467 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:28:56.467 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:28:56.467 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:28:56.467 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:28:56.467 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:28:56.467 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:28:56.467 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:28:56.467 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:28:56.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:28:56.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:28:56.467 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 -- 172.21.15.112:0/3916489735 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3af80b65f0 msgr2=0x7f3af80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3af80b65f0 0x7f3af80b8b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3b08031440 tx=0x7f3b08009640 comp rx=0 tx=0).stop 2024-09-21T03:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 -- 172.21.15.112:0/3916489735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1094d0 msgr2=0x7f3b0c19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1094d0 0x7f3b0c19aa70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c970 tx=0x7f3afc00ce30 comp rx=0 tx=0).stop 2024-09-21T03:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 -- 172.21.15.112:0/3916489735 shutdown_connections 2024-09-21T03:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3af80b65f0 0x7f3af80b8b00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b0c109e20 0x7f3b0c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b0c1094d0 0x7f3b0c19aa70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 --2- 172.21.15.112:0/3916489735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c103b90 0x7f3b0c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:28:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 -- 172.21.15.112:0/3916489735 >> 172.21.15.112:0/3916489735 conn(0x7f3b0c0ff5e0 msgr2=0x7f3b0c101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:28:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 -- 172.21.15.112:0/3916489735 shutdown_connections 2024-09-21T03:28:56.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:28:56.461+0000 7f3b13f2d640 1 -- 172.21.15.112:0/3916489735 wait complete. 2024-09-21T03:28:56.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:56 smithi112 ceph-mon[31442]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:56.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:56 smithi112 ceph-mon[31442]: pgmap v316: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 153 KiB/s wr, 25 op/s 2024-09-21T03:28:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:56 smithi203 ceph-mon[34723]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:56 smithi203 ceph-mon[34723]: pgmap v316: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 153 KiB/s wr, 25 op/s 2024-09-21T03:28:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:56 smithi187 ceph-mon[34603]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:28:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:56 smithi187 ceph-mon[34603]: pgmap v316: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 153 KiB/s wr, 25 op/s 2024-09-21T03:28:57.268 INFO:tasks.cephfs.filesystem:scrub status for tag:790a90df-67c8-4c06-8295-aa8220726acb - {'path': '/', 'tag': '790a90df-67c8-4c06-8295-aa8220726acb', 'options': 'recursive,force'} 2024-09-21T03:28:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:28:59 smithi187 ceph-mon[34603]: pgmap v317: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 153 KiB/s wr, 24 op/s 2024-09-21T03:28:59.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:28:59 smithi112 ceph-mon[31442]: pgmap v317: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 153 KiB/s wr, 24 op/s 2024-09-21T03:28:59.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:28:59 smithi203 ceph-mon[34723]: pgmap v317: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 153 KiB/s wr, 24 op/s 2024-09-21T03:29:01.287 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:29:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:29:00.875+0000 7f3805f5e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:29:01.287 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:29:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:29:00.875+0000 7f3805f5e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:29:01.288 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:29:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:29:00.875+0000 7f3805f5e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:29:01.288 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:29:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:29:00.875+0000 7f3805f5e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:29:01.288 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:29:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:29:00.875+0000 7f3805f5e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:29:01.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:01 smithi187 ceph-mon[34603]: pgmap v318: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 176 KiB/s wr, 54 op/s 2024-09-21T03:29:01.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:01 smithi112 ceph-mon[31442]: pgmap v318: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 176 KiB/s wr, 54 op/s 2024-09-21T03:29:01.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:01 smithi203 ceph-mon[34723]: pgmap v318: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 176 KiB/s wr, 54 op/s 2024-09-21T03:29:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:03 smithi187 ceph-mon[34603]: pgmap v319: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-21T03:29:03.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:03 smithi112 ceph-mon[31442]: pgmap v319: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-21T03:29:03.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:03 smithi203 ceph-mon[34723]: pgmap v319: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-09-21T03:29:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:04 smithi112 ceph-mon[31442]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T03:29:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:04 smithi112 ceph-mon[31442]: pgmap v320: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 176 KiB/s wr, 54 op/s 2024-09-21T03:29:04.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:04 smithi203 ceph-mon[34723]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T03:29:04.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:04 smithi203 ceph-mon[34723]: pgmap v320: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 176 KiB/s wr, 54 op/s 2024-09-21T03:29:04.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:04 smithi187 ceph-mon[34603]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T03:29:04.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:04 smithi187 ceph-mon[34603]: pgmap v320: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 176 KiB/s wr, 54 op/s 2024-09-21T03:29:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:07 smithi187 ceph-mon[34603]: pgmap v321: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 69 op/s 2024-09-21T03:29:07.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:07 smithi112 ceph-mon[31442]: pgmap v321: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 69 op/s 2024-09-21T03:29:07.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:07 smithi203 ceph-mon[34723]: pgmap v321: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 69 op/s 2024-09-21T03:29:08.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:08 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:08 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:08.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:08 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:29:09.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:09 smithi187 ceph-mon[34603]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 KiB/s wr, 49 op/s 2024-09-21T03:29:09.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:09 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:29:09.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:29:09.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:29:09.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:09 smithi112 ceph-mon[31442]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 KiB/s wr, 49 op/s 2024-09-21T03:29:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:09 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:29:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:29:09.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:29:09.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:09 smithi203 ceph-mon[34723]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 KiB/s wr, 49 op/s 2024-09-21T03:29:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:09 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:29:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:29:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:29:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:29:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:10 smithi112 ceph-mon[31442]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 KiB/s wr, 51 op/s 2024-09-21T03:29:10.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:10 smithi203 ceph-mon[34723]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 KiB/s wr, 51 op/s 2024-09-21T03:29:10.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:10 smithi187 ceph-mon[34603]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 46 KiB/s wr, 51 op/s 2024-09-21T03:29:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:13 smithi187 ceph-mon[34603]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T03:29:13.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:13 smithi112 ceph-mon[31442]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T03:29:13.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:13 smithi203 ceph-mon[34723]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T03:29:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:15 smithi187 ceph-mon[34603]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T03:29:15.713 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:15 smithi203 ceph-mon[34723]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T03:29:15.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:15 smithi112 ceph-mon[31442]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T03:29:16.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:16 smithi112 ceph-mon[31442]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T03:29:16.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:16 smithi203 ceph-mon[34723]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T03:29:16.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:16 smithi187 ceph-mon[34603]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T03:29:19.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:19 smithi187 ceph-mon[34603]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-21T03:29:19.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:19 smithi112 ceph-mon[31442]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-21T03:29:19.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:19 smithi203 ceph-mon[34723]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-21T03:29:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:21 smithi187 ceph-mon[34603]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T03:29:21.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:21 smithi112 ceph-mon[31442]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T03:29:21.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:21 smithi203 ceph-mon[34723]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-21T03:29:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:22 smithi112 ceph-mon[31442]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:29:22.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:22 smithi203 ceph-mon[34723]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:29:22.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:22 smithi187 ceph-mon[34603]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:29:25.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:25 smithi187 ceph-mon[34603]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T03:29:25.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:25 smithi112 ceph-mon[31442]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T03:29:25.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:25 smithi203 ceph-mon[34723]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T03:29:27.269 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:29:27.634 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:27.754 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:27 smithi112 ceph-mon[31442]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:29:27.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:27 smithi203 ceph-mon[34723]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:29:27.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:27 smithi187 ceph-mon[34603]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T03:29:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.160+0000 7ff6a7123640 1 -- 172.21.15.112:0/3051208903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0105300 msgr2=0x7ff6a010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.160+0000 7ff6a7123640 1 --2- 172.21.15.112:0/3051208903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0105300 0x7ff6a010d850 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009900 tx=0x7ff68c02a410 comp rx=0 tx=0).stop 2024-09-21T03:29:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.160+0000 7ff6a7123640 1 -- 172.21.15.112:0/3051208903 shutdown_connections 2024-09-21T03:29:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.160+0000 7ff6a7123640 1 --2- 172.21.15.112:0/3051208903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a010de10 0x7ff6a0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.160+0000 7ff6a7123640 1 --2- 172.21.15.112:0/3051208903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0105300 0x7ff6a010d850 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.160+0000 7ff6a7123640 1 --2- 172.21.15.112:0/3051208903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a01049b0 0x7ff6a0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.160+0000 7ff6a7123640 1 -- 172.21.15.112:0/3051208903 >> 172.21.15.112:0/3051208903 conn(0x7ff6a0079730 msgr2=0x7ff6a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.160+0000 7ff6a7123640 1 -- 172.21.15.112:0/3051208903 shutdown_connections 2024-09-21T03:29:28.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.160+0000 7ff6a7123640 1 -- 172.21.15.112:0/3051208903 wait complete. 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.161+0000 7ff6a7123640 1 Processor -- start 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.161+0000 7ff6a7123640 1 -- start start 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.162+0000 7ff6a7123640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a01049b0 0x7ff6a019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.162+0000 7ff6a7123640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0105300 0x7ff6a019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.162+0000 7ff6a7123640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a010de10 0x7ff6a019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.162+0000 7ff69ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0105300 0x7ff6a019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.162+0000 7ff69ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0105300 0x7ff6a019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50004/0 (socket says 172.21.15.112:50004) 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.162+0000 7ff69ffff640 1 -- 172.21.15.112:0/4228079461 learned_addr learned my addr 172.21.15.112:0/4228079461 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.162+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a01049b0 0x7ff6a019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.162+0000 7ff6a7123640 1 -- 172.21.15.112:0/4228079461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0113ac0 con 0x7ff6a0105300 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.162+0000 7ff6a56c0640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a010de10 0x7ff6a019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.162+0000 7ff6a7123640 1 -- 172.21.15.112:0/4228079461 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0113940 con 0x7ff6a01049b0 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.162+0000 7ff6a7123640 1 -- 172.21.15.112:0/4228079461 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0113c40 con 0x7ff6a010de10 2024-09-21T03:29:28.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff69ffff640 1 -- 172.21.15.112:0/4228079461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a010de10 msgr2=0x7ff6a019fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff69ffff640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a010de10 0x7ff6a019fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff69ffff640 1 -- 172.21.15.112:0/4228079461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a01049b0 msgr2=0x7ff6a019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff69ffff640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a01049b0 0x7ff6a019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff69ffff640 1 -- 172.21.15.112:0/4228079461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a01a01b0 con 0x7ff6a0105300 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a01049b0 0x7ff6a019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff69ffff640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0105300 0x7ff6a019a8a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff68c02a930 tx=0x7ff68c030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff69dffb640 1 -- 172.21.15.112:0/4228079461 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c03f070 con 0x7ff6a0105300 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff6a7123640 1 -- 172.21.15.112:0/4228079461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a01a0440 con 0x7ff6a0105300 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff69dffb640 1 -- 172.21.15.112:0/4228079461 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff68c004460 con 0x7ff6a0105300 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff69dffb640 1 -- 172.21.15.112:0/4228079461 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c035030 con 0x7ff6a0105300 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.163+0000 7ff6a7123640 1 -- 172.21.15.112:0/4228079461 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a01a0970 con 0x7ff6a0105300 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.166+0000 7ff6ad697740 1 Processor -- start 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.166+0000 7ff6ad697740 1 -- start start 2024-09-21T03:29:28.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.166+0000 7ff6ad697740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f721070 0x55a37fac8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.166+0000 7ff6ad697740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6f81b0 0x55a37fac8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.166+0000 7ff6ad697740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 0x55a37fac90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.166+0000 7ff6ad697740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a37fa3ee50 con 0x55a37f700390 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.166+0000 7ff6ad697740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a37fa2e280 con 0x55a37f721070 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.166+0000 7ff6ad697740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a37fa32090 con 0x55a37f6f81b0 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.167+0000 7ff69dffb640 1 -- 172.21.15.112:0/4228079461 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff68c042070 con 0x7ff6a0105300 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.167+0000 7ff69ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 0x55a37fac90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.167+0000 7ff69ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 0x55a37fac90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50016/0 (socket says 172.21.15.112:50016) 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.167+0000 7ff69ffff640 1 -- 172.21.15.112:0/1817463296 learned_addr learned my addr 172.21.15.112:0/1817463296 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.167+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/1817463296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6f81b0 0x55a37fac8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.167+0000 7ff6a56c0640 1 --2- 172.21.15.112:0/1817463296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f721070 0x55a37fac8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.167+0000 7ff6a4ebf640 1 -- 172.21.15.112:0/1817463296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f721070 msgr2=0x55a37fac8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:28.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.167+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/1817463296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f721070 0x55a37fac8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.167+0000 7ff6a4ebf640 1 -- 172.21.15.112:0/1817463296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 msgr2=0x55a37fac90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.167+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/1817463296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 0x55a37fac90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.167+0000 7ff6a4ebf640 1 -- 172.21.15.112:0/1817463296 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a37f78c650 con 0x55a37f6f81b0 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.168+0000 7ff69dffb640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6900bab40 0x7ff6900bd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.168+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/1817463296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6f81b0 0x55a37fac8b90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff694008ba0 tx=0x7ff69400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3889de1be4834563 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.168+0000 7ff6a56c0640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6900bab40 0x7ff6900bd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.168+0000 7ff69dffb640 1 -- 172.21.15.112:0/4228079461 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff68c108180 con 0x7ff6a0105300 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.168+0000 7ff68affd640 1 -- 172.21.15.112:0/1817463296 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694014070 con 0x55a37f6f81b0 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.168+0000 7ff68affd640 1 -- 172.21.15.112:0/1817463296 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6940054d0 con 0x55a37f6f81b0 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.169+0000 7ff68affd640 1 -- 172.21.15.112:0/1817463296 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694004bc0 con 0x55a37f6f81b0 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.169+0000 7ff6ad697740 1 -- 172.21.15.112:0/1817463296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6f81b0 msgr2=0x55a37fac8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.169+0000 7ff6ad697740 1 --2- 172.21.15.112:0/1817463296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6f81b0 0x55a37fac8b90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff694008ba0 tx=0x7ff69400e720 comp rx=0 tx=0).stop 2024-09-21T03:29:28.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.169+0000 7ff6a56c0640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6900bab40 0x7ff6900bd050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff698005fd0 tx=0x7ff698009600 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.169+0000 7ff6ad697740 1 -- 172.21.15.112:0/1817463296 shutdown_connections 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.169+0000 7ff6ad697740 1 --2- 172.21.15.112:0/1817463296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 0x55a37fac90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.169+0000 7ff6ad697740 1 --2- 172.21.15.112:0/1817463296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6f81b0 0x55a37fac8b90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.169+0000 7ff6ad697740 1 --2- 172.21.15.112:0/1817463296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f721070 0x55a37fac8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.169+0000 7ff6ad697740 1 -- 172.21.15.112:0/1817463296 >> 172.21.15.112:0/1817463296 conn(0x55a37f715460 msgr2=0x55a37fac6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.169+0000 7ff6ad697740 1 -- 172.21.15.112:0/1817463296 shutdown_connections 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.169+0000 7ff6ad697740 1 -- 172.21.15.112:0/1817463296 wait complete. 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.170+0000 7ff6ad697740 1 -- start start 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.170+0000 7ff6ad697740 1 Processor -- start 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.170+0000 7ff6ad697740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fad3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.170+0000 7ff6ad697740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f700390 0x55a37fad4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6ad697740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f721070 0x55a37fad46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6ad697740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a37f88f700 con 0x55a37f6f81b0 2024-09-21T03:29:28.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6ad697740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a37f9c9110 con 0x55a37f700390 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6ad697740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a37fad6ed0 con 0x55a37f721070 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6a4ebf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fad3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6ad697740 10 client.0 _ll_register_callbacks cb 0x55a37f9e2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6a4ebf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fad3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50028/0 (socket says 172.21.15.112:50028) 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6a4ebf640 1 -- 172.21.15.112:0/3808030825 learned_addr learned my addr 172.21.15.112:0/3808030825 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6a56c0640 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f721070 0x55a37fad46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff69ffff640 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f700390 0x55a37fad4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6a4ebf640 1 -- 172.21.15.112:0/3808030825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f721070 msgr2=0x55a37fad46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f721070 0x55a37fad46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6a4ebf640 1 -- 172.21.15.112:0/3808030825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f700390 msgr2=0x55a37fad4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f700390 0x55a37fad4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6a4ebf640 1 -- 172.21.15.112:0/3808030825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a37f9f6c40 con 0x55a37f6f81b0 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fad3c50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff694005e30 tx=0x7ff694004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33f825770118ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:28.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.171+0000 7ff67ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.172+0000 7ff67ffff640 1 -- 172.21.15.112:0/3808030825 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694014040 con 0x55a37f6f81b0 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.172+0000 7ff67ffff640 1 -- 172.21.15.112:0/3808030825 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff6940050a0 con 0x55a37f6f81b0 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.172+0000 7ff6ad697740 1 -- 172.21.15.112:0/3808030825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6940054d0 con 0x55a37f6f81b0 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.172+0000 7ff67ffff640 1 -- 172.21.15.112:0/3808030825 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6940052a0 con 0x55a37f6f81b0 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.172+0000 7ff6ad697740 10 client.15519 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.172+0000 7ff6ad697740 1 -- 172.21.15.112:0/3808030825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a37fae2b30 con 0x55a37f6f81b0 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.172+0000 7ff67ffff640 1 -- 172.21.15.112:0/3808030825 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6940054d0 con 0x55a37f6f81b0 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.172+0000 7ff67ffff640 1 -- 172.21.15.112:0/3808030825 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff694024420 con 0x55a37f6f81b0 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.172+0000 7ff6ad697740 10 client.15519 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.172+0000 7ff6ad697740 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.173+0000 7ff6ad697740 20 client.15519 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.173+0000 7ff6ad697740 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a37f6df9f0 0x55a37fae5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.173+0000 7ff6ad697740 4 client.15519 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.173+0000 7ff6ad697740 1 -- 172.21.15.112:0/3808030825 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a37fa40da0 con 0x55a37f6df9f0 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.173+0000 7ff69ffff640 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a37f6df9f0 0x55a37fae5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.174+0000 7ff69ffff640 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a37f6df9f0 0x55a37fae5e90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bde800629d51617 server_cookie=91b32650ca66e769 in_seq=0 out_seq=0 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.174+0000 7ff67ffff640 10 client.15519 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.174+0000 7ff67ffff640 1 -- 172.21.15.112:0/3808030825 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a37fa40da0 con 0x55a37f6df9f0 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.174+0000 7ff67ffff640 10 client.15519 handle_command_reply: tid=0 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.174+0000 7ff6ad697740 1 client.15519 shutdown 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff6ad697740 1 -- 172.21.15.112:0/3808030825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 msgr2=0x55a37fad3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff6ad697740 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fad3c50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff694005e30 tx=0x7ff694004f80 comp rx=0 tx=0).stop 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff6ad697740 1 -- 172.21.15.112:0/3808030825 shutdown_connections 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff6ad697740 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a37f6df9f0 0x55a37fae5e90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff67ffff640 0 client.15519 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff6ad697740 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f721070 0x55a37fad46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff6ad697740 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f700390 0x55a37fad4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff6ad697740 1 --2- 172.21.15.112:0/3808030825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fad3c50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff6ad697740 1 -- 172.21.15.112:0/3808030825 >> 172.21.15.112:0/3808030825 conn(0x55a37f715460 msgr2=0x55a37faca040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff6ad697740 1 -- 172.21.15.112:0/3808030825 shutdown_connections 2024-09-21T03:29:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff6ad697740 1 -- 172.21.15.112:0/3808030825 wait complete. 2024-09-21T03:29:28.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.175+0000 7ff6ad697740 20 client.15519 trim_cache size 0 max 16384 2024-09-21T03:29:28.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.203+0000 7ff6ad697740 1 Processor -- start 2024-09-21T03:29:28.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.203+0000 7ff6ad697740 1 -- start start 2024-09-21T03:29:28.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.203+0000 7ff6ad697740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f6df9f0 0x55a37fad5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.203+0000 7ff6ad697740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6f81b0 0x55a37fad0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.203+0000 7ff6ad697740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 0x55a37fad1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.203+0000 7ff6ad697740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01afce0 con 0x55a37f700390 2024-09-21T03:29:28.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.203+0000 7ff6ad697740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01ad5b0 con 0x55a37f6df9f0 2024-09-21T03:29:28.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.203+0000 7ff6ad697740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01ab420 con 0x55a37f6f81b0 2024-09-21T03:29:28.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff6a56c0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 0x55a37fad1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff6a56c0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 0x55a37fad1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50044/0 (socket says 172.21.15.112:50044) 2024-09-21T03:29:28.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff6a56c0640 1 -- 172.21.15.112:0/875959903 learned_addr learned my addr 172.21.15.112:0/875959903 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:28.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/875959903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f6df9f0 0x55a37fad5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff69ffff640 1 --2- 172.21.15.112:0/875959903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6f81b0 0x55a37fad0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff6a56c0640 1 -- 172.21.15.112:0/875959903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6f81b0 msgr2=0x55a37fad0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:28.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff6a56c0640 1 --2- 172.21.15.112:0/875959903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6f81b0 0x55a37fad0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff6a56c0640 1 -- 172.21.15.112:0/875959903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f6df9f0 msgr2=0x55a37fad5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:28.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff6a56c0640 1 --2- 172.21.15.112:0/875959903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f6df9f0 0x55a37fad5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff6a56c0640 1 -- 172.21.15.112:0/875959903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a37fae62f0 con 0x55a37f700390 2024-09-21T03:29:28.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/875959903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f6df9f0 0x55a37fad5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:28.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff6a56c0640 1 --2- 172.21.15.112:0/875959903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 0x55a37fad1380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff6980117f0 tx=0x7ff698016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de7146b843f9bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:28.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.204+0000 7ff68a7fc640 1 -- 172.21.15.112:0/875959903 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698011de0 con 0x55a37f700390 2024-09-21T03:29:28.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.205+0000 7ff68a7fc640 1 -- 172.21.15.112:0/875959903 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff698003e20 con 0x55a37f700390 2024-09-21T03:29:28.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.205+0000 7ff68a7fc640 1 -- 172.21.15.112:0/875959903 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69802a6d0 con 0x55a37f700390 2024-09-21T03:29:28.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.205+0000 7ff6ad697740 1 -- 172.21.15.112:0/875959903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 msgr2=0x55a37fad1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.206+0000 7ff6ad697740 1 --2- 172.21.15.112:0/875959903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 0x55a37fad1380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff6980117f0 tx=0x7ff698016000 comp rx=0 tx=0).stop 2024-09-21T03:29:28.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.206+0000 7ff6ad697740 1 -- 172.21.15.112:0/875959903 shutdown_connections 2024-09-21T03:29:28.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.206+0000 7ff6ad697740 1 --2- 172.21.15.112:0/875959903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f700390 0x55a37fad1380 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.206+0000 7ff6ad697740 1 --2- 172.21.15.112:0/875959903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6f81b0 0x55a37fad0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.206+0000 7ff6ad697740 1 --2- 172.21.15.112:0/875959903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f6df9f0 0x55a37fad5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.206+0000 7ff6ad697740 1 -- 172.21.15.112:0/875959903 >> 172.21.15.112:0/875959903 conn(0x55a37f715460 msgr2=0x55a37faca040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:28.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.206+0000 7ff6ad697740 1 -- 172.21.15.112:0/875959903 shutdown_connections 2024-09-21T03:29:28.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.206+0000 7ff6ad697740 1 -- 172.21.15.112:0/875959903 wait complete. 2024-09-21T03:29:28.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.206+0000 7ff6ad697740 1 -- start start 2024-09-21T03:29:28.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.206+0000 7ff6ad697740 1 Processor -- start 2024-09-21T03:29:28.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.207+0000 7ff6ad697740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6df9f0 0x55a37fada7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.207+0000 7ff6ad697740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fadad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.207+0000 7ff69ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fadad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.207+0000 7ff69ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fadad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50046/0 (socket says 172.21.15.112:50046) 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.207+0000 7ff69ffff640 1 -- 172.21.15.112:0/877329321 learned_addr learned my addr 172.21.15.112:0/877329321 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.207+0000 7ff6ad697740 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f700390 0x55a37fb4f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.207+0000 7ff6ad697740 1 -- 172.21.15.112:0/877329321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01aba70 con 0x55a37f6f81b0 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.207+0000 7ff6ad697740 1 -- 172.21.15.112:0/877329321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01acee0 con 0x55a37f700390 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.207+0000 7ff6ad697740 1 -- 172.21.15.112:0/877329321 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01ac140 con 0x55a37f6df9f0 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.207+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6df9f0 0x55a37fada7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.207+0000 7ff6ad697740 10 client.0 _ll_register_callbacks cb 0x7ff68c10bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.208+0000 7ff6a56c0640 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f700390 0x55a37fb4f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.208+0000 7ff6a4ebf640 1 -- 172.21.15.112:0/877329321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f700390 msgr2=0x55a37fb4f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:28.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.208+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f700390 0x55a37fb4f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.208+0000 7ff6a4ebf640 1 -- 172.21.15.112:0/877329321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 msgr2=0x55a37fadad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.208+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fadad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.208+0000 7ff6a4ebf640 1 -- 172.21.15.112:0/877329321 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a37fb4fc70 con 0x55a37f6df9f0 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.208+0000 7ff69ffff640 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fadad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.208+0000 7ff6a4ebf640 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6df9f0 0x55a37fada7d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff694024810 tx=0x7ff694008310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb108989292c3d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.208+0000 7ff67ffff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.209+0000 7ff67ffff640 1 -- 172.21.15.112:0/877329321 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694031070 con 0x55a37f6df9f0 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.209+0000 7ff6ad697740 1 -- 172.21.15.112:0/877329321 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff698003e20 con 0x55a37f6df9f0 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.209+0000 7ff67ffff640 1 -- 172.21.15.112:0/877329321 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff694005440 con 0x55a37f6df9f0 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.209+0000 7ff67ffff640 1 -- 172.21.15.112:0/877329321 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff69403b050 con 0x55a37f6df9f0 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.209+0000 7ff6ad697740 10 client.25147 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:28.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.209+0000 7ff6ad697740 1 -- 172.21.15.112:0/877329321 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a37fb50e20 con 0x55a37f6df9f0 2024-09-21T03:29:28.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.210+0000 7ff67ffff640 1 -- 172.21.15.112:0/877329321 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff698003e20 con 0x55a37f6df9f0 2024-09-21T03:29:28.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.210+0000 7ff67ffff640 1 -- 172.21.15.112:0/877329321 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff6940035f0 con 0x55a37f6df9f0 2024-09-21T03:29:28.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.210+0000 7ff6ad697740 10 client.25147 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.210+0000 7ff6ad697740 10 client.25147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.210+0000 7ff6ad697740 20 client.25147 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.210+0000 7ff6ad697740 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a37f721070 0x55a37fb53540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.210+0000 7ff6ad697740 4 client.25147 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.210+0000 7ff6ad697740 1 -- 172.21.15.112:0/877329321 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a37fa3c850 con 0x55a37f721070 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.210+0000 7ff69ffff640 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a37f721070 0x55a37fb53540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.211+0000 7ff67ffff640 10 client.25147 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.211+0000 7ff69ffff640 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a37f721070 0x55a37fb53540 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9869ef68445e8d7 server_cookie=a31b333c111737d7 in_seq=0 out_seq=0 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.211+0000 7ff67ffff640 1 -- 172.21.15.112:0/877329321 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55a37fa3c850 con 0x55a37f721070 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.211+0000 7ff67ffff640 10 client.25147 handle_command_reply: tid=0 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.211+0000 7ff6ad697740 1 client.25147 shutdown 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff6ad697740 1 -- 172.21.15.112:0/877329321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6df9f0 msgr2=0x55a37fada7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff6ad697740 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6df9f0 0x55a37fada7d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff694024810 tx=0x7ff694008310 comp rx=0 tx=0).stop 2024-09-21T03:29:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff6ad697740 1 -- 172.21.15.112:0/877329321 shutdown_connections 2024-09-21T03:29:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff6ad697740 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a37f721070 0x55a37fb53540 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff6ad697740 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a37f700390 0x55a37fb4f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff6ad697740 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a37f6f81b0 0x55a37fadad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff6ad697740 1 --2- 172.21.15.112:0/877329321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a37f6df9f0 0x55a37fada7d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff6ad697740 1 -- 172.21.15.112:0/877329321 >> 172.21.15.112:0/877329321 conn(0x55a37f715460 msgr2=0x55a37fac9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff67ffff640 0 client.25147 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff6ad697740 1 -- 172.21.15.112:0/877329321 shutdown_connections 2024-09-21T03:29:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff6ad697740 1 -- 172.21.15.112:0/877329321 wait complete. 2024-09-21T03:29:28.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.212+0000 7ff6ad697740 20 client.25147 trim_cache size 0 max 16384 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:29:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:29:28.218 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:29:28.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 -- 172.21.15.112:0/4228079461 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6900bab40 msgr2=0x7ff6900bd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6900bab40 0x7ff6900bd050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff698005fd0 tx=0x7ff698009600 comp rx=0 tx=0).stop 2024-09-21T03:29:28.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 -- 172.21.15.112:0/4228079461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0105300 msgr2=0x7ff6a019a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:28.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0105300 0x7ff6a019a8a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff68c02a930 tx=0x7ff68c030650 comp rx=0 tx=0).stop 2024-09-21T03:29:28.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 -- 172.21.15.112:0/4228079461 shutdown_connections 2024-09-21T03:29:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6900bab40 0x7ff6900bd050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6a010de10 0x7ff6a019fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6a0105300 0x7ff6a019a8a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 --2- 172.21.15.112:0/4228079461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a01049b0 0x7ff6a019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 -- 172.21.15.112:0/4228079461 >> 172.21.15.112:0/4228079461 conn(0x7ff6a0079730 msgr2=0x7ff6a01057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 -- 172.21.15.112:0/4228079461 shutdown_connections 2024-09-21T03:29:28.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:28.213+0000 7ff6a7123640 1 -- 172.21.15.112:0/4228079461 wait complete. 2024-09-21T03:29:28.757 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:28 smithi112 ceph-mon[31442]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:29:28.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:28 smithi203 ceph-mon[34723]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:29:28.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:28 smithi187 ceph-mon[34603]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T03:29:29.010 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:29:29.011 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:29:29.290 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.793+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/2886706475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d0107b30 msgr2=0x7fa3d0107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.793+0000 7fa3d6dbe640 1 --2- 172.21.15.112:0/2886706475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d0107b30 0x7fa3d0107f40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009900 tx=0x7fa3bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:29:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.794+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/2886706475 shutdown_connections 2024-09-21T03:29:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.794+0000 7fa3d6dbe640 1 --2- 172.21.15.112:0/2886706475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3d0108480 0x7fa3d010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.794+0000 7fa3d6dbe640 1 --2- 172.21.15.112:0/2886706475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d0107b30 0x7fa3d0107f40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.794+0000 7fa3d6dbe640 1 --2- 172.21.15.112:0/2886706475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d01021f0 0x7fa3d0102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.794+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/2886706475 >> 172.21.15.112:0/2886706475 conn(0x7fa3d0077c60 msgr2=0x7fa3d00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.794+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/2886706475 shutdown_connections 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.794+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/2886706475 wait complete. 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.795+0000 7fa3d6dbe640 1 Processor -- start 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.795+0000 7fa3d6dbe640 1 -- start start 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.795+0000 7fa3d6dbe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3d01021f0 0x7fa3d019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d6dbe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d0107b30 0x7fa3d019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d6dbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d0108480 0x7fa3d01a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d6dbe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0112160 con 0x7fa3d0108480 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d6dbe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0111fe0 con 0x7fa3d01021f0 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d6dbe640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01122e0 con 0x7fa3d0107b30 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d4b5a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3d01021f0 0x7fa3d019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3cffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d0107b30 0x7fa3d019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d535b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d0108480 0x7fa3d01a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3cffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d0107b30 0x7fa3d019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:51508/0 (socket says 172.21.15.112:51508) 2024-09-21T03:29:29.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3cffff640 1 -- 172.21.15.112:0/123694563 learned_addr learned my addr 172.21.15.112:0/123694563 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:29.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d535b640 1 -- 172.21.15.112:0/123694563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d0107b30 msgr2=0x7fa3d019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d535b640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d0107b30 0x7fa3d019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d535b640 1 -- 172.21.15.112:0/123694563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3d01021f0 msgr2=0x7fa3d019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:29.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d535b640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3d01021f0 0x7fa3d019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:29.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.796+0000 7fa3d535b640 1 -- 172.21.15.112:0/123694563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d01a2640 con 0x7fa3d0108480 2024-09-21T03:29:29.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.797+0000 7fa3d535b640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d0108480 0x7fa3d01a1f10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009fd0 tx=0x7fa3c800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:29.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.797+0000 7fa3cdffb640 1 -- 172.21.15.112:0/123694563 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800ce20 con 0x7fa3d0108480 2024-09-21T03:29:29.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.797+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/123694563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d01a2930 con 0x7fa3d0108480 2024-09-21T03:29:29.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.797+0000 7fa3cdffb640 1 -- 172.21.15.112:0/123694563 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c8005020 con 0x7fa3d0108480 2024-09-21T03:29:29.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.797+0000 7fa3cdffb640 1 -- 172.21.15.112:0/123694563 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8010690 con 0x7fa3d0108480 2024-09-21T03:29:29.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.797+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/123694563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d01a9090 con 0x7fa3d0108480 2024-09-21T03:29:29.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.799+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/123694563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d0103470 con 0x7fa3d0108480 2024-09-21T03:29:29.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.800+0000 7fa3cdffb640 1 -- 172.21.15.112:0/123694563 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa3c80054d0 con 0x7fa3d0108480 2024-09-21T03:29:29.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.801+0000 7fa3cdffb640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa3c00b6880 0x7fa3c00b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:29.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.802+0000 7fa3cdffb640 1 -- 172.21.15.112:0/123694563 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa3c80e7a10 con 0x7fa3d0108480 2024-09-21T03:29:29.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.802+0000 7fa3d4b5a640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa3c00b6880 0x7fa3c00b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:29.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.802+0000 7fa3d4b5a640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa3c00b6880 0x7fa3c00b8d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005fd0 tx=0x7fa3c4005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:29.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:29.806+0000 7fa3cdffb640 1 -- 172.21.15.112:0/123694563 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3c80adf60 con 0x7fa3d0108480 2024-09-21T03:29:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.160+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/123694563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3d01a92c0 con 0x7fa3d0108480 2024-09-21T03:29:30.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.160+0000 7fa3cdffb640 1 -- 172.21.15.112:0/123694563 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11929 (secure 0 0 0) 0x7fa3c80b2340 con 0x7fa3d0108480 2024-09-21T03:29:30.162 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:29:30.162 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"btime":"2024-09-21T03:28:46:418112+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:28:46.048810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:29:30.163 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-09-21T03:29:30.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.163+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/123694563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa3c00b6880 msgr2=0x7fa3c00b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.164+0000 7fa3d6dbe640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa3c00b6880 0x7fa3c00b8d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005fd0 tx=0x7fa3c4005850 comp rx=0 tx=0).stop 2024-09-21T03:29:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.164+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/123694563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d0108480 msgr2=0x7fa3d01a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.164+0000 7fa3d6dbe640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d0108480 0x7fa3d01a1f10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009fd0 tx=0x7fa3c800ef10 comp rx=0 tx=0).stop 2024-09-21T03:29:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.164+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/123694563 shutdown_connections 2024-09-21T03:29:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.164+0000 7fa3d6dbe640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa3c00b6880 0x7fa3c00b8d90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.164+0000 7fa3d6dbe640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3d0108480 0x7fa3d01a1f10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.164+0000 7fa3d6dbe640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d0107b30 0x7fa3d019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.164+0000 7fa3d6dbe640 1 --2- 172.21.15.112:0/123694563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3d01021f0 0x7fa3d019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.164+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/123694563 >> 172.21.15.112:0/123694563 conn(0x7fa3d0077c60 msgr2=0x7fa3d00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.164+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/123694563 shutdown_connections 2024-09-21T03:29:30.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:30.164+0000 7fa3d6dbe640 1 -- 172.21.15.112:0/123694563 wait complete. 2024-09-21T03:29:30.486 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:30 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/123694563' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:30.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:30 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/123694563' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:30 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/123694563' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:30.778 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:29:31.060 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:31.523 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:31 smithi112 ceph-mon[31442]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T03:29:31.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.546+0000 7fb7567d2640 1 -- 172.21.15.112:0/1907713026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb75010ddf0 msgr2=0x7fb750110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.546+0000 7fb7567d2640 1 --2- 172.21.15.112:0/1907713026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb75010ddf0 0x7fb750110300 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb748009ca0 tx=0x7fb74802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.546+0000 7fb7567d2640 1 -- 172.21.15.112:0/1907713026 shutdown_connections 2024-09-21T03:29:31.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.546+0000 7fb7567d2640 1 --2- 172.21.15.112:0/1907713026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb75010ddf0 0x7fb750110300 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.546+0000 7fb7567d2640 1 --2- 172.21.15.112:0/1907713026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb750105310 0x7fb75010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.546+0000 7fb7567d2640 1 --2- 172.21.15.112:0/1907713026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7501049c0 0x7fb750104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.546+0000 7fb7567d2640 1 -- 172.21.15.112:0/1907713026 >> 172.21.15.112:0/1907713026 conn(0x7fb750079680 msgr2=0x7fb750079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.546+0000 7fb7567d2640 1 -- 172.21.15.112:0/1907713026 shutdown_connections 2024-09-21T03:29:31.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.547+0000 7fb7567d2640 1 -- 172.21.15.112:0/1907713026 wait complete. 2024-09-21T03:29:31.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.547+0000 7fb7567d2640 1 Processor -- start 2024-09-21T03:29:31.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.547+0000 7fb7567d2640 1 -- start start 2024-09-21T03:29:31.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb7567d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7501049c0 0x7fb750104110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb7567d2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb750105310 0x7fb750102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb7567d2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb75010ddf0 0x7fb750102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb7567d2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb750113a90 con 0x7fb7501049c0 2024-09-21T03:29:31.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb7567d2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb750113910 con 0x7fb750105310 2024-09-21T03:29:31.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb7567d2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb750113c10 con 0x7fb75010ddf0 2024-09-21T03:29:31.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb74ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7501049c0 0x7fb750104110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb74ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7501049c0 0x7fb750104110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50084/0 (socket says 172.21.15.112:50084) 2024-09-21T03:29:31.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb74ffff640 1 -- 172.21.15.112:0/858082570 learned_addr learned my addr 172.21.15.112:0/858082570 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:31.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb74f7fe640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb750105310 0x7fb750102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb74ffff640 1 -- 172.21.15.112:0/858082570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb75010ddf0 msgr2=0x7fb750102c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:29:31.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb74ffff640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb75010ddf0 0x7fb750102c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb74ffff640 1 -- 172.21.15.112:0/858082570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb750105310 msgr2=0x7fb750102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb74ffff640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb750105310 0x7fb750102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.548+0000 7fb74ffff640 1 -- 172.21.15.112:0/858082570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb750103460 con 0x7fb7501049c0 2024-09-21T03:29:31.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.549+0000 7fb74ffff640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7501049c0 0x7fb750104110 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb74000d950 tx=0x7fb74000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.549+0000 7fb74d7fa640 1 -- 172.21.15.112:0/858082570 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740017030 con 0x7fb7501049c0 2024-09-21T03:29:31.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.549+0000 7fb74d7fa640 1 -- 172.21.15.112:0/858082570 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb740004020 con 0x7fb7501049c0 2024-09-21T03:29:31.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.549+0000 7fb7567d2640 1 -- 172.21.15.112:0/858082570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7501acda0 con 0x7fb7501049c0 2024-09-21T03:29:31.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.549+0000 7fb74d7fa640 1 -- 172.21.15.112:0/858082570 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7400047a0 con 0x7fb7501049c0 2024-09-21T03:29:31.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.549+0000 7fb7567d2640 1 -- 172.21.15.112:0/858082570 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7501ad200 con 0x7fb7501049c0 2024-09-21T03:29:31.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.552+0000 7fb74d7fa640 1 -- 172.21.15.112:0/858082570 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb740009a30 con 0x7fb7501049c0 2024-09-21T03:29:31.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.553+0000 7fb75cd46740 1 Processor -- start 2024-09-21T03:29:31.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.553+0000 7fb75cd46740 1 -- start start 2024-09-21T03:29:31.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.553+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be330bf070 0x55be33466290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.553+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be330961b0 0x55be334667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.553+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3309e390 0x55be33466d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.553+0000 7fb75cd46740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be333dcea0 con 0x55be3309e390 2024-09-21T03:29:31.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.553+0000 7fb75cd46740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55be333cc2d0 con 0x55be330961b0 2024-09-21T03:29:31.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.553+0000 7fb75cd46740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55be333d00e0 con 0x55be330bf070 2024-09-21T03:29:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3309e390 0x55be33466d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb754d6f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be330bf070 0x55be33466290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3309e390 0x55be33466d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50092/0 (socket says 172.21.15.112:50092) 2024-09-21T03:29:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb754d6f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be330bf070 0x55be33466290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:51530/0 (socket says 172.21.15.112:51530) 2024-09-21T03:29:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74f7fe640 1 -- 172.21.15.112:0/3076174214 learned_addr learned my addr 172.21.15.112:0/3076174214 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:31.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74ffff640 1 --2- 172.21.15.112:0/3076174214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be330961b0 0x55be334667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74f7fe640 1 -- 172.21.15.112:0/3076174214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be330bf070 msgr2=0x55be33466290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74f7fe640 1 --2- 172.21.15.112:0/3076174214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be330bf070 0x55be33466290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74f7fe640 1 -- 172.21.15.112:0/3076174214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be330961b0 msgr2=0x55be334667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74f7fe640 1 --2- 172.21.15.112:0/3076174214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be330961b0 0x55be334667d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74f7fe640 1 -- 172.21.15.112:0/3076174214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3312a650 con 0x55be3309e390 2024-09-21T03:29:31.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74d7fa640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7380b6700 0x7fb7380b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74f7fe640 1 --2- 172.21.15.112:0/3076174214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3309e390 0x55be33466d10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb74400bb90 tx=0x7fb74400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74770d1a9621a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb74d7fa640 1 -- 172.21.15.112:0/858082570 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fb7400e7240 con 0x7fb7501049c0 2024-09-21T03:29:31.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.554+0000 7fb7367fc640 1 -- 172.21.15.112:0/3076174214 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744015070 con 0x55be3309e390 2024-09-21T03:29:31.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.555+0000 7fb7367fc640 1 -- 172.21.15.112:0/3076174214 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7440054d0 con 0x55be3309e390 2024-09-21T03:29:31.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.555+0000 7fb7367fc640 1 -- 172.21.15.112:0/3076174214 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744011440 con 0x55be3309e390 2024-09-21T03:29:31.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.555+0000 7fb74ffff640 1 --2- 172.21.15.112:0/3076174214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be330961b0 0x55be334667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:31.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.556+0000 7fb754d6f640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7380b6700 0x7fb7380b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.556+0000 7fb75cd46740 1 -- 172.21.15.112:0/3076174214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3309e390 msgr2=0x55be33466d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.556+0000 7fb75cd46740 1 --2- 172.21.15.112:0/3076174214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3309e390 0x55be33466d10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb74400bb90 tx=0x7fb74400f730 comp rx=0 tx=0).stop 2024-09-21T03:29:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.556+0000 7fb75cd46740 1 -- 172.21.15.112:0/3076174214 shutdown_connections 2024-09-21T03:29:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.556+0000 7fb75cd46740 1 --2- 172.21.15.112:0/3076174214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be3309e390 0x55be33466d10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.556+0000 7fb75cd46740 1 --2- 172.21.15.112:0/3076174214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be330961b0 0x55be334667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.556+0000 7fb75cd46740 1 --2- 172.21.15.112:0/3076174214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be330bf070 0x55be33466290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.556+0000 7fb75cd46740 1 -- 172.21.15.112:0/3076174214 >> 172.21.15.112:0/3076174214 conn(0x55be330b3460 msgr2=0x55be33464180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.556+0000 7fb754d6f640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7380b6700 0x7fb7380b8c10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb748006010 tx=0x7fb748009360 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.556+0000 7fb75cd46740 1 -- 172.21.15.112:0/3076174214 shutdown_connections 2024-09-21T03:29:31.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.556+0000 7fb75cd46740 1 -- 172.21.15.112:0/3076174214 wait complete. 2024-09-21T03:29:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.557+0000 7fb75cd46740 1 -- start start 2024-09-21T03:29:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.557+0000 7fb75cd46740 1 Processor -- start 2024-09-21T03:29:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.558+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330961b0 0x55be33471920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.558+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be3309e390 0x55be33471e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.558+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be330bf070 0x55be3347dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.558+0000 7fb74ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330961b0 0x55be33471920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.558+0000 7fb75cd46740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be3322d700 con 0x55be330961b0 2024-09-21T03:29:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.558+0000 7fb74ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330961b0 0x55be33471920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50096/0 (socket says 172.21.15.112:50096) 2024-09-21T03:29:31.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.558+0000 7fb74ffff640 1 -- 172.21.15.112:0/1114648347 learned_addr learned my addr 172.21.15.112:0/1114648347 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:31.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.558+0000 7fb75cd46740 1 -- 172.21.15.112:0/1114648347 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55be33367160 con 0x55be330bf070 2024-09-21T03:29:31.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.558+0000 7fb75cd46740 1 -- 172.21.15.112:0/1114648347 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55be33474a70 con 0x55be3309e390 2024-09-21T03:29:31.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.559+0000 7fb74f7fe640 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be3309e390 0x55be33471e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.559+0000 7fb754d6f640 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be330bf070 0x55be3347dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.559+0000 7fb75cd46740 10 client.0 _ll_register_callbacks cb 0x55be333809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.559+0000 7fb74ffff640 1 -- 172.21.15.112:0/1114648347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be3309e390 msgr2=0x55be33471e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.559+0000 7fb74ffff640 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be3309e390 0x55be33471e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.559+0000 7fb74ffff640 1 -- 172.21.15.112:0/1114648347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be330bf070 msgr2=0x55be3347dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.559+0000 7fb74ffff640 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be330bf070 0x55be3347dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.559+0000 7fb74ffff640 1 -- 172.21.15.112:0/1114648347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be33394c90 con 0x55be330961b0 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.559+0000 7fb74ffff640 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330961b0 0x55be33471920 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb7400aa300 tx=0x7fb740009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36828a040e5cac5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.559+0000 7fb72f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.560+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1114648347 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740092e60 con 0x55be330961b0 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.560+0000 7fb75cd46740 1 -- 172.21.15.112:0/1114648347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7440054d0 con 0x55be330961b0 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.560+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1114648347 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fb7400a9300 con 0x55be330961b0 2024-09-21T03:29:31.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.560+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1114648347 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb7400ad790 con 0x55be330961b0 2024-09-21T03:29:31.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.560+0000 7fb75cd46740 10 client.15543 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:31.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.560+0000 7fb75cd46740 1 -- 172.21.15.112:0/1114648347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be33480750 con 0x55be330961b0 2024-09-21T03:29:31.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.560+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1114648347 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7440054d0 con 0x55be330961b0 2024-09-21T03:29:31.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.560+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1114648347 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb7400e9110 con 0x55be330961b0 2024-09-21T03:29:31.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.560+0000 7fb75cd46740 10 client.15543 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:31.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.561+0000 7fb75cd46740 10 client.15543 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:29:31.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.561+0000 7fb75cd46740 20 client.15543 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:31.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.561+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55be3307d9f0 0x55be33484150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.561+0000 7fb75cd46740 4 client.15543 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:31.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.561+0000 7fb75cd46740 1 -- 172.21.15.112:0/1114648347 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be333dedf0 con 0x55be3307d9f0 2024-09-21T03:29:31.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.561+0000 7fb74f7fe640 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55be3307d9f0 0x55be33484150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.562+0000 7fb72f7fe640 10 client.15543 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.562+0000 7fb74f7fe640 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55be3307d9f0 0x55be33484150 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8a45884f8d07008 server_cookie=8c9bca28dd7dc6a0 in_seq=0 out_seq=0 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.562+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1114648347 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55be333dedf0 con 0x55be3307d9f0 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.562+0000 7fb72f7fe640 10 client.15543 handle_command_reply: tid=0 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.562+0000 7fb75cd46740 1 client.15543 shutdown 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.563+0000 7fb75cd46740 1 -- 172.21.15.112:0/1114648347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330961b0 msgr2=0x55be33471920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.563+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330961b0 0x55be33471920 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb7400aa300 tx=0x7fb740009f50 comp rx=0 tx=0).stop 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.563+0000 7fb75cd46740 1 -- 172.21.15.112:0/1114648347 shutdown_connections 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.563+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55be3307d9f0 0x55be33484150 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.563+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be330bf070 0x55be3347dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.563+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be3309e390 0x55be33471e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.563+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1114648347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330961b0 0x55be33471920 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.563+0000 7fb75cd46740 1 -- 172.21.15.112:0/1114648347 >> 172.21.15.112:0/1114648347 conn(0x55be330b3460 msgr2=0x55be334672c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.563+0000 7fb72f7fe640 0 client.15543 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:31.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.563+0000 7fb75cd46740 1 -- 172.21.15.112:0/1114648347 shutdown_connections 2024-09-21T03:29:31.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.563+0000 7fb75cd46740 1 -- 172.21.15.112:0/1114648347 wait complete. 2024-09-21T03:29:31.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.564+0000 7fb75cd46740 20 client.15543 trim_cache size 0 max 16384 2024-09-21T03:29:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.589+0000 7fb75cd46740 1 Processor -- start 2024-09-21T03:29:31.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.589+0000 7fb75cd46740 1 -- start start 2024-09-21T03:29:31.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.589+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 0x55be33471de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.589+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be330bf070 0x55be3347dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.589+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be33052da0 0x55be33463aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.589+0000 7fb75cd46740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501b8440 con 0x55be33052da0 2024-09-21T03:29:31.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.589+0000 7fb75cd46740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501b5d10 con 0x55be3309e390 2024-09-21T03:29:31.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.589+0000 7fb75cd46740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501b3b60 con 0x55be330bf070 2024-09-21T03:29:31.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.589+0000 7fb754d6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be33052da0 0x55be33463aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.590+0000 7fb754d6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be33052da0 0x55be33463aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50112/0 (socket says 172.21.15.112:50112) 2024-09-21T03:29:31.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.590+0000 7fb754d6f640 1 -- 172.21.15.112:0/3941629502 learned_addr learned my addr 172.21.15.112:0/3941629502 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:31.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.590+0000 7fb74ffff640 1 --2- 172.21.15.112:0/3941629502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 0x55be33471de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.590+0000 7fb754d6f640 1 -- 172.21.15.112:0/3941629502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be330bf070 msgr2=0x55be3347dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.590+0000 7fb754d6f640 1 --2- 172.21.15.112:0/3941629502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be330bf070 0x55be3347dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.590+0000 7fb754d6f640 1 -- 172.21.15.112:0/3941629502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 msgr2=0x55be33471de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.590+0000 7fb754d6f640 1 --2- 172.21.15.112:0/3941629502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 0x55be33471de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.590+0000 7fb754d6f640 1 -- 172.21.15.112:0/3941629502 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be33468580 con 0x55be33052da0 2024-09-21T03:29:31.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.590+0000 7fb74ffff640 1 --2- 172.21.15.112:0/3941629502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 0x55be33471de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:31.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.590+0000 7fb754d6f640 1 --2- 172.21.15.112:0/3941629502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be33052da0 0x55be33463aa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb748031c80 tx=0x7fb748036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37eb23db7d8053b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.590+0000 7fb735ffb640 1 -- 172.21.15.112:0/3941629502 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748054070 con 0x55be33052da0 2024-09-21T03:29:31.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.591+0000 7fb735ffb640 1 -- 172.21.15.112:0/3941629502 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb748036070 con 0x55be33052da0 2024-09-21T03:29:31.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.591+0000 7fb735ffb640 1 -- 172.21.15.112:0/3941629502 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74802f8a0 con 0x55be33052da0 2024-09-21T03:29:31.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.592+0000 7fb75cd46740 1 -- 172.21.15.112:0/3941629502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be33052da0 msgr2=0x55be33463aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.592+0000 7fb75cd46740 1 --2- 172.21.15.112:0/3941629502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be33052da0 0x55be33463aa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb748031c80 tx=0x7fb748036000 comp rx=0 tx=0).stop 2024-09-21T03:29:31.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.592+0000 7fb75cd46740 1 -- 172.21.15.112:0/3941629502 shutdown_connections 2024-09-21T03:29:31.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.592+0000 7fb75cd46740 1 --2- 172.21.15.112:0/3941629502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be33052da0 0x55be33463aa0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.592+0000 7fb75cd46740 1 --2- 172.21.15.112:0/3941629502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be330bf070 0x55be3347dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.592+0000 7fb75cd46740 1 --2- 172.21.15.112:0/3941629502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 0x55be33471de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.592+0000 7fb75cd46740 1 -- 172.21.15.112:0/3941629502 >> 172.21.15.112:0/3941629502 conn(0x55be330b3460 msgr2=0x55be334716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.592+0000 7fb75cd46740 1 -- 172.21.15.112:0/3941629502 shutdown_connections 2024-09-21T03:29:31.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.592+0000 7fb75cd46740 1 -- 172.21.15.112:0/3941629502 wait complete. 2024-09-21T03:29:31.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.592+0000 7fb75cd46740 1 -- start start 2024-09-21T03:29:31.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.593+0000 7fb75cd46740 1 Processor -- start 2024-09-21T03:29:31.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.593+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be33052da0 0x55be3346c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.593+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 0x55be3346cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.593+0000 7fb75cd46740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330bf070 0x55be334e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.593+0000 7fb75cd46740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501b41d0 con 0x55be330bf070 2024-09-21T03:29:31.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.593+0000 7fb75cd46740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501b5640 con 0x55be3309e390 2024-09-21T03:29:31.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.593+0000 7fb75cd46740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb7501b48a0 con 0x55be33052da0 2024-09-21T03:29:31.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb75cd46740 10 client.0 _ll_register_callbacks cb 0x7fb7400046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb74f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 0x55be3346cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb74f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 0x55be3346cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:58570/0 (socket says 172.21.15.112:58570) 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb74f7fe640 1 -- 172.21.15.112:0/1690747791 learned_addr learned my addr 172.21.15.112:0/1690747791 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb74ffff640 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be33052da0 0x55be3346c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb754d6f640 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330bf070 0x55be334e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb74f7fe640 1 -- 172.21.15.112:0/1690747791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be33052da0 msgr2=0x55be3346c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb74f7fe640 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be33052da0 0x55be3346c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb74f7fe640 1 -- 172.21.15.112:0/1690747791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330bf070 msgr2=0x55be334e6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb74f7fe640 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330bf070 0x55be334e6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb74f7fe640 1 -- 172.21.15.112:0/1690747791 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be334e7360 con 0x55be3309e390 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb74f7fe640 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 0x55be3346cd00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb744004a70 tx=0x7fb74400bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e2a2849eace360b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.594+0000 7fb72f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.595+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1690747791 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744015040 con 0x55be3309e390 2024-09-21T03:29:31.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.595+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1690747791 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fb744011640 con 0x55be3309e390 2024-09-21T03:29:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.595+0000 7fb75cd46740 1 -- 172.21.15.112:0/1690747791 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb748036070 con 0x55be3309e390 2024-09-21T03:29:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.595+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1690747791 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb744039050 con 0x55be3309e390 2024-09-21T03:29:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.595+0000 7fb75cd46740 10 client.25148 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.595+0000 7fb75cd46740 1 -- 172.21.15.112:0/1690747791 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be334ee750 con 0x55be3309e390 2024-09-21T03:29:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.595+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1690747791 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb744030070 con 0x55be3309e390 2024-09-21T03:29:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.596+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1690747791 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb74402b7b0 con 0x55be3309e390 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.596+0000 7fb75cd46740 10 client.25148 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.596+0000 7fb75cd46740 10 client.25148 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.596+0000 7fb75cd46740 20 client.25148 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.596+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55be334eebf0 0x55be334f1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.596+0000 7fb75cd46740 4 client.25148 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.596+0000 7fb75cd46740 1 -- 172.21.15.112:0/1690747791 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb748036070 con 0x55be334eebf0 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.596+0000 7fb74ffff640 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55be334eebf0 0x55be334f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.596+0000 7fb72f7fe640 10 client.25148 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.596+0000 7fb74ffff640 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55be334eebf0 0x55be334f1010 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85ca83354db06119 server_cookie=3bf261f3dc30ed52 in_seq=0 out_seq=0 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb72f7fe640 1 -- 172.21.15.112:0/1690747791 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb748036070 con 0x55be334eebf0 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb72f7fe640 10 client.25148 handle_command_reply: tid=0 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 1 client.25148 shutdown 2024-09-21T03:29:31.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 1 -- 172.21.15.112:0/1690747791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 msgr2=0x55be3346cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 0x55be3346cd00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb744004a70 tx=0x7fb74400bde0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 1 -- 172.21.15.112:0/1690747791 shutdown_connections 2024-09-21T03:29:31.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55be334eebf0 0x55be334f1010 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be330bf070 0x55be334e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be3309e390 0x55be3346cd00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 1 --2- 172.21.15.112:0/1690747791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be33052da0 0x55be3346c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 1 -- 172.21.15.112:0/1690747791 >> 172.21.15.112:0/1690747791 conn(0x55be330b3460 msgr2=0x55be33463a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb72f7fe640 0 client.25148 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:31.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 1 -- 172.21.15.112:0/1690747791 shutdown_connections 2024-09-21T03:29:31.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 1 -- 172.21.15.112:0/1690747791 wait complete. 2024-09-21T03:29:31.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.597+0000 7fb75cd46740 20 client.25148 trim_cache size 0 max 16384 2024-09-21T03:29:31.603 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:29:31.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.598+0000 7fb7567d2640 1 -- 172.21.15.112:0/858082570 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7380b6700 msgr2=0x7fb7380b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.598+0000 7fb7567d2640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7380b6700 0x7fb7380b8c10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb748006010 tx=0x7fb748009360 comp rx=0 tx=0).stop 2024-09-21T03:29:31.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.598+0000 7fb7567d2640 1 -- 172.21.15.112:0/858082570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7501049c0 msgr2=0x7fb750104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:31.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.598+0000 7fb7567d2640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7501049c0 0x7fb750104110 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb74000d950 tx=0x7fb74000de10 comp rx=0 tx=0).stop 2024-09-21T03:29:31.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.598+0000 7fb7567d2640 1 -- 172.21.15.112:0/858082570 shutdown_connections 2024-09-21T03:29:31.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.598+0000 7fb7567d2640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7380b6700 0x7fb7380b8c10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.598+0000 7fb7567d2640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb75010ddf0 0x7fb750102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.598+0000 7fb7567d2640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb750105310 0x7fb750102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.598+0000 7fb7567d2640 1 --2- 172.21.15.112:0/858082570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7501049c0 0x7fb750104110 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:31.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.598+0000 7fb7567d2640 1 -- 172.21.15.112:0/858082570 >> 172.21.15.112:0/858082570 conn(0x7fb750079680 msgr2=0x7fb7500ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:31.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.599+0000 7fb7567d2640 1 -- 172.21.15.112:0/858082570 shutdown_connections 2024-09-21T03:29:31.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:31.599+0000 7fb7567d2640 1 -- 172.21.15.112:0/858082570 wait complete. 2024-09-21T03:29:31.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:31 smithi203 ceph-mon[34723]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T03:29:31.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:31 smithi187 ceph-mon[34603]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-21T03:29:32.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:29:32.622 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:33.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.126+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/927994247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08102630 msgr2=0x7f6a08102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.126+0000 7f6a0d6e1640 1 --2- 172.21.15.112:0/927994247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08102630 0x7f6a08102a40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009900 tx=0x7f69f802a480 comp rx=0 tx=0).stop 2024-09-21T03:29:33.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.126+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/927994247 shutdown_connections 2024-09-21T03:29:33.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.126+0000 7f6a0d6e1640 1 --2- 172.21.15.112:0/927994247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a081101e0 0x7f6a08112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.126+0000 7f6a0d6e1640 1 --2- 172.21.15.112:0/927994247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a08102f80 0x7f6a0810fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.126+0000 7f6a0d6e1640 1 --2- 172.21.15.112:0/927994247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08102630 0x7f6a08102a40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.126+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/927994247 >> 172.21.15.112:0/927994247 conn(0x7f6a08079730 msgr2=0x7f6a08079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:33.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.126+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/927994247 shutdown_connections 2024-09-21T03:29:33.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.126+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/927994247 wait complete. 2024-09-21T03:29:33.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.128+0000 7f6a0d6e1640 1 Processor -- start 2024-09-21T03:29:33.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.128+0000 7f6a0d6e1640 1 -- start start 2024-09-21T03:29:33.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.128+0000 7f6a0d6e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08102630 0x7f6a0819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.129+0000 7f6a0d6e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a08102f80 0x7f6a0819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.129+0000 7f6a0d6e1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a081101e0 0x7f6a081a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.129+0000 7f6a06ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08102630 0x7f6a0819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.129+0000 7f6a06ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08102630 0x7f6a0819c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50138/0 (socket says 172.21.15.112:50138) 2024-09-21T03:29:33.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.129+0000 7f6a06ffd640 1 -- 172.21.15.112:0/3721065791 learned_addr learned my addr 172.21.15.112:0/3721065791 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:33.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.129+0000 7f6a067fc640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a08102f80 0x7f6a0819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.129+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/3721065791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a08115c50 con 0x7f6a08102630 2024-09-21T03:29:33.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.129+0000 7f6a077fe640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a081101e0 0x7f6a081a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.129+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/3721065791 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a08115ad0 con 0x7f6a081101e0 2024-09-21T03:29:33.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.129+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/3721065791 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6a08115dd0 con 0x7f6a08102f80 2024-09-21T03:29:33.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.130+0000 7f6a077fe640 1 -- 172.21.15.112:0/3721065791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a08102f80 msgr2=0x7f6a0819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.130+0000 7f6a077fe640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a08102f80 0x7f6a0819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.130+0000 7f6a077fe640 1 -- 172.21.15.112:0/3721065791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08102630 msgr2=0x7f6a0819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.130+0000 7f6a077fe640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08102630 0x7f6a0819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.130+0000 7f6a077fe640 1 -- 172.21.15.112:0/3721065791 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a081a25b0 con 0x7f6a081101e0 2024-09-21T03:29:33.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.130+0000 7f6a077fe640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a081101e0 0x7f6a081a1e80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009fd0 tx=0x7f6a0000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:33.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.130+0000 7f69effff640 1 -- 172.21.15.112:0/3721065791 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000ce20 con 0x7f6a081101e0 2024-09-21T03:29:33.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.131+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/3721065791 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a081a28a0 con 0x7f6a081101e0 2024-09-21T03:29:33.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.131+0000 7f69effff640 1 -- 172.21.15.112:0/3721065791 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a00005020 con 0x7f6a081101e0 2024-09-21T03:29:33.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.131+0000 7f69effff640 1 -- 172.21.15.112:0/3721065791 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00010690 con 0x7f6a081101e0 2024-09-21T03:29:33.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.131+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/3721065791 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a081a9000 con 0x7f6a081101e0 2024-09-21T03:29:33.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.134+0000 7f69effff640 1 -- 172.21.15.112:0/3721065791 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f6a000054d0 con 0x7f6a081101e0 2024-09-21T03:29:33.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.134+0000 7f6a13c55740 1 Processor -- start 2024-09-21T03:29:33.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.134+0000 7f6a13c55740 1 -- start start 2024-09-21T03:29:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.134+0000 7f69effff640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69f00b6700 0x7f69f00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.134+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf39f070 0x55ecaf7462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf3761b0 0x55ecaf746800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f69effff640 1 -- 172.21.15.112:0/3721065791 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f6a000e7d40 con 0x7f6a081101e0 2024-09-21T03:29:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f6a067fc640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69f00b6700 0x7f69f00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf37e390 0x55ecaf746d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f6a067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf3761b0 0x55ecaf746800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f6a067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf3761b0 0x55ecaf746800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50146/0 (socket says 172.21.15.112:50146) 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f6a067fc640 1 -- 172.21.15.112:0/2250857208 learned_addr learned my addr 172.21.15.112:0/2250857208 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f6a13c55740 1 -- 172.21.15.112:0/2250857208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ecaf6bcea0 con 0x55ecaf3761b0 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/2250857208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf39f070 0x55ecaf7462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f6a13c55740 1 -- 172.21.15.112:0/2250857208 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ecaf6ac2d0 con 0x55ecaf39f070 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f6a13c55740 1 -- 172.21.15.112:0/2250857208 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ecaf6b00e0 con 0x55ecaf37e390 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.135+0000 7f6a077fe640 1 --2- 172.21.15.112:0/2250857208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf37e390 0x55ecaf746d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.136+0000 7f6a067fc640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69f00b6700 0x7f69f00b8c10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f69fc011550 tx=0x7f69fc011950 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.136+0000 7f6a077fe640 1 -- 172.21.15.112:0/2250857208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf39f070 msgr2=0x55ecaf7462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.136+0000 7f6a077fe640 1 --2- 172.21.15.112:0/2250857208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf39f070 0x55ecaf7462c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.136+0000 7f6a077fe640 1 -- 172.21.15.112:0/2250857208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf3761b0 msgr2=0x55ecaf746800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.136+0000 7f6a077fe640 1 --2- 172.21.15.112:0/2250857208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf3761b0 0x55ecaf746800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.136+0000 7f6a077fe640 1 -- 172.21.15.112:0/2250857208 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecaf40a650 con 0x55ecaf37e390 2024-09-21T03:29:33.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.136+0000 7f6a077fe640 1 --2- 172.21.15.112:0/2250857208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf37e390 0x55ecaf746d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a000e54f0 tx=0x7f6a000ad450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb58fb07d030c7bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.136+0000 7f69ed7fa640 1 -- 172.21.15.112:0/2250857208 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a000ae2f0 con 0x55ecaf37e390 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.136+0000 7f69ed7fa640 1 -- 172.21.15.112:0/2250857208 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a000b26d0 con 0x55ecaf37e390 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.136+0000 7f69ed7fa640 1 -- 172.21.15.112:0/2250857208 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a000a9540 con 0x55ecaf37e390 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.137+0000 7f6a13c55740 1 -- 172.21.15.112:0/2250857208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf37e390 msgr2=0x55ecaf746d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.137+0000 7f6a13c55740 1 --2- 172.21.15.112:0/2250857208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf37e390 0x55ecaf746d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a000e54f0 tx=0x7f6a000ad450 comp rx=0 tx=0).stop 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.137+0000 7f6a13c55740 1 -- 172.21.15.112:0/2250857208 shutdown_connections 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.137+0000 7f6a13c55740 1 --2- 172.21.15.112:0/2250857208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf37e390 0x55ecaf746d40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.137+0000 7f6a13c55740 1 --2- 172.21.15.112:0/2250857208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf3761b0 0x55ecaf746800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.137+0000 7f6a13c55740 1 --2- 172.21.15.112:0/2250857208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf39f070 0x55ecaf7462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.137+0000 7f6a13c55740 1 -- 172.21.15.112:0/2250857208 >> 172.21.15.112:0/2250857208 conn(0x55ecaf393460 msgr2=0x55ecaf7441b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.137+0000 7f6a13c55740 1 -- 172.21.15.112:0/2250857208 shutdown_connections 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.137+0000 7f6a13c55740 1 -- 172.21.15.112:0/2250857208 wait complete. 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.138+0000 7f6a13c55740 1 -- start start 2024-09-21T03:29:33.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.138+0000 7f6a13c55740 1 Processor -- start 2024-09-21T03:29:33.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.138+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf3761b0 0x55ecaf7518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.138+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf751df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf39f070 0x55ecaf752330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf751df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf751df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50150/0 (socket says 172.21.15.112:50150) 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a067fc640 1 -- 172.21.15.112:0/2764624091 learned_addr learned my addr 172.21.15.112:0/2764624091 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a13c55740 1 -- 172.21.15.112:0/2764624091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ecaf50d700 con 0x55ecaf37e390 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a13c55740 1 -- 172.21.15.112:0/2764624091 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ecaf647160 con 0x55ecaf39f070 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a13c55740 1 -- 172.21.15.112:0/2764624091 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ecaf754af0 con 0x55ecaf3761b0 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a077fe640 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf39f070 0x55ecaf752330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf3761b0 0x55ecaf7518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a13c55740 10 client.0 _ll_register_callbacks cb 0x55ecaf6609e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a077fe640 1 -- 172.21.15.112:0/2764624091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf3761b0 msgr2=0x55ecaf7518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a077fe640 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf3761b0 0x55ecaf7518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a077fe640 1 -- 172.21.15.112:0/2764624091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 msgr2=0x55ecaf751df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a077fe640 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf751df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.139+0000 7f6a077fe640 1 -- 172.21.15.112:0/2764624091 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecaf674c90 con 0x55ecaf39f070 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.140+0000 7f6a077fe640 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf39f070 0x55ecaf752330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a000e5520 tx=0x7f6a000057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d01e216593cffdce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:33.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.140+0000 7f69eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.140+0000 7f69eaff5640 1 -- 172.21.15.112:0/2764624091 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a000d51c0 con 0x55ecaf39f070 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.140+0000 7f6a13c55740 1 -- 172.21.15.112:0/2764624091 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a000b26d0 con 0x55ecaf39f070 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.140+0000 7f69eaff5640 1 -- 172.21.15.112:0/2764624091 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f6a000ad050 con 0x55ecaf39f070 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.140+0000 7f69eaff5640 1 -- 172.21.15.112:0/2764624091 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6a000ae2f0 con 0x55ecaf39f070 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.140+0000 7f6a13c55740 10 client.25160 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.140+0000 7f6a13c55740 1 -- 172.21.15.112:0/2764624091 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ecaf7606c0 con 0x55ecaf39f070 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.140+0000 7f69eaff5640 1 -- 172.21.15.112:0/2764624091 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a000b26d0 con 0x55ecaf39f070 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.141+0000 7f69eaff5640 1 -- 172.21.15.112:0/2764624091 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6a000a9040 con 0x55ecaf39f070 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.141+0000 7f6a13c55740 10 client.25160 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.141+0000 7f6a13c55740 10 client.25160 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.141+0000 7f6a13c55740 20 client.25160 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.141+0000 7f6a13c55740 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55ecaf35d9f0 0x55ecaf763a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.141+0000 7f6a13c55740 4 client.25160 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.141+0000 7f6a13c55740 1 -- 172.21.15.112:0/2764624091 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecaf6bedf0 con 0x55ecaf35d9f0 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.141+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55ecaf35d9f0 0x55ecaf763a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.142+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55ecaf35d9f0 0x55ecaf763a20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41270e0f517d06c3 server_cookie=ac9c554564d5f4d6 in_seq=0 out_seq=0 2024-09-21T03:29:33.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.142+0000 7f69eaff5640 10 client.25160 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:33.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.142+0000 7f69eaff5640 1 -- 172.21.15.112:0/2764624091 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ecaf6bedf0 con 0x55ecaf35d9f0 2024-09-21T03:29:33.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.142+0000 7f69eaff5640 10 client.25160 handle_command_reply: tid=0 2024-09-21T03:29:33.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.142+0000 7f6a13c55740 1 client.25160 shutdown 2024-09-21T03:29:33.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.143+0000 7f6a13c55740 1 -- 172.21.15.112:0/2764624091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf39f070 msgr2=0x55ecaf752330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.143+0000 7f6a13c55740 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf39f070 0x55ecaf752330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a000e5520 tx=0x7f6a000057c0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.143+0000 7f6a13c55740 1 -- 172.21.15.112:0/2764624091 shutdown_connections 2024-09-21T03:29:33.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.143+0000 7f6a13c55740 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55ecaf35d9f0 0x55ecaf763a20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.143+0000 7f6a13c55740 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf39f070 0x55ecaf752330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.143+0000 7f6a13c55740 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf751df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.143+0000 7f6a13c55740 1 --2- 172.21.15.112:0/2764624091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf3761b0 0x55ecaf7518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.143+0000 7f6a13c55740 1 -- 172.21.15.112:0/2764624091 >> 172.21.15.112:0/2764624091 conn(0x55ecaf393460 msgr2=0x55ecaf749450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:33.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.143+0000 7f69eaff5640 0 client.25160 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:33.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.144+0000 7f6a13c55740 1 -- 172.21.15.112:0/2764624091 shutdown_connections 2024-09-21T03:29:33.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.144+0000 7f6a13c55740 1 -- 172.21.15.112:0/2764624091 wait complete. 2024-09-21T03:29:33.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.144+0000 7f6a13c55740 20 client.25160 trim_cache size 0 max 16384 2024-09-21T03:29:33.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a13c55740 1 Processor -- start 2024-09-21T03:29:33.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a13c55740 1 -- start start 2024-09-21T03:29:33.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 0x55ecaf753790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf75d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf39f070 0x55ecaf743760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf75d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf75d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50162/0 (socket says 172.21.15.112:50162) 2024-09-21T03:29:33.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a067fc640 1 -- 172.21.15.112:0/1586922996 learned_addr learned my addr 172.21.15.112:0/1586922996 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:33.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a13c55740 1 -- 172.21.15.112:0/1586922996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081b3fb0 con 0x55ecaf37e390 2024-09-21T03:29:33.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a13c55740 1 -- 172.21.15.112:0/1586922996 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081b1880 con 0x55ecaf3761b0 2024-09-21T03:29:33.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a13c55740 1 -- 172.21.15.112:0/1586922996 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081af6d0 con 0x55ecaf39f070 2024-09-21T03:29:33.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.157+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/1586922996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 0x55ecaf753790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f6a077fe640 1 --2- 172.21.15.112:0/1586922996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf39f070 0x55ecaf743760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f6a06ffd640 1 -- 172.21.15.112:0/1586922996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf39f070 msgr2=0x55ecaf743760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/1586922996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf39f070 0x55ecaf743760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f6a06ffd640 1 -- 172.21.15.112:0/1586922996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 msgr2=0x55ecaf75d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/1586922996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf75d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f6a06ffd640 1 -- 172.21.15.112:0/1586922996 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecaf7485b0 con 0x55ecaf3761b0 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f6a077fe640 1 --2- 172.21.15.112:0/1586922996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf39f070 0x55ecaf743760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f6a067fc640 1 --2- 172.21.15.112:0/1586922996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf75d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/1586922996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 0x55ecaf753790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f69f8004770 tx=0x7f69f80058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67277f8987afec4d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f69ecff9640 1 -- 172.21.15.112:0/1586922996 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f804a070 con 0x55ecaf3761b0 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f69ecff9640 1 -- 172.21.15.112:0/1586922996 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f8031950 con 0x55ecaf3761b0 2024-09-21T03:29:33.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.158+0000 7f69ecff9640 1 -- 172.21.15.112:0/1586922996 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8035040 con 0x55ecaf3761b0 2024-09-21T03:29:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.159+0000 7f6a13c55740 1 -- 172.21.15.112:0/1586922996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 msgr2=0x55ecaf753790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.159+0000 7f6a13c55740 1 --2- 172.21.15.112:0/1586922996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 0x55ecaf753790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f69f8004770 tx=0x7f69f80058d0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.159+0000 7f6a13c55740 1 -- 172.21.15.112:0/1586922996 shutdown_connections 2024-09-21T03:29:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.159+0000 7f6a13c55740 1 --2- 172.21.15.112:0/1586922996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf39f070 0x55ecaf743760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.159+0000 7f6a13c55740 1 --2- 172.21.15.112:0/1586922996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf75d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.159+0000 7f6a13c55740 1 --2- 172.21.15.112:0/1586922996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 0x55ecaf753790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.159+0000 7f6a13c55740 1 -- 172.21.15.112:0/1586922996 >> 172.21.15.112:0/1586922996 conn(0x55ecaf393460 msgr2=0x55ecaf75cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:33.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.159+0000 7f6a13c55740 1 -- 172.21.15.112:0/1586922996 shutdown_connections 2024-09-21T03:29:33.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.159+0000 7f6a13c55740 1 -- 172.21.15.112:0/1586922996 wait complete. 2024-09-21T03:29:33.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a13c55740 1 -- start start 2024-09-21T03:29:33.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a13c55740 1 Processor -- start 2024-09-21T03:29:33.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 0x55ecaf764d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf765290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a13c55740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf39f070 0x55ecaf7cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a13c55740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081afd40 con 0x55ecaf37e390 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a13c55740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081b11b0 con 0x55ecaf3761b0 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a13c55740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081b0410 con 0x55ecaf39f070 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf765290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a06ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 0x55ecaf764d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a13c55740 10 client.0 _ll_register_callbacks cb 0x7f6a000aa330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf765290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50172/0 (socket says 172.21.15.112:50172) 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.160+0000 7f6a06ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 0x55ecaf764d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:58624/0 (socket says 172.21.15.112:58624) 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.161+0000 7f6a067fc640 1 -- 172.21.15.112:0/4156845678 learned_addr learned my addr 172.21.15.112:0/4156845678 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.161+0000 7f6a077fe640 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf39f070 0x55ecaf7cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.161+0000 7f6a06ffd640 1 -- 172.21.15.112:0/4156845678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf39f070 msgr2=0x55ecaf7cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.161+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf39f070 0x55ecaf7cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.161+0000 7f6a06ffd640 1 -- 172.21.15.112:0/4156845678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 msgr2=0x55ecaf765290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.161+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf765290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.161+0000 7f6a06ffd640 1 -- 172.21.15.112:0/4156845678 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecaf765b30 con 0x55ecaf3761b0 2024-09-21T03:29:33.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.161+0000 7f6a067fc640 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf765290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:33.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.162+0000 7f6a06ffd640 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 0x55ecaf764d50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f69f802a4c0 tx=0x7f69f8004490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8efa73befda2906a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:33.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.162+0000 7f69eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:29:33.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.162+0000 7f69eaff5640 1 -- 172.21.15.112:0/4156845678 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8035050 con 0x55ecaf3761b0 2024-09-21T03:29:33.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.162+0000 7f69eaff5640 1 -- 172.21.15.112:0/4156845678 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f69f80314a0 con 0x55ecaf3761b0 2024-09-21T03:29:33.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.162+0000 7f6a13c55740 1 -- 172.21.15.112:0/4156845678 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ecaf6ba8a0 con 0x55ecaf3761b0 2024-09-21T03:29:33.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.164+0000 7f69eaff5640 1 -- 172.21.15.112:0/4156845678 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69f804a040 con 0x55ecaf3761b0 2024-09-21T03:29:33.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.164+0000 7f6a13c55740 10 client.25166 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:33.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.164+0000 7f6a13c55740 1 -- 172.21.15.112:0/4156845678 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ecaf7ce700 con 0x55ecaf3761b0 2024-09-21T03:29:33.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.164+0000 7f69eaff5640 1 -- 172.21.15.112:0/4156845678 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ecaf6ba8a0 con 0x55ecaf3761b0 2024-09-21T03:29:33.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.164+0000 7f69eaff5640 1 -- 172.21.15.112:0/4156845678 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f69f803d450 con 0x55ecaf3761b0 2024-09-21T03:29:33.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.164+0000 7f6a13c55740 10 client.25166 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:33.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.164+0000 7f6a13c55740 10 client.25166 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:29:33.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.164+0000 7f6a13c55740 20 client.25166 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:33.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.164+0000 7f6a13c55740 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55ecaf332da0 0x55ecaf7d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:33.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.164+0000 7f6a13c55740 4 client.25166 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:33.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.164+0000 7f6a13c55740 1 -- 172.21.15.112:0/4156845678 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ecaf6b7cd0 con 0x55ecaf332da0 2024-09-21T03:29:33.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.165+0000 7f6a067fc640 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55ecaf332da0 0x55ecaf7d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:33.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.165+0000 7f69eaff5640 10 client.25166 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:33.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.165+0000 7f6a067fc640 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55ecaf332da0 0x55ecaf7d0ee0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35ff88095e7dbc2 server_cookie=a3c91dd9426b60eb in_seq=0 out_seq=0 2024-09-21T03:29:33.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f69eaff5640 1 -- 172.21.15.112:0/4156845678 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ecaf6b7cd0 con 0x55ecaf332da0 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f69eaff5640 10 client.25166 handle_command_reply: tid=0 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 1 client.25166 shutdown 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 1 -- 172.21.15.112:0/4156845678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 msgr2=0x55ecaf764d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 0x55ecaf764d50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f69f802a4c0 tx=0x7f69f8004490 comp rx=0 tx=0).stop 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 1 -- 172.21.15.112:0/4156845678 shutdown_connections 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55ecaf332da0 0x55ecaf7d0ee0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecaf39f070 0x55ecaf7cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecaf37e390 0x55ecaf765290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f69eaff5640 0 client.25166 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 1 --2- 172.21.15.112:0/4156845678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecaf3761b0 0x55ecaf764d50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 1 -- 172.21.15.112:0/4156845678 >> 172.21.15.112:0/4156845678 conn(0x55ecaf393460 msgr2=0x55ecaf75d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 1 -- 172.21.15.112:0/4156845678 shutdown_connections 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 1 -- 172.21.15.112:0/4156845678 wait complete. 2024-09-21T03:29:33.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.166+0000 7f6a13c55740 20 client.25166 trim_cache size 0 max 16384 2024-09-21T03:29:33.170 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:29:33.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/3721065791 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69f00b6700 msgr2=0x7f69f00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69f00b6700 0x7f69f00b8c10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f69fc011550 tx=0x7f69fc011950 comp rx=0 tx=0).stop 2024-09-21T03:29:33.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/3721065791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a081101e0 msgr2=0x7f6a081a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:33.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a081101e0 0x7f6a081a1e80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009fd0 tx=0x7f6a0000ef10 comp rx=0 tx=0).stop 2024-09-21T03:29:33.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/3721065791 shutdown_connections 2024-09-21T03:29:33.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f69f00b6700 0x7f69f00b8c10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a081101e0 0x7f6a081a1e80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a08102f80 0x7f6a0819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 --2- 172.21.15.112:0/3721065791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a08102630 0x7f6a0819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:33.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/3721065791 >> 172.21.15.112:0/3721065791 conn(0x7f6a08079730 msgr2=0x7f6a08108cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:33.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/3721065791 shutdown_connections 2024-09-21T03:29:33.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:33.167+0000 7f6a0d6e1640 1 -- 172.21.15.112:0/3721065791 wait complete. 2024-09-21T03:29:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:33 smithi112 ceph-mon[31442]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:29:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:33 smithi203 ceph-mon[34723]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:29:33.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:33 smithi187 ceph-mon[34603]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-21T03:29:33.948 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:29:34.243 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:34.667 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:34 smithi112 ceph-mon[31442]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T03:29:34.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.810+0000 7fa58c335640 1 -- 172.21.15.112:0/2163992581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa584102a60 msgr2=0x7fa584102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:34.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.810+0000 7fa58c335640 1 --2- 172.21.15.112:0/2163992581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa584102a60 0x7fa584102e70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa580009900 tx=0x7fa58002a410 comp rx=0 tx=0).stop 2024-09-21T03:29:34.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.810+0000 7fa58c335640 1 -- 172.21.15.112:0/2163992581 shutdown_connections 2024-09-21T03:29:34.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.810+0000 7fa58c335640 1 --2- 172.21.15.112:0/2163992581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa584108cf0 0x7fa58410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.810+0000 7fa58c335640 1 --2- 172.21.15.112:0/2163992581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5841083a0 0x7fa5841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.810+0000 7fa58c335640 1 --2- 172.21.15.112:0/2163992581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa584102a60 0x7fa584102e70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.810+0000 7fa58c335640 1 -- 172.21.15.112:0/2163992581 >> 172.21.15.112:0/2163992581 conn(0x7fa584079730 msgr2=0x7fa584079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:34.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.810+0000 7fa58c335640 1 -- 172.21.15.112:0/2163992581 shutdown_connections 2024-09-21T03:29:34.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.810+0000 7fa58c335640 1 -- 172.21.15.112:0/2163992581 wait complete. 2024-09-21T03:29:34.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.812+0000 7fa58c335640 1 Processor -- start 2024-09-21T03:29:34.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.812+0000 7fa58c335640 1 -- start start 2024-09-21T03:29:34.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.812+0000 7fa58c335640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa584102a60 0x7fa584109610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.812+0000 7fa58c335640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5841083a0 0x7fa584109b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.812+0000 7fa58c335640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa584108cf0 0x7fa58410a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.812+0000 7fa58c335640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841129b0 con 0x7fa584102a60 2024-09-21T03:29:34.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.812+0000 7fa58c335640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa584112830 con 0x7fa584108cf0 2024-09-21T03:29:34.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.812+0000 7fa58c335640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa584112b30 con 0x7fa5841083a0 2024-09-21T03:29:34.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.812+0000 7fa58a0d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa584102a60 0x7fa584109610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.812+0000 7fa58a0d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa584102a60 0x7fa584109610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50184/0 (socket says 172.21.15.112:50184) 2024-09-21T03:29:34.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.812+0000 7fa58a0d1640 1 -- 172.21.15.112:0/1847983999 learned_addr learned my addr 172.21.15.112:0/1847983999 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:34.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.813+0000 7fa58a8d2640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa584108cf0 0x7fa58410a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.813+0000 7fa5898d0640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5841083a0 0x7fa584109b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.813+0000 7fa58a8d2640 1 -- 172.21.15.112:0/1847983999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5841083a0 msgr2=0x7fa584109b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:34.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.813+0000 7fa58a8d2640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5841083a0 0x7fa584109b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.813+0000 7fa58a8d2640 1 -- 172.21.15.112:0/1847983999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa584102a60 msgr2=0x7fa584109610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:34.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.813+0000 7fa58a8d2640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa584102a60 0x7fa584109610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.813+0000 7fa58a8d2640 1 -- 172.21.15.112:0/1847983999 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5841a86f0 con 0x7fa584108cf0 2024-09-21T03:29:34.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.813+0000 7fa5898d0640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5841083a0 0x7fa584109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:34.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.813+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa584102a60 0x7fa584109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:34.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.813+0000 7fa58a8d2640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa584108cf0 0x7fa58410a090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009fd0 tx=0x7fa57c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:34.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.814+0000 7fa57b7fe640 1 -- 172.21.15.112:0/1847983999 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c00ce20 con 0x7fa584108cf0 2024-09-21T03:29:34.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.814+0000 7fa57b7fe640 1 -- 172.21.15.112:0/1847983999 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa57c005020 con 0x7fa584108cf0 2024-09-21T03:29:34.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.814+0000 7fa57b7fe640 1 -- 172.21.15.112:0/1847983999 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c010690 con 0x7fa584108cf0 2024-09-21T03:29:34.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.814+0000 7fa58c335640 1 -- 172.21.15.112:0/1847983999 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5841a89e0 con 0x7fa584108cf0 2024-09-21T03:29:34.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.814+0000 7fa58c335640 1 -- 172.21.15.112:0/1847983999 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5841a8ed0 con 0x7fa584108cf0 2024-09-21T03:29:34.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.817+0000 7fa5928a9740 1 Processor -- start 2024-09-21T03:29:34.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.818+0000 7fa57b7fe640 1 -- 172.21.15.112:0/1847983999 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa57c0054d0 con 0x7fa584108cf0 2024-09-21T03:29:34.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.819+0000 7fa5928a9740 1 -- start start 2024-09-21T03:29:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.819+0000 7fa57b7fe640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5700bee90 0x7fa5700c13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.819+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac4282c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.819+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac428800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.819+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac428d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.819+0000 7fa5928a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac39eea0 con 0x55f9ac060390 2024-09-21T03:29:34.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.819+0000 7fa5928a9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac38e2d0 con 0x55f9ac081070 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.819+0000 7fa5928a9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac3920e0 con 0x55f9ac0581b0 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.819+0000 7fa57b7fe640 1 -- 172.21.15.112:0/1847983999 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa57c0e7df0 con 0x7fa584108cf0 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa58a8d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac428d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa58a8d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac428d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50188/0 (socket says 172.21.15.112:50188) 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa58a8d2640 1 -- 172.21.15.112:0/2749482796 learned_addr learned my addr 172.21.15.112:0/2749482796 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/2749482796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac4282c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa5898d0640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5700bee90 0x7fa5700c13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa58a0d1640 1 -- 172.21.15.112:0/2749482796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 msgr2=0x55f9ac428800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/2749482796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac428800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa58a0d1640 1 -- 172.21.15.112:0/2749482796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 msgr2=0x55f9ac428d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/2749482796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac428d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa58a0d1640 1 -- 172.21.15.112:0/2749482796 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9ac0ec650 con 0x55f9ac081070 2024-09-21T03:29:34.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa58a8d2640 1 --2- 172.21.15.112:0/2749482796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac428d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.820+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/2749482796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac4282c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa58410b060 tx=0x7fa580005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be53337997671faf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.821+0000 7fa578d9a640 1 -- 172.21.15.112:0/2749482796 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580040070 con 0x55f9ac081070 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.821+0000 7fa578d9a640 1 -- 172.21.15.112:0/2749482796 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5800047a0 con 0x55f9ac081070 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.821+0000 7fa578d9a640 1 -- 172.21.15.112:0/2749482796 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580035040 con 0x55f9ac081070 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.822+0000 7fa5928a9740 1 -- 172.21.15.112:0/2749482796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 msgr2=0x55f9ac4282c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.822+0000 7fa5928a9740 1 --2- 172.21.15.112:0/2749482796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac4282c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa58410b060 tx=0x7fa580005f30 comp rx=0 tx=0).stop 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.822+0000 7fa5898d0640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5700bee90 0x7fa5700c13a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa58406be20 tx=0x7fa574005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.822+0000 7fa5928a9740 1 -- 172.21.15.112:0/2749482796 shutdown_connections 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.822+0000 7fa5928a9740 1 --2- 172.21.15.112:0/2749482796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac428d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.822+0000 7fa5928a9740 1 --2- 172.21.15.112:0/2749482796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac428800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.822+0000 7fa5928a9740 1 --2- 172.21.15.112:0/2749482796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac4282c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.822+0000 7fa5928a9740 1 -- 172.21.15.112:0/2749482796 >> 172.21.15.112:0/2749482796 conn(0x55f9ac075460 msgr2=0x55f9ac4261b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:34.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.822+0000 7fa5928a9740 1 -- 172.21.15.112:0/2749482796 shutdown_connections 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.822+0000 7fa5928a9740 1 -- 172.21.15.112:0/2749482796 wait complete. 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.823+0000 7fa5928a9740 1 -- start start 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.823+0000 7fa5928a9740 1 Processor -- start 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.823+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac433980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.823+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac060390 0x55f9ac433ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.823+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac081070 0x55f9ac43fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.823+0000 7fa5928a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac1ef700 con 0x55f9ac081070 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.823+0000 7fa5928a9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac329160 con 0x55f9ac060390 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.823+0000 7fa5928a9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac436a40 con 0x55f9ac0581b0 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa5928a9740 10 client.0 _ll_register_callbacks cb 0x55f9ac3429e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa58a0d1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac433980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa5898d0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac060390 0x55f9ac433ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa58a8d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac081070 0x55f9ac43fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa5898d0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac060390 0x55f9ac433ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:58666/0 (socket says 172.21.15.112:58666) 2024-09-21T03:29:34.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa5898d0640 1 -- 172.21.15.112:0/3976271416 learned_addr learned my addr 172.21.15.112:0/3976271416 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:34.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa58a0d1640 1 -- 172.21.15.112:0/3976271416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac060390 msgr2=0x55f9ac433ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:34.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac060390 0x55f9ac433ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa58a0d1640 1 -- 172.21.15.112:0/3976271416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac081070 msgr2=0x55f9ac43fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:34.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac081070 0x55f9ac43fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa58a0d1640 1 -- 172.21.15.112:0/3976271416 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9ac356c90 con 0x55f9ac0581b0 2024-09-21T03:29:34.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa5898d0640 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac060390 0x55f9ac433ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:34.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa58a8d2640 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac081070 0x55f9ac43fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:34.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.824+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac433980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa580000c00 tx=0x7fa580004e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7179eae630735520 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:34.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.825+0000 7fa56dffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:29:34.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.825+0000 7fa56dffb640 1 -- 172.21.15.112:0/3976271416 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580035070 con 0x55f9ac0581b0 2024-09-21T03:29:34.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.825+0000 7fa5928a9740 1 -- 172.21.15.112:0/3976271416 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5800047a0 con 0x55f9ac0581b0 2024-09-21T03:29:34.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.825+0000 7fa56dffb640 1 -- 172.21.15.112:0/3976271416 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa580005410 con 0x55f9ac0581b0 2024-09-21T03:29:34.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.825+0000 7fa56dffb640 1 -- 172.21.15.112:0/3976271416 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa580040040 con 0x55f9ac0581b0 2024-09-21T03:29:34.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.825+0000 7fa5928a9740 10 client.25192 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:34.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.825+0000 7fa5928a9740 1 -- 172.21.15.112:0/3976271416 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9ac442750 con 0x55f9ac0581b0 2024-09-21T03:29:34.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.826+0000 7fa56dffb640 1 -- 172.21.15.112:0/3976271416 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5800047a0 con 0x55f9ac0581b0 2024-09-21T03:29:34.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.826+0000 7fa56dffb640 1 -- 172.21.15.112:0/3976271416 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa5800315a0 con 0x55f9ac0581b0 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.826+0000 7fa5928a9740 10 client.25192 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.826+0000 7fa5928a9740 10 client.25192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.826+0000 7fa5928a9740 20 client.25192 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.826+0000 7fa5928a9740 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f9ac03f9f0 0x55f9ac445c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.826+0000 7fa5928a9740 4 client.25192 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.826+0000 7fa5928a9740 1 -- 172.21.15.112:0/3976271416 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9ac3a0df0 con 0x55f9ac03f9f0 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.826+0000 7fa5898d0640 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f9ac03f9f0 0x55f9ac445c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.827+0000 7fa56dffb640 10 client.25192 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.827+0000 7fa5898d0640 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f9ac03f9f0 0x55f9ac445c60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0abb9f331ce7fc6 server_cookie=a67be4d005e94c16 in_seq=0 out_seq=0 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa56dffb640 1 -- 172.21.15.112:0/3976271416 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9ac3a0df0 con 0x55f9ac03f9f0 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa56dffb640 10 client.25192 handle_command_reply: tid=0 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa5928a9740 1 client.25192 shutdown 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa5928a9740 1 -- 172.21.15.112:0/3976271416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 msgr2=0x55f9ac433980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:34.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa5928a9740 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac433980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa580000c00 tx=0x7fa580004e20 comp rx=0 tx=0).stop 2024-09-21T03:29:34.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa5928a9740 1 -- 172.21.15.112:0/3976271416 shutdown_connections 2024-09-21T03:29:34.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa5928a9740 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f9ac03f9f0 0x55f9ac445c60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa5928a9740 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac081070 0x55f9ac43fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa5928a9740 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac060390 0x55f9ac433ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa5928a9740 1 --2- 172.21.15.112:0/3976271416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac433980 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa5928a9740 1 -- 172.21.15.112:0/3976271416 >> 172.21.15.112:0/3976271416 conn(0x55f9ac075460 msgr2=0x55f9ac42b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:34.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa56dffb640 0 client.25192 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:34.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa5928a9740 1 -- 172.21.15.112:0/3976271416 shutdown_connections 2024-09-21T03:29:34.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.828+0000 7fa5928a9740 1 -- 172.21.15.112:0/3976271416 wait complete. 2024-09-21T03:29:34.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.829+0000 7fa5928a9740 20 client.25192 trim_cache size 0 max 16384 2024-09-21T03:29:34.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.854+0000 7fa5928a9740 1 Processor -- start 2024-09-21T03:29:34.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.854+0000 7fa5928a9740 1 -- start start 2024-09-21T03:29:34.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.855+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac42f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.855+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac433e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.855+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac43fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.855+0000 7fa5928a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841afba0 con 0x55f9ac060390 2024-09-21T03:29:34.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.855+0000 7fa5928a9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841ad470 con 0x55f9ac081070 2024-09-21T03:29:34.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.855+0000 7fa5928a9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841ab320 con 0x55f9ac0581b0 2024-09-21T03:29:34.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.855+0000 7fa5898d0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac433e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.855+0000 7fa58a0d1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac42f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.856+0000 7fa58a8d2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac43fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.856+0000 7fa58a0d1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac42f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:51646/0 (socket says 172.21.15.112:51646) 2024-09-21T03:29:34.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.856+0000 7fa58a0d1640 1 -- 172.21.15.112:0/712740897 learned_addr learned my addr 172.21.15.112:0/712740897 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:34.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.856+0000 7fa58a0d1640 1 -- 172.21.15.112:0/712740897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 msgr2=0x55f9ac43fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:34.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.856+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/712740897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac43fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.856+0000 7fa58a0d1640 1 -- 172.21.15.112:0/712740897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 msgr2=0x55f9ac433e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:34.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.856+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/712740897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac433e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.856+0000 7fa58a0d1640 1 -- 172.21.15.112:0/712740897 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9ac42a8d0 con 0x55f9ac0581b0 2024-09-21T03:29:34.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.856+0000 7fa5898d0640 1 --2- 172.21.15.112:0/712740897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac433e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:34.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.856+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/712740897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac42f130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa580005a60 tx=0x7fa580031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea3cb5f6c6ad7f28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:34.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.856+0000 7fa56ffff640 1 -- 172.21.15.112:0/712740897 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580035040 con 0x55f9ac0581b0 2024-09-21T03:29:34.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.857+0000 7fa56ffff640 1 -- 172.21.15.112:0/712740897 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa580009520 con 0x55f9ac0581b0 2024-09-21T03:29:34.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.857+0000 7fa56ffff640 1 -- 172.21.15.112:0/712740897 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580003970 con 0x55f9ac0581b0 2024-09-21T03:29:34.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.858+0000 7fa5928a9740 1 -- 172.21.15.112:0/712740897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 msgr2=0x55f9ac42f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.858+0000 7fa5928a9740 1 --2- 172.21.15.112:0/712740897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac42f130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa580005a60 tx=0x7fa580031400 comp rx=0 tx=0).stop 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.858+0000 7fa5928a9740 1 -- 172.21.15.112:0/712740897 shutdown_connections 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.858+0000 7fa5928a9740 1 --2- 172.21.15.112:0/712740897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac43fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.858+0000 7fa5928a9740 1 --2- 172.21.15.112:0/712740897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac433e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.858+0000 7fa5928a9740 1 --2- 172.21.15.112:0/712740897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac42f130 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.858+0000 7fa5928a9740 1 -- 172.21.15.112:0/712740897 >> 172.21.15.112:0/712740897 conn(0x55f9ac075460 msgr2=0x55f9ac433730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.858+0000 7fa5928a9740 1 -- 172.21.15.112:0/712740897 shutdown_connections 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.858+0000 7fa5928a9740 1 -- 172.21.15.112:0/712740897 wait complete. 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.858+0000 7fa5928a9740 1 -- start start 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.858+0000 7fa5928a9740 1 Processor -- start 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac43c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac437e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5928a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac4383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5898d0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac437e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5898d0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac437e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50228/0 (socket says 172.21.15.112:50228) 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5898d0640 1 -- 172.21.15.112:0/2928430404 learned_addr learned my addr 172.21.15.112:0/2928430404 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5928a9740 1 -- 172.21.15.112:0/2928430404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841ab970 con 0x55f9ac060390 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5928a9740 1 -- 172.21.15.112:0/2928430404 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841acda0 con 0x55f9ac081070 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5928a9740 1 -- 172.21.15.112:0/2928430404 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa5841ac000 con 0x55f9ac0581b0 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5928a9740 10 client.0 _ll_register_callbacks cb 0x7fa57c0aa2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac43c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa58a8d2640 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac4383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5898d0640 1 -- 172.21.15.112:0/2928430404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 msgr2=0x55f9ac43c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5898d0640 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac43c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5898d0640 1 -- 172.21.15.112:0/2928430404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 msgr2=0x55f9ac4383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:34.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5898d0640 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac4383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.859+0000 7fa5898d0640 1 -- 172.21.15.112:0/2928430404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9ac438b40 con 0x55f9ac060390 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.860+0000 7fa5898d0640 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac437e90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55f9ac446400 tx=0x7fa574014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e07ea9ffb9364 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.860+0000 7fa56dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.860+0000 7fa56dffb640 1 -- 172.21.15.112:0/2928430404 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57401fd30 con 0x55f9ac060390 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.860+0000 7fa56dffb640 1 -- 172.21.15.112:0/2928430404 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa57400f3d0 con 0x55f9ac060390 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.860+0000 7fa5928a9740 1 -- 172.21.15.112:0/2928430404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa580009520 con 0x55f9ac060390 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.860+0000 7fa56dffb640 1 -- 172.21.15.112:0/2928430404 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa574048050 con 0x55f9ac060390 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.860+0000 7fa5928a9740 10 client.15582 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.860+0000 7fa5928a9740 1 -- 172.21.15.112:0/2928430404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9ac4af5e0 con 0x55f9ac060390 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.861+0000 7fa56dffb640 1 -- 172.21.15.112:0/2928430404 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa57400fa80 con 0x55f9ac060390 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.861+0000 7fa56dffb640 1 -- 172.21.15.112:0/2928430404 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa5740437f0 con 0x55f9ac060390 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.861+0000 7fa5928a9740 10 client.15582 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.861+0000 7fa5928a9740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:34.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.861+0000 7fa5928a9740 20 client.15582 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.861+0000 7fa5928a9740 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f9ac014da0 0x55f9ac4b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.861+0000 7fa5928a9740 4 client.15582 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.861+0000 7fa5928a9740 1 -- 172.21.15.112:0/2928430404 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa580009520 con 0x55f9ac014da0 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.862+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f9ac014da0 0x55f9ac4b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.862+0000 7fa56dffb640 10 client.15582 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.862+0000 7fa58a0d1640 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f9ac014da0 0x55f9ac4b1e20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fdd3a346abc1303 server_cookie=ee1b393ec22d4df0 in_seq=0 out_seq=0 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.862+0000 7fa56dffb640 1 -- 172.21.15.112:0/2928430404 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa580009520 con 0x55f9ac014da0 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.862+0000 7fa56dffb640 10 client.15582 handle_command_reply: tid=0 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.862+0000 7fa5928a9740 1 client.15582 shutdown 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa5928a9740 1 -- 172.21.15.112:0/2928430404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 msgr2=0x55f9ac437e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa5928a9740 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac437e90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55f9ac446400 tx=0x7fa574014070 comp rx=0 tx=0).stop 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa5928a9740 1 -- 172.21.15.112:0/2928430404 shutdown_connections 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa5928a9740 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f9ac014da0 0x55f9ac4b1e20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa5928a9740 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f9ac081070 0x55f9ac4383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.870 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa5928a9740 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f9ac060390 0x55f9ac437e90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa5928a9740 1 --2- 172.21.15.112:0/2928430404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f9ac0581b0 0x55f9ac43c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa5928a9740 1 -- 172.21.15.112:0/2928430404 >> 172.21.15.112:0/2928430404 conn(0x55f9ac075460 msgr2=0x55f9ac4292f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa56dffb640 0 client.15582 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa5928a9740 1 -- 172.21.15.112:0/2928430404 shutdown_connections 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa5928a9740 1 -- 172.21.15.112:0/2928430404 wait complete. 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.863+0000 7fa5928a9740 20 client.15582 trim_cache size 0 max 16384 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 -- 172.21.15.112:0/1847983999 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5700bee90 msgr2=0x7fa5700c13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5700bee90 0x7fa5700c13a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa58406be20 tx=0x7fa574005f90 comp rx=0 tx=0).stop 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 -- 172.21.15.112:0/1847983999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa584108cf0 msgr2=0x7fa58410a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa584108cf0 0x7fa58410a090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009fd0 tx=0x7fa57c00ef10 comp rx=0 tx=0).stop 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 -- 172.21.15.112:0/1847983999 shutdown_connections 2024-09-21T03:29:34.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5700bee90 0x7fa5700c13a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa584108cf0 0x7fa58410a090 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5841083a0 0x7fa584109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 --2- 172.21.15.112:0/1847983999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa584102a60 0x7fa584109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:34.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 -- 172.21.15.112:0/1847983999 >> 172.21.15.112:0/1847983999 conn(0x7fa584079730 msgr2=0x7fa5841016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:34.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 -- 172.21.15.112:0/1847983999 shutdown_connections 2024-09-21T03:29:34.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:34.864+0000 7fa58c335640 1 -- 172.21.15.112:0/1847983999 wait complete. 2024-09-21T03:29:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:34 smithi203 ceph-mon[34723]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T03:29:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:34 smithi187 ceph-mon[34603]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T03:29:36.576 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:29:36.577 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:29:36.858 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:37.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.360+0000 7f3183a3b640 1 -- 172.21.15.112:0/3582816009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f317c1049b0 msgr2=0x7f317c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:37.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.360+0000 7f3183a3b640 1 --2- 172.21.15.112:0/3582816009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f317c1049b0 0x7f317c104dc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3178009900 tx=0x7f317802a410 comp rx=0 tx=0).stop 2024-09-21T03:29:37.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.360+0000 7f3183a3b640 1 -- 172.21.15.112:0/3582816009 shutdown_connections 2024-09-21T03:29:37.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.360+0000 7f3183a3b640 1 --2- 172.21.15.112:0/3582816009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f317c10de10 0x7f317c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.360+0000 7f3183a3b640 1 --2- 172.21.15.112:0/3582816009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317c105300 0x7f317c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.360+0000 7f3183a3b640 1 --2- 172.21.15.112:0/3582816009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f317c1049b0 0x7f317c104dc0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.360+0000 7f3183a3b640 1 -- 172.21.15.112:0/3582816009 >> 172.21.15.112:0/3582816009 conn(0x7f317c079730 msgr2=0x7f317c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.360+0000 7f3183a3b640 1 -- 172.21.15.112:0/3582816009 shutdown_connections 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.360+0000 7f3183a3b640 1 -- 172.21.15.112:0/3582816009 wait complete. 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.361+0000 7f3183a3b640 1 Processor -- start 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.361+0000 7f3183a3b640 1 -- start start 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.362+0000 7f3183a3b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317c105300 0x7f317c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.362+0000 7f3183a3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f317c10de10 0x7f317c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.362+0000 7f3183a3b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f317c19fda0 0x7f317c1a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.362+0000 7f3180fd6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f317c10de10 0x7f317c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.362+0000 7f31817d7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317c105300 0x7f317c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.362+0000 7f3181fd8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f317c19fda0 0x7f317c1a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.362+0000 7f3181fd8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f317c19fda0 0x7f317c1a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:59190/0 (socket says 172.21.15.112:59190) 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.362+0000 7f3181fd8640 1 -- 172.21.15.112:0/3819738194 learned_addr learned my addr 172.21.15.112:0/3819738194 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:37.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.362+0000 7f3183a3b640 1 -- 172.21.15.112:0/3819738194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f317c113a90 con 0x7f317c10de10 2024-09-21T03:29:37.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.362+0000 7f3183a3b640 1 -- 172.21.15.112:0/3819738194 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f317c113910 con 0x7f317c19fda0 2024-09-21T03:29:37.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.362+0000 7f3183a3b640 1 -- 172.21.15.112:0/3819738194 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f317c113c10 con 0x7f317c105300 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.363+0000 7f3180fd6640 1 -- 172.21.15.112:0/3819738194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317c105300 msgr2=0x7f317c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.363+0000 7f3180fd6640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317c105300 0x7f317c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.363+0000 7f3180fd6640 1 -- 172.21.15.112:0/3819738194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f317c19fda0 msgr2=0x7f317c1a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.363+0000 7f3180fd6640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f317c19fda0 0x7f317c1a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.363+0000 7f3180fd6640 1 -- 172.21.15.112:0/3819738194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c1a0980 con 0x7f317c10de10 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.363+0000 7f3181fd8640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f317c19fda0 0x7f317c1a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.363+0000 7f3180fd6640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f317c10de10 0x7f317c19abc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f317c06ad30 tx=0x7f316c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.364+0000 7f31727fc640 1 -- 172.21.15.112:0/3819738194 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c00ccc0 con 0x7f317c10de10 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.364+0000 7f31727fc640 1 -- 172.21.15.112:0/3819738194 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f316c005020 con 0x7f317c10de10 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.364+0000 7f31727fc640 1 -- 172.21.15.112:0/3819738194 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c00ce60 con 0x7f317c10de10 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.364+0000 7f3183a3b640 1 -- 172.21.15.112:0/3819738194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c1025d0 con 0x7f317c10de10 2024-09-21T03:29:37.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.364+0000 7f3183a3b640 1 -- 172.21.15.112:0/3819738194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c102ac0 con 0x7f317c10de10 2024-09-21T03:29:37.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.368+0000 7f3189faf740 1 Processor -- start 2024-09-21T03:29:37.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.368+0000 7f3189faf740 1 -- start start 2024-09-21T03:29:37.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.368+0000 7f3189faf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44421350 0x55bc447d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.368+0000 7f31727fc640 1 -- 172.21.15.112:0/3819738194 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f316c022020 con 0x7f317c10de10 2024-09-21T03:29:37.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.369+0000 7f3189faf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc443f8490 0x55bc447d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.369+0000 7f3189faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.369+0000 7f3189faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc4473ee60 con 0x55bc44400670 2024-09-21T03:29:37.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.369+0000 7f3189faf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bc4472e290 con 0x55bc44421350 2024-09-21T03:29:37.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.369+0000 7f3189faf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bc447320a0 con 0x55bc443f8490 2024-09-21T03:29:37.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.369+0000 7f3180fd6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.369+0000 7f3180fd6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46254/0 (socket says 172.21.15.112:46254) 2024-09-21T03:29:37.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.369+0000 7f3180fd6640 1 -- 172.21.15.112:0/2861932219 learned_addr learned my addr 172.21.15.112:0/2861932219 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:37.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.369+0000 7f3181fd8640 1 --2- 172.21.15.112:0/2861932219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44421350 0x55bc447d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.369+0000 7f31817d7640 1 --2- 172.21.15.112:0/2861932219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc443f8490 0x55bc447d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.370+0000 7f3180fd6640 1 -- 172.21.15.112:0/2861932219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc443f8490 msgr2=0x55bc447d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:37.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.370+0000 7f3180fd6640 1 --2- 172.21.15.112:0/2861932219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc443f8490 0x55bc447d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.370+0000 7f3180fd6640 1 -- 172.21.15.112:0/2861932219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44421350 msgr2=0x55bc447d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:37.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.370+0000 7f3180fd6640 1 --2- 172.21.15.112:0/2861932219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44421350 0x55bc447d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.370+0000 7f3180fd6640 1 -- 172.21.15.112:0/2861932219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc4448c930 con 0x55bc44400670 2024-09-21T03:29:37.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.370+0000 7f3181fd8640 1 --2- 172.21.15.112:0/2861932219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44421350 0x55bc447d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:37.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.370+0000 7f31727fc640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31680b67d0 0x7f31680b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.370+0000 7f3180fd6640 1 --2- 172.21.15.112:0/2861932219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447d16c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f316c0e4620 tx=0x7f316c0ef0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cff4caec60a7974 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.370+0000 7f31727fc640 1 -- 172.21.15.112:0/3819738194 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f316c0e6f00 con 0x7f317c10de10 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.370+0000 7f3181fd8640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31680b67d0 0x7f31680b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.370+0000 7f31677fe640 1 -- 172.21.15.112:0/2861932219 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c0ad810 con 0x55bc44400670 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.371+0000 7f31677fe640 1 -- 172.21.15.112:0/2861932219 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f316c0b0cd0 con 0x55bc44400670 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.371+0000 7f31677fe640 1 -- 172.21.15.112:0/2861932219 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c0b0930 con 0x55bc44400670 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.371+0000 7f3189faf740 1 -- 172.21.15.112:0/2861932219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 msgr2=0x55bc447d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.371+0000 7f3189faf740 1 --2- 172.21.15.112:0/2861932219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447d16c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f316c0e4620 tx=0x7f316c0ef0e0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.371+0000 7f3181fd8640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31680b67d0 0x7f31680b8ce0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f31740049d0 tx=0x7f3174009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.372+0000 7f3189faf740 1 -- 172.21.15.112:0/2861932219 shutdown_connections 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.372+0000 7f3189faf740 1 --2- 172.21.15.112:0/2861932219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447d16c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.372+0000 7f3189faf740 1 --2- 172.21.15.112:0/2861932219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc443f8490 0x55bc447d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.372+0000 7f3189faf740 1 --2- 172.21.15.112:0/2861932219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44421350 0x55bc447d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.372+0000 7f3189faf740 1 -- 172.21.15.112:0/2861932219 >> 172.21.15.112:0/2861932219 conn(0x55bc44415740 msgr2=0x55bc447ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:37.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.372+0000 7f3189faf740 1 -- 172.21.15.112:0/2861932219 shutdown_connections 2024-09-21T03:29:37.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.372+0000 7f3189faf740 1 -- 172.21.15.112:0/2861932219 wait complete. 2024-09-21T03:29:37.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.372+0000 7f3189faf740 1 -- start start 2024-09-21T03:29:37.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.372+0000 7f3189faf740 1 Processor -- start 2024-09-21T03:29:37.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3189faf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc443f8490 0x55bc447dc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3189faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3189faf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44421350 0x55bc447f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3189faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc4458f890 con 0x55bc44400670 2024-09-21T03:29:37.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3189faf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bc446c8d40 con 0x55bc443f8490 2024-09-21T03:29:37.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3189faf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bc447df470 con 0x55bc44421350 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3180fd6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3189faf740 10 client.0 _ll_register_callbacks cb 0x55bc4472cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3180fd6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447dcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46262/0 (socket says 172.21.15.112:46262) 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3180fd6640 1 -- 172.21.15.112:0/1893625687 learned_addr learned my addr 172.21.15.112:0/1893625687 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f31817d7640 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc443f8490 0x55bc447dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3181fd8640 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44421350 0x55bc447f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3180fd6640 1 -- 172.21.15.112:0/1893625687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44421350 msgr2=0x55bc447f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3180fd6640 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44421350 0x55bc447f97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3180fd6640 1 -- 172.21.15.112:0/1893625687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc443f8490 msgr2=0x55bc447dc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3180fd6640 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc443f8490 0x55bc447dc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3180fd6640 1 -- 172.21.15.112:0/1893625687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc447f9dc0 con 0x55bc44400670 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.373+0000 7f3181fd8640 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44421350 0x55bc447f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.374+0000 7f3180fd6640 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447dcda0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f316c0e4760 tx=0x7f316c0ba960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283ce67f4b8e4a48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:37.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.374+0000 7f3164ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.374+0000 7f3164ff9640 1 -- 172.21.15.112:0/1893625687 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c01f070 con 0x55bc44400670 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.374+0000 7f3164ff9640 1 -- 172.21.15.112:0/1893625687 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f316c0ea2e0 con 0x55bc44400670 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.374+0000 7f3189faf740 1 -- 172.21.15.112:0/1893625687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f316c0b0cd0 con 0x55bc44400670 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.374+0000 7f3164ff9640 1 -- 172.21.15.112:0/1893625687 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f316c0b0b10 con 0x55bc44400670 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.374+0000 7f3189faf740 10 client.15594 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.374+0000 7f3189faf740 1 -- 172.21.15.112:0/1893625687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bc447df880 con 0x55bc44400670 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.375+0000 7f3164ff9640 1 -- 172.21.15.112:0/1893625687 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f316c0b0cd0 con 0x55bc44400670 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.376+0000 7f3164ff9640 1 -- 172.21.15.112:0/1893625687 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f316c0d41a0 con 0x55bc44400670 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.376+0000 7f3189faf740 10 client.15594 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.376+0000 7f3189faf740 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.376+0000 7f3189faf740 20 client.15594 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.376+0000 7f3189faf740 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55bc443dfcd0 0x55bc447e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.376+0000 7f3189faf740 4 client.15594 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.376+0000 7f3189faf740 1 -- 172.21.15.112:0/1893625687 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc44740db0 con 0x55bc443dfcd0 2024-09-21T03:29:37.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.376+0000 7f31817d7640 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55bc443dfcd0 0x55bc447e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.376+0000 7f3164ff9640 10 client.15594 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:37.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.376+0000 7f31817d7640 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55bc443dfcd0 0x55bc447e2c40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56084dc71e38468 server_cookie=4313520a4692204a in_seq=0 out_seq=0 2024-09-21T03:29:37.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3164ff9640 1 -- 172.21.15.112:0/1893625687 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc44740db0 con 0x55bc443dfcd0 2024-09-21T03:29:37.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3164ff9640 10 client.15594 handle_command_reply: tid=0 2024-09-21T03:29:37.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3189faf740 1 client.15594 shutdown 2024-09-21T03:29:37.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3189faf740 1 -- 172.21.15.112:0/1893625687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 msgr2=0x55bc447dcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:37.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3189faf740 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447dcda0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f316c0e4760 tx=0x7f316c0ba960 comp rx=0 tx=0).stop 2024-09-21T03:29:37.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3189faf740 1 -- 172.21.15.112:0/1893625687 shutdown_connections 2024-09-21T03:29:37.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3189faf740 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55bc443dfcd0 0x55bc447e2c40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3189faf740 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44421350 0x55bc447f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3189faf740 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc44400670 0x55bc447dcda0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3189faf740 1 --2- 172.21.15.112:0/1893625687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc443f8490 0x55bc447dc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3189faf740 1 -- 172.21.15.112:0/1893625687 >> 172.21.15.112:0/1893625687 conn(0x55bc44415740 msgr2=0x55bc447d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:37.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.377+0000 7f3164ff9640 0 client.15594 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:37.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.378+0000 7f3189faf740 1 -- 172.21.15.112:0/1893625687 shutdown_connections 2024-09-21T03:29:37.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.378+0000 7f3189faf740 1 -- 172.21.15.112:0/1893625687 wait complete. 2024-09-21T03:29:37.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.378+0000 7f3189faf740 20 client.15594 trim_cache size 0 max 16384 2024-09-21T03:29:37.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.410+0000 7f3189faf740 1 Processor -- start 2024-09-21T03:29:37.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.410+0000 7f3189faf740 1 -- start start 2024-09-21T03:29:37.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.410+0000 7f3189faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 0x55bc447f9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.410+0000 7f3189faf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44400670 0x55bc447f9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.410+0000 7f3189faf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44421350 0x55bc447d5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.410+0000 7f3189faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3160010eb0 con 0x55bc443f8490 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.410+0000 7f3189faf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f316000e780 con 0x55bc44421350 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.410+0000 7f3189faf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f316000c5d0 con 0x55bc44400670 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.411+0000 7f3180fd6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44400670 0x55bc447f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.411+0000 7f31817d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 0x55bc447f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.411+0000 7f3180fd6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44400670 0x55bc447f9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:38130/0 (socket says 172.21.15.112:38130) 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.411+0000 7f3180fd6640 1 -- 172.21.15.112:0/3884355990 learned_addr learned my addr 172.21.15.112:0/3884355990 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.411+0000 7f3180fd6640 1 -- 172.21.15.112:0/3884355990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44421350 msgr2=0x55bc447d5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.411+0000 7f3180fd6640 1 --2- 172.21.15.112:0/3884355990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44421350 0x55bc447d5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.411+0000 7f3180fd6640 1 -- 172.21.15.112:0/3884355990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 msgr2=0x55bc447f9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.411+0000 7f3180fd6640 1 --2- 172.21.15.112:0/3884355990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 0x55bc447f9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.411+0000 7f3180fd6640 1 -- 172.21.15.112:0/3884355990 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc447d2f30 con 0x55bc44400670 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.411+0000 7f3180fd6640 1 --2- 172.21.15.112:0/3884355990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44400670 0x55bc447f9650 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f316c0ae430 tx=0x7f316c0e7240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbac8e060c91f406 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:37.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.412+0000 7f3166ffd640 1 -- 172.21.15.112:0/3884355990 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c01f070 con 0x55bc44400670 2024-09-21T03:29:37.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.412+0000 7f3166ffd640 1 -- 172.21.15.112:0/3884355990 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f316c0ba9d0 con 0x55bc44400670 2024-09-21T03:29:37.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.412+0000 7f3166ffd640 1 -- 172.21.15.112:0/3884355990 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c0b0f90 con 0x55bc44400670 2024-09-21T03:29:37.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.412+0000 7f3189faf740 1 -- 172.21.15.112:0/3884355990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44400670 msgr2=0x55bc447f9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:37.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.412+0000 7f3189faf740 1 --2- 172.21.15.112:0/3884355990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44400670 0x55bc447f9650 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f316c0ae430 tx=0x7f316c0e7240 comp rx=0 tx=0).stop 2024-09-21T03:29:37.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.413+0000 7f3189faf740 1 -- 172.21.15.112:0/3884355990 shutdown_connections 2024-09-21T03:29:37.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.413+0000 7f3189faf740 1 --2- 172.21.15.112:0/3884355990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44421350 0x55bc447d5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.413+0000 7f3189faf740 1 --2- 172.21.15.112:0/3884355990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44400670 0x55bc447f9650 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.413+0000 7f3189faf740 1 --2- 172.21.15.112:0/3884355990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 0x55bc447f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.413+0000 7f3189faf740 1 -- 172.21.15.112:0/3884355990 >> 172.21.15.112:0/3884355990 conn(0x55bc44415740 msgr2=0x55bc447daee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:37.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.413+0000 7f3189faf740 1 -- 172.21.15.112:0/3884355990 shutdown_connections 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.413+0000 7f3189faf740 1 -- 172.21.15.112:0/3884355990 wait complete. 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.413+0000 7f3189faf740 1 -- start start 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.414+0000 7f3189faf740 1 Processor -- start 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.414+0000 7f3189faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 0x55bc447e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.414+0000 7f3189faf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44400670 0x55bc447e4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.414+0000 7f3189faf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44421350 0x55bc447e5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.414+0000 7f3189faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f316000cc40 con 0x55bc443f8490 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.414+0000 7f3189faf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f316000e0b0 con 0x55bc44400670 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.414+0000 7f3189faf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f316000d310 con 0x55bc44421350 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.414+0000 7f3189faf740 10 client.0 _ll_register_callbacks cb 0x7f316c0ead70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.414+0000 7f31817d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 0x55bc447e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f31817d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 0x55bc447e66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46278/0 (socket says 172.21.15.112:46278) 2024-09-21T03:29:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f31817d7640 1 -- 172.21.15.112:0/1689302258 learned_addr learned my addr 172.21.15.112:0/1689302258 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:37.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f3181fd8640 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44421350 0x55bc447e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f3180fd6640 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44400670 0x55bc447e4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f31817d7640 1 -- 172.21.15.112:0/1689302258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44421350 msgr2=0x55bc447e5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:37.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f31817d7640 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44421350 0x55bc447e5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f31817d7640 1 -- 172.21.15.112:0/1689302258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44400670 msgr2=0x55bc447e4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:37.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f31817d7640 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44400670 0x55bc447e4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f31817d7640 1 -- 172.21.15.112:0/1689302258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc448587e0 con 0x55bc443f8490 2024-09-21T03:29:37.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f31817d7640 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 0x55bc447e66b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3178009230 tx=0x7f3178032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40156951fb166e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:37.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f3164ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:37.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.415+0000 7f3164ff9640 1 -- 172.21.15.112:0/1689302258 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178035040 con 0x55bc443f8490 2024-09-21T03:29:37.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.416+0000 7f3164ff9640 1 -- 172.21.15.112:0/1689302258 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f31780318b0 con 0x55bc443f8490 2024-09-21T03:29:37.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.416+0000 7f3189faf740 1 -- 172.21.15.112:0/1689302258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f316c0ba9d0 con 0x55bc443f8490 2024-09-21T03:29:37.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.416+0000 7f3164ff9640 1 -- 172.21.15.112:0/1689302258 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3178004040 con 0x55bc443f8490 2024-09-21T03:29:37.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.416+0000 7f3189faf740 10 client.15600 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.416+0000 7f3189faf740 1 -- 172.21.15.112:0/1689302258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bc44859a40 con 0x55bc443f8490 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.416+0000 7f3164ff9640 1 -- 172.21.15.112:0/1689302258 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31780054f0 con 0x55bc443f8490 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.417+0000 7f3164ff9640 1 -- 172.21.15.112:0/1689302258 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3178004a30 con 0x55bc443f8490 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.417+0000 7f3189faf740 10 client.15600 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.417+0000 7f3189faf740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.417+0000 7f3189faf740 20 client.15600 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.417+0000 7f3189faf740 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55bc443b5080 0x55bc4485c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.417+0000 7f3189faf740 4 client.15600 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.417+0000 7f3189faf740 1 -- 172.21.15.112:0/1689302258 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f316c0ba9d0 con 0x55bc443b5080 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.417+0000 7f3180fd6640 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55bc443b5080 0x55bc4485c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.418+0000 7f3164ff9640 10 client.15600 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.418+0000 7f3180fd6640 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55bc443b5080 0x55bc4485c200 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1730d246b24107 server_cookie=afc359e2c579e619 in_seq=0 out_seq=0 2024-09-21T03:29:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.419+0000 7f3164ff9640 1 -- 172.21.15.112:0/1689302258 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f316c0ba9d0 con 0x55bc443b5080 2024-09-21T03:29:37.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.419+0000 7f3164ff9640 10 client.15600 handle_command_reply: tid=0 2024-09-21T03:29:37.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.419+0000 7f3189faf740 1 client.15600 shutdown 2024-09-21T03:29:37.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.420+0000 7f3189faf740 1 -- 172.21.15.112:0/1689302258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 msgr2=0x55bc447e66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:37.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.420+0000 7f3189faf740 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 0x55bc447e66b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3178009230 tx=0x7f3178032e80 comp rx=0 tx=0).stop 2024-09-21T03:29:37.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.421+0000 7f3189faf740 1 -- 172.21.15.112:0/1689302258 shutdown_connections 2024-09-21T03:29:37.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.421+0000 7f3189faf740 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55bc443b5080 0x55bc4485c200 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.421+0000 7f3189faf740 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc44421350 0x55bc447e5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.421+0000 7f3189faf740 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc44400670 0x55bc447e4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.421+0000 7f3189faf740 1 --2- 172.21.15.112:0/1689302258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc443f8490 0x55bc447e66b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.421+0000 7f3189faf740 1 -- 172.21.15.112:0/1689302258 >> 172.21.15.112:0/1689302258 conn(0x55bc44415740 msgr2=0x55bc447f9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:37.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.421+0000 7f3164ff9640 0 client.15600 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:37.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3189faf740 1 -- 172.21.15.112:0/1689302258 shutdown_connections 2024-09-21T03:29:37.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3189faf740 1 -- 172.21.15.112:0/1689302258 wait complete. 2024-09-21T03:29:37.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3189faf740 20 client.15600 trim_cache size 0 max 16384 2024-09-21T03:29:37.425 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:29:37.425 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:29:37.425 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "82573db5-77b6-4d68-9155-414eebbfe779", 2024-09-21T03:29:37.425 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:29:37.425 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:29:37.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3183a3b640 1 -- 172.21.15.112:0/3819738194 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31680b67d0 msgr2=0x7f31680b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:37.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3183a3b640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31680b67d0 0x7f31680b8ce0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f31740049d0 tx=0x7f3174009500 comp rx=0 tx=0).stop 2024-09-21T03:29:37.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3183a3b640 1 -- 172.21.15.112:0/3819738194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f317c10de10 msgr2=0x7f317c19abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:37.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3183a3b640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f317c10de10 0x7f317c19abc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f317c06ad30 tx=0x7f316c00ed90 comp rx=0 tx=0).stop 2024-09-21T03:29:37.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3183a3b640 1 -- 172.21.15.112:0/3819738194 shutdown_connections 2024-09-21T03:29:37.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3183a3b640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31680b67d0 0x7f31680b8ce0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3183a3b640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f317c19fda0 0x7f317c1a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3183a3b640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f317c10de10 0x7f317c19abc0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3183a3b640 1 --2- 172.21.15.112:0/3819738194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317c105300 0x7f317c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:37.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.422+0000 7f3183a3b640 1 -- 172.21.15.112:0/3819738194 >> 172.21.15.112:0/3819738194 conn(0x7f317c079730 msgr2=0x7f317c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:37.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.423+0000 7f3183a3b640 1 -- 172.21.15.112:0/3819738194 shutdown_connections 2024-09-21T03:29:37.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:37.423+0000 7f3183a3b640 1 -- 172.21.15.112:0/3819738194 wait complete. 2024-09-21T03:29:37.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:37 smithi112 ceph-mon[31442]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T03:29:37.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:37 smithi203 ceph-mon[34723]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T03:29:37.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:37 smithi187 ceph-mon[34603]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-21T03:29:38.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:29:38.447 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:38.561 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:38 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:29:38.561 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:38 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:38.562 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:38 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:29:38.562 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:38 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:38.562 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:38 smithi112 ceph-mon[31442]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-21T03:29:38.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:38 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:29:38.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:38 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:38.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:38 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:29:38.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:38 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:38.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:38 smithi203 ceph-mon[34723]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-21T03:29:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:38 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:29:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:38 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:38 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:29:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:38 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:38.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:38 smithi187 ceph-mon[34603]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-21T03:29:38.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.982+0000 7f6e7820c640 1 -- 172.21.15.112:0/1515522164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e701094b0 msgr2=0x7f6e701098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.982+0000 7f6e7820c640 1 --2- 172.21.15.112:0/1515522164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e701094b0 0x7f6e701098c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009900 tx=0x7f6e6002a410 comp rx=0 tx=0).stop 2024-09-21T03:29:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.982+0000 7f6e7820c640 1 -- 172.21.15.112:0/1515522164 shutdown_connections 2024-09-21T03:29:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.982+0000 7f6e7820c640 1 --2- 172.21.15.112:0/1515522164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e70109e00 0x7f6e70110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.982+0000 7f6e7820c640 1 --2- 172.21.15.112:0/1515522164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e701094b0 0x7f6e701098c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.982+0000 7f6e7820c640 1 --2- 172.21.15.112:0/1515522164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e70103bc0 0x7f6e70103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.982+0000 7f6e7820c640 1 -- 172.21.15.112:0/1515522164 >> 172.21.15.112:0/1515522164 conn(0x7f6e7007b910 msgr2=0x7f6e7007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.982+0000 7f6e7820c640 1 -- 172.21.15.112:0/1515522164 shutdown_connections 2024-09-21T03:29:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.982+0000 7f6e7820c640 1 -- 172.21.15.112:0/1515522164 wait complete. 2024-09-21T03:29:38.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.983+0000 7f6e7820c640 1 Processor -- start 2024-09-21T03:29:38.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e7820c640 1 -- start start 2024-09-21T03:29:38.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e7820c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e70103bc0 0x7f6e70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:38.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e7820c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e701094b0 0x7f6e70075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:38.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e7820c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e70109e00 0x7f6e70076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:38.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e7820c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e70113b10 con 0x7f6e70109e00 2024-09-21T03:29:38.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e7820c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e70113990 con 0x7f6e70103bc0 2024-09-21T03:29:38.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e7820c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6e70113c90 con 0x7f6e701094b0 2024-09-21T03:29:38.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e75fa8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e70103bc0 0x7f6e70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:38.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e767a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e70109e00 0x7f6e70076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:38.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e757a7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e701094b0 0x7f6e70075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:38.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e75fa8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e70103bc0 0x7f6e70075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:59254/0 (socket says 172.21.15.112:59254) 2024-09-21T03:29:38.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.984+0000 7f6e75fa8640 1 -- 172.21.15.112:0/1623751089 learned_addr learned my addr 172.21.15.112:0/1623751089 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:38.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.985+0000 7f6e757a7640 1 -- 172.21.15.112:0/1623751089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e70103bc0 msgr2=0x7f6e70075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:38.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.985+0000 7f6e757a7640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e70103bc0 0x7f6e70075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.985+0000 7f6e757a7640 1 -- 172.21.15.112:0/1623751089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e70109e00 msgr2=0x7f6e70076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:38.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.985+0000 7f6e757a7640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e70109e00 0x7f6e70076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.985+0000 7f6e757a7640 1 -- 172.21.15.112:0/1623751089 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e701a8760 con 0x7f6e701094b0 2024-09-21T03:29:38.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.985+0000 7f6e767a9640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e70109e00 0x7f6e70076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:38.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.985+0000 7f6e757a7640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e701094b0 0x7f6e70075c50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009230 tx=0x7f6e60003be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:38.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.985+0000 7f6e66ffd640 1 -- 172.21.15.112:0/1623751089 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6003f070 con 0x7f6e701094b0 2024-09-21T03:29:38.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.985+0000 7f6e7820c640 1 -- 172.21.15.112:0/1623751089 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e701a89f0 con 0x7f6e701094b0 2024-09-21T03:29:38.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.985+0000 7f6e66ffd640 1 -- 172.21.15.112:0/1623751089 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e60003e10 con 0x7f6e701094b0 2024-09-21T03:29:38.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.987+0000 7f6e66ffd640 1 -- 172.21.15.112:0/1623751089 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60035030 con 0x7f6e701094b0 2024-09-21T03:29:38.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.987+0000 7f6e7820c640 1 -- 172.21.15.112:0/1623751089 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e701a8ef0 con 0x7f6e701094b0 2024-09-21T03:29:38.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.990+0000 7f6e66ffd640 1 -- 172.21.15.112:0/1623751089 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f6e60042070 con 0x7f6e701094b0 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.990+0000 7f6e7e780740 1 Processor -- start 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.990+0000 7f6e7e780740 1 -- start start 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.991+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 0x557155986270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.991+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 0x5571559867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.991+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555be390 0x557155986cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.991+0000 7f6e7e780740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5571558fce50 con 0x5571555be390 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.991+0000 7f6e7e780740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5571558ec280 con 0x5571555b61b0 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.991+0000 7f6e7e780740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5571558f0090 con 0x5571555df070 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.991+0000 7f6e757a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555be390 0x557155986cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.991+0000 7f6e757a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555be390 0x557155986cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46310/0 (socket says 172.21.15.112:46310) 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.991+0000 7f6e757a7640 1 -- 172.21.15.112:0/1431615372 learned_addr learned my addr 172.21.15.112:0/1431615372 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:38.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.991+0000 7f6e767a9640 1 --2- 172.21.15.112:0/1431615372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 0x557155986270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:38.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.991+0000 7f6e75fa8640 1 --2- 172.21.15.112:0/1431615372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 0x5571559867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:38.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e757a7640 1 -- 172.21.15.112:0/1431615372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 msgr2=0x557155986270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:38.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e757a7640 1 --2- 172.21.15.112:0/1431615372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 0x557155986270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e757a7640 1 -- 172.21.15.112:0/1431615372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 msgr2=0x5571559867b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:38.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e757a7640 1 --2- 172.21.15.112:0/1431615372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 0x5571559867b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e757a7640 1 -- 172.21.15.112:0/1431615372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715564a650 con 0x5571555be390 2024-09-21T03:29:38.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e767a9640 1 --2- 172.21.15.112:0/1431615372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 0x557155986270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:38.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e66ffd640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e5c0b66b0 0x7f6e5c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:38.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e75fa8640 1 --2- 172.21.15.112:0/1431615372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 0x5571559867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:38.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e767a9640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e5c0b66b0 0x7f6e5c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:38.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e66ffd640 1 -- 172.21.15.112:0/1623751089 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f6e60107a20 con 0x7f6e701094b0 2024-09-21T03:29:38.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e757a7640 1 --2- 172.21.15.112:0/1431615372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555be390 0x557155986cf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6e60105140 tx=0x7f6e60091a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97310555ca13438 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:38.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.992+0000 7f6e767a9640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e5c0b66b0 0x7f6e5c0b8bc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009fd0 tx=0x7f6e68009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:38.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.993+0000 7f6e5bfff640 1 -- 172.21.15.112:0/1431615372 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e600cd750 con 0x5571555be390 2024-09-21T03:29:38.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.993+0000 7f6e5bfff640 1 -- 172.21.15.112:0/1431615372 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e60110070 con 0x5571555be390 2024-09-21T03:29:38.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.993+0000 7f6e5bfff640 1 -- 172.21.15.112:0/1431615372 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e600cd0f0 con 0x5571555be390 2024-09-21T03:29:38.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.994+0000 7f6e7e780740 1 -- 172.21.15.112:0/1431615372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555be390 msgr2=0x557155986cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:38.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.994+0000 7f6e7e780740 1 --2- 172.21.15.112:0/1431615372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555be390 0x557155986cf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6e60105140 tx=0x7f6e60091a70 comp rx=0 tx=0).stop 2024-09-21T03:29:38.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.994+0000 7f6e7e780740 1 -- 172.21.15.112:0/1431615372 shutdown_connections 2024-09-21T03:29:38.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.994+0000 7f6e7e780740 1 --2- 172.21.15.112:0/1431615372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555be390 0x557155986cf0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.994+0000 7f6e7e780740 1 --2- 172.21.15.112:0/1431615372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 0x5571559867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.994+0000 7f6e7e780740 1 --2- 172.21.15.112:0/1431615372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 0x557155986270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:38.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.995+0000 7f6e7e780740 1 -- 172.21.15.112:0/1431615372 >> 172.21.15.112:0/1431615372 conn(0x5571555d3460 msgr2=0x557155984160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:38.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.995+0000 7f6e7e780740 1 -- 172.21.15.112:0/1431615372 shutdown_connections 2024-09-21T03:29:38.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.995+0000 7f6e7e780740 1 -- 172.21.15.112:0/1431615372 wait complete. 2024-09-21T03:29:38.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.995+0000 7f6e7e780740 1 -- start start 2024-09-21T03:29:38.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.995+0000 7f6e7e780740 1 Processor -- start 2024-09-21T03:29:38.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.996+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 0x5571559919c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:39.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.996+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555be390 0x557155991f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:39.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.996+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 0x55715599d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:39.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.996+0000 7f6e75fa8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 0x5571559919c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:39.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.996+0000 7f6e767a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 0x55715599d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:39.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.996+0000 7f6e75fa8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 0x5571559919c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:59276/0 (socket says 172.21.15.112:59276) 2024-09-21T03:29:39.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.996+0000 7f6e767a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 0x55715599d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46326/0 (socket says 172.21.15.112:46326) 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e75fa8640 1 -- 172.21.15.112:0/2427054015 learned_addr learned my addr 172.21.15.112:0/2427054015 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e7e780740 1 -- 172.21.15.112:0/2427054015 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55715574d700 con 0x5571555df070 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e7e780740 1 -- 172.21.15.112:0/2427054015 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557155887110 con 0x5571555b61b0 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e757a7640 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555be390 0x557155991f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e7e780740 1 -- 172.21.15.112:0/2427054015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5571558fce50 con 0x5571555be390 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e75fa8640 1 -- 172.21.15.112:0/2427054015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555be390 msgr2=0x557155991f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e75fa8640 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555be390 0x557155991f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e75fa8640 1 -- 172.21.15.112:0/2427054015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 msgr2=0x55715599d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e75fa8640 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 0x55715599d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e75fa8640 1 -- 172.21.15.112:0/2427054015 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571558b4c40 con 0x5571555b61b0 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e767a9640 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 0x55715599d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e7e780740 10 client.0 _ll_register_callbacks cb 0x5571558a0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e757a7640 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555be390 0x557155991f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e7e780740 1 -- 172.21.15.112:0/2427054015 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e60110070 con 0x5571555b61b0 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e75fa8640 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 0x5571559919c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00ba90 tx=0x7f6e6c00bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96d51762a49396e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.997+0000 7f6e597fa640 10 client.25202 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e597fa640 1 -- 172.21.15.112:0/2427054015 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c005220 con 0x5571555b61b0 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e597fa640 1 -- 172.21.15.112:0/2427054015 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f6e6c0053c0 con 0x5571555b61b0 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e597fa640 1 -- 172.21.15.112:0/2427054015 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6e6c005600 con 0x5571555b61b0 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e7e780740 10 client.25202 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e7e780740 1 -- 172.21.15.112:0/2427054015 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5571559a02e0 con 0x5571555b61b0 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e597fa640 1 -- 172.21.15.112:0/2427054015 <== mon.2 v2:172.21.15.187:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6e6c0047a0 con 0x5571555b61b0 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e7e780740 10 client.25202 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e7e780740 10 client.25202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e7e780740 20 client.25202 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e7e780740 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55715559d9f0 0x5571559a3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e7e780740 4 client.25202 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.998+0000 7f6e7e780740 1 -- 172.21.15.112:0/2427054015 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571558feda0 con 0x55715559d9f0 2024-09-21T03:29:39.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.999+0000 7f6e757a7640 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55715559d9f0 0x5571559a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.999+0000 7f6e597fa640 10 client.25202 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:38.999+0000 7f6e757a7640 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55715559d9f0 0x5571559a3750 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f043b8241ae73b2 server_cookie=e5ee22d952aedbb6 in_seq=0 out_seq=0 2024-09-21T03:29:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e597fa640 1 -- 172.21.15.112:0/2427054015 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5571558feda0 con 0x55715559d9f0 2024-09-21T03:29:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e597fa640 10 client.25202 handle_command_reply: tid=0 2024-09-21T03:29:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e7e780740 1 client.25202 shutdown 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e7e780740 1 -- 172.21.15.112:0/2427054015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 msgr2=0x5571559919c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e7e780740 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 0x5571559919c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00ba90 tx=0x7f6e6c00bf50 comp rx=0 tx=0).stop 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e7e780740 1 -- 172.21.15.112:0/2427054015 shutdown_connections 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e7e780740 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55715559d9f0 0x5571559a3750 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e7e780740 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 0x55715599d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e7e780740 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555be390 0x557155991f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e597fa640 0 client.25202 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e7e780740 1 --2- 172.21.15.112:0/2427054015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555b61b0 0x5571559919c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.000+0000 7f6e7e780740 1 -- 172.21.15.112:0/2427054015 >> 172.21.15.112:0/2427054015 conn(0x5571555d3460 msgr2=0x557155989400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.001+0000 7f6e7e780740 1 -- 172.21.15.112:0/2427054015 shutdown_connections 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.001+0000 7f6e7e780740 1 -- 172.21.15.112:0/2427054015 wait complete. 2024-09-21T03:29:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.001+0000 7f6e7e780740 20 client.25202 trim_cache size 0 max 16384 2024-09-21T03:29:39.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.037+0000 7f6e7e780740 1 Processor -- start 2024-09-21T03:29:39.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.037+0000 7f6e7e780740 1 -- start start 2024-09-21T03:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.037+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555b61b0 0x5571559879b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.037+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555be390 0x55715598ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.037+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 0x55715598f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.037+0000 7f6e7e780740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e701b3f30 con 0x5571555df070 2024-09-21T03:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.037+0000 7f6e7e780740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e701b1800 con 0x5571555be390 2024-09-21T03:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.037+0000 7f6e7e780740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6e701af650 con 0x5571555b61b0 2024-09-21T03:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e767a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 0x55715598f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e75fa8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555b61b0 0x5571559879b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e75fa8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555b61b0 0x5571559879b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:38184/0 (socket says 172.21.15.112:38184) 2024-09-21T03:29:39.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e75fa8640 1 -- 172.21.15.112:0/3125001185 learned_addr learned my addr 172.21.15.112:0/3125001185 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e767a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 0x55715598f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46340/0 (socket says 172.21.15.112:46340) 2024-09-21T03:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e757a7640 1 --2- 172.21.15.112:0/3125001185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555be390 0x55715598ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e75fa8640 1 -- 172.21.15.112:0/3125001185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555be390 msgr2=0x55715598ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e75fa8640 1 --2- 172.21.15.112:0/3125001185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555be390 0x55715598ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e75fa8640 1 -- 172.21.15.112:0/3125001185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 msgr2=0x55715598f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e75fa8640 1 --2- 172.21.15.112:0/3125001185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 0x55715598f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e75fa8640 1 -- 172.21.15.112:0/3125001185 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715598fec0 con 0x5571555b61b0 2024-09-21T03:29:39.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.038+0000 7f6e75fa8640 1 --2- 172.21.15.112:0/3125001185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555b61b0 0x5571559879b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c005c20 tx=0x7f6e6c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=159df8a215e51ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.039+0000 7f6e5b7fe640 1 -- 172.21.15.112:0/3125001185 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c02b040 con 0x5571555b61b0 2024-09-21T03:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.039+0000 7f6e5b7fe640 1 -- 172.21.15.112:0/3125001185 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e60110070 con 0x5571555b61b0 2024-09-21T03:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.039+0000 7f6e5b7fe640 1 -- 172.21.15.112:0/3125001185 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c041ba0 con 0x5571555b61b0 2024-09-21T03:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.039+0000 7f6e7e780740 1 -- 172.21.15.112:0/3125001185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555b61b0 msgr2=0x5571559879b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.039+0000 7f6e7e780740 1 --2- 172.21.15.112:0/3125001185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555b61b0 0x5571559879b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c005c20 tx=0x7f6e6c014000 comp rx=0 tx=0).stop 2024-09-21T03:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.040+0000 7f6e7e780740 1 -- 172.21.15.112:0/3125001185 shutdown_connections 2024-09-21T03:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.040+0000 7f6e7e780740 1 --2- 172.21.15.112:0/3125001185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555df070 0x55715598f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.040+0000 7f6e7e780740 1 --2- 172.21.15.112:0/3125001185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555be390 0x55715598ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.040+0000 7f6e7e780740 1 --2- 172.21.15.112:0/3125001185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555b61b0 0x5571559879b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.040+0000 7f6e7e780740 1 -- 172.21.15.112:0/3125001185 >> 172.21.15.112:0/3125001185 conn(0x5571555d3460 msgr2=0x5571559872a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:39.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.040+0000 7f6e7e780740 1 -- 172.21.15.112:0/3125001185 shutdown_connections 2024-09-21T03:29:39.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.040+0000 7f6e7e780740 1 -- 172.21.15.112:0/3125001185 wait complete. 2024-09-21T03:29:39.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.040+0000 7f6e7e780740 1 -- start start 2024-09-21T03:29:39.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.040+0000 7f6e7e780740 1 Processor -- start 2024-09-21T03:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555b61b0 0x557155997b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555be390 0x5571559980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e7e780740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 0x557155a0d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e7e780740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e701afcc0 con 0x5571555b61b0 2024-09-21T03:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e7e780740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e701b1130 con 0x5571555be390 2024-09-21T03:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e7e780740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6e701b0390 con 0x5571555df070 2024-09-21T03:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e7e780740 10 client.0 _ll_register_callbacks cb 0x7f6e6c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e75fa8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555b61b0 0x557155997b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:39.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e75fa8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555b61b0 0x557155997b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46348/0 (socket says 172.21.15.112:46348) 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e75fa8640 1 -- 172.21.15.112:0/4262689921 learned_addr learned my addr 172.21.15.112:0/4262689921 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e757a7640 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555be390 0x5571559980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.041+0000 7f6e767a9640 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 0x557155a0d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e767a9640 1 -- 172.21.15.112:0/4262689921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555be390 msgr2=0x5571559980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e767a9640 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555be390 0x5571559980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e767a9640 1 -- 172.21.15.112:0/4262689921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555b61b0 msgr2=0x557155997b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e767a9640 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555b61b0 0x557155997b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e767a9640 1 -- 172.21.15.112:0/4262689921 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571559988d0 con 0x5571555df070 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e75fa8640 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555b61b0 0x557155997b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e757a7640 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555be390 0x5571559980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e767a9640 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 0x557155a0d120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55715599e8f0 tx=0x7f6e68004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4015767601d3e871 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e597fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e597fa640 1 -- 172.21.15.112:0/4262689921 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6802f060 con 0x5571555df070 2024-09-21T03:29:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e597fa640 1 -- 172.21.15.112:0/4262689921 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f6e68011de0 con 0x5571555df070 2024-09-21T03:29:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.042+0000 7f6e7e780740 1 -- 172.21.15.112:0/4262689921 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e60110070 con 0x5571555df070 2024-09-21T03:29:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e597fa640 1 -- 172.21.15.112:0/4262689921 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6e68034450 con 0x5571555df070 2024-09-21T03:29:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e7e780740 10 client.25228 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e7e780740 1 -- 172.21.15.112:0/4262689921 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557155a0e6a0 con 0x5571555df070 2024-09-21T03:29:39.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e597fa640 1 -- 172.21.15.112:0/4262689921 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e60110070 con 0x5571555df070 2024-09-21T03:29:39.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e597fa640 1 -- 172.21.15.112:0/4262689921 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6e6801e510 con 0x5571555df070 2024-09-21T03:29:39.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e7e780740 10 client.25228 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:39.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e7e780740 10 client.25228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e7e780740 20 client.25228 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e7e780740 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x557155572da0 0x557155a10e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e7e780740 4 client.25228 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e7e780740 1 -- 172.21.15.112:0/4262689921 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5571558fa850 con 0x557155572da0 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.043+0000 7f6e75fa8640 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x557155572da0 0x557155a10e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.044+0000 7f6e597fa640 10 client.25228 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.044+0000 7f6e75fa8640 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x557155572da0 0x557155a10e30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f83b91663ca5592 server_cookie=711317b2f4a72ef9 in_seq=0 out_seq=0 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.044+0000 7f6e597fa640 1 -- 172.21.15.112:0/4262689921 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x5571558fa850 con 0x557155572da0 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.044+0000 7f6e597fa640 10 client.25228 handle_command_reply: tid=0 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.044+0000 7f6e7e780740 1 client.25228 shutdown 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e7e780740 1 -- 172.21.15.112:0/4262689921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 msgr2=0x557155a0d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e7e780740 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 0x557155a0d120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55715599e8f0 tx=0x7f6e68004a40 comp rx=0 tx=0).stop 2024-09-21T03:29:39.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e7e780740 1 -- 172.21.15.112:0/4262689921 shutdown_connections 2024-09-21T03:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e7e780740 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x557155572da0 0x557155a10e30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e7e780740 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571555df070 0x557155a0d120 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e7e780740 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571555be390 0x5571559980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e597fa640 0 client.25228 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e7e780740 1 --2- 172.21.15.112:0/4262689921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5571555b61b0 0x557155997b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e7e780740 1 -- 172.21.15.112:0/4262689921 >> 172.21.15.112:0/4262689921 conn(0x5571555d3460 msgr2=0x557155987c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e7e780740 1 -- 172.21.15.112:0/4262689921 shutdown_connections 2024-09-21T03:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e7e780740 1 -- 172.21.15.112:0/4262689921 wait complete. 2024-09-21T03:29:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.045+0000 7f6e7e780740 20 client.25228 trim_cache size 0 max 16384 2024-09-21T03:29:39.049 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:29:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:29:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:29:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:29:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:29:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:29:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:29:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:29:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:29:39.051 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.046+0000 7f6e7820c640 1 -- 172.21.15.112:0/1623751089 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e5c0b66b0 msgr2=0x7f6e5c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.046+0000 7f6e7820c640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e5c0b66b0 0x7f6e5c0b8bc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009fd0 tx=0x7f6e68009500 comp rx=0 tx=0).stop 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.046+0000 7f6e7820c640 1 -- 172.21.15.112:0/1623751089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e701094b0 msgr2=0x7f6e70075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.046+0000 7f6e7820c640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e701094b0 0x7f6e70075c50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009230 tx=0x7f6e60003be0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.046+0000 7f6e7820c640 1 -- 172.21.15.112:0/1623751089 shutdown_connections 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.046+0000 7f6e7820c640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6e5c0b66b0 0x7f6e5c0b8bc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.046+0000 7f6e7820c640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e70109e00 0x7f6e70076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.046+0000 7f6e7820c640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e701094b0 0x7f6e70075c50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.046+0000 7f6e7820c640 1 --2- 172.21.15.112:0/1623751089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e70103bc0 0x7f6e70075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.046+0000 7f6e7820c640 1 -- 172.21.15.112:0/1623751089 >> 172.21.15.112:0/1623751089 conn(0x7f6e7007b910 msgr2=0x7f6e70107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.046+0000 7f6e7820c640 1 -- 172.21.15.112:0/1623751089 shutdown_connections 2024-09-21T03:29:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:39.047+0000 7f6e7820c640 1 -- 172.21.15.112:0/1623751089 wait complete. 2024-09-21T03:29:39.753 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:39 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:29:39.793 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:29:39.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:29:39.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:39 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:29:39.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:39 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:29:40.072 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:40.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:40 smithi112 ceph-mon[31442]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T03:29:40.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.635+0000 7fc793e32640 1 -- 172.21.15.112:0/1699800785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc78c1083a0 msgr2=0x7fc78c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:40.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.635+0000 7fc793e32640 1 --2- 172.21.15.112:0/1699800785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc78c1083a0 0x7fc78c1087b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009900 tx=0x7fc77c02a410 comp rx=0 tx=0).stop 2024-09-21T03:29:40.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.635+0000 7fc793e32640 1 -- 172.21.15.112:0/1699800785 shutdown_connections 2024-09-21T03:29:40.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.635+0000 7fc793e32640 1 --2- 172.21.15.112:0/1699800785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc78c108cf0 0x7fc78c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.635+0000 7fc793e32640 1 --2- 172.21.15.112:0/1699800785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc78c1083a0 0x7fc78c1087b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.635+0000 7fc793e32640 1 --2- 172.21.15.112:0/1699800785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78c102a60 0x7fc78c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.635+0000 7fc793e32640 1 -- 172.21.15.112:0/1699800785 >> 172.21.15.112:0/1699800785 conn(0x7fc78c079730 msgr2=0x7fc78c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:40.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.636+0000 7fc793e32640 1 -- 172.21.15.112:0/1699800785 shutdown_connections 2024-09-21T03:29:40.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.636+0000 7fc793e32640 1 -- 172.21.15.112:0/1699800785 wait complete. 2024-09-21T03:29:40.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc793e32640 1 Processor -- start 2024-09-21T03:29:40.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc793e32640 1 -- start start 2024-09-21T03:29:40.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc793e32640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc78c102a60 0x7fc78c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:40.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc793e32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc78c1083a0 0x7fc78c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:40.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc793e32640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78c108cf0 0x7fc78c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc793e32640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c112960 con 0x7fc78c1083a0 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc793e32640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c1127e0 con 0x7fc78c108cf0 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc793e32640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c112ae0 con 0x7fc78c102a60 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc791bce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc78c102a60 0x7fc78c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc7913cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc78c1083a0 0x7fc78c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc791bce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc78c102a60 0x7fc78c1990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:38210/0 (socket says 172.21.15.112:38210) 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.637+0000 7fc791bce640 1 -- 172.21.15.112:0/3329629241 learned_addr learned my addr 172.21.15.112:0/3329629241 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.638+0000 7fc7923cf640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78c108cf0 0x7fc78c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.638+0000 7fc791bce640 1 -- 172.21.15.112:0/3329629241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78c108cf0 msgr2=0x7fc78c199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.638+0000 7fc791bce640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78c108cf0 0x7fc78c199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.638+0000 7fc791bce640 1 -- 172.21.15.112:0/3329629241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc78c1083a0 msgr2=0x7fc78c199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.638+0000 7fc791bce640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc78c1083a0 0x7fc78c199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.638+0000 7fc791bce640 1 -- 172.21.15.112:0/3329629241 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c1a8790 con 0x7fc78c102a60 2024-09-21T03:29:40.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.638+0000 7fc791bce640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc78c102a60 0x7fc78c1990e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc78800e9f0 tx=0x7fc78800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:40.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.638+0000 7fc782ffd640 1 -- 172.21.15.112:0/3329629241 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78800cd80 con 0x7fc78c102a60 2024-09-21T03:29:40.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.638+0000 7fc793e32640 1 -- 172.21.15.112:0/3329629241 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c1a8a80 con 0x7fc78c102a60 2024-09-21T03:29:40.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.639+0000 7fc793e32640 1 -- 172.21.15.112:0/3329629241 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c1a8fe0 con 0x7fc78c102a60 2024-09-21T03:29:40.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.639+0000 7fc782ffd640 1 -- 172.21.15.112:0/3329629241 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7880108e0 con 0x7fc78c102a60 2024-09-21T03:29:40.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.639+0000 7fc782ffd640 1 -- 172.21.15.112:0/3329629241 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788018860 con 0x7fc78c102a60 2024-09-21T03:29:40.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.640+0000 7fc793e32640 1 -- 172.21.15.112:0/3329629241 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7700052c0 con 0x7fc78c102a60 2024-09-21T03:29:40.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.642+0000 7fc782ffd640 1 -- 172.21.15.112:0/3329629241 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc788010430 con 0x7fc78c102a60 2024-09-21T03:29:40.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.643+0000 7fc782ffd640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc7780b6700 0x7fc7780b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:40.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.643+0000 7fc782ffd640 1 -- 172.21.15.112:0/3329629241 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc7880e7860 con 0x7fc78c102a60 2024-09-21T03:29:40.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.643+0000 7fc7913cd640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc7780b6700 0x7fc7780b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:40.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.644+0000 7fc7913cd640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc7780b6700 0x7fc7780b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:40.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.648+0000 7fc782ffd640 1 -- 172.21.15.112:0/3329629241 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7880ade10 con 0x7fc78c102a60 2024-09-21T03:29:40.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:40 smithi203 ceph-mon[34723]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T03:29:40.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:40 smithi187 ceph-mon[34603]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-21T03:29:40.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.969+0000 7fc793e32640 1 -- 172.21.15.112:0/3329629241 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc770005880 con 0x7fc78c102a60 2024-09-21T03:29:40.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.970+0000 7fc782ffd640 1 -- 172.21.15.112:0/3329629241 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11929 (secure 0 0 0) 0x7fc7880b21f0 con 0x7fc78c102a60 2024-09-21T03:29:40.972 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:29:40.972 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"btime":"2024-09-21T03:28:46:418112+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:28:46.048810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:29:40.972 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-09-21T03:29:40.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 -- 172.21.15.112:0/3329629241 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc7780b6700 msgr2=0x7fc7780b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc7780b6700 0x7fc7780b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c031400 comp rx=0 tx=0).stop 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 -- 172.21.15.112:0/3329629241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc78c102a60 msgr2=0x7fc78c1990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc78c102a60 0x7fc78c1990e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc78800e9f0 tx=0x7fc78800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 -- 172.21.15.112:0/3329629241 shutdown_connections 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc7780b6700 0x7fc7780b8c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78c108cf0 0x7fc78c199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc78c1083a0 0x7fc78c199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 --2- 172.21.15.112:0/3329629241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc78c102a60 0x7fc78c1990e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 -- 172.21.15.112:0/3329629241 >> 172.21.15.112:0/3329629241 conn(0x7fc78c079730 msgr2=0x7fc78c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:40.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 -- 172.21.15.112:0/3329629241 shutdown_connections 2024-09-21T03:29:40.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:40.973+0000 7fc793e32640 1 -- 172.21.15.112:0/3329629241 wait complete. 2024-09-21T03:29:41.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:41 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3329629241' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:41.668 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:29:41.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:41 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3329629241' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:41.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:41 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3329629241' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:42.044 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:42.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.578+0000 7efda74b4640 1 -- 172.21.15.112:0/3149039346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda0102090 msgr2=0x7efda01024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.578+0000 7efda74b4640 1 --2- 172.21.15.112:0/3149039346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda0102090 0x7efda01024a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009900 tx=0x7efd9c02a410 comp rx=0 tx=0).stop 2024-09-21T03:29:42.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.578+0000 7efda74b4640 1 -- 172.21.15.112:0/3149039346 shutdown_connections 2024-09-21T03:29:42.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.578+0000 7efda74b4640 1 --2- 172.21.15.112:0/3149039346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efda0106bd0 0x7efda0110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.578+0000 7efda74b4640 1 --2- 172.21.15.112:0/3149039346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda0106280 0x7efda0106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.578+0000 7efda74b4640 1 --2- 172.21.15.112:0/3149039346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda0102090 0x7efda01024a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.578+0000 7efda74b4640 1 -- 172.21.15.112:0/3149039346 >> 172.21.15.112:0/3149039346 conn(0x7efda0077c60 msgr2=0x7efda00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:42.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.578+0000 7efda74b4640 1 -- 172.21.15.112:0/3149039346 shutdown_connections 2024-09-21T03:29:42.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.578+0000 7efda74b4640 1 -- 172.21.15.112:0/3149039346 wait complete. 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.579+0000 7efda74b4640 1 Processor -- start 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.579+0000 7efda74b4640 1 -- start start 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.579+0000 7efda74b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda0102090 0x7efda0103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.579+0000 7efda74b4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efda0106280 0x7efda0103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.579+0000 7efda74b4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda0106bd0 0x7efda0103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.579+0000 7efda74b4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efda0113ac0 con 0x7efda0102090 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.579+0000 7efda74b4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efda0113940 con 0x7efda0106bd0 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.579+0000 7efda74b4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efda0113c40 con 0x7efda0106280 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.580+0000 7efda5250640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda0102090 0x7efda0103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.580+0000 7efda5250640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda0102090 0x7efda0103030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46372/0 (socket says 172.21.15.112:46372) 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.580+0000 7efda5250640 1 -- 172.21.15.112:0/884554472 learned_addr learned my addr 172.21.15.112:0/884554472 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.580+0000 7efda4a4f640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efda0106280 0x7efda0103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.580+0000 7efda5a51640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda0106bd0 0x7efda0103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.580+0000 7efda5a51640 1 -- 172.21.15.112:0/884554472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efda0106280 msgr2=0x7efda0103570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.580+0000 7efda5a51640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efda0106280 0x7efda0103570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.580+0000 7efda5a51640 1 -- 172.21.15.112:0/884554472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda0102090 msgr2=0x7efda0103030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.580+0000 7efda5a51640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda0102090 0x7efda0103030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.580+0000 7efda5a51640 1 -- 172.21.15.112:0/884554472 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda01a8810 con 0x7efda0106bd0 2024-09-21T03:29:42.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.580+0000 7efda5250640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda0102090 0x7efda0103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:42.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.581+0000 7efda5a51640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda0106bd0 0x7efda0103ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efd9800e940 tx=0x7efd9800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:42.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.581+0000 7efd967fc640 1 -- 172.21.15.112:0/884554472 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98004d20 con 0x7efda0106bd0 2024-09-21T03:29:42.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.581+0000 7efd967fc640 1 -- 172.21.15.112:0/884554472 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd98005020 con 0x7efda0106bd0 2024-09-21T03:29:42.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.581+0000 7efda74b4640 1 -- 172.21.15.112:0/884554472 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda01a8b00 con 0x7efda0106bd0 2024-09-21T03:29:42.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.581+0000 7efd967fc640 1 -- 172.21.15.112:0/884554472 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd980116d0 con 0x7efda0106bd0 2024-09-21T03:29:42.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.581+0000 7efda74b4640 1 -- 172.21.15.112:0/884554472 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda01a9030 con 0x7efda0106bd0 2024-09-21T03:29:42.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.585+0000 7efdada28740 1 Processor -- start 2024-09-21T03:29:42.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.585+0000 7efd967fc640 1 -- 172.21.15.112:0/884554472 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7efd98005460 con 0x7efda0106bd0 2024-09-21T03:29:42.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.586+0000 7efdada28740 1 -- start start 2024-09-21T03:29:42.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.586+0000 7efdada28740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a58070 0x5629a7dff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.586+0000 7efdada28740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a2f1b0 0x5629a7dffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.586+0000 7efdada28740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a37390 0x5629a7e00120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.586+0000 7efdada28740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5629a7d75ea0 con 0x5629a7a58070 2024-09-21T03:29:42.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.586+0000 7efdada28740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5629a7d652d0 con 0x5629a7a37390 2024-09-21T03:29:42.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.586+0000 7efdada28740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5629a7d690e0 con 0x5629a7a2f1b0 2024-09-21T03:29:42.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.586+0000 7efda4a4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a58070 0x5629a7dff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.586+0000 7efda4a4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a58070 0x5629a7dff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46388/0 (socket says 172.21.15.112:46388) 2024-09-21T03:29:42.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.586+0000 7efda4a4f640 1 -- 172.21.15.112:0/3406128304 learned_addr learned my addr 172.21.15.112:0/3406128304 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:42.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.586+0000 7efda5250640 1 --2- 172.21.15.112:0/3406128304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a2f1b0 0x5629a7dffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.587+0000 7efda4a4f640 1 -- 172.21.15.112:0/3406128304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a2f1b0 msgr2=0x5629a7dffbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:42.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.587+0000 7efda4a4f640 1 --2- 172.21.15.112:0/3406128304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a2f1b0 0x5629a7dffbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.587+0000 7efda4a4f640 1 -- 172.21.15.112:0/3406128304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a37390 msgr2=0x5629a7e00120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.587+0000 7efda4a4f640 1 --2- 172.21.15.112:0/3406128304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a37390 0x5629a7e00120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.587+0000 7efda4a4f640 1 -- 172.21.15.112:0/3406128304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a7ac3650 con 0x5629a7a58070 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.587+0000 7efda5250640 1 --2- 172.21.15.112:0/3406128304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a2f1b0 0x5629a7dffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.587+0000 7efda4a4f640 1 --2- 172.21.15.112:0/3406128304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a58070 0x5629a7dff6a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efd9000bc00 tx=0x7efd900103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c170abfa944864 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.587+0000 7efd967fc640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd8c0b6700 0x7efd8c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.588+0000 7efd967fc640 1 -- 172.21.15.112:0/884554472 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7efd980e7960 con 0x7efda0106bd0 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.588+0000 7efda5250640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd8c0b6700 0x7efd8c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.588+0000 7efda5250640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd8c0b6700 0x7efd8c0b8c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5629a7dfe320 tx=0x7efd9c0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.589+0000 7efd8b7fe640 1 -- 172.21.15.112:0/3406128304 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90010e00 con 0x5629a7a58070 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.589+0000 7efd8b7fe640 1 -- 172.21.15.112:0/3406128304 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9001c930 con 0x5629a7a58070 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.589+0000 7efd8b7fe640 1 -- 172.21.15.112:0/3406128304 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90011aa0 con 0x5629a7a58070 2024-09-21T03:29:42.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.590+0000 7efdada28740 1 -- 172.21.15.112:0/3406128304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a58070 msgr2=0x5629a7dff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.590+0000 7efdada28740 1 --2- 172.21.15.112:0/3406128304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a58070 0x5629a7dff6a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efd9000bc00 tx=0x7efd900103d0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.590+0000 7efdada28740 1 -- 172.21.15.112:0/3406128304 shutdown_connections 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.590+0000 7efdada28740 1 --2- 172.21.15.112:0/3406128304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a37390 0x5629a7e00120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.590+0000 7efdada28740 1 --2- 172.21.15.112:0/3406128304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a2f1b0 0x5629a7dffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.590+0000 7efdada28740 1 --2- 172.21.15.112:0/3406128304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a58070 0x5629a7dff6a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.590+0000 7efdada28740 1 -- 172.21.15.112:0/3406128304 >> 172.21.15.112:0/3406128304 conn(0x5629a7a4c460 msgr2=0x5629a7dfd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:42.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.590+0000 7efdada28740 1 -- 172.21.15.112:0/3406128304 shutdown_connections 2024-09-21T03:29:42.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.590+0000 7efdada28740 1 -- 172.21.15.112:0/3406128304 wait complete. 2024-09-21T03:29:42.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.591+0000 7efdada28740 1 -- start start 2024-09-21T03:29:42.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.591+0000 7efdada28740 1 Processor -- start 2024-09-21T03:29:42.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.591+0000 7efdada28740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a2f1b0 0x5629a7e0ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.591+0000 7efdada28740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a37390 0x5629a7e0b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.591+0000 7efdada28740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a58070 0x5629a7e0b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efda4a4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a37390 0x5629a7e0b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efda5250640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a2f1b0 0x5629a7e0ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efda4a4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a37390 0x5629a7e0b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46392/0 (socket says 172.21.15.112:46392) 2024-09-21T03:29:42.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efda4a4f640 1 -- 172.21.15.112:0/2678904798 learned_addr learned my addr 172.21.15.112:0/2678904798 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:42.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efda5a51640 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a58070 0x5629a7e0b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efdada28740 1 -- 172.21.15.112:0/2678904798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5629a7bc6700 con 0x5629a7a37390 2024-09-21T03:29:42.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efdada28740 1 -- 172.21.15.112:0/2678904798 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5629a7d00160 con 0x5629a7a2f1b0 2024-09-21T03:29:42.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efdada28740 1 -- 172.21.15.112:0/2678904798 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5629a7e0de80 con 0x5629a7a58070 2024-09-21T03:29:42.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efda4a4f640 1 -- 172.21.15.112:0/2678904798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a58070 msgr2=0x5629a7e0b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:42.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efda4a4f640 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a58070 0x5629a7e0b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efda4a4f640 1 -- 172.21.15.112:0/2678904798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a2f1b0 msgr2=0x5629a7e0ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:42.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efda4a4f640 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a2f1b0 0x5629a7e0ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efda4a4f640 1 -- 172.21.15.112:0/2678904798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a7d2dc90 con 0x5629a7a37390 2024-09-21T03:29:42.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efdada28740 10 client.0 _ll_register_callbacks cb 0x5629a7d199e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:42.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.592+0000 7efdada28740 1 -- 172.21.15.112:0/2678904798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9001c930 con 0x5629a7a37390 2024-09-21T03:29:42.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.593+0000 7efda4a4f640 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a37390 0x5629a7e0b280 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efd90011580 tx=0x7efd90010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e3805e6675e036 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:42.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.593+0000 7efd88ff9640 10 client.15636 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:42.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.593+0000 7efd88ff9640 1 -- 172.21.15.112:0/2678904798 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9002a060 con 0x5629a7a37390 2024-09-21T03:29:42.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.593+0000 7efd88ff9640 1 -- 172.21.15.112:0/2678904798 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7efd9001cce0 con 0x5629a7a37390 2024-09-21T03:29:42.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.593+0000 7efd88ff9640 1 -- 172.21.15.112:0/2678904798 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efd9001c790 con 0x5629a7a37390 2024-09-21T03:29:42.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.593+0000 7efdada28740 10 client.15636 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:42.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.593+0000 7efdada28740 1 -- 172.21.15.112:0/2678904798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5629a7e19940 con 0x5629a7a37390 2024-09-21T03:29:42.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.594+0000 7efd88ff9640 1 -- 172.21.15.112:0/2678904798 <== mon.0 v2:172.21.15.112:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efd90011820 con 0x5629a7a37390 2024-09-21T03:29:42.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.594+0000 7efdada28740 10 client.15636 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:42.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.594+0000 7efdada28740 10 client.15636 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:29:42.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.594+0000 7efdada28740 20 client.15636 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:42.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.594+0000 7efdada28740 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5629a7a169f0 0x5629a7e1d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.594+0000 7efdada28740 4 client.15636 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:42.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.594+0000 7efdada28740 1 -- 172.21.15.112:0/2678904798 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629a7d77df0 con 0x5629a7a169f0 2024-09-21T03:29:42.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.594+0000 7efda5250640 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5629a7a169f0 0x5629a7e1d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.594+0000 7efda5250640 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5629a7a169f0 0x5629a7e1d1b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d9fcb92555e367b server_cookie=7db4739cad548b92 in_seq=0 out_seq=0 2024-09-21T03:29:42.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.594+0000 7efd88ff9640 10 client.15636 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:42.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.595+0000 7efd88ff9640 1 -- 172.21.15.112:0/2678904798 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5629a7d77df0 con 0x5629a7a169f0 2024-09-21T03:29:42.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.595+0000 7efd88ff9640 10 client.15636 handle_command_reply: tid=0 2024-09-21T03:29:42.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.595+0000 7efdada28740 1 client.15636 shutdown 2024-09-21T03:29:42.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.595+0000 7efdada28740 1 -- 172.21.15.112:0/2678904798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a37390 msgr2=0x5629a7e0b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.595+0000 7efdada28740 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a37390 0x5629a7e0b280 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efd90011580 tx=0x7efd90010eb0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.596+0000 7efdada28740 1 -- 172.21.15.112:0/2678904798 shutdown_connections 2024-09-21T03:29:42.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.596+0000 7efdada28740 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5629a7a169f0 0x5629a7e1d1b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.596+0000 7efdada28740 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a58070 0x5629a7e0b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.596+0000 7efd88ff9640 0 client.15636 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:42.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.596+0000 7efdada28740 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a37390 0x5629a7e0b280 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.596+0000 7efdada28740 1 --2- 172.21.15.112:0/2678904798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a2f1b0 0x5629a7e0ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.596+0000 7efdada28740 1 -- 172.21.15.112:0/2678904798 >> 172.21.15.112:0/2678904798 conn(0x5629a7a4c460 msgr2=0x5629a7e02880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:42.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.596+0000 7efdada28740 1 -- 172.21.15.112:0/2678904798 shutdown_connections 2024-09-21T03:29:42.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.596+0000 7efdada28740 1 -- 172.21.15.112:0/2678904798 wait complete. 2024-09-21T03:29:42.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.596+0000 7efdada28740 20 client.15636 trim_cache size 0 max 16384 2024-09-21T03:29:42.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.609+0000 7efdada28740 1 Processor -- start 2024-09-21T03:29:42.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.609+0000 7efdada28740 1 -- start start 2024-09-21T03:29:42.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efdada28740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 0x5629a7e0cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efdada28740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a2f1b0 0x5629a7e07f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efdada28740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a37390 0x5629a7e08470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efdada28740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efda01b41a0 con 0x5629a7a169f0 2024-09-21T03:29:42.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efdada28740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efda01b1a70 con 0x5629a7a37390 2024-09-21T03:29:42.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efdada28740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efda01af8c0 con 0x5629a7a2f1b0 2024-09-21T03:29:42.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efda5a51640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a37390 0x5629a7e08470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efda4a4f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a2f1b0 0x5629a7e07f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efda5a51640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a37390 0x5629a7e08470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:59356/0 (socket says 172.21.15.112:59356) 2024-09-21T03:29:42.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efda5a51640 1 -- 172.21.15.112:0/3626091803 learned_addr learned my addr 172.21.15.112:0/3626091803 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:42.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efda5250640 1 --2- 172.21.15.112:0/3626091803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 0x5629a7e0cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efda5a51640 1 -- 172.21.15.112:0/3626091803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a2f1b0 msgr2=0x5629a7e07f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:42.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efda5a51640 1 --2- 172.21.15.112:0/3626091803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a2f1b0 0x5629a7e07f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efda5a51640 1 -- 172.21.15.112:0/3626091803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 msgr2=0x5629a7e0cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:42.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efda5a51640 1 --2- 172.21.15.112:0/3626091803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 0x5629a7e0cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.610+0000 7efda5a51640 1 -- 172.21.15.112:0/3626091803 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a7e01bc0 con 0x5629a7a37390 2024-09-21T03:29:42.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.611+0000 7efda5a51640 1 --2- 172.21.15.112:0/3626091803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a37390 0x5629a7e08470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd980e51c0 tx=0x7efd980ad710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc0d3589be61f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:42.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.611+0000 7efd8affd640 1 -- 172.21.15.112:0/3626091803 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9801f040 con 0x5629a7a37390 2024-09-21T03:29:42.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.611+0000 7efd8affd640 1 -- 172.21.15.112:0/3626091803 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd980ad3d0 con 0x5629a7a37390 2024-09-21T03:29:42.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.611+0000 7efd8affd640 1 -- 172.21.15.112:0/3626091803 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd980aeda0 con 0x5629a7a37390 2024-09-21T03:29:42.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.612+0000 7efdada28740 1 -- 172.21.15.112:0/3626091803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a37390 msgr2=0x5629a7e08470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.612+0000 7efdada28740 1 --2- 172.21.15.112:0/3626091803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a37390 0x5629a7e08470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd980e51c0 tx=0x7efd980ad710 comp rx=0 tx=0).stop 2024-09-21T03:29:42.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.612+0000 7efdada28740 1 -- 172.21.15.112:0/3626091803 shutdown_connections 2024-09-21T03:29:42.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.612+0000 7efdada28740 1 --2- 172.21.15.112:0/3626091803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a37390 0x5629a7e08470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.612+0000 7efdada28740 1 --2- 172.21.15.112:0/3626091803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a2f1b0 0x5629a7e07f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.612+0000 7efdada28740 1 --2- 172.21.15.112:0/3626091803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 0x5629a7e0cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.612+0000 7efdada28740 1 -- 172.21.15.112:0/3626091803 >> 172.21.15.112:0/3626091803 conn(0x5629a7a4c460 msgr2=0x5629a7e095c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:42.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.612+0000 7efdada28740 1 -- 172.21.15.112:0/3626091803 shutdown_connections 2024-09-21T03:29:42.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.612+0000 7efdada28740 1 -- 172.21.15.112:0/3626091803 wait complete. 2024-09-21T03:29:42.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.612+0000 7efdada28740 1 -- start start 2024-09-21T03:29:42.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efdada28740 1 Processor -- start 2024-09-21T03:29:42.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efdada28740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 0x5629a7e0f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efdada28740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a2f1b0 0x5629a7e0f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efda5250640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 0x5629a7e0f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efdada28740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a37390 0x5629a7e86bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efdada28740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efda01aff30 con 0x5629a7a169f0 2024-09-21T03:29:42.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efdada28740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efda01b13a0 con 0x5629a7a2f1b0 2024-09-21T03:29:42.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efdada28740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efda01b0600 con 0x5629a7a37390 2024-09-21T03:29:42.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efda5250640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 0x5629a7e0f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46402/0 (socket says 172.21.15.112:46402) 2024-09-21T03:29:42.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efda5250640 1 -- 172.21.15.112:0/3298769917 learned_addr learned my addr 172.21.15.112:0/3298769917 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:42.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efda4a4f640 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a2f1b0 0x5629a7e0f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.613+0000 7efdada28740 10 client.0 _ll_register_callbacks cb 0x7efd9000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:42.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.614+0000 7efda5a51640 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a37390 0x5629a7e86bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.614+0000 7efda5250640 1 -- 172.21.15.112:0/3298769917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a37390 msgr2=0x5629a7e86bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:42.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.614+0000 7efda5250640 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a37390 0x5629a7e86bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.614+0000 7efda5250640 1 -- 172.21.15.112:0/3298769917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a2f1b0 msgr2=0x5629a7e0f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:42.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.614+0000 7efda5250640 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a2f1b0 0x5629a7e0f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.614+0000 7efda5250640 1 -- 172.21.15.112:0/3298769917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a7e100e0 con 0x5629a7a169f0 2024-09-21T03:29:42.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.614+0000 7efda5250640 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 0x5629a7e0f390 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005940 tx=0x7efd9c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c1481285c10956 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:42.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.614+0000 7efd88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:42.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.614+0000 7efd88ff9640 1 -- 172.21.15.112:0/3298769917 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c043070 con 0x5629a7a169f0 2024-09-21T03:29:42.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.614+0000 7efd88ff9640 1 -- 172.21.15.112:0/3298769917 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7efd9c005220 con 0x5629a7a169f0 2024-09-21T03:29:42.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efdada28740 1 -- 172.21.15.112:0/3298769917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd980ad3d0 con 0x5629a7a169f0 2024-09-21T03:29:42.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efd88ff9640 1 -- 172.21.15.112:0/3298769917 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efd9c035040 con 0x5629a7a169f0 2024-09-21T03:29:42.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efdada28740 10 client.15642 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:42.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efdada28740 1 -- 172.21.15.112:0/3298769917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5629a7e880f0 con 0x5629a7a169f0 2024-09-21T03:29:42.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efd88ff9640 1 -- 172.21.15.112:0/3298769917 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd980ad3d0 con 0x5629a7a169f0 2024-09-21T03:29:42.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efd88ff9640 1 -- 172.21.15.112:0/3298769917 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efd9c004960 con 0x5629a7a169f0 2024-09-21T03:29:42.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efdada28740 10 client.15642 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:42.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efdada28740 10 client.15642 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:29:42.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efdada28740 20 client.15642 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:42.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efdada28740 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5629a7a58070 0x5629a7e8a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:42.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efdada28740 4 client.15642 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:42.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.615+0000 7efdada28740 1 -- 172.21.15.112:0/3298769917 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629a7d738a0 con 0x5629a7a58070 2024-09-21T03:29:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.616+0000 7efda4a4f640 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5629a7a58070 0x5629a7e8a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.616+0000 7efd88ff9640 10 client.15642 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.616+0000 7efda4a4f640 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5629a7a58070 0x5629a7e8a830 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcabd09af56c6c86 server_cookie=e1a564a1dc5174c6 in_seq=0 out_seq=0 2024-09-21T03:29:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efd88ff9640 1 -- 172.21.15.112:0/3298769917 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629a7d738a0 con 0x5629a7a58070 2024-09-21T03:29:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efd88ff9640 10 client.15642 handle_command_reply: tid=0 2024-09-21T03:29:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efdada28740 1 client.15642 shutdown 2024-09-21T03:29:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efdada28740 1 -- 172.21.15.112:0/3298769917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 msgr2=0x5629a7e0f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efdada28740 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 0x5629a7e0f390 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005940 tx=0x7efd9c009260 comp rx=0 tx=0).stop 2024-09-21T03:29:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efdada28740 1 -- 172.21.15.112:0/3298769917 shutdown_connections 2024-09-21T03:29:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efdada28740 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5629a7a58070 0x5629a7e8a830 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efdada28740 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629a7a37390 0x5629a7e86bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efdada28740 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629a7a2f1b0 0x5629a7e0f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efdada28740 1 --2- 172.21.15.112:0/3298769917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5629a7a169f0 0x5629a7e0f390 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efdada28740 1 -- 172.21.15.112:0/3298769917 >> 172.21.15.112:0/3298769917 conn(0x5629a7a4c460 msgr2=0x5629a7e00640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.617+0000 7efd88ff9640 0 client.15642 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.618+0000 7efdada28740 1 -- 172.21.15.112:0/3298769917 shutdown_connections 2024-09-21T03:29:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.618+0000 7efdada28740 1 -- 172.21.15.112:0/3298769917 wait complete. 2024-09-21T03:29:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.618+0000 7efdada28740 20 client.15642 trim_cache size 0 max 16384 2024-09-21T03:29:42.632 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:29:42.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.618+0000 7efda74b4640 1 -- 172.21.15.112:0/884554472 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd8c0b6700 msgr2=0x7efd8c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.618+0000 7efda74b4640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd8c0b6700 0x7efd8c0b8c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5629a7dfe320 tx=0x7efd9c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.618+0000 7efda74b4640 1 -- 172.21.15.112:0/884554472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda0106bd0 msgr2=0x7efda0103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:42.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.618+0000 7efda74b4640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda0106bd0 0x7efda0103ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efd9800e940 tx=0x7efd9800ee00 comp rx=0 tx=0).stop 2024-09-21T03:29:42.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.619+0000 7efda74b4640 1 -- 172.21.15.112:0/884554472 shutdown_connections 2024-09-21T03:29:42.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.619+0000 7efda74b4640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd8c0b6700 0x7efd8c0b8c10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.619+0000 7efda74b4640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efda0106bd0 0x7efda0103ab0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.619+0000 7efda74b4640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efda0106280 0x7efda0103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.619+0000 7efda74b4640 1 --2- 172.21.15.112:0/884554472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efda0102090 0x7efda0103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:42.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.619+0000 7efda74b4640 1 -- 172.21.15.112:0/884554472 >> 172.21.15.112:0/884554472 conn(0x7efda0077c60 msgr2=0x7efda0107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:42.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.619+0000 7efda74b4640 1 -- 172.21.15.112:0/884554472 shutdown_connections 2024-09-21T03:29:42.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:42.619+0000 7efda74b4640 1 -- 172.21.15.112:0/884554472 wait complete. 2024-09-21T03:29:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:42 smithi112 ceph-mon[31442]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-21T03:29:42.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:42 smithi203 ceph-mon[34723]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-21T03:29:42.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:42 smithi187 ceph-mon[34603]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-21T03:29:43.308 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:29:43.580 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:44.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.136+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/4039979111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0b4102630 msgr2=0x7fd0b4102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.136+0000 7fd0b8d2c640 1 --2- 172.21.15.112:0/4039979111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0b4102630 0x7fd0b4102a40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009900 tx=0x7fd0a402a480 comp rx=0 tx=0).stop 2024-09-21T03:29:44.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.136+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/4039979111 shutdown_connections 2024-09-21T03:29:44.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.136+0000 7fd0b8d2c640 1 --2- 172.21.15.112:0/4039979111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0b41101e0 0x7fd0b4112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.136+0000 7fd0b8d2c640 1 --2- 172.21.15.112:0/4039979111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0b4102f80 0x7fd0b410fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.136+0000 7fd0b8d2c640 1 --2- 172.21.15.112:0/4039979111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0b4102630 0x7fd0b4102a40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.136+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/4039979111 >> 172.21.15.112:0/4039979111 conn(0x7fd0b4079730 msgr2=0x7fd0b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:44.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.136+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/4039979111 shutdown_connections 2024-09-21T03:29:44.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.136+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/4039979111 wait complete. 2024-09-21T03:29:44.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.137+0000 7fd0b8d2c640 1 Processor -- start 2024-09-21T03:29:44.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.137+0000 7fd0b8d2c640 1 -- start start 2024-09-21T03:29:44.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b8d2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0b4102630 0x7fd0b4074e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b8d2c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0b4102f80 0x7fd0b406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b8d2c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0b41101e0 0x7fd0b406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b8d2c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4115c50 con 0x7fd0b4102630 2024-09-21T03:29:44.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b8d2c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4115ad0 con 0x7fd0b41101e0 2024-09-21T03:29:44.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b8d2c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4115dd0 con 0x7fd0b4102f80 2024-09-21T03:29:44.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0b4102630 0x7fd0b4074e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0b4102630 0x7fd0b4074e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46422/0 (socket says 172.21.15.112:46422) 2024-09-21T03:29:44.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b259c640 1 -- 172.21.15.112:0/3878221504 learned_addr learned my addr 172.21.15.112:0/3878221504 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:44.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b1d9b640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0b4102f80 0x7fd0b406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0b41101e0 0x7fd0b406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b259c640 1 -- 172.21.15.112:0/3878221504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0b4102f80 msgr2=0x7fd0b406e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.138+0000 7fd0b259c640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0b4102f80 0x7fd0b406e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.139+0000 7fd0b259c640 1 -- 172.21.15.112:0/3878221504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0b41101e0 msgr2=0x7fd0b406e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.139+0000 7fd0b259c640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0b41101e0 0x7fd0b406e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.139+0000 7fd0b259c640 1 -- 172.21.15.112:0/3878221504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b406eed0 con 0x7fd0b4102630 2024-09-21T03:29:44.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.139+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0b41101e0 0x7fd0b406e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:44.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.139+0000 7fd0b259c640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0b4102630 0x7fd0b4074e60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4005940 tx=0x7fd0a40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:44.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.139+0000 7fd0a37fe640 1 -- 172.21.15.112:0/3878221504 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a403f030 con 0x7fd0b4102630 2024-09-21T03:29:44.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.139+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/3878221504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b4073b30 con 0x7fd0b4102630 2024-09-21T03:29:44.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.139+0000 7fd0a37fe640 1 -- 172.21.15.112:0/3878221504 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a40044c0 con 0x7fd0b4102630 2024-09-21T03:29:44.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.139+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/3878221504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b4074060 con 0x7fd0b4102630 2024-09-21T03:29:44.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.139+0000 7fd0a37fe640 1 -- 172.21.15.112:0/3878221504 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4035070 con 0x7fd0b4102630 2024-09-21T03:29:44.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.143+0000 7fd0bf2a0740 1 Processor -- start 2024-09-21T03:29:44.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.143+0000 7fd0bf2a0740 1 -- start start 2024-09-21T03:29:44.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca61eea0 con 0x55a7ca2d81b0 2024-09-21T03:29:44.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca60e2d0 con 0x55a7ca2e0390 2024-09-21T03:29:44.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca6120e0 con 0x55a7ca301070 2024-09-21T03:29:44.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0a37fe640 1 -- 172.21.15.112:0/3878221504 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fd0a4042030 con 0x7fd0b4102630 2024-09-21T03:29:44.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0b2d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0b2d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6a86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:38316/0 (socket says 172.21.15.112:38316) 2024-09-21T03:29:44.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0b2d9d640 1 -- 172.21.15.112:0/1124485099 learned_addr learned my addr 172.21.15.112:0/1124485099 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:44.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.144+0000 7fd0b1d9b640 1 --2- 172.21.15.112:0/1124485099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.145+0000 7fd0b2d9d640 1 -- 172.21.15.112:0/1124485099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 msgr2=0x55a7ca6a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.145+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/1124485099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.145+0000 7fd0b2d9d640 1 -- 172.21.15.112:0/1124485099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 msgr2=0x55a7ca6a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.145+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/1124485099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.145+0000 7fd0b2d9d640 1 -- 172.21.15.112:0/1124485099 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ca36c650 con 0x55a7ca301070 2024-09-21T03:29:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.145+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/1124485099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6a86a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0117e0 tx=0x7fd0ac011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90353148e7c3bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.145+0000 7fd0a0ff9640 1 -- 172.21.15.112:0/1124485099 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac011310 con 0x55a7ca301070 2024-09-21T03:29:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.146+0000 7fd0a0ff9640 1 -- 172.21.15.112:0/1124485099 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ac003e20 con 0x55a7ca301070 2024-09-21T03:29:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.146+0000 7fd0a0ff9640 1 -- 172.21.15.112:0/1124485099 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0044f0 con 0x55a7ca301070 2024-09-21T03:29:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.146+0000 7fd0a37fe640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd09c0b6700 0x7fd09c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.146+0000 7fd0b259c640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd09c0b6700 0x7fd09c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.147+0000 7fd0a37fe640 1 -- 172.21.15.112:0/3878221504 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd0a4107af0 con 0x7fd0b4102630 2024-09-21T03:29:44.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.147+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/1124485099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 msgr2=0x55a7ca6a86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.147+0000 7fd0b259c640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd09c0b6700 0x7fd09c0b8c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55a7ca6a7320 tx=0x7fd0a40ca040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.147+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/1124485099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6a86a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0117e0 tx=0x7fd0ac011ca0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.147+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/1124485099 shutdown_connections 2024-09-21T03:29:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.147+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/1124485099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.147+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/1124485099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.147+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/1124485099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6a86a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.147+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/1124485099 >> 172.21.15.112:0/1124485099 conn(0x55a7ca2f5460 msgr2=0x55a7ca6a6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.147+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/1124485099 shutdown_connections 2024-09-21T03:29:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.147+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/1124485099 wait complete. 2024-09-21T03:29:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.148+0000 7fd0bf2a0740 1 -- start start 2024-09-21T03:29:44.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.148+0000 7fd0bf2a0740 1 Processor -- start 2024-09-21T03:29:44.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2e0390 0x55a7ca6b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca46f700 con 0x55a7ca2e0390 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca5a9160 con 0x55a7ca2d81b0 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca6b6e80 con 0x55a7ca301070 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0bf2a0740 10 client.0 _ll_register_callbacks cb 0x55a7ca5c29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0b1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2e0390 0x55a7ca6b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0b2d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0b1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2e0390 0x55a7ca6b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46446/0 (socket says 172.21.15.112:46446) 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0b1d9b640 1 -- 172.21.15.112:0/4294488199 learned_addr learned my addr 172.21.15.112:0/4294488199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0b1d9b640 1 -- 172.21.15.112:0/4294488199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 msgr2=0x55a7ca6c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0b1d9b640 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0b1d9b640 1 -- 172.21.15.112:0/4294488199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2d81b0 msgr2=0x55a7ca6b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.149+0000 7fd0b1d9b640 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.150+0000 7fd0b1d9b640 1 -- 172.21.15.112:0/4294488199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ca5d6c90 con 0x55a7ca2e0390 2024-09-21T03:29:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.150+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.150+0000 7fd0b1d9b640 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2e0390 0x55a7ca6b42f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80126e0 tx=0x7fd0a8012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458cf72b3d3d4b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.150+0000 7fd099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.150+0000 7fd099ffb640 1 -- 172.21.15.112:0/4294488199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8012190 con 0x55a7ca2e0390 2024-09-21T03:29:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.150+0000 7fd099ffb640 1 -- 172.21.15.112:0/4294488199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd0a800f920 con 0x55a7ca2e0390 2024-09-21T03:29:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.150+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/4294488199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ac003e20 con 0x55a7ca2e0390 2024-09-21T03:29:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.150+0000 7fd099ffb640 1 -- 172.21.15.112:0/4294488199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0a800fb60 con 0x55a7ca2e0390 2024-09-21T03:29:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.150+0000 7fd0bf2a0740 10 client.15654 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.150+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/4294488199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a7ca6c2c50 con 0x55a7ca2e0390 2024-09-21T03:29:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.151+0000 7fd099ffb640 1 -- 172.21.15.112:0/4294488199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ac003e20 con 0x55a7ca2e0390 2024-09-21T03:29:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.151+0000 7fd099ffb640 1 -- 172.21.15.112:0/4294488199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd0a801c470 con 0x55a7ca2e0390 2024-09-21T03:29:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.151+0000 7fd0bf2a0740 10 client.15654 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.151+0000 7fd0bf2a0740 10 client.15654 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:29:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.151+0000 7fd0bf2a0740 20 client.15654 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.151+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a7ca2bf9f0 0x55a7ca6c6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.151+0000 7fd0bf2a0740 4 client.15654 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.151+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/4294488199 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7ca620df0 con 0x55a7ca2bf9f0 2024-09-21T03:29:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.151+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a7ca2bf9f0 0x55a7ca6c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.152+0000 7fd099ffb640 10 client.15654 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.152+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a7ca2bf9f0 0x55a7ca6c6510 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1df77c204e44bd8 server_cookie=aa47a96ae46b75c9 in_seq=0 out_seq=0 2024-09-21T03:29:44.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.153+0000 7fd099ffb640 1 -- 172.21.15.112:0/4294488199 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7ca620df0 con 0x55a7ca2bf9f0 2024-09-21T03:29:44.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.153+0000 7fd099ffb640 10 client.15654 handle_command_reply: tid=0 2024-09-21T03:29:44.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.153+0000 7fd0bf2a0740 1 client.15654 shutdown 2024-09-21T03:29:44.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.153+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/4294488199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2e0390 msgr2=0x55a7ca6b42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.153+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2e0390 0x55a7ca6b42f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80126e0 tx=0x7fd0a8012ba0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.153+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/4294488199 shutdown_connections 2024-09-21T03:29:44.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.153+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a7ca2bf9f0 0x55a7ca6c6510 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.153+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.153+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2e0390 0x55a7ca6b42f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.153+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/4294488199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.153+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/4294488199 >> 172.21.15.112:0/4294488199 conn(0x55a7ca2f5460 msgr2=0x55a7ca6ab830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:44.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.154+0000 7fd099ffb640 0 client.15654 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:44.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.154+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/4294488199 shutdown_connections 2024-09-21T03:29:44.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.154+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/4294488199 wait complete. 2024-09-21T03:29:44.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.154+0000 7fd0bf2a0740 20 client.15654 trim_cache size 0 max 16384 2024-09-21T03:29:44.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.179+0000 7fd0bf2a0740 1 Processor -- start 2024-09-21T03:29:44.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.179+0000 7fd0bf2a0740 1 -- start start 2024-09-21T03:29:44.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.180+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.180+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6c00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.180+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.180+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41bc890 con 0x55a7ca2d81b0 2024-09-21T03:29:44.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.180+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41ba160 con 0x55a7ca2e0390 2024-09-21T03:29:44.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.180+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41b7fb0 con 0x55a7ca301070 2024-09-21T03:29:44.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.180+0000 7fd0b2d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.180+0000 7fd0b2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.180+0000 7fd0b2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46458/0 (socket says 172.21.15.112:46458) 2024-09-21T03:29:44.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.180+0000 7fd0b2d9d640 1 -- 172.21.15.112:0/2892674 learned_addr learned my addr 172.21.15.112:0/2892674 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:44.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.180+0000 7fd0b1d9b640 1 --2- 172.21.15.112:0/2892674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6c00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.181+0000 7fd0b2d9d640 1 -- 172.21.15.112:0/2892674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 msgr2=0x55a7ca6a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.181+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/2892674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.181+0000 7fd0b2d9d640 1 -- 172.21.15.112:0/2892674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 msgr2=0x55a7ca6c00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.181+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/2892674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6c00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.181+0000 7fd0b2d9d640 1 -- 172.21.15.112:0/2892674 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ca6aaee0 con 0x55a7ca2d81b0 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.181+0000 7fd0b1d9b640 1 --2- 172.21.15.112:0/2892674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6c00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.182+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/2892674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6b4270 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac000f80 tx=0x7fd0ac005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eefc24681ba9a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.182+0000 7fd09bfff640 1 -- 172.21.15.112:0/2892674 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac022040 con 0x55a7ca2d81b0 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.182+0000 7fd09bfff640 1 -- 172.21.15.112:0/2892674 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ac00fe50 con 0x55a7ca2d81b0 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.182+0000 7fd09bfff640 1 -- 172.21.15.112:0/2892674 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac02c450 con 0x55a7ca2d81b0 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.183+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/2892674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 msgr2=0x55a7ca6b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.183+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/2892674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6b4270 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac000f80 tx=0x7fd0ac005ab0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.183+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/2892674 shutdown_connections 2024-09-21T03:29:44.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.183+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/2892674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca301070 0x55a7ca6a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.183+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/2892674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6c00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.183+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/2892674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6b4270 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.183+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/2892674 >> 172.21.15.112:0/2892674 conn(0x55a7ca2f5460 msgr2=0x55a7ca6b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:44.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.184+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/2892674 shutdown_connections 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.184+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/2892674 wait complete. 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.184+0000 7fd0bf2a0740 1 -- start start 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.185+0000 7fd0bf2a0740 1 Processor -- start 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.185+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6bc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.185+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6bcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.185+0000 7fd0bf2a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca301070 0x55a7ca6b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41b8620 con 0x55a7ca301070 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41b9a90 con 0x55a7ca2e0390 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0bf2a0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41b8cf0 con 0x55a7ca2d81b0 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0bf2a0740 10 client.0 _ll_register_callbacks cb 0x7fd0a8009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0b1d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6bcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0b1d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6bcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:59420/0 (socket says 172.21.15.112:59420) 2024-09-21T03:29:44.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0b1d9b640 1 -- 172.21.15.112:0/197028591 learned_addr learned my addr 172.21.15.112:0/197028591 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:44.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca301070 0x55a7ca6b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0b1d9b640 1 -- 172.21.15.112:0/197028591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca2d81b0 msgr2=0x55a7ca6bc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:44.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0b1d9b640 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6bc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0b1d9b640 1 -- 172.21.15.112:0/197028591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca301070 msgr2=0x55a7ca6b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:44.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0b1d9b640 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca301070 0x55a7ca6b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0b1d9b640 1 -- 172.21.15.112:0/197028591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ca6b8340 con 0x55a7ca2e0390 2024-09-21T03:29:44.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.186+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca301070 0x55a7ca6b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:44.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.187+0000 7fd0b1d9b640 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6bcb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80126e0 tx=0x7fd0a80123c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=687438212a52cfed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:44.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.187+0000 7fd099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:29:44.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.187+0000 7fd099ffb640 1 -- 172.21.15.112:0/197028591 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a801cd40 con 0x55a7ca2e0390 2024-09-21T03:29:44.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.187+0000 7fd099ffb640 1 -- 172.21.15.112:0/197028591 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd0a801c470 con 0x55a7ca2e0390 2024-09-21T03:29:44.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.187+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/197028591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ac00fe50 con 0x55a7ca2e0390 2024-09-21T03:29:44.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.187+0000 7fd099ffb640 1 -- 172.21.15.112:0/197028591 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0a8036050 con 0x55a7ca2e0390 2024-09-21T03:29:44.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.187+0000 7fd0bf2a0740 10 client.25247 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:44.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.187+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/197028591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a7ca730ab0 con 0x55a7ca2e0390 2024-09-21T03:29:44.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.188+0000 7fd099ffb640 1 -- 172.21.15.112:0/197028591 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ac00fe50 con 0x55a7ca2e0390 2024-09-21T03:29:44.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.188+0000 7fd099ffb640 1 -- 172.21.15.112:0/197028591 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd0a80039d0 con 0x55a7ca2e0390 2024-09-21T03:29:44.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.188+0000 7fd0bf2a0740 10 client.25247 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:44.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.188+0000 7fd0bf2a0740 10 client.25247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:29:44.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.188+0000 7fd0bf2a0740 20 client.25247 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:44.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.188+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a7ca294da0 0x55a7ca733260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:44.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.188+0000 7fd0bf2a0740 4 client.25247 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:44.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.188+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/197028591 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7ca61c8a0 con 0x55a7ca294da0 2024-09-21T03:29:44.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.188+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a7ca294da0 0x55a7ca733260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:44.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd099ffb640 10 client.25247 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:44.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd0b2d9d640 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a7ca294da0 0x55a7ca733260 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5df4737d3fc401c0 server_cookie=fb984a88a82d7ad0 in_seq=0 out_seq=0 2024-09-21T03:29:44.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd099ffb640 1 -- 172.21.15.112:0/197028591 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7ca61c8a0 con 0x55a7ca294da0 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd099ffb640 10 client.25247 handle_command_reply: tid=0 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd0bf2a0740 1 client.25247 shutdown 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/197028591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 msgr2=0x55a7ca6bcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6bcb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80126e0 tx=0x7fd0a80123c0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/197028591 shutdown_connections 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a7ca294da0 0x55a7ca733260 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7ca301070 0x55a7ca6b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a7ca2e0390 0x55a7ca6bcb30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd0bf2a0740 1 --2- 172.21.15.112:0/197028591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7ca2d81b0 0x55a7ca6bc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/197028591 >> 172.21.15.112:0/197028591 conn(0x55a7ca2f5460 msgr2=0x55a7ca6a96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.189+0000 7fd099ffb640 0 client.25247 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:44.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.190+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/197028591 shutdown_connections 2024-09-21T03:29:44.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.190+0000 7fd0bf2a0740 1 -- 172.21.15.112:0/197028591 wait complete. 2024-09-21T03:29:44.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.190+0000 7fd0bf2a0740 20 client.25247 trim_cache size 0 max 16384 2024-09-21T03:29:44.197 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:29:44.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.190+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/3878221504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd09c0b6700 msgr2=0x7fd09c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.190+0000 7fd0b8d2c640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd09c0b6700 0x7fd09c0b8c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55a7ca6a7320 tx=0x7fd0a40ca040 comp rx=0 tx=0).stop 2024-09-21T03:29:44.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.190+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/3878221504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0b4102630 msgr2=0x7fd0b4074e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:44.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.190+0000 7fd0b8d2c640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0b4102630 0x7fd0b4074e60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4005940 tx=0x7fd0a40040c0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.190+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/3878221504 shutdown_connections 2024-09-21T03:29:44.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.190+0000 7fd0b8d2c640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd09c0b6700 0x7fd09c0b8c10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.190+0000 7fd0b8d2c640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0b41101e0 0x7fd0b406e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.191+0000 7fd0b8d2c640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0b4102f80 0x7fd0b406e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.191+0000 7fd0b8d2c640 1 --2- 172.21.15.112:0/3878221504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0b4102630 0x7fd0b4074e60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:44.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.191+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/3878221504 >> 172.21.15.112:0/3878221504 conn(0x7fd0b4079730 msgr2=0x7fd0b4108cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:44.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.191+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/3878221504 shutdown_connections 2024-09-21T03:29:44.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:44.191+0000 7fd0b8d2c640 1 -- 172.21.15.112:0/3878221504 wait complete. 2024-09-21T03:29:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:44 smithi112 ceph-mon[31442]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-21T03:29:44.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:44 smithi203 ceph-mon[34723]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-21T03:29:44.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:44 smithi187 ceph-mon[34603]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-21T03:29:45.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:29:45.296 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:45.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.831+0000 7fa5af108640 1 -- 172.21.15.112:0/1504585816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8101fa0 msgr2=0x7fa5a81023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:45.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.831+0000 7fa5af108640 1 --2- 172.21.15.112:0/1504585816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8101fa0 0x7fa5a81023b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009860 tx=0x7fa59c02a340 comp rx=0 tx=0).stop 2024-09-21T03:29:45.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.832+0000 7fa5af108640 1 -- 172.21.15.112:0/1504585816 shutdown_connections 2024-09-21T03:29:45.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.832+0000 7fa5af108640 1 --2- 172.21.15.112:0/1504585816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a8106bd0 0x7fa5a8110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.832+0000 7fa5af108640 1 --2- 172.21.15.112:0/1504585816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5a8106280 0x7fa5a8106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.832+0000 7fa5af108640 1 --2- 172.21.15.112:0/1504585816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8101fa0 0x7fa5a81023b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.832+0000 7fa5af108640 1 -- 172.21.15.112:0/1504585816 >> 172.21.15.112:0/1504585816 conn(0x7fa5a8077ca0 msgr2=0x7fa5a80780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:45.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.832+0000 7fa5af108640 1 -- 172.21.15.112:0/1504585816 shutdown_connections 2024-09-21T03:29:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.832+0000 7fa5af108640 1 -- 172.21.15.112:0/1504585816 wait complete. 2024-09-21T03:29:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.833+0000 7fa5af108640 1 Processor -- start 2024-09-21T03:29:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.834+0000 7fa5af108640 1 -- start start 2024-09-21T03:29:45.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.834+0000 7fa5af108640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a8101fa0 0x7fa5a819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.834+0000 7fa5af108640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5a8106280 0x7fa5a819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.834+0000 7fa5af108640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8106bd0 0x7fa5a819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.834+0000 7fa5af108640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8113ad0 con 0x7fa5a8101fa0 2024-09-21T03:29:45.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.834+0000 7fa5af108640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8113950 con 0x7fa5a8106bd0 2024-09-21T03:29:45.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.834+0000 7fa5af108640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8113c50 con 0x7fa5a8106280 2024-09-21T03:29:45.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.834+0000 7fa5acea4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a8101fa0 0x7fa5a819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5acea4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a8101fa0 0x7fa5a819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45636/0 (socket says 172.21.15.112:45636) 2024-09-21T03:29:45.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5acea4640 1 -- 172.21.15.112:0/3099357787 learned_addr learned my addr 172.21.15.112:0/3099357787 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:45.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5ad6a5640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8106bd0 0x7fa5a819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5a7fff640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5a8106280 0x7fa5a819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5ad6a5640 1 -- 172.21.15.112:0/3099357787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5a8106280 msgr2=0x7fa5a819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:45.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5ad6a5640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5a8106280 0x7fa5a819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5ad6a5640 1 -- 172.21.15.112:0/3099357787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a8101fa0 msgr2=0x7fa5a819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:45.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5ad6a5640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a8101fa0 0x7fa5a819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5ad6a5640 1 -- 172.21.15.112:0/3099357787 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a81a0410 con 0x7fa5a8106bd0 2024-09-21T03:29:45.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5acea4640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a8101fa0 0x7fa5a819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:45.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5ad6a5640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8106bd0 0x7fa5a819fce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000efc0 tx=0x7fa5a000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:45.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.835+0000 7fa5a5ffb640 1 -- 172.21.15.112:0/3099357787 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0003970 con 0x7fa5a8106bd0 2024-09-21T03:29:45.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.836+0000 7fa5a5ffb640 1 -- 172.21.15.112:0/3099357787 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5a0004020 con 0x7fa5a8106bd0 2024-09-21T03:29:45.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.836+0000 7fa5af108640 1 -- 172.21.15.112:0/3099357787 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a81a0700 con 0x7fa5a8106bd0 2024-09-21T03:29:45.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.836+0000 7fa5a5ffb640 1 -- 172.21.15.112:0/3099357787 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a000b940 con 0x7fa5a8106bd0 2024-09-21T03:29:45.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.837+0000 7fa5af108640 1 -- 172.21.15.112:0/3099357787 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a81a6e60 con 0x7fa5a8106bd0 2024-09-21T03:29:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.840+0000 7fa5a5ffb640 1 -- 172.21.15.112:0/3099357787 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa5a001d020 con 0x7fa5a8106bd0 2024-09-21T03:29:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.840+0000 7fa5b567c740 1 Processor -- start 2024-09-21T03:29:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.840+0000 7fa5b567c740 1 -- start start 2024-09-21T03:29:45.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd81070 0x56435e12cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e12cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd60390 0x56435e12d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5b567c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56435e09eea0 con 0x56435dd60390 2024-09-21T03:29:45.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5b567c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56435e08e2d0 con 0x56435dd581b0 2024-09-21T03:29:45.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5b567c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56435e0920e0 con 0x56435dd81070 2024-09-21T03:29:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5acea4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e12cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5acea4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e12cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:37320/0 (socket says 172.21.15.112:37320) 2024-09-21T03:29:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5acea4640 1 -- 172.21.15.112:0/3321595916 learned_addr learned my addr 172.21.15.112:0/3321595916 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5a5ffb640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5980b6700 0x7fa5980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5ad6a5640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5980b6700 0x7fa5980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5acea4640 1 -- 172.21.15.112:0/3321595916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd81070 msgr2=0x56435e12cab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:29:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.841+0000 7fa5acea4640 1 --2- 172.21.15.112:0/3321595916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd81070 0x56435e12cab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.842+0000 7fa5acea4640 1 -- 172.21.15.112:0/3321595916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd60390 msgr2=0x56435e12d530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:29:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.842+0000 7fa5a5ffb640 1 -- 172.21.15.112:0/3099357787 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa5a00e75b0 con 0x7fa5a8106bd0 2024-09-21T03:29:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.842+0000 7fa5a7fff640 1 --2- 172.21.15.112:0/3321595916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd60390 0x56435e12d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.842+0000 7fa5acea4640 1 --2- 172.21.15.112:0/3321595916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd60390 0x56435e12d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.842+0000 7fa5acea4640 1 -- 172.21.15.112:0/3321595916 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435ddec650 con 0x56435dd581b0 2024-09-21T03:29:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.842+0000 7fa5acea4640 1 --2- 172.21.15.112:0/3321595916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e12cff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa59c000c00 tx=0x7fa59c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e858479c564313b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.842+0000 7fa5ad6a5640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5980b6700 0x7fa5980b8c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00ad3f0 tx=0x7fa5a00a9b90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:45.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.843+0000 7fa592ffd640 1 -- 172.21.15.112:0/3321595916 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c037040 con 0x56435dd581b0 2024-09-21T03:29:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.843+0000 7fa592ffd640 1 -- 172.21.15.112:0/3321595916 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa59c030450 con 0x56435dd581b0 2024-09-21T03:29:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.843+0000 7fa592ffd640 1 -- 172.21.15.112:0/3321595916 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c032a30 con 0x56435dd581b0 2024-09-21T03:29:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.844+0000 7fa5b567c740 1 -- 172.21.15.112:0/3321595916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 msgr2=0x56435e12cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.844+0000 7fa5b567c740 1 --2- 172.21.15.112:0/3321595916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e12cff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa59c000c00 tx=0x7fa59c009510 comp rx=0 tx=0).stop 2024-09-21T03:29:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.844+0000 7fa5b567c740 1 -- 172.21.15.112:0/3321595916 shutdown_connections 2024-09-21T03:29:45.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.844+0000 7fa5b567c740 1 --2- 172.21.15.112:0/3321595916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd60390 0x56435e12d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.844+0000 7fa5b567c740 1 --2- 172.21.15.112:0/3321595916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e12cff0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.844+0000 7fa5b567c740 1 --2- 172.21.15.112:0/3321595916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd81070 0x56435e12cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.844+0000 7fa5b567c740 1 -- 172.21.15.112:0/3321595916 >> 172.21.15.112:0/3321595916 conn(0x56435dd75460 msgr2=0x56435e12a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:45.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.844+0000 7fa5b567c740 1 -- 172.21.15.112:0/3321595916 shutdown_connections 2024-09-21T03:29:45.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.844+0000 7fa5b567c740 1 -- 172.21.15.112:0/3321595916 wait complete. 2024-09-21T03:29:45.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.845+0000 7fa5b567c740 1 -- start start 2024-09-21T03:29:45.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.845+0000 7fa5b567c740 1 Processor -- start 2024-09-21T03:29:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.845+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e13ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 0x56435e13d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e142480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5acea4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 0x56435e13d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5acea4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 0x56435e13d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:48020/0 (socket says 172.21.15.112:48020) 2024-09-21T03:29:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5a7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e13ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5a7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e13ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:37336/0 (socket says 172.21.15.112:37336) 2024-09-21T03:29:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5acea4640 1 -- 172.21.15.112:0/46491454 learned_addr learned my addr 172.21.15.112:0/46491454 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5b567c740 1 -- 172.21.15.112:0/46491454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56435deef700 con 0x56435dd81070 2024-09-21T03:29:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5b567c740 1 -- 172.21.15.112:0/46491454 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56435e029160 con 0x56435dd581b0 2024-09-21T03:29:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5b567c740 1 -- 172.21.15.112:0/46491454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56435e13b230 con 0x56435dd60390 2024-09-21T03:29:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5b567c740 10 client.0 _ll_register_callbacks cb 0x56435e0429e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5acea4640 1 -- 172.21.15.112:0/46491454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 msgr2=0x56435e13ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5acea4640 1 --2- 172.21.15.112:0/46491454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e13ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5acea4640 1 -- 172.21.15.112:0/46491454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 msgr2=0x56435e142480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5acea4640 1 --2- 172.21.15.112:0/46491454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.846+0000 7fa5acea4640 1 -- 172.21.15.112:0/46491454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56435e056c90 con 0x56435dd60390 2024-09-21T03:29:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.847+0000 7fa5acea4640 1 --2- 172.21.15.112:0/46491454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 0x56435e13d460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009980 tx=0x7fa59c005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e3c3902ee965ab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.847+0000 7fa587fff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:29:45.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.847+0000 7fa587fff640 1 -- 172.21.15.112:0/46491454 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c037030 con 0x56435dd60390 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.847+0000 7fa587fff640 1 -- 172.21.15.112:0/46491454 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa59c032d80 con 0x56435dd60390 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.847+0000 7fa5b567c740 1 -- 172.21.15.112:0/46491454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa59c030450 con 0x56435dd60390 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.847+0000 7fa587fff640 1 -- 172.21.15.112:0/46491454 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa59c055050 con 0x56435dd60390 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.847+0000 7fa5b567c740 10 client.25270 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.847+0000 7fa5b567c740 1 -- 172.21.15.112:0/46491454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56435e14b390 con 0x56435dd60390 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.848+0000 7fa587fff640 1 -- 172.21.15.112:0/46491454 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa59c03d930 con 0x56435dd60390 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.848+0000 7fa587fff640 1 -- 172.21.15.112:0/46491454 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa59c046810 con 0x56435dd60390 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.848+0000 7fa5b567c740 10 client.25270 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.848+0000 7fa5b567c740 10 client.25270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.848+0000 7fa5b567c740 20 client.25270 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.848+0000 7fa5b567c740 1 --2- 172.21.15.112:0/46491454 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56435dd3f9f0 0x56435e14e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.848+0000 7fa5b567c740 4 client.25270 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.848+0000 7fa5b567c740 1 -- 172.21.15.112:0/46491454 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa59c030450 con 0x56435dd3f9f0 2024-09-21T03:29:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.848+0000 7fa5a7fff640 1 --2- 172.21.15.112:0/46491454 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56435dd3f9f0 0x56435e14e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.849+0000 7fa587fff640 10 client.25270 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.849+0000 7fa5a7fff640 1 --2- 172.21.15.112:0/46491454 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56435dd3f9f0 0x56435e14e6f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc7acef293699de4 server_cookie=da058ae7dbb3d92 in_seq=0 out_seq=0 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa587fff640 1 -- 172.21.15.112:0/46491454 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa59c030450 con 0x56435dd3f9f0 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa587fff640 10 client.25270 handle_command_reply: tid=0 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa5b567c740 1 client.25270 shutdown 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa5b567c740 1 -- 172.21.15.112:0/46491454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 msgr2=0x56435e13d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa5b567c740 1 --2- 172.21.15.112:0/46491454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 0x56435e13d460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009980 tx=0x7fa59c005f80 comp rx=0 tx=0).stop 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa5b567c740 1 -- 172.21.15.112:0/46491454 shutdown_connections 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa5b567c740 1 --2- 172.21.15.112:0/46491454 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56435dd3f9f0 0x56435e14e6f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa5b567c740 1 --2- 172.21.15.112:0/46491454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa5b567c740 1 --2- 172.21.15.112:0/46491454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 0x56435e13d460 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa5b567c740 1 --2- 172.21.15.112:0/46491454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e13ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa5b567c740 1 -- 172.21.15.112:0/46491454 >> 172.21.15.112:0/46491454 conn(0x56435dd75460 msgr2=0x56435e12dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.850+0000 7fa587fff640 0 client.25270 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.851+0000 7fa5b567c740 1 -- 172.21.15.112:0/46491454 shutdown_connections 2024-09-21T03:29:45.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.851+0000 7fa5b567c740 1 -- 172.21.15.112:0/46491454 wait complete. 2024-09-21T03:29:45.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.851+0000 7fa5b567c740 20 client.25270 trim_cache size 0 max 16384 2024-09-21T03:29:45.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5b567c740 1 Processor -- start 2024-09-21T03:29:45.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5b567c740 1 -- start start 2024-09-21T03:29:45.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e142a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 0x56435e12a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e125bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5b567c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81afbf0 con 0x56435dd81070 2024-09-21T03:29:45.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5b567c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81ad4c0 con 0x56435dd581b0 2024-09-21T03:29:45.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5b567c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81ab310 con 0x56435dd60390 2024-09-21T03:29:45.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e125bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e125bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45648/0 (socket says 172.21.15.112:45648) 2024-09-21T03:29:45.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5acea4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 0x56435e12a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.877+0000 7fa5a7fff640 1 -- 172.21.15.112:0/1171138015 learned_addr learned my addr 172.21.15.112:0/1171138015 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:45.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.878+0000 7fa5acea4640 1 -- 172.21.15.112:0/1171138015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 msgr2=0x56435e142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:45.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.878+0000 7fa5acea4640 1 --2- 172.21.15.112:0/1171138015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.878+0000 7fa5acea4640 1 -- 172.21.15.112:0/1171138015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 msgr2=0x56435e125bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:45.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.878+0000 7fa5acea4640 1 --2- 172.21.15.112:0/1171138015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e125bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.878+0000 7fa5acea4640 1 -- 172.21.15.112:0/1171138015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435e13df40 con 0x56435dd60390 2024-09-21T03:29:45.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.878+0000 7fa5a7fff640 1 --2- 172.21.15.112:0/1171138015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e125bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:45.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.878+0000 7fa5acea4640 1 --2- 172.21.15.112:0/1171138015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 0x56435e12a890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa59c046c60 tx=0x7fa59c02a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dada2ae5e7078976 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:45.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.879+0000 7fa5927fc640 1 -- 172.21.15.112:0/1171138015 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c037040 con 0x56435dd60390 2024-09-21T03:29:45.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.879+0000 7fa5927fc640 1 -- 172.21.15.112:0/1171138015 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa59c046e50 con 0x56435dd60390 2024-09-21T03:29:45.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.879+0000 7fa5927fc640 1 -- 172.21.15.112:0/1171138015 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c03d6e0 con 0x56435dd60390 2024-09-21T03:29:45.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.880+0000 7fa5b567c740 1 -- 172.21.15.112:0/1171138015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 msgr2=0x56435e12a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:45.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.880+0000 7fa5b567c740 1 --2- 172.21.15.112:0/1171138015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 0x56435e12a890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa59c046c60 tx=0x7fa59c02a340 comp rx=0 tx=0).stop 2024-09-21T03:29:45.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.880+0000 7fa5b567c740 1 -- 172.21.15.112:0/1171138015 shutdown_connections 2024-09-21T03:29:45.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.880+0000 7fa5b567c740 1 --2- 172.21.15.112:0/1171138015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e125bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.880+0000 7fa5b567c740 1 --2- 172.21.15.112:0/1171138015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd60390 0x56435e12a890 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.880+0000 7fa5b567c740 1 --2- 172.21.15.112:0/1171138015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd581b0 0x56435e142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.880+0000 7fa5b567c740 1 -- 172.21.15.112:0/1171138015 >> 172.21.15.112:0/1171138015 conn(0x56435dd75460 msgr2=0x56435e12a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:45.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.880+0000 7fa5b567c740 1 -- 172.21.15.112:0/1171138015 shutdown_connections 2024-09-21T03:29:45.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.880+0000 7fa5b567c740 1 -- 172.21.15.112:0/1171138015 wait complete. 2024-09-21T03:29:45.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.881+0000 7fa5b567c740 1 -- start start 2024-09-21T03:29:45.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.881+0000 7fa5b567c740 1 Processor -- start 2024-09-21T03:29:45.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.881+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd581b0 0x56435e148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.881+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd60390 0x56435e146500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.881+0000 7fa5b567c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e146a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5b567c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81ab980 con 0x56435dd81070 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5b567c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81acdf0 con 0x56435dd60390 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5b567c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81ac050 con 0x56435dd581b0 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5b567c740 10 client.0 _ll_register_callbacks cb 0x7fa594005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e146a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45656/0 (socket says 172.21.15.112:45656) 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5a7fff640 1 -- 172.21.15.112:0/3490293851 learned_addr learned my addr 172.21.15.112:0/3490293851 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5a7fff640 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd581b0 0x56435e148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5acea4640 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd60390 0x56435e146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5a7fff640 1 -- 172.21.15.112:0/3490293851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd60390 msgr2=0x56435e146500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5a7fff640 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd60390 0x56435e146500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5a7fff640 1 -- 172.21.15.112:0/3490293851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 msgr2=0x56435e146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5a7fff640 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.882+0000 7fa5a7fff640 1 -- 172.21.15.112:0/3490293851 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56435e147180 con 0x56435dd581b0 2024-09-21T03:29:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.883+0000 7fa5acea4640 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd60390 0x56435e146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.883+0000 7fa5a7fff640 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd581b0 0x56435e148fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa594004700 tx=0x7fa594015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bb837c267ca5c76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.883+0000 7fa587fff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:29:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.883+0000 7fa587fff640 1 -- 172.21.15.112:0/3490293851 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594021040 con 0x56435dd581b0 2024-09-21T03:29:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.883+0000 7fa587fff640 1 -- 172.21.15.112:0/3490293851 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa594005220 con 0x56435dd581b0 2024-09-21T03:29:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.883+0000 7fa5b567c740 1 -- 172.21.15.112:0/3490293851 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa59c046e50 con 0x56435dd581b0 2024-09-21T03:29:45.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.883+0000 7fa587fff640 1 -- 172.21.15.112:0/3490293851 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa594009d90 con 0x56435dd581b0 2024-09-21T03:29:45.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.883+0000 7fa5b567c740 10 client.25276 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:45.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.883+0000 7fa5b567c740 1 -- 172.21.15.112:0/3490293851 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56435e1b8db0 con 0x56435dd581b0 2024-09-21T03:29:45.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.884+0000 7fa587fff640 1 -- 172.21.15.112:0/3490293851 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa59400ecf0 con 0x56435dd581b0 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.884+0000 7fa587fff640 1 -- 172.21.15.112:0/3490293851 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa59400f040 con 0x56435dd581b0 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.884+0000 7fa5b567c740 10 client.25276 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.884+0000 7fa5b567c740 10 client.25276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.884+0000 7fa5b567c740 20 client.25276 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.884+0000 7fa5b567c740 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56435dd14da0 0x56435e1bb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.884+0000 7fa5b567c740 4 client.25276 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.884+0000 7fa5b567c740 1 -- 172.21.15.112:0/3490293851 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa59c046e50 con 0x56435dd14da0 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.885+0000 7fa5acea4640 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56435dd14da0 0x56435e1bb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.885+0000 7fa587fff640 10 client.25276 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.885+0000 7fa5acea4640 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56435dd14da0 0x56435e1bb580 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b1ef0d18c2f2b4e server_cookie=87939be1b971ca4b in_seq=0 out_seq=0 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.885+0000 7fa587fff640 1 -- 172.21.15.112:0/3490293851 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa59c046e50 con 0x56435dd14da0 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.885+0000 7fa587fff640 10 client.25276 handle_command_reply: tid=0 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.885+0000 7fa5b567c740 1 client.25276 shutdown 2024-09-21T03:29:45.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.885+0000 7fa5b567c740 1 -- 172.21.15.112:0/3490293851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd581b0 msgr2=0x56435e148fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.885+0000 7fa5b567c740 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd581b0 0x56435e148fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa594004700 tx=0x7fa594015040 comp rx=0 tx=0).stop 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.886+0000 7fa5b567c740 1 -- 172.21.15.112:0/3490293851 shutdown_connections 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.886+0000 7fa5b567c740 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56435dd14da0 0x56435e1bb580 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.886+0000 7fa5b567c740 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56435dd81070 0x56435e146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.886+0000 7fa5b567c740 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56435dd60390 0x56435e146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.886+0000 7fa587fff640 0 client.25276 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.886+0000 7fa5b567c740 1 --2- 172.21.15.112:0/3490293851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56435dd581b0 0x56435e148fe0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.886+0000 7fa5b567c740 1 -- 172.21.15.112:0/3490293851 >> 172.21.15.112:0/3490293851 conn(0x56435dd75460 msgr2=0x56435e129e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.886+0000 7fa5b567c740 1 -- 172.21.15.112:0/3490293851 shutdown_connections 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.886+0000 7fa5b567c740 1 -- 172.21.15.112:0/3490293851 wait complete. 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.886+0000 7fa5b567c740 20 client.25276 trim_cache size 0 max 16384 2024-09-21T03:29:45.890 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:29:45.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.887+0000 7fa5af108640 1 -- 172.21.15.112:0/3099357787 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5980b6700 msgr2=0x7fa5980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:45.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.887+0000 7fa5af108640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5980b6700 0x7fa5980b8c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00ad3f0 tx=0x7fa5a00a9b90 comp rx=0 tx=0).stop 2024-09-21T03:29:45.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.887+0000 7fa5af108640 1 -- 172.21.15.112:0/3099357787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8106bd0 msgr2=0x7fa5a819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:45.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.887+0000 7fa5af108640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8106bd0 0x7fa5a819fce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000efc0 tx=0x7fa5a000c800 comp rx=0 tx=0).stop 2024-09-21T03:29:45.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.887+0000 7fa5af108640 1 -- 172.21.15.112:0/3099357787 shutdown_connections 2024-09-21T03:29:45.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.887+0000 7fa5af108640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa5980b6700 0x7fa5980b8c10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.887+0000 7fa5af108640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5a8106bd0 0x7fa5a819fce0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.887+0000 7fa5af108640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5a8106280 0x7fa5a819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.887+0000 7fa5af108640 1 --2- 172.21.15.112:0/3099357787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5a8101fa0 0x7fa5a819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:45.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.887+0000 7fa5af108640 1 -- 172.21.15.112:0/3099357787 >> 172.21.15.112:0/3099357787 conn(0x7fa5a8077ca0 msgr2=0x7fa5a8107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:45.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.888+0000 7fa5af108640 1 -- 172.21.15.112:0/3099357787 shutdown_connections 2024-09-21T03:29:45.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:45.888+0000 7fa5af108640 1 -- 172.21.15.112:0/3099357787 wait complete. 2024-09-21T03:29:46.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:46 smithi112 ceph-mon[31442]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-21T03:29:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:46 smithi203 ceph-mon[34723]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-21T03:29:46.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:46 smithi187 ceph-mon[34603]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-21T03:29:47.582 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:29:47.583 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:29:47.943 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.532+0000 7f8dc5176640 1 -- 172.21.15.112:0/1861691671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0109e00 msgr2=0x7f8dc0110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.532+0000 7f8dc5176640 1 --2- 172.21.15.112:0/1861691671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0109e00 0x7f8dc0110630 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009ca0 tx=0x7f8db802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.533+0000 7f8dc5176640 1 -- 172.21.15.112:0/1861691671 shutdown_connections 2024-09-21T03:29:48.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.533+0000 7f8dc5176640 1 --2- 172.21.15.112:0/1861691671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc0109e00 0x7f8dc0110630 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.533+0000 7f8dc5176640 1 --2- 172.21.15.112:0/1861691671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc01094b0 0x7f8dc01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.533+0000 7f8dc5176640 1 --2- 172.21.15.112:0/1861691671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0101f70 0x7f8dc0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.533+0000 7f8dc5176640 1 -- 172.21.15.112:0/1861691671 >> 172.21.15.112:0/1861691671 conn(0x7f8dc0077c60 msgr2=0x7f8dc00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.533+0000 7f8dc5176640 1 -- 172.21.15.112:0/1861691671 shutdown_connections 2024-09-21T03:29:48.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.534+0000 7f8dc5176640 1 -- 172.21.15.112:0/1861691671 wait complete. 2024-09-21T03:29:48.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.534+0000 7f8dc5176640 1 Processor -- start 2024-09-21T03:29:48.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.535+0000 7f8dc5176640 1 -- start start 2024-09-21T03:29:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.535+0000 7f8dc5176640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0101f70 0x7f8dc0102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.535+0000 7f8dc5176640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc01094b0 0x7f8dc0103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.535+0000 7f8dc5176640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0109e00 0x7f8dc0103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.535+0000 7f8dc5176640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc0113ac0 con 0x7f8dc01094b0 2024-09-21T03:29:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.535+0000 7f8dc5176640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc0113940 con 0x7f8dc0109e00 2024-09-21T03:29:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.535+0000 7f8dc5176640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc0113c40 con 0x7f8dc0101f70 2024-09-21T03:29:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.535+0000 7f8dbe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc01094b0 0x7f8dc0103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.535+0000 7f8dbf59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0109e00 0x7f8dc0103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.535+0000 7f8dbe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc01094b0 0x7f8dc0103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45664/0 (socket says 172.21.15.112:45664) 2024-09-21T03:29:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.535+0000 7f8dbe59c640 1 -- 172.21.15.112:0/152247160 learned_addr learned my addr 172.21.15.112:0/152247160 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8dbed9d640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0101f70 0x7f8dc0102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8dbe59c640 1 -- 172.21.15.112:0/152247160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0101f70 msgr2=0x7f8dc0102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0101f70 0x7f8dc0102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8dbe59c640 1 -- 172.21.15.112:0/152247160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0109e00 msgr2=0x7f8dc0103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:48.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0109e00 0x7f8dc0103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8dbe59c640 1 -- 172.21.15.112:0/152247160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc01a8750 con 0x7f8dc01094b0 2024-09-21T03:29:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8dbf59e640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0109e00 0x7f8dc0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc01094b0 0x7f8dc0103450 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8db400e9f0 tx=0x7f8db400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8da7fff640 1 -- 172.21.15.112:0/152247160 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db400ccc0 con 0x7f8dc01094b0 2024-09-21T03:29:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8dc5176640 1 -- 172.21.15.112:0/152247160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc01a8a40 con 0x7f8dc01094b0 2024-09-21T03:29:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8da7fff640 1 -- 172.21.15.112:0/152247160 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db40108e0 con 0x7f8dc01094b0 2024-09-21T03:29:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.536+0000 7f8da7fff640 1 -- 172.21.15.112:0/152247160 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4018780 con 0x7f8dc01094b0 2024-09-21T03:29:48.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.537+0000 7f8dc5176640 1 -- 172.21.15.112:0/152247160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc01a8fd0 con 0x7f8dc01094b0 2024-09-21T03:29:48.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.541+0000 7f8da7fff640 1 -- 172.21.15.112:0/152247160 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8db4010430 con 0x7f8dc01094b0 2024-09-21T03:29:48.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.541+0000 7f8dcb6ea740 1 Processor -- start 2024-09-21T03:29:48.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.541+0000 7f8dcb6ea740 1 -- start start 2024-09-21T03:29:48.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.541+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96c6350 0x55a2e9a6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.541+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a6d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.541+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a6deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.541+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a2e99e3e60 con 0x55a2e96a5670 2024-09-21T03:29:48.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.541+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a2e99d3290 con 0x55a2e96c6350 2024-09-21T03:29:48.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.541+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a2e99d70a0 con 0x55a2e969d490 2024-09-21T03:29:48.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.542+0000 7f8dbe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a6deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.542+0000 7f8dbed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a6d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.542+0000 7f8dbe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a6deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45676/0 (socket says 172.21.15.112:45676) 2024-09-21T03:29:48.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.542+0000 7f8dbe59c640 1 -- 172.21.15.112:0/1065165121 learned_addr learned my addr 172.21.15.112:0/1065165121 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:48.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.542+0000 7f8dbf59e640 1 --2- 172.21.15.112:0/1065165121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96c6350 0x55a2e9a6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.542+0000 7f8dbe59c640 1 -- 172.21.15.112:0/1065165121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 msgr2=0x55a2e9a6d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:48.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.542+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/1065165121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a6d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.542+0000 7f8dbe59c640 1 -- 172.21.15.112:0/1065165121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96c6350 msgr2=0x55a2e9a6d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:48.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.542+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/1065165121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96c6350 0x55a2e9a6d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.542+0000 7f8dbe59c640 1 -- 172.21.15.112:0/1065165121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e9731930 con 0x55a2e96a5670 2024-09-21T03:29:48.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.543+0000 7f8da7fff640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8da80b6700 0x7f8da80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.543+0000 7f8da7fff640 1 -- 172.21.15.112:0/152247160 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f8db40e7b40 con 0x7f8dc01094b0 2024-09-21T03:29:48.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.543+0000 7f8dbf59e640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8da80b6700 0x7f8da80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.543+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/1065165121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a6deb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8db40e5260 tx=0x7f8db40f0680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b5a2229678e078 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:48.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.544+0000 7f8dbf59e640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8da80b6700 0x7f8da80b8c10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8db8006010 tx=0x7f8db80094a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:48.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.544+0000 7f8da57fa640 1 -- 172.21.15.112:0/1065165121 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db40aa040 con 0x55a2e96a5670 2024-09-21T03:29:48.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.544+0000 7f8da57fa640 1 -- 172.21.15.112:0/1065165121 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db40a9150 con 0x55a2e96a5670 2024-09-21T03:29:48.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.544+0000 7f8da57fa640 1 -- 172.21.15.112:0/1065165121 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db40ad160 con 0x55a2e96a5670 2024-09-21T03:29:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.545+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/1065165121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 msgr2=0x55a2e9a6deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.545+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/1065165121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a6deb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8db40e5260 tx=0x7f8db40f0680 comp rx=0 tx=0).stop 2024-09-21T03:29:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.545+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/1065165121 shutdown_connections 2024-09-21T03:29:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.545+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/1065165121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a6deb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.545+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/1065165121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a6d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.545+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/1065165121 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96c6350 0x55a2e9a6d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.545+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/1065165121 >> 172.21.15.112:0/1065165121 conn(0x55a2e96ba740 msgr2=0x55a2e9a6b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.545+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/1065165121 shutdown_connections 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.545+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/1065165121 wait complete. 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.546+0000 7f8dcb6ea740 1 -- start start 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.546+0000 7f8dcb6ea740 1 Processor -- start 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.546+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e969d490 0x55a2e9a81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.546+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e96a5670 0x55a2e9a81e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.546+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a8d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.546+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a2e9834890 con 0x55a2e96c6350 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.546+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a2e996dd40 con 0x55a2e969d490 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.546+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a2e9a84340 con 0x55a2e96a5670 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.546+0000 7f8dcb6ea740 10 client.0 _ll_register_callbacks cb 0x55a2e99d1c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:48.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.546+0000 7f8dbf59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a8d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8dbf59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a8d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45684/0 (socket says 172.21.15.112:45684) 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8dbf59e640 1 -- 172.21.15.112:0/3626050271 learned_addr learned my addr 172.21.15.112:0/3626050271 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8dbed9d640 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e969d490 0x55a2e9a81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8dbf59e640 1 -- 172.21.15.112:0/3626050271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e96a5670 msgr2=0x55a2e9a81e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e96a5670 0x55a2e9a81e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8dbf59e640 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e96a5670 0x55a2e9a81e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8dbf59e640 1 -- 172.21.15.112:0/3626050271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e969d490 msgr2=0x55a2e9a81890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8dbf59e640 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e969d490 0x55a2e9a81890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8dbf59e640 1 -- 172.21.15.112:0/3626050271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e9a8dd10 con 0x55a2e96c6350 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e96a5670 0x55a2e9a81e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8dbf59e640 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a8d6a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8db8031a30 tx=0x7f8db802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946ecb1999f725ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.547+0000 7f8da2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.548+0000 7f8da2ff5640 1 -- 172.21.15.112:0/3626050271 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8004040 con 0x55a2e96c6350 2024-09-21T03:29:48.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.548+0000 7f8da2ff5640 1 -- 172.21.15.112:0/3626050271 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f8db80041e0 con 0x55a2e96c6350 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.548+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/3626050271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8db40a9150 con 0x55a2e96c6350 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.548+0000 7f8da2ff5640 1 -- 172.21.15.112:0/3626050271 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8db80043e0 con 0x55a2e96c6350 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.548+0000 7f8dcb6ea740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.548+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/3626050271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2e9a90620 con 0x55a2e96c6350 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.548+0000 7f8da2ff5640 1 -- 172.21.15.112:0/3626050271 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db40a9150 con 0x55a2e96c6350 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.549+0000 7f8da2ff5640 1 -- 172.21.15.112:0/3626050271 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8db805b020 con 0x55a2e96c6350 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.549+0000 7f8dcb6ea740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.549+0000 7f8dcb6ea740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.549+0000 7f8dcb6ea740 20 client.15684 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.549+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a2e9684cd0 0x55a2e9a93b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.549+0000 7f8dcb6ea740 4 client.15684 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.549+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/3626050271 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2e99e5db0 con 0x55a2e9684cd0 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.549+0000 7f8dbed9d640 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a2e9684cd0 0x55a2e9a93b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.550+0000 7f8da2ff5640 10 client.15684 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:48.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.550+0000 7f8dbed9d640 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a2e9684cd0 0x55a2e9a93b60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b34c96500511283b server_cookie=10014225c536e30e in_seq=0 out_seq=0 2024-09-21T03:29:48.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8da2ff5640 1 -- 172.21.15.112:0/3626050271 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2e99e5db0 con 0x55a2e9684cd0 2024-09-21T03:29:48.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8da2ff5640 10 client.15684 handle_command_reply: tid=0 2024-09-21T03:29:48.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 1 client.15684 shutdown 2024-09-21T03:29:48.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/3626050271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 msgr2=0x55a2e9a8d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:48.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a8d6a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8db8031a30 tx=0x7f8db802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/3626050271 shutdown_connections 2024-09-21T03:29:48.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a2e9684cd0 0x55a2e9a93b60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a8d6a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e96a5670 0x55a2e9a81e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/3626050271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e969d490 0x55a2e9a81890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/3626050271 >> 172.21.15.112:0/3626050271 conn(0x55a2e96ba740 msgr2=0x55a2e9a69850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:48.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8da2ff5640 0 client.15684 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:48.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/3626050271 shutdown_connections 2024-09-21T03:29:48.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/3626050271 wait complete. 2024-09-21T03:29:48.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.551+0000 7f8dcb6ea740 20 client.15684 trim_cache size 0 max 16384 2024-09-21T03:29:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.569+0000 7f8dcb6ea740 1 Processor -- start 2024-09-21T03:29:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.569+0000 7f8dcb6ea740 1 -- start start 2024-09-21T03:29:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.569+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a7ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a80450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96c6350 0x55a2e9a6ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01afdc0 con 0x55a2e96a5670 2024-09-21T03:29:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01ad690 con 0x55a2e96c6350 2024-09-21T03:29:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01ab4e0 con 0x55a2e969d490 2024-09-21T03:29:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dbe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a80450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dbe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a80450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45698/0 (socket says 172.21.15.112:45698) 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dbe59c640 1 -- 172.21.15.112:0/4252581802 learned_addr learned my addr 172.21.15.112:0/4252581802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dbed9d640 1 --2- 172.21.15.112:0/4252581802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a7ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dbf59e640 1 --2- 172.21.15.112:0/4252581802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96c6350 0x55a2e9a6ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dbe59c640 1 -- 172.21.15.112:0/4252581802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 msgr2=0x55a2e9a7ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/4252581802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a7ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dbe59c640 1 -- 172.21.15.112:0/4252581802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96c6350 msgr2=0x55a2e9a6ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/4252581802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96c6350 0x55a2e9a6ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dbe59c640 1 -- 172.21.15.112:0/4252581802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e9a940d0 con 0x55a2e96a5670 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.570+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/4252581802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a80450 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8db40e53a0 tx=0x7f8db40ebca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2033878e2d437d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.571+0000 7f8da4ff9640 1 -- 172.21.15.112:0/4252581802 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db40b2030 con 0x55a2e96a5670 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.571+0000 7f8da4ff9640 1 -- 172.21.15.112:0/4252581802 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db40eb4e0 con 0x55a2e96a5670 2024-09-21T03:29:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.571+0000 7f8da4ff9640 1 -- 172.21.15.112:0/4252581802 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db40ad340 con 0x55a2e96a5670 2024-09-21T03:29:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.572+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/4252581802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 msgr2=0x55a2e9a80450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.572+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/4252581802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a80450 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8db40e53a0 tx=0x7f8db40ebca0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.572+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/4252581802 shutdown_connections 2024-09-21T03:29:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.572+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/4252581802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96c6350 0x55a2e9a6ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.572+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/4252581802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96a5670 0x55a2e9a80450 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.572+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/4252581802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a7ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.572+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/4252581802 >> 172.21.15.112:0/4252581802 conn(0x55a2e96ba740 msgr2=0x55a2e9a7c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.572+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/4252581802 shutdown_connections 2024-09-21T03:29:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.572+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/4252581802 wait complete. 2024-09-21T03:29:48.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.572+0000 7f8dcb6ea740 1 -- start start 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.572+0000 7f8dcb6ea740 1 Processor -- start 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a89a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96a5670 0x55a2e9a89fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dcb6ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a84720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01abb50 con 0x55a2e96c6350 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01acfc0 con 0x55a2e96a5670 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dcb6ea740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01ac220 con 0x55a2e969d490 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dcb6ea740 10 client.0 _ll_register_callbacks cb 0x7f8db8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbf59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a84720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbf59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a84720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45710/0 (socket says 172.21.15.112:45710) 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbf59e640 1 -- 172.21.15.112:0/2984434166 learned_addr learned my addr 172.21.15.112:0/2984434166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96a5670 0x55a2e9a89fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbed9d640 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a89a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbf59e640 1 -- 172.21.15.112:0/2984434166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 msgr2=0x55a2e9a89a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbf59e640 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a89a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbf59e640 1 -- 172.21.15.112:0/2984434166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96a5670 msgr2=0x55a2e9a89fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbf59e640 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96a5670 0x55a2e9a89fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbf59e640 1 -- 172.21.15.112:0/2984434166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e9a84e90 con 0x55a2e96c6350 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbe59c640 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96a5670 0x55a2e9a89fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbed9d640 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a89a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8dbf59e640 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a84720 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8db80305c0 tx=0x7f8db80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec69606442f8242 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.573+0000 7f8da2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8da2ff5640 1 -- 172.21.15.112:0/2984434166 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db80309b0 con 0x55a2e96c6350 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/2984434166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8db40eb4e0 con 0x55a2e96c6350 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8da2ff5640 1 -- 172.21.15.112:0/2984434166 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f8db8030b50 con 0x55a2e96c6350 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8da2ff5640 1 -- 172.21.15.112:0/2984434166 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8db8030d50 con 0x55a2e96c6350 2024-09-21T03:29:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8dcb6ea740 10 client.15690 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/2984434166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a2e9aff0e0 con 0x55a2e96c6350 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8da2ff5640 1 -- 172.21.15.112:0/2984434166 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db40eb4e0 con 0x55a2e96c6350 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8da2ff5640 1 -- 172.21.15.112:0/2984434166 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8db805b070 con 0x55a2e96c6350 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8dcb6ea740 10 client.15690 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8dcb6ea740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8dcb6ea740 20 client.15690 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a2e965a080 0x55a2e9b01870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8dcb6ea740 4 client.15690 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.574+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/2984434166 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a2e99e1860 con 0x55a2e965a080 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.576+0000 7f8dbed9d640 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a2e965a080 0x55a2e9b01870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.576+0000 7f8da2ff5640 10 client.15690 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.576+0000 7f8dbed9d640 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a2e965a080 0x55a2e9b01870 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2913548f43dc4bc4 server_cookie=6f2059a5ad3b5492 in_seq=0 out_seq=0 2024-09-21T03:29:48.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8da2ff5640 1 -- 172.21.15.112:0/2984434166 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a2e99e1860 con 0x55a2e965a080 2024-09-21T03:29:48.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8da2ff5640 10 client.15690 handle_command_reply: tid=0 2024-09-21T03:29:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8dcb6ea740 1 client.15690 shutdown 2024-09-21T03:29:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/2984434166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 msgr2=0x55a2e9a84720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a84720 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8db80305c0 tx=0x7f8db80047b0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/2984434166 shutdown_connections 2024-09-21T03:29:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a2e965a080 0x55a2e9b01870 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8da2ff5640 0 client.15690 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2e96c6350 0x55a2e9a84720 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2e96a5670 0x55a2e9a89fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8dcb6ea740 1 --2- 172.21.15.112:0/2984434166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a2e969d490 0x55a2e9a89a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.578+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/2984434166 >> 172.21.15.112:0/2984434166 conn(0x55a2e96ba740 msgr2=0x55a2e9a6abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:48.583 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:29:48.584 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:29:48.584 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "11f692fb-1419-449e-b703-28887d34acb4", 2024-09-21T03:29:48.584 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:29:48.584 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:29:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/2984434166 shutdown_connections 2024-09-21T03:29:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dcb6ea740 1 -- 172.21.15.112:0/2984434166 wait complete. 2024-09-21T03:29:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dcb6ea740 20 client.15690 trim_cache size 0 max 16384 2024-09-21T03:29:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dc5176640 1 -- 172.21.15.112:0/152247160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8da80b6700 msgr2=0x7f8da80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dc5176640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8da80b6700 0x7f8da80b8c10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8db8006010 tx=0x7f8db80094a0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dc5176640 1 -- 172.21.15.112:0/152247160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc01094b0 msgr2=0x7f8dc0103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dc5176640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc01094b0 0x7f8dc0103450 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8db400e9f0 tx=0x7f8db400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dc5176640 1 -- 172.21.15.112:0/152247160 shutdown_connections 2024-09-21T03:29:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dc5176640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8da80b6700 0x7f8da80b8c10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dc5176640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0109e00 0x7f8dc0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dc5176640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dc01094b0 0x7f8dc0103450 secure :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8db400e9f0 tx=0x7f8db400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dc5176640 1 --2- 172.21.15.112:0/152247160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc0101f70 0x7f8dc0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.579+0000 7f8dc5176640 1 -- 172.21.15.112:0/152247160 >> 172.21.15.112:0/152247160 conn(0x7f8dc0077c60 msgr2=0x7f8dc0105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.580+0000 7f8dc5176640 1 -- 172.21.15.112:0/152247160 shutdown_connections 2024-09-21T03:29:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:48.580+0000 7f8dc5176640 1 -- 172.21.15.112:0/152247160 wait complete. 2024-09-21T03:29:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:48 smithi112 ceph-mon[31442]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-21T03:29:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:48 smithi203 ceph-mon[34723]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-21T03:29:48.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:48 smithi187 ceph-mon[34603]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-21T03:29:49.288 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:29:49.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:49 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:29:49.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:49 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:49.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:49 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:29:49.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:49 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:49.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:49 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:29:49.644 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:49 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:29:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:49 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:49 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:29:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:49 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:49 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:29:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:49 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:29:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:49 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:49 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:29:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:49 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:29:49.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:49 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:29:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.180+0000 7fc238f1b640 1 -- 172.21.15.112:0/3097504577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc234102a60 msgr2=0x7fc234102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.180+0000 7fc238f1b640 1 --2- 172.21.15.112:0/3097504577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc234102a60 0x7fc234102e70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc224009950 tx=0x7fc22402a440 comp rx=0 tx=0).stop 2024-09-21T03:29:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.180+0000 7fc238f1b640 1 -- 172.21.15.112:0/3097504577 shutdown_connections 2024-09-21T03:29:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.180+0000 7fc238f1b640 1 --2- 172.21.15.112:0/3097504577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc234108cf0 0x7fc23410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.180+0000 7fc238f1b640 1 --2- 172.21.15.112:0/3097504577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2341083a0 0x7fc2341087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.180+0000 7fc238f1b640 1 --2- 172.21.15.112:0/3097504577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc234102a60 0x7fc234102e70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.180+0000 7fc238f1b640 1 -- 172.21.15.112:0/3097504577 >> 172.21.15.112:0/3097504577 conn(0x7fc234079730 msgr2=0x7fc234079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.181+0000 7fc238f1b640 1 -- 172.21.15.112:0/3097504577 shutdown_connections 2024-09-21T03:29:50.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.181+0000 7fc238f1b640 1 -- 172.21.15.112:0/3097504577 wait complete. 2024-09-21T03:29:50.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.182+0000 7fc238f1b640 1 Processor -- start 2024-09-21T03:29:50.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.182+0000 7fc238f1b640 1 -- start start 2024-09-21T03:29:50.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.182+0000 7fc238f1b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2341083a0 0x7fc23419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.182+0000 7fc238f1b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc234108cf0 0x7fc23419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.182+0000 7fc238f1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23419fd60 0x7fc2341a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.182+0000 7fc238f1b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc234112960 con 0x7fc23419fd60 2024-09-21T03:29:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.182+0000 7fc238f1b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341127e0 con 0x7fc2341083a0 2024-09-21T03:29:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.182+0000 7fc238f1b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc234112ae0 con 0x7fc234108cf0 2024-09-21T03:29:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.183+0000 7fc232d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23419fd60 0x7fc2341a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.183+0000 7fc232d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23419fd60 0x7fc2341a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45724/0 (socket says 172.21.15.112:45724) 2024-09-21T03:29:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.183+0000 7fc232d9d640 1 -- 172.21.15.112:0/748659801 learned_addr learned my addr 172.21.15.112:0/748659801 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.183+0000 7fc23259c640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2341083a0 0x7fc23419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.183+0000 7fc229d9b640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc234108cf0 0x7fc23419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.183+0000 7fc232d9d640 1 -- 172.21.15.112:0/748659801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc234108cf0 msgr2=0x7fc23419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.183+0000 7fc232d9d640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc234108cf0 0x7fc23419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.183+0000 7fc232d9d640 1 -- 172.21.15.112:0/748659801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2341083a0 msgr2=0x7fc23419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.183+0000 7fc232d9d640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2341083a0 0x7fc23419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.183+0000 7fc232d9d640 1 -- 172.21.15.112:0/748659801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2341a0940 con 0x7fc23419fd60 2024-09-21T03:29:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.184+0000 7fc232d9d640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23419fd60 0x7fc2341a0170 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00ed70 tx=0x7fc22c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.184+0000 7fc22bfff640 1 -- 172.21.15.112:0/748659801 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c004b20 con 0x7fc23419fd60 2024-09-21T03:29:50.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.184+0000 7fc22bfff640 1 -- 172.21.15.112:0/748659801 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc22c004cc0 con 0x7fc23419fd60 2024-09-21T03:29:50.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.184+0000 7fc22bfff640 1 -- 172.21.15.112:0/748659801 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c010690 con 0x7fc23419fd60 2024-09-21T03:29:50.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.184+0000 7fc238f1b640 1 -- 172.21.15.112:0/748659801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2341a6ee0 con 0x7fc23419fd60 2024-09-21T03:29:50.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.184+0000 7fc238f1b640 1 -- 172.21.15.112:0/748659801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23410b6d0 con 0x7fc23419fd60 2024-09-21T03:29:50.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.187+0000 7fc23f48f740 1 Processor -- start 2024-09-21T03:29:50.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.188+0000 7fc22bfff640 1 -- 172.21.15.112:0/748659801 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc22c0108f0 con 0x7fc23419fd60 2024-09-21T03:29:50.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.189+0000 7fc23f48f740 1 -- start start 2024-09-21T03:29:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.189+0000 7fc23f48f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563494403070 0x5634947aa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.189+0000 7fc23f48f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943da1b0 0x5634947aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.189+0000 7fc23f48f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943e2390 0x5634947aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.189+0000 7fc23f48f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563494720e50 con 0x563494403070 2024-09-21T03:29:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.189+0000 7fc23f48f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563494710280 con 0x5634943da1b0 2024-09-21T03:29:50.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.189+0000 7fc23f48f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563494714090 con 0x5634943e2390 2024-09-21T03:29:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.189+0000 7fc229d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563494403070 0x5634947aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.189+0000 7fc229d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563494403070 0x5634947aa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45734/0 (socket says 172.21.15.112:45734) 2024-09-21T03:29:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.189+0000 7fc229d9b640 1 -- 172.21.15.112:0/2860347968 learned_addr learned my addr 172.21.15.112:0/2860347968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.190+0000 7fc23259c640 1 --2- 172.21.15.112:0/2860347968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943da1b0 0x5634947aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.190+0000 7fc229d9b640 1 --2- 172.21.15.112:0/2860347968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943e2390 0x5634947aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.190+0000 7fc229d9b640 1 -- 172.21.15.112:0/2860347968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943e2390 msgr2=0x5634947aacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.190+0000 7fc229d9b640 1 --2- 172.21.15.112:0/2860347968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943e2390 0x5634947aacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.190+0000 7fc229d9b640 1 -- 172.21.15.112:0/2860347968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943da1b0 msgr2=0x5634947aa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.190+0000 7fc229d9b640 1 --2- 172.21.15.112:0/2860347968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943da1b0 0x5634947aa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.190+0000 7fc229d9b640 1 -- 172.21.15.112:0/2860347968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349446e650 con 0x563494403070 2024-09-21T03:29:50.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.190+0000 7fc23259c640 1 --2- 172.21.15.112:0/2860347968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943da1b0 0x5634947aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.190+0000 7fc22bfff640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2200bef10 0x7fc2200c1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.190+0000 7fc229d9b640 1 --2- 172.21.15.112:0/2860347968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563494403070 0x5634947aa270 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc21c013780 tx=0x7fc21c009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2534ba76bd0cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.191+0000 7fc22bfff640 1 -- 172.21.15.112:0/748659801 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc22c0e7c10 con 0x7fc23419fd60 2024-09-21T03:29:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.191+0000 7fc23259c640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2200bef10 0x7fc2200c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.191+0000 7fc228d99640 1 -- 172.21.15.112:0/2860347968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c013130 con 0x563494403070 2024-09-21T03:29:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.191+0000 7fc228d99640 1 -- 172.21.15.112:0/2860347968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc21c005020 con 0x563494403070 2024-09-21T03:29:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.191+0000 7fc228d99640 1 -- 172.21.15.112:0/2860347968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00faa0 con 0x563494403070 2024-09-21T03:29:50.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.191+0000 7fc23259c640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2200bef10 0x7fc2200c1420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc224031440 tx=0x7fc224005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.193+0000 7fc23f48f740 1 -- 172.21.15.112:0/2860347968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563494403070 msgr2=0x5634947aa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.193+0000 7fc23f48f740 1 --2- 172.21.15.112:0/2860347968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563494403070 0x5634947aa270 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc21c013780 tx=0x7fc21c009830 comp rx=0 tx=0).stop 2024-09-21T03:29:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.193+0000 7fc23f48f740 1 -- 172.21.15.112:0/2860347968 shutdown_connections 2024-09-21T03:29:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.193+0000 7fc23f48f740 1 --2- 172.21.15.112:0/2860347968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943e2390 0x5634947aacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.193+0000 7fc23f48f740 1 --2- 172.21.15.112:0/2860347968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943da1b0 0x5634947aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.193+0000 7fc23f48f740 1 --2- 172.21.15.112:0/2860347968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563494403070 0x5634947aa270 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.193+0000 7fc23f48f740 1 -- 172.21.15.112:0/2860347968 >> 172.21.15.112:0/2860347968 conn(0x5634943f7460 msgr2=0x5634947a8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.193+0000 7fc23f48f740 1 -- 172.21.15.112:0/2860347968 shutdown_connections 2024-09-21T03:29:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.193+0000 7fc23f48f740 1 -- 172.21.15.112:0/2860347968 wait complete. 2024-09-21T03:29:50.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.194+0000 7fc23f48f740 1 -- start start 2024-09-21T03:29:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.194+0000 7fc23f48f740 1 Processor -- start 2024-09-21T03:29:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.194+0000 7fc23f48f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943da1b0 0x5634947b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.195+0000 7fc23f48f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 0x5634947b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.195+0000 7fc229d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 0x5634947b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.195+0000 7fc229d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 0x5634947b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45744/0 (socket says 172.21.15.112:45744) 2024-09-21T03:29:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.195+0000 7fc229d9b640 1 -- 172.21.15.112:0/3471340239 learned_addr learned my addr 172.21.15.112:0/3471340239 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.195+0000 7fc23259c640 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943da1b0 0x5634947b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.195+0000 7fc23f48f740 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563494403070 0x5634947c1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.195+0000 7fc23f48f740 1 -- 172.21.15.112:0/3471340239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563494571700 con 0x5634943e2390 2024-09-21T03:29:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.195+0000 7fc23f48f740 1 -- 172.21.15.112:0/3471340239 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5634946ab110 con 0x5634943da1b0 2024-09-21T03:29:50.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.195+0000 7fc23f48f740 1 -- 172.21.15.112:0/3471340239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5634947b89f0 con 0x563494403070 2024-09-21T03:29:50.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.195+0000 7fc23f48f740 10 client.0 _ll_register_callbacks cb 0x5634946c4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:50.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.196+0000 7fc229d9b640 1 -- 172.21.15.112:0/3471340239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563494403070 msgr2=0x5634947c1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:50.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.196+0000 7fc229d9b640 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563494403070 0x5634947c1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.196+0000 7fc229d9b640 1 -- 172.21.15.112:0/3471340239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943da1b0 msgr2=0x5634947b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.196+0000 7fc229d9b640 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943da1b0 0x5634947b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.196+0000 7fc229d9b640 1 -- 172.21.15.112:0/3471340239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634946d8c40 con 0x5634943e2390 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.196+0000 7fc229d9b640 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 0x5634947b5ea0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc21c005760 tx=0x7fc21c00f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad50aca261cd657 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.196+0000 7fc219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.196+0000 7fc23f48f740 1 -- 172.21.15.112:0/3471340239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc21c005020 con 0x5634943e2390 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.196+0000 7fc219ffb640 1 -- 172.21.15.112:0/3471340239 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00fe70 con 0x5634943e2390 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.197+0000 7fc219ffb640 1 -- 172.21.15.112:0/3471340239 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc21c0035f0 con 0x5634943e2390 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.197+0000 7fc219ffb640 1 -- 172.21.15.112:0/3471340239 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc21c003870 con 0x5634943e2390 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.197+0000 7fc23f48f740 10 client.15702 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.197+0000 7fc23f48f740 1 -- 172.21.15.112:0/3471340239 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5634947c4540 con 0x5634943e2390 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.197+0000 7fc219ffb640 1 -- 172.21.15.112:0/3471340239 <== mon.0 v2:172.21.15.112:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc21c003a10 con 0x5634943e2390 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.197+0000 7fc23f48f740 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:50.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.197+0000 7fc23f48f740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.198+0000 7fc23f48f740 20 client.15702 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.198+0000 7fc23f48f740 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5634943c19f0 0x5634947c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.198+0000 7fc23f48f740 4 client.15702 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.198+0000 7fc23f48f740 1 -- 172.21.15.112:0/3471340239 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc21c005020 con 0x5634943c19f0 2024-09-21T03:29:50.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.198+0000 7fc23259c640 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5634943c19f0 0x5634947c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.198+0000 7fc219ffb640 10 client.15702 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.198+0000 7fc23259c640 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5634943c19f0 0x5634947c78a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3027d17fcbb7c3b2 server_cookie=d9b898efe5a27531 in_seq=0 out_seq=0 2024-09-21T03:29:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.199+0000 7fc219ffb640 1 -- 172.21.15.112:0/3471340239 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc21c005020 con 0x5634943c19f0 2024-09-21T03:29:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.199+0000 7fc219ffb640 10 client.15702 handle_command_reply: tid=0 2024-09-21T03:29:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.199+0000 7fc23f48f740 1 client.15702 shutdown 2024-09-21T03:29:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.199+0000 7fc23f48f740 1 -- 172.21.15.112:0/3471340239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 msgr2=0x5634947b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.199+0000 7fc23f48f740 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 0x5634947b5ea0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc21c005760 tx=0x7fc21c00f840 comp rx=0 tx=0).stop 2024-09-21T03:29:50.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.200+0000 7fc23f48f740 1 -- 172.21.15.112:0/3471340239 shutdown_connections 2024-09-21T03:29:50.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.200+0000 7fc23f48f740 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5634943c19f0 0x5634947c78a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.200+0000 7fc23f48f740 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563494403070 0x5634947c1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.200+0000 7fc23f48f740 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 0x5634947b5ea0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.200+0000 7fc219ffb640 0 client.15702 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:50.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.200+0000 7fc23f48f740 1 --2- 172.21.15.112:0/3471340239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943da1b0 0x5634947b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.200+0000 7fc23f48f740 1 -- 172.21.15.112:0/3471340239 >> 172.21.15.112:0/3471340239 conn(0x5634943f7460 msgr2=0x5634947ad450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:50.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.200+0000 7fc23f48f740 1 -- 172.21.15.112:0/3471340239 shutdown_connections 2024-09-21T03:29:50.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.200+0000 7fc23f48f740 1 -- 172.21.15.112:0/3471340239 wait complete. 2024-09-21T03:29:50.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.200+0000 7fc23f48f740 20 client.15702 trim_cache size 0 max 16384 2024-09-21T03:29:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc23f48f740 1 Processor -- start 2024-09-21T03:29:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc23f48f740 1 -- start start 2024-09-21T03:29:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc23f48f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 0x5634947ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc23f48f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943da1b0 0x5634947c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc23f48f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943e2390 0x5634947a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc229d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943da1b0 0x5634947c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc229d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943da1b0 0x5634947c1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45748/0 (socket says 172.21.15.112:45748) 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc23f48f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341b0350 con 0x5634943da1b0 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc229d9b640 1 -- 172.21.15.112:0/3015279250 learned_addr learned my addr 172.21.15.112:0/3015279250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc23f48f740 1 -- 172.21.15.112:0/3015279250 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341adc20 con 0x5634943c19f0 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc23f48f740 1 -- 172.21.15.112:0/3015279250 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341aba70 con 0x5634943e2390 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.235+0000 7fc23259c640 1 --2- 172.21.15.112:0/3015279250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 0x5634947ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.236+0000 7fc23259c640 1 -- 172.21.15.112:0/3015279250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943e2390 msgr2=0x5634947a7ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.236+0000 7fc23259c640 1 --2- 172.21.15.112:0/3015279250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943e2390 0x5634947a7ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.236+0000 7fc23259c640 1 -- 172.21.15.112:0/3015279250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943da1b0 msgr2=0x5634947c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.236+0000 7fc23259c640 1 --2- 172.21.15.112:0/3015279250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943da1b0 0x5634947c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.236+0000 7fc23259c640 1 -- 172.21.15.112:0/3015279250 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634947ac9a0 con 0x5634943c19f0 2024-09-21T03:29:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.236+0000 7fc229d9b640 1 --2- 172.21.15.112:0/3015279250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943da1b0 0x5634947c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.236+0000 7fc23259c640 1 --2- 172.21.15.112:0/3015279250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 0x5634947ada80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc224008000 tx=0x7fc22403e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9714d0c452db4cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.236+0000 7fc21bfff640 1 -- 172.21.15.112:0/3015279250 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224035050 con 0x5634943c19f0 2024-09-21T03:29:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.236+0000 7fc21bfff640 1 -- 172.21.15.112:0/3015279250 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc224004df0 con 0x5634943c19f0 2024-09-21T03:29:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.236+0000 7fc21bfff640 1 -- 172.21.15.112:0/3015279250 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240314a0 con 0x5634943c19f0 2024-09-21T03:29:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.238+0000 7fc23f48f740 1 -- 172.21.15.112:0/3015279250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 msgr2=0x5634947ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.238+0000 7fc23f48f740 1 --2- 172.21.15.112:0/3015279250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 0x5634947ada80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc224008000 tx=0x7fc22403e000 comp rx=0 tx=0).stop 2024-09-21T03:29:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.238+0000 7fc23f48f740 1 -- 172.21.15.112:0/3015279250 shutdown_connections 2024-09-21T03:29:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.238+0000 7fc23f48f740 1 --2- 172.21.15.112:0/3015279250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943e2390 0x5634947a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.238+0000 7fc23f48f740 1 --2- 172.21.15.112:0/3015279250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943da1b0 0x5634947c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.238+0000 7fc23f48f740 1 --2- 172.21.15.112:0/3015279250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 0x5634947ada80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.238+0000 7fc23f48f740 1 -- 172.21.15.112:0/3015279250 >> 172.21.15.112:0/3015279250 conn(0x5634943f7460 msgr2=0x5634947ad370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.238+0000 7fc23f48f740 1 -- 172.21.15.112:0/3015279250 shutdown_connections 2024-09-21T03:29:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.238+0000 7fc23f48f740 1 -- 172.21.15.112:0/3015279250 wait complete. 2024-09-21T03:29:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.238+0000 7fc23f48f740 1 -- start start 2024-09-21T03:29:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc23f48f740 1 Processor -- start 2024-09-21T03:29:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc23f48f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 0x5634947b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc23f48f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943da1b0 0x5634947ba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc23f48f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 0x5634947ba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc23f48f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341ac0e0 con 0x5634943e2390 2024-09-21T03:29:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc23f48f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341ad550 con 0x5634943c19f0 2024-09-21T03:29:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc23f48f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc2341ac7b0 con 0x5634943da1b0 2024-09-21T03:29:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc23f48f740 10 client.0 _ll_register_callbacks cb 0x7fc21c009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc23259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 0x5634947b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc23259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 0x5634947b9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:37462/0 (socket says 172.21.15.112:37462) 2024-09-21T03:29:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc23259c640 1 -- 172.21.15.112:0/1964970925 learned_addr learned my addr 172.21.15.112:0/1964970925 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc229d9b640 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943da1b0 0x5634947ba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.239+0000 7fc232d9d640 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 0x5634947ba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.240+0000 7fc23259c640 1 -- 172.21.15.112:0/1964970925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943da1b0 msgr2=0x5634947ba2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.240+0000 7fc23259c640 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943da1b0 0x5634947ba2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.240+0000 7fc23259c640 1 -- 172.21.15.112:0/1964970925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 msgr2=0x5634947ba830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.240+0000 7fc23259c640 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 0x5634947ba830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.240+0000 7fc23259c640 1 -- 172.21.15.112:0/1964970925 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563494831200 con 0x5634943c19f0 2024-09-21T03:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.240+0000 7fc23259c640 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 0x5634947b9db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc224004650 tx=0x7fc224009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d21120017cff65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.240+0000 7fc219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.240+0000 7fc219ffb640 1 -- 172.21.15.112:0/1964970925 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22405c020 con 0x5634943c19f0 2024-09-21T03:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.240+0000 7fc23f48f740 1 -- 172.21.15.112:0/1964970925 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc224004df0 con 0x5634943c19f0 2024-09-21T03:29:50.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.240+0000 7fc219ffb640 1 -- 172.21.15.112:0/1964970925 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc2240035f0 con 0x5634943c19f0 2024-09-21T03:29:50.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.241+0000 7fc219ffb640 1 -- 172.21.15.112:0/1964970925 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc224035070 con 0x5634943c19f0 2024-09-21T03:29:50.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.241+0000 7fc23f48f740 10 client.25292 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:50.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.241+0000 7fc23f48f740 1 -- 172.21.15.112:0/1964970925 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563494832470 con 0x5634943c19f0 2024-09-21T03:29:50.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.241+0000 7fc219ffb640 1 -- 172.21.15.112:0/1964970925 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc224004df0 con 0x5634943c19f0 2024-09-21T03:29:50.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.242+0000 7fc219ffb640 1 -- 172.21.15.112:0/1964970925 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc2240316d0 con 0x5634943c19f0 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.242+0000 7fc23f48f740 10 client.25292 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.242+0000 7fc23f48f740 10 client.25292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.242+0000 7fc23f48f740 20 client.25292 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.242+0000 7fc23f48f740 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563494403070 0x563494834bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.242+0000 7fc23f48f740 4 client.25292 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.242+0000 7fc23f48f740 1 -- 172.21.15.112:0/1964970925 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563494722da0 con 0x563494403070 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.242+0000 7fc229d9b640 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563494403070 0x563494834bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc229d9b640 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563494403070 0x563494834bb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254f7e457698ba29 server_cookie=6fe4224e9581d135 in_seq=0 out_seq=0 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc219ffb640 10 client.25292 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc219ffb640 1 -- 172.21.15.112:0/1964970925 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x563494722da0 con 0x563494403070 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc219ffb640 10 client.25292 handle_command_reply: tid=0 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 1 client.25292 shutdown 2024-09-21T03:29:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 1 -- 172.21.15.112:0/1964970925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 msgr2=0x5634947b9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 0x5634947b9db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc224004650 tx=0x7fc224009630 comp rx=0 tx=0).stop 2024-09-21T03:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 1 -- 172.21.15.112:0/1964970925 shutdown_connections 2024-09-21T03:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563494403070 0x563494834bb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc219ffb640 0 client.25292 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634943e2390 0x5634947ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5634943da1b0 0x5634947ba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 1 --2- 172.21.15.112:0/1964970925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634943c19f0 0x5634947b9db0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 1 -- 172.21.15.112:0/1964970925 >> 172.21.15.112:0/1964970925 conn(0x5634943f7460 msgr2=0x5634947a7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 1 -- 172.21.15.112:0/1964970925 shutdown_connections 2024-09-21T03:29:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 1 -- 172.21.15.112:0/1964970925 wait complete. 2024-09-21T03:29:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.243+0000 7fc23f48f740 20 client.25292 trim_cache size 0 max 16384 2024-09-21T03:29:50.247 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:29:50.247 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:29:50.247 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:29:50.247 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:29:50.247 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:29:50.247 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:29:50.247 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:29:50.248 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:29:50.249 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:29:50.249 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:29:50.249 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:29:50.249 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:29:50.249 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:29:50.249 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:29:50.249 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:29:50.249 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:29:50.249 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.244+0000 7fc238f1b640 1 -- 172.21.15.112:0/748659801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2200bef10 msgr2=0x7fc2200c1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.244+0000 7fc238f1b640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2200bef10 0x7fc2200c1420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc224031440 tx=0x7fc224005a20 comp rx=0 tx=0).stop 2024-09-21T03:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.244+0000 7fc238f1b640 1 -- 172.21.15.112:0/748659801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23419fd60 msgr2=0x7fc2341a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.244+0000 7fc238f1b640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23419fd60 0x7fc2341a0170 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00ed70 tx=0x7fc22c00c6a0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.245+0000 7fc238f1b640 1 -- 172.21.15.112:0/748659801 shutdown_connections 2024-09-21T03:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.245+0000 7fc238f1b640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2200bef10 0x7fc2200c1420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.245+0000 7fc238f1b640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc23419fd60 0x7fc2341a0170 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.245+0000 7fc238f1b640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc234108cf0 0x7fc23419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.245+0000 7fc238f1b640 1 --2- 172.21.15.112:0/748659801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2341083a0 0x7fc23419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.245+0000 7fc238f1b640 1 -- 172.21.15.112:0/748659801 >> 172.21.15.112:0/748659801 conn(0x7fc234079730 msgr2=0x7fc2340ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.246+0000 7fc238f1b640 1 -- 172.21.15.112:0/748659801 shutdown_connections 2024-09-21T03:29:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:50.246+0000 7fc238f1b640 1 -- 172.21.15.112:0/748659801 wait complete. 2024-09-21T03:29:50.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:50 smithi112 ceph-mon[31442]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-21T03:29:50.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:50 smithi203 ceph-mon[34723]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-21T03:29:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:50 smithi187 ceph-mon[34603]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-21T03:29:51.022 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:29:51.022 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:29:51.314 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:51.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.861+0000 7f86366df640 1 -- 172.21.15.112:0/727763087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8630102030 msgr2=0x7f8630102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.861+0000 7f86366df640 1 --2- 172.21.15.112:0/727763087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8630102030 0x7f8630102440 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8620009900 tx=0x7f862002a410 comp rx=0 tx=0).stop 2024-09-21T03:29:51.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.861+0000 7f86366df640 1 -- 172.21.15.112:0/727763087 shutdown_connections 2024-09-21T03:29:51.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.861+0000 7f86366df640 1 --2- 172.21.15.112:0/727763087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8630106bd0 0x7f8630110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.861+0000 7f86366df640 1 --2- 172.21.15.112:0/727763087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8630106280 0x7f8630106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.861+0000 7f86366df640 1 --2- 172.21.15.112:0/727763087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8630102030 0x7f8630102440 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.861+0000 7f86366df640 1 -- 172.21.15.112:0/727763087 >> 172.21.15.112:0/727763087 conn(0x7f8630077c60 msgr2=0x7f86300780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:51.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.861+0000 7f86366df640 1 -- 172.21.15.112:0/727763087 shutdown_connections 2024-09-21T03:29:51.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.861+0000 7f86366df640 1 -- 172.21.15.112:0/727763087 wait complete. 2024-09-21T03:29:51.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.862+0000 7f86366df640 1 Processor -- start 2024-09-21T03:29:51.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.862+0000 7f86366df640 1 -- start start 2024-09-21T03:29:51.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.863+0000 7f86366df640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8630102030 0x7f863019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.863+0000 7f86366df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8630106280 0x7f863019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.863+0000 7f86366df640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8630106bd0 0x7f863019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.863+0000 7f86366df640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8630113ac0 con 0x7f8630106280 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.863+0000 7f86366df640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8630113940 con 0x7f8630102030 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.863+0000 7f86366df640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8630113c40 con 0x7f8630106bd0 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.863+0000 7f862ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8630102030 0x7f863019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.864+0000 7f862ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8630102030 0x7f863019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:37472/0 (socket says 172.21.15.112:37472) 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.864+0000 7f862ffff640 1 -- 172.21.15.112:0/2761680491 learned_addr learned my addr 172.21.15.112:0/2761680491 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.864+0000 7f8634c7c640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8630106bd0 0x7f863019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.864+0000 7f862f7fe640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8630106280 0x7f863019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.864+0000 7f8634c7c640 1 -- 172.21.15.112:0/2761680491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8630102030 msgr2=0x7f863019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.864+0000 7f8634c7c640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8630102030 0x7f863019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.864+0000 7f8634c7c640 1 -- 172.21.15.112:0/2761680491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8630106280 msgr2=0x7f863019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:51.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.864+0000 7f8634c7c640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8630106280 0x7f863019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:51.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.864+0000 7f8634c7c640 1 -- 172.21.15.112:0/2761680491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86301a0420 con 0x7f8630106bd0 2024-09-21T03:29:51.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.864+0000 7f862f7fe640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8630106280 0x7f863019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:51.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.865+0000 7f8634c7c640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8630106bd0 0x7f863019fcf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f862800e940 tx=0x7f862800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:51.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.865+0000 7f862d7fa640 1 -- 172.21.15.112:0/2761680491 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628004d20 con 0x7f8630106bd0 2024-09-21T03:29:51.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.865+0000 7f862d7fa640 1 -- 172.21.15.112:0/2761680491 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8628005020 con 0x7f8630106bd0 2024-09-21T03:29:51.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.865+0000 7f862d7fa640 1 -- 172.21.15.112:0/2761680491 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86280116d0 con 0x7f8630106bd0 2024-09-21T03:29:51.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.865+0000 7f86366df640 1 -- 172.21.15.112:0/2761680491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86301a0710 con 0x7f8630106bd0 2024-09-21T03:29:51.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.865+0000 7f86366df640 1 -- 172.21.15.112:0/2761680491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86301a6e70 con 0x7f8630106bd0 2024-09-21T03:29:51.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.866+0000 7f86366df640 1 -- 172.21.15.112:0/2761680491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8630102440 con 0x7f8630106bd0 2024-09-21T03:29:51.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.868+0000 7f862d7fa640 1 -- 172.21.15.112:0/2761680491 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8628005460 con 0x7f8630106bd0 2024-09-21T03:29:51.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.869+0000 7f862d7fa640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f86180b6680 0x7f86180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:51.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.869+0000 7f862ffff640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f86180b6680 0x7f86180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:51.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.869+0000 7f862d7fa640 1 -- 172.21.15.112:0/2761680491 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f86280e7b70 con 0x7f8630106bd0 2024-09-21T03:29:51.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.870+0000 7f862ffff640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f86180b6680 0x7f86180b8b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f86200096c0 tx=0x7f86200058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:51.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:51.872+0000 7f862d7fa640 1 -- 172.21.15.112:0/2761680491 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8628016030 con 0x7f8630106bd0 2024-09-21T03:29:52.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.165+0000 7f86366df640 1 -- 172.21.15.112:0/2761680491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f863019b5e0 con 0x7f8630106bd0 2024-09-21T03:29:52.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.167+0000 7f862d7fa640 1 -- 172.21.15.112:0/2761680491 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11929 (secure 0 0 0) 0x7f863019b5e0 con 0x7f8630106bd0 2024-09-21T03:29:52.170 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:29:52.170 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"btime":"2024-09-21T03:28:46:418112+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:28:46.048810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:29:52.170 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 25 2024-09-21T03:29:52.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.169+0000 7f86366df640 1 -- 172.21.15.112:0/2761680491 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f86180b6680 msgr2=0x7f86180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.169+0000 7f86366df640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f86180b6680 0x7f86180b8b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f86200096c0 tx=0x7f86200058d0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.169+0000 7f86366df640 1 -- 172.21.15.112:0/2761680491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8630106bd0 msgr2=0x7f863019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:52.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.169+0000 7f86366df640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8630106bd0 0x7f863019fcf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f862800e940 tx=0x7f862800ee00 comp rx=0 tx=0).stop 2024-09-21T03:29:52.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.169+0000 7f86366df640 1 -- 172.21.15.112:0/2761680491 shutdown_connections 2024-09-21T03:29:52.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.169+0000 7f86366df640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f86180b6680 0x7f86180b8b90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.169+0000 7f86366df640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8630106bd0 0x7f863019fcf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.169+0000 7f86366df640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8630106280 0x7f863019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.169+0000 7f86366df640 1 --2- 172.21.15.112:0/2761680491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8630102030 0x7f863019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:52.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.169+0000 7f86366df640 1 -- 172.21.15.112:0/2761680491 >> 172.21.15.112:0/2761680491 conn(0x7f8630077c60 msgr2=0x7f8630107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:52.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.169+0000 7f86366df640 1 -- 172.21.15.112:0/2761680491 shutdown_connections 2024-09-21T03:29:52.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:52.170+0000 7f86366df640 1 -- 172.21.15.112:0/2761680491 wait complete. 2024-09-21T03:29:52.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:52 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2761680491' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:52.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:52 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2761680491' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:52.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:52 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2761680491' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:29:52.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:29:53.228 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:53 smithi112 ceph-mon[31442]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T03:29:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:53.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:53 smithi203 ceph-mon[34723]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T03:29:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:53 smithi187 ceph-mon[34603]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T03:29:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.908+0000 7fd2aa990640 1 -- 172.21.15.112:0/270033433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a4107b80 msgr2=0x7fd2a4107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.908+0000 7fd2aa990640 1 --2- 172.21.15.112:0/270033433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a4107b80 0x7fd2a4107f90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd29800aa10 tx=0x7fd29802b720 comp rx=0 tx=0).stop 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.908+0000 7fd2aa990640 1 -- 172.21.15.112:0/270033433 shutdown_connections 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.908+0000 7fd2aa990640 1 --2- 172.21.15.112:0/270033433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a41084d0 0x7fd2a410ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.908+0000 7fd2aa990640 1 --2- 172.21.15.112:0/270033433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a4107b80 0x7fd2a4107f90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.908+0000 7fd2aa990640 1 --2- 172.21.15.112:0/270033433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2a4102240 0x7fd2a4102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.908+0000 7fd2aa990640 1 -- 172.21.15.112:0/270033433 >> 172.21.15.112:0/270033433 conn(0x7fd2a4079f90 msgr2=0x7fd2a407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.908+0000 7fd2aa990640 1 -- 172.21.15.112:0/270033433 shutdown_connections 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.908+0000 7fd2aa990640 1 -- 172.21.15.112:0/270033433 wait complete. 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.909+0000 7fd2aa990640 1 Processor -- start 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.909+0000 7fd2aa990640 1 -- start start 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.909+0000 7fd2aa990640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2a4102240 0x7fd2a419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.909+0000 7fd2aa990640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a4107b80 0x7fd2a41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.909+0000 7fd2aa990640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a41084d0 0x7fd2a41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2aa990640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4111960 con 0x7fd2a41084d0 2024-09-21T03:29:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2aa990640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a41117e0 con 0x7fd2a4107b80 2024-09-21T03:29:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2aa990640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4111ae0 con 0x7fd2a4102240 2024-09-21T03:29:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a8f2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a41084d0 0x7fd2a41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a8f2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a41084d0 0x7fd2a41987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45786/0 (socket says 172.21.15.112:45786) 2024-09-21T03:29:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2a4102240 0x7fd2a419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a8f2d640 1 -- 172.21.15.112:0/3859379460 learned_addr learned my addr 172.21.15.112:0/3859379460 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:53.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a4107b80 0x7fd2a41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a8f2d640 1 -- 172.21.15.112:0/3859379460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2a4102240 msgr2=0x7fd2a419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a8f2d640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2a4102240 0x7fd2a419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a8f2d640 1 -- 172.21.15.112:0/3859379460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a4107b80 msgr2=0x7fd2a41981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a8f2d640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a4107b80 0x7fd2a41981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a8f2d640 1 -- 172.21.15.112:0/3859379460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a4198fc0 con 0x7fd2a41084d0 2024-09-21T03:29:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a8f2d640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a41084d0 0x7fd2a41987a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00eb10 tx=0x7fd29c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:53.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.910+0000 7fd2a17fa640 1 -- 172.21.15.112:0/3859379460 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c004b20 con 0x7fd2a41084d0 2024-09-21T03:29:53.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.911+0000 7fd2aa990640 1 -- 172.21.15.112:0/3859379460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a419dc70 con 0x7fd2a41084d0 2024-09-21T03:29:53.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.911+0000 7fd2a17fa640 1 -- 172.21.15.112:0/3859379460 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd29c004cc0 con 0x7fd2a41084d0 2024-09-21T03:29:53.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.911+0000 7fd2aa990640 1 -- 172.21.15.112:0/3859379460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a419e180 con 0x7fd2a41084d0 2024-09-21T03:29:53.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.911+0000 7fd2a17fa640 1 -- 172.21.15.112:0/3859379460 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c010670 con 0x7fd2a41084d0 2024-09-21T03:29:53.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.914+0000 7fd2a17fa640 1 -- 172.21.15.112:0/3859379460 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fd29c005020 con 0x7fd2a41084d0 2024-09-21T03:29:53.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.914+0000 7fd2b0f04740 1 Processor -- start 2024-09-21T03:29:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.914+0000 7fd2b0f04740 1 -- start start 2024-09-21T03:29:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.914+0000 7fd2b0f04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f95e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.914+0000 7fd2b0f04740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f58e1b0 0x56315f95e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.914+0000 7fd2b0f04740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f596390 0x56315f95ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.914+0000 7fd2b0f04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56315f8d4ea0 con 0x56315f5b7070 2024-09-21T03:29:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.914+0000 7fd2b0f04740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56315f8c42d0 con 0x56315f58e1b0 2024-09-21T03:29:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.914+0000 7fd2b0f04740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56315f8c80e0 con 0x56315f596390 2024-09-21T03:29:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a17fa640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd28c0b6700 0x7fd28c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f95e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f95e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45792/0 (socket says 172.21.15.112:45792) 2024-09-21T03:29:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a37fe640 1 -- 172.21.15.112:0/2145819066 learned_addr learned my addr 172.21.15.112:0/2145819066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a3fff640 1 --2- 172.21.15.112:0/2145819066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f58e1b0 0x56315f95e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a17fa640 1 -- 172.21.15.112:0/3859379460 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd29c0e7a30 con 0x7fd2a41084d0 2024-09-21T03:29:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a8f2d640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd28c0b6700 0x7fd28c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a37fe640 1 -- 172.21.15.112:0/2145819066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f596390 msgr2=0x56315f95ed40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:29:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/2145819066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f596390 0x56315f95ed40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a37fe640 1 -- 172.21.15.112:0/2145819066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f58e1b0 msgr2=0x56315f95e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/2145819066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f58e1b0 0x56315f95e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a37fe640 1 -- 172.21.15.112:0/2145819066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315f622650 con 0x56315f5b7070 2024-09-21T03:29:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/2145819066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f95e2c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd298005ab0 tx=0x7fd2980040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b06b15f7cd9965 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.916+0000 7fd2a8f2d640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd28c0b6700 0x7fd28c0b8c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0a9eb0 tx=0x7fd29c0e7d30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.915+0000 7fd28a7fc640 1 -- 172.21.15.112:0/2145819066 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298041070 con 0x56315f5b7070 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.916+0000 7fd28a7fc640 1 -- 172.21.15.112:0/2145819066 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2980054f0 con 0x56315f5b7070 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.916+0000 7fd28a7fc640 1 -- 172.21.15.112:0/2145819066 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298036040 con 0x56315f5b7070 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.917+0000 7fd2b0f04740 1 -- 172.21.15.112:0/2145819066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 msgr2=0x56315f95e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.917+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/2145819066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f95e2c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd298005ab0 tx=0x7fd2980040f0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.917+0000 7fd2b0f04740 1 -- 172.21.15.112:0/2145819066 shutdown_connections 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.917+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/2145819066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f596390 0x56315f95ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.917+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/2145819066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f58e1b0 0x56315f95e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.917+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/2145819066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f95e2c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.917+0000 7fd2b0f04740 1 -- 172.21.15.112:0/2145819066 >> 172.21.15.112:0/2145819066 conn(0x56315f5ab460 msgr2=0x56315f95c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.917+0000 7fd2b0f04740 1 -- 172.21.15.112:0/2145819066 shutdown_connections 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.917+0000 7fd2b0f04740 1 -- 172.21.15.112:0/2145819066 wait complete. 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.917+0000 7fd2b0f04740 1 -- start start 2024-09-21T03:29:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2b0f04740 1 Processor -- start 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2b0f04740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f58e1b0 0x56315f9698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2b0f04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f596390 0x56315f969e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2b0f04740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f5b7070 0x56315f96a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2b0f04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56315f725700 con 0x56315f596390 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2b0f04740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56315f85f160 con 0x56315f5b7070 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2b0f04740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56315f96ca40 con 0x56315f58e1b0 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2b0f04740 10 client.0 _ll_register_callbacks cb 0x56315f8789e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f596390 0x56315f969e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f596390 0x56315f969e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45796/0 (socket says 172.21.15.112:45796) 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2a3fff640 1 -- 172.21.15.112:0/865640863 learned_addr learned my addr 172.21.15.112:0/865640863 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.918+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f58e1b0 0x56315f9698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f5b7070 0x56315f96a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2a3fff640 1 -- 172.21.15.112:0/865640863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f58e1b0 msgr2=0x56315f9698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2a3fff640 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f58e1b0 0x56315f9698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2a3fff640 1 -- 172.21.15.112:0/865640863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f5b7070 msgr2=0x56315f96a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2a3fff640 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f5b7070 0x56315f96a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2a3fff640 1 -- 172.21.15.112:0/865640863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56315f88cc90 con 0x56315f596390 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f5b7070 0x56315f96a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2a3fff640 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f596390 0x56315f969e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd29400b960 tx=0x7fd29400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b074beaec039a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2837fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2837fe640 1 -- 172.21.15.112:0/865640863 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294003ea0 con 0x56315f596390 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2837fe640 1 -- 172.21.15.112:0/865640863 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd294005220 con 0x56315f596390 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.919+0000 7fd2b0f04740 1 -- 172.21.15.112:0/865640863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2980054f0 con 0x56315f596390 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.920+0000 7fd2837fe640 1 -- 172.21.15.112:0/865640863 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd294005420 con 0x56315f596390 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.920+0000 7fd2b0f04740 10 client.15726 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.920+0000 7fd2b0f04740 1 -- 172.21.15.112:0/865640863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56315f9787b0 con 0x56315f596390 2024-09-21T03:29:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.920+0000 7fd2837fe640 1 -- 172.21.15.112:0/865640863 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2980054f0 con 0x56315f596390 2024-09-21T03:29:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.920+0000 7fd2837fe640 1 -- 172.21.15.112:0/865640863 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd294019030 con 0x56315f596390 2024-09-21T03:29:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.920+0000 7fd2b0f04740 10 client.15726 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.920+0000 7fd2b0f04740 10 client.15726 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:29:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.920+0000 7fd2b0f04740 20 client.15726 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.920+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56315f5759f0 0x56315f97c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.920+0000 7fd2b0f04740 4 client.15726 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.920+0000 7fd2b0f04740 1 -- 172.21.15.112:0/865640863 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56315f8d6df0 con 0x56315f5759f0 2024-09-21T03:29:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.921+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56315f5759f0 0x56315f97c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.921+0000 7fd2837fe640 10 client.15726 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.922+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56315f5759f0 0x56315f97c140 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffe075e25338d393 server_cookie=62b35c35500973ce in_seq=0 out_seq=0 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.923+0000 7fd2837fe640 1 -- 172.21.15.112:0/865640863 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56315f8d6df0 con 0x56315f5759f0 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.923+0000 7fd2837fe640 10 client.15726 handle_command_reply: tid=0 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.923+0000 7fd2b0f04740 1 client.15726 shutdown 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2b0f04740 1 -- 172.21.15.112:0/865640863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f596390 msgr2=0x56315f969e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f596390 0x56315f969e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd29400b960 tx=0x7fd29400be20 comp rx=0 tx=0).stop 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2b0f04740 1 -- 172.21.15.112:0/865640863 shutdown_connections 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56315f5759f0 0x56315f97c140 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f5b7070 0x56315f96a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f596390 0x56315f969e00 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/865640863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f58e1b0 0x56315f9698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2b0f04740 1 -- 172.21.15.112:0/865640863 >> 172.21.15.112:0/865640863 conn(0x56315f5ab460 msgr2=0x56315f961450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2837fe640 0 client.15726 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2b0f04740 1 -- 172.21.15.112:0/865640863 shutdown_connections 2024-09-21T03:29:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2b0f04740 1 -- 172.21.15.112:0/865640863 wait complete. 2024-09-21T03:29:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.924+0000 7fd2b0f04740 20 client.15726 trim_cache size 0 max 16384 2024-09-21T03:29:53.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.949+0000 7fd2b0f04740 1 Processor -- start 2024-09-21T03:29:53.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.949+0000 7fd2b0f04740 1 -- start start 2024-09-21T03:29:53.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.950+0000 7fd2b0f04740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f58e1b0 0x56315f975d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.950+0000 7fd2b0f04740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f596390 0x56315f95fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.950+0000 7fd2b0f04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f966ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.950+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f596390 0x56315f95fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.950+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f596390 0x56315f95fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:37516/0 (socket says 172.21.15.112:37516) 2024-09-21T03:29:53.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.950+0000 7fd2a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f966ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.950+0000 7fd2a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f966ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45798/0 (socket says 172.21.15.112:45798) 2024-09-21T03:29:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.950+0000 7fd2a3fff640 1 -- 172.21.15.112:0/3037132460 learned_addr learned my addr 172.21.15.112:0/3037132460 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.950+0000 7fd2b0f04740 1 -- 172.21.15.112:0/3037132460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd290010f30 con 0x56315f5b7070 2024-09-21T03:29:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.950+0000 7fd2b0f04740 1 -- 172.21.15.112:0/3037132460 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd29000e800 con 0x56315f596390 2024-09-21T03:29:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.950+0000 7fd2b0f04740 1 -- 172.21.15.112:0/3037132460 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd29000c650 con 0x56315f58e1b0 2024-09-21T03:29:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.951+0000 7fd2a37fe640 1 -- 172.21.15.112:0/3037132460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f58e1b0 msgr2=0x56315f975d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.951+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/3037132460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f58e1b0 0x56315f975d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.951+0000 7fd2a37fe640 1 -- 172.21.15.112:0/3037132460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f596390 msgr2=0x56315f95fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:53.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.951+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/3037132460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f596390 0x56315f95fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.951+0000 7fd2a37fe640 1 -- 172.21.15.112:0/3037132460 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315f96ab80 con 0x56315f5b7070 2024-09-21T03:29:53.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.951+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/3037132460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f966ab0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd2980057c0 tx=0x7fd2980040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73a7ae5c3cdaf90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:53.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.951+0000 7fd289ffb640 1 -- 172.21.15.112:0/3037132460 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298036040 con 0x56315f5b7070 2024-09-21T03:29:53.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.951+0000 7fd289ffb640 1 -- 172.21.15.112:0/3037132460 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2980054f0 con 0x56315f5b7070 2024-09-21T03:29:53.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.951+0000 7fd289ffb640 1 -- 172.21.15.112:0/3037132460 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298041040 con 0x56315f5b7070 2024-09-21T03:29:53.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.952+0000 7fd2b0f04740 1 -- 172.21.15.112:0/3037132460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 msgr2=0x56315f966ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:53.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.953+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/3037132460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f966ab0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd2980057c0 tx=0x7fd2980040f0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.953+0000 7fd2b0f04740 1 -- 172.21.15.112:0/3037132460 shutdown_connections 2024-09-21T03:29:53.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.953+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/3037132460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f5b7070 0x56315f966ab0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.953+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/3037132460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f596390 0x56315f95fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.953+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/3037132460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f58e1b0 0x56315f975d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.953+0000 7fd2b0f04740 1 -- 172.21.15.112:0/3037132460 >> 172.21.15.112:0/3037132460 conn(0x56315f5ab460 msgr2=0x56315f95f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:53.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.953+0000 7fd2b0f04740 1 -- 172.21.15.112:0/3037132460 shutdown_connections 2024-09-21T03:29:53.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.953+0000 7fd2b0f04740 1 -- 172.21.15.112:0/3037132460 wait complete. 2024-09-21T03:29:53.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.953+0000 7fd2b0f04740 1 -- start start 2024-09-21T03:29:53.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.954+0000 7fd2b0f04740 1 Processor -- start 2024-09-21T03:29:53.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.954+0000 7fd2b0f04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f58e1b0 0x56315f96d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.955+0000 7fd2a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f58e1b0 0x56315f96d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.955+0000 7fd2a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f58e1b0 0x56315f96d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45808/0 (socket says 172.21.15.112:45808) 2024-09-21T03:29:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.955+0000 7fd2a37fe640 1 -- 172.21.15.112:0/4123233031 learned_addr learned my addr 172.21.15.112:0/4123233031 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.955+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f596390 0x56315f96dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.955+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f5b7070 0x56315f9deb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.955+0000 7fd2b0f04740 1 -- 172.21.15.112:0/4123233031 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd29000ccc0 con 0x56315f58e1b0 2024-09-21T03:29:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.955+0000 7fd2b0f04740 1 -- 172.21.15.112:0/4123233031 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd29000e130 con 0x56315f596390 2024-09-21T03:29:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.955+0000 7fd2b0f04740 1 -- 172.21.15.112:0/4123233031 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd29000d390 con 0x56315f5b7070 2024-09-21T03:29:53.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.955+0000 7fd2a3fff640 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f596390 0x56315f96dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.956+0000 7fd2a37fe640 1 -- 172.21.15.112:0/4123233031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f5b7070 msgr2=0x56315f9deb30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:29:53.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.956+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f5b7070 0x56315f9deb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.956+0000 7fd2a37fe640 1 -- 172.21.15.112:0/4123233031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f596390 msgr2=0x56315f96dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.956+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f596390 0x56315f96dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.956+0000 7fd2a37fe640 1 -- 172.21.15.112:0/4123233031 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56315f9df1a0 con 0x56315f58e1b0 2024-09-21T03:29:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.956+0000 7fd2a37fe640 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f58e1b0 0x56315f96d6a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd29800a960 tx=0x7fd298004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82719cddd81f01df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.956+0000 7fd2b0f04740 10 client.0 _ll_register_callbacks cb 0x7fd2940041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.956+0000 7fd2b0f04740 1 -- 172.21.15.112:0/4123233031 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2980054f0 con 0x56315f58e1b0 2024-09-21T03:29:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.956+0000 7fd2837fe640 10 client.15732 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.956+0000 7fd2837fe640 1 -- 172.21.15.112:0/4123233031 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298041030 con 0x56315f58e1b0 2024-09-21T03:29:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.956+0000 7fd2837fe640 1 -- 172.21.15.112:0/4123233031 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd298005250 con 0x56315f58e1b0 2024-09-21T03:29:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.957+0000 7fd2837fe640 1 -- 172.21.15.112:0/4123233031 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd298036040 con 0x56315f58e1b0 2024-09-21T03:29:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.957+0000 7fd2b0f04740 10 client.15732 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:53.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.957+0000 7fd2b0f04740 1 -- 172.21.15.112:0/4123233031 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=25}) -- 0x56315f9e6290 con 0x56315f58e1b0 2024-09-21T03:29:53.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.957+0000 7fd2837fe640 1 -- 172.21.15.112:0/4123233031 <== mon.0 v2:172.21.15.112:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd29803b040 con 0x56315f58e1b0 2024-09-21T03:29:53.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.957+0000 7fd2b0f04740 10 client.15732 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:53.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.957+0000 7fd2b0f04740 10 client.15732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:29:53.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.957+0000 7fd2b0f04740 20 client.15732 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:53.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.957+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56315f54ada0 0x56315f9e84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:53.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.957+0000 7fd2b0f04740 4 client.15732 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:53.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.957+0000 7fd2b0f04740 1 -- 172.21.15.112:0/4123233031 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56315f8d28a0 con 0x56315f54ada0 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.958+0000 7fd2a3fff640 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56315f54ada0 0x56315f9e84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.958+0000 7fd2a3fff640 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56315f54ada0 0x56315f9e84a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14587e502a37c9a5 server_cookie=f23f1be45187c82c in_seq=0 out_seq=0 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.958+0000 7fd2837fe640 10 client.15732 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.958+0000 7fd2837fe640 1 -- 172.21.15.112:0/4123233031 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56315f8d28a0 con 0x56315f54ada0 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.958+0000 7fd2837fe640 10 client.15732 handle_command_reply: tid=0 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.958+0000 7fd2b0f04740 1 client.15732 shutdown 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2b0f04740 1 -- 172.21.15.112:0/4123233031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f58e1b0 msgr2=0x56315f96d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f58e1b0 0x56315f96d6a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd29800a960 tx=0x7fd298004e30 comp rx=0 tx=0).stop 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2b0f04740 1 -- 172.21.15.112:0/4123233031 shutdown_connections 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56315f54ada0 0x56315f9e84a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56315f5b7070 0x56315f9deb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56315f596390 0x56315f96dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2837fe640 0 client.15732 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:29:53.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2b0f04740 1 --2- 172.21.15.112:0/4123233031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56315f58e1b0 0x56315f96d6a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2b0f04740 1 -- 172.21.15.112:0/4123233031 >> 172.21.15.112:0/4123233031 conn(0x56315f5ab460 msgr2=0x56315f96ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:53.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2b0f04740 1 -- 172.21.15.112:0/4123233031 shutdown_connections 2024-09-21T03:29:53.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2b0f04740 1 -- 172.21.15.112:0/4123233031 wait complete. 2024-09-21T03:29:53.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.959+0000 7fd2b0f04740 20 client.15732 trim_cache size 0 max 16384 2024-09-21T03:29:53.964 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:29:53.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.960+0000 7fd2aa990640 1 -- 172.21.15.112:0/3859379460 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd28c0b6700 msgr2=0x7fd28c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:53.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.960+0000 7fd2aa990640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd28c0b6700 0x7fd28c0b8c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0a9eb0 tx=0x7fd29c0e7d30 comp rx=0 tx=0).stop 2024-09-21T03:29:53.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.960+0000 7fd2aa990640 1 -- 172.21.15.112:0/3859379460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a41084d0 msgr2=0x7fd2a41987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:53.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.960+0000 7fd2aa990640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a41084d0 0x7fd2a41987a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00eb10 tx=0x7fd29c00c2d0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.960+0000 7fd2aa990640 1 -- 172.21.15.112:0/3859379460 shutdown_connections 2024-09-21T03:29:53.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.960+0000 7fd2aa990640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd28c0b6700 0x7fd28c0b8c10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.960+0000 7fd2aa990640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2a41084d0 0x7fd2a41987a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.960+0000 7fd2aa990640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2a4107b80 0x7fd2a41981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.960+0000 7fd2aa990640 1 --2- 172.21.15.112:0/3859379460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2a4102240 0x7fd2a419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:53.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.960+0000 7fd2aa990640 1 -- 172.21.15.112:0/3859379460 >> 172.21.15.112:0/3859379460 conn(0x7fd2a4079f90 msgr2=0x7fd2a4105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:53.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.961+0000 7fd2aa990640 1 -- 172.21.15.112:0/3859379460 shutdown_connections 2024-09-21T03:29:53.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:53.961+0000 7fd2aa990640 1 -- 172.21.15.112:0/3859379460 wait complete. 2024-09-21T03:29:54.754 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:54 smithi112 ceph-mon[31442]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-21T03:29:54.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:29:54.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:54 smithi203 ceph-mon[34723]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-21T03:29:54.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:54 smithi187 ceph-mon[34603]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-21T03:29:55.081 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.565+0000 7f3cbe436640 1 -- 172.21.15.112:0/3630520030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb81084b0 msgr2=0x7f3cb810ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.565+0000 7f3cbe436640 1 --2- 172.21.15.112:0/3630520030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb81084b0 0x7f3cb810ece0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009ca0 tx=0x7f3cb002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.565+0000 7f3cbe436640 1 -- 172.21.15.112:0/3630520030 shutdown_connections 2024-09-21T03:29:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.565+0000 7f3cbe436640 1 --2- 172.21.15.112:0/3630520030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb81084b0 0x7f3cb810ece0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.565+0000 7f3cbe436640 1 --2- 172.21.15.112:0/3630520030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cb8107b60 0x7f3cb8107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.565+0000 7f3cbe436640 1 --2- 172.21.15.112:0/3630520030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb8102270 0x7f3cb8102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.565+0000 7f3cbe436640 1 -- 172.21.15.112:0/3630520030 >> 172.21.15.112:0/3630520030 conn(0x7f3cb8077cc0 msgr2=0x7f3cb8078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:55.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.565+0000 7f3cbe436640 1 -- 172.21.15.112:0/3630520030 shutdown_connections 2024-09-21T03:29:55.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.566+0000 7f3cbe436640 1 -- 172.21.15.112:0/3630520030 wait complete. 2024-09-21T03:29:55.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.566+0000 7f3cbe436640 1 Processor -- start 2024-09-21T03:29:55.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.566+0000 7f3cbe436640 1 -- start start 2024-09-21T03:29:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cbe436640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8102270 0x7f3cb819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cbe436640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb8107b60 0x7f3cb819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cbe436640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cb81084b0 0x7f3cb819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cbe436640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb81129f0 con 0x7f3cb8102270 2024-09-21T03:29:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cbe436640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8112870 con 0x7f3cb81084b0 2024-09-21T03:29:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cbe436640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8112b70 con 0x7f3cb8107b60 2024-09-21T03:29:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8102270 0x7f3cb819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8102270 0x7f3cb819b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56582/0 (socket says 172.21.15.112:56582) 2024-09-21T03:29:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cb7fff640 1 -- 172.21.15.112:0/4110254812 learned_addr learned my addr 172.21.15.112:0/4110254812 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cbc9d3640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cb81084b0 0x7f3cb819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cb77fe640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb8107b60 0x7f3cb819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cb7fff640 1 -- 172.21.15.112:0/4110254812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb8107b60 msgr2=0x7f3cb819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb8107b60 0x7f3cb819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cb7fff640 1 -- 172.21.15.112:0/4110254812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cb81084b0 msgr2=0x7f3cb819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cb81084b0 0x7f3cb819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.567+0000 7f3cb7fff640 1 -- 172.21.15.112:0/4110254812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb81a65d0 con 0x7f3cb8102270 2024-09-21T03:29:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.568+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8102270 0x7f3cb819b2c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800e9f0 tx=0x7f3ca800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.568+0000 7f3cb57fa640 1 -- 172.21.15.112:0/4110254812 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca800cd80 con 0x7f3cb8102270 2024-09-21T03:29:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.568+0000 7f3cbe436640 1 -- 172.21.15.112:0/4110254812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb81a68c0 con 0x7f3cb8102270 2024-09-21T03:29:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.568+0000 7f3cbe436640 1 -- 172.21.15.112:0/4110254812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb81a6e00 con 0x7f3cb8102270 2024-09-21T03:29:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.568+0000 7f3cb57fa640 1 -- 172.21.15.112:0/4110254812 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca80054d0 con 0x7f3cb8102270 2024-09-21T03:29:55.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.568+0000 7f3cb57fa640 1 -- 172.21.15.112:0/4110254812 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8010690 con 0x7f3cb8102270 2024-09-21T03:29:55.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.571+0000 7f3cb57fa640 1 -- 172.21.15.112:0/4110254812 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3ca8010830 con 0x7f3cb8102270 2024-09-21T03:29:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.571+0000 7f3cc49aa740 1 Processor -- start 2024-09-21T03:29:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cc49aa740 1 -- start start 2024-09-21T03:29:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52bc070 0x5582d56636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52931b0 0x5582d5663be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5664120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5582d55d9ea0 con 0x5582d52bc070 2024-09-21T03:29:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5582d55c92d0 con 0x5582d52931b0 2024-09-21T03:29:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5582d55cd0e0 con 0x5582d529b390 2024-09-21T03:29:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cbc9d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52bc070 0x5582d56636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cbc9d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52bc070 0x5582d56636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56584/0 (socket says 172.21.15.112:56584) 2024-09-21T03:29:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cbc9d3640 1 -- 172.21.15.112:0/1508489931 learned_addr learned my addr 172.21.15.112:0/1508489931 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cb57fa640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3ca00b6700 0x7f3ca00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cb77fe640 1 --2- 172.21.15.112:0/1508489931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5664120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cbc9d3640 1 -- 172.21.15.112:0/1508489931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 msgr2=0x5582d5664120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.572+0000 7f3cbc9d3640 1 --2- 172.21.15.112:0/1508489931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5664120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.573+0000 7f3cbc9d3640 1 -- 172.21.15.112:0/1508489931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52931b0 msgr2=0x5582d5663be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:29:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.573+0000 7f3cbc9d3640 1 --2- 172.21.15.112:0/1508489931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52931b0 0x5582d5663be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.573+0000 7f3cbc9d3640 1 -- 172.21.15.112:0/1508489931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d5327650 con 0x5582d52bc070 2024-09-21T03:29:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.573+0000 7f3cb77fe640 1 --2- 172.21.15.112:0/1508489931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5664120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:29:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.573+0000 7f3cbc9d3640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3ca00b6700 0x7f3ca00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.573+0000 7f3cb57fa640 1 -- 172.21.15.112:0/4110254812 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3ca80e77d0 con 0x7f3cb8102270 2024-09-21T03:29:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.573+0000 7f3cbc9d3640 1 --2- 172.21.15.112:0/1508489931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52bc070 0x5582d56636a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0035000 tx=0x7f3cb00098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f011388fb4ef905d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.573+0000 7f3c9e7fc640 1 -- 172.21.15.112:0/1508489931 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0042040 con 0x5582d52bc070 2024-09-21T03:29:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.573+0000 7f3cbc9d3640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3ca00b6700 0x7f3ca00b8c10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3cb004d000 tx=0x7f3cb004e000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:55.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.573+0000 7f3c9e7fc640 1 -- 172.21.15.112:0/1508489931 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cb0036070 con 0x5582d52bc070 2024-09-21T03:29:55.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.573+0000 7f3c9e7fc640 1 -- 172.21.15.112:0/1508489931 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0004400 con 0x5582d52bc070 2024-09-21T03:29:55.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.574+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1508489931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52bc070 msgr2=0x5582d56636a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.574+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/1508489931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52bc070 0x5582d56636a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0035000 tx=0x7f3cb00098d0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.574+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1508489931 shutdown_connections 2024-09-21T03:29:55.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.574+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/1508489931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5664120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.574+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/1508489931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52931b0 0x5582d5663be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.574+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/1508489931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52bc070 0x5582d56636a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.574+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1508489931 >> 172.21.15.112:0/1508489931 conn(0x5582d52b0460 msgr2=0x5582d5661590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:55.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.575+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1508489931 shutdown_connections 2024-09-21T03:29:55.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.575+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1508489931 wait complete. 2024-09-21T03:29:55.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.575+0000 7f3cc49aa740 1 -- start start 2024-09-21T03:29:55.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.575+0000 7f3cc49aa740 1 Processor -- start 2024-09-21T03:29:55.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d56731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5673710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52bc070 0x5582d567f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5582d542a700 con 0x5582d52931b0 2024-09-21T03:29:55.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5582d5564160 con 0x5582d52bc070 2024-09-21T03:29:55.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5582d56761e0 con 0x5582d529b390 2024-09-21T03:29:55.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cc49aa740 10 client.0 _ll_register_callbacks cb 0x5582d557d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:55.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d56731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d56731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56592/0 (socket says 172.21.15.112:56592) 2024-09-21T03:29:55.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cb7fff640 1 -- 172.21.15.112:0/2259112396 learned_addr learned my addr 172.21.15.112:0/2259112396 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:55.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cb77fe640 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5673710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cbc9d3640 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52bc070 0x5582d567f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cb7fff640 1 -- 172.21.15.112:0/2259112396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 msgr2=0x5582d5673710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:55.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5673710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cb7fff640 1 -- 172.21.15.112:0/2259112396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52bc070 msgr2=0x5582d567f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:55.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.576+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52bc070 0x5582d567f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.577+0000 7f3cb7fff640 1 -- 172.21.15.112:0/2259112396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d5591c90 con 0x5582d52931b0 2024-09-21T03:29:55.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.577+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d56731d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80e5030 tx=0x7f3ca80b6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fde0c0ed692ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:55.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.577+0000 7f3c977fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.577+0000 7f3c977fe640 1 -- 172.21.15.112:0/2259112396 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80acfd0 con 0x5582d52931b0 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.577+0000 7f3c977fe640 1 -- 172.21.15.112:0/2259112396 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3ca80a8fd0 con 0x5582d52931b0 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.577+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2259112396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cb0036070 con 0x5582d52931b0 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.577+0000 7f3c977fe640 1 -- 172.21.15.112:0/2259112396 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ca80acdf0 con 0x5582d52931b0 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.577+0000 7f3cc49aa740 10 client.15744 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.577+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2259112396 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582d5682010 con 0x5582d52931b0 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.577+0000 7f3c977fe640 1 -- 172.21.15.112:0/2259112396 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca80ad190 con 0x5582d52931b0 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.578+0000 7f3c977fe640 1 -- 172.21.15.112:0/2259112396 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3ca80ac350 con 0x5582d52931b0 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.578+0000 7f3cc49aa740 10 client.15744 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.578+0000 7f3cc49aa740 10 client.15744 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.578+0000 7f3cc49aa740 20 client.15744 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.578+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5582d527a9f0 0x5582d56854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.578+0000 7f3cc49aa740 4 client.15744 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.578+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2259112396 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3cb0036070 con 0x5582d527a9f0 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.578+0000 7f3cb77fe640 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5582d527a9f0 0x5582d56854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.579+0000 7f3c977fe640 10 client.15744 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.579+0000 7f3cb77fe640 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5582d527a9f0 0x5582d56854b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcfe8047dd38bbdd server_cookie=85f2da74fdff59c3 in_seq=0 out_seq=0 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.579+0000 7f3c977fe640 1 -- 172.21.15.112:0/2259112396 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f3cb0036070 con 0x5582d527a9f0 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.579+0000 7f3c977fe640 10 client.15744 handle_command_reply: tid=0 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.579+0000 7f3cc49aa740 1 client.15744 shutdown 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2259112396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 msgr2=0x5582d56731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d56731d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80e5030 tx=0x7f3ca80b6f90 comp rx=0 tx=0).stop 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2259112396 shutdown_connections 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5582d527a9f0 0x5582d56854b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3c977fe640 0 client.15744 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52bc070 0x5582d567f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5673710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/2259112396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d56731d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2259112396 >> 172.21.15.112:0/2259112396 conn(0x5582d52b0460 msgr2=0x5582d5665090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:55.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2259112396 shutdown_connections 2024-09-21T03:29:55.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2259112396 wait complete. 2024-09-21T03:29:55.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.580+0000 7f3cc49aa740 20 client.15744 trim_cache size 0 max 16384 2024-09-21T03:29:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.604+0000 7f3cc49aa740 1 Processor -- start 2024-09-21T03:29:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.604+0000 7f3cc49aa740 1 -- start start 2024-09-21T03:29:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.605+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d527a9f0 0x5582d5674e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.605+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52931b0 0x5582d56703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.605+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5670900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.605+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4010ec0 con 0x5582d527a9f0 2024-09-21T03:29:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.605+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca400e790 con 0x5582d52931b0 2024-09-21T03:29:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.605+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca400c5e0 con 0x5582d529b390 2024-09-21T03:29:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.605+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d527a9f0 0x5582d5674e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.605+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d527a9f0 0x5582d5674e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56602/0 (socket says 172.21.15.112:56602) 2024-09-21T03:29:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.605+0000 7f3cb7fff640 1 -- 172.21.15.112:0/2480283427 learned_addr learned my addr 172.21.15.112:0/2480283427 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.605+0000 7f3cb77fe640 1 --2- 172.21.15.112:0/2480283427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52931b0 0x5582d56703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.605+0000 7f3cbc9d3640 1 --2- 172.21.15.112:0/2480283427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5670900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.606+0000 7f3cb7fff640 1 -- 172.21.15.112:0/2480283427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 msgr2=0x5582d5670900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.606+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/2480283427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5670900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.606+0000 7f3cb7fff640 1 -- 172.21.15.112:0/2480283427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52931b0 msgr2=0x5582d56703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.606+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/2480283427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52931b0 0x5582d56703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.606+0000 7f3cb7fff640 1 -- 172.21.15.112:0/2480283427 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d5674490 con 0x5582d527a9f0 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.606+0000 7f3cbc9d3640 1 --2- 172.21.15.112:0/2480283427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5670900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.606+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/2480283427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d527a9f0 0x5582d5674e00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80ad380 tx=0x7f3ca800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1f6c5a35880029 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.606+0000 7f3c9dffb640 1 -- 172.21.15.112:0/2480283427 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80cc070 con 0x5582d527a9f0 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.606+0000 7f3c9dffb640 1 -- 172.21.15.112:0/2480283427 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca80ad190 con 0x5582d527a9f0 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.606+0000 7f3c9dffb640 1 -- 172.21.15.112:0/2480283427 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80b3550 con 0x5582d527a9f0 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.607+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2480283427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d527a9f0 msgr2=0x5582d5674e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.607+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/2480283427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d527a9f0 0x5582d5674e00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80ad380 tx=0x7f3ca800cf80 comp rx=0 tx=0).stop 2024-09-21T03:29:55.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.607+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2480283427 shutdown_connections 2024-09-21T03:29:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.607+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/2480283427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d5670900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.607+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/2480283427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d52931b0 0x5582d56703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.607+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/2480283427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d527a9f0 0x5582d5674e00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.607+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2480283427 >> 172.21.15.112:0/2480283427 conn(0x5582d52b0460 msgr2=0x5582d56746f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.607+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2480283427 shutdown_connections 2024-09-21T03:29:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.607+0000 7f3cc49aa740 1 -- 172.21.15.112:0/2480283427 wait complete. 2024-09-21T03:29:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.608+0000 7f3cc49aa740 1 -- start start 2024-09-21T03:29:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.608+0000 7f3cc49aa740 1 Processor -- start 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.608+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d527a9f0 0x5582d5679850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.608+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d5679d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cc49aa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d56eee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca400cc50 con 0x5582d52931b0 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca400e0c0 con 0x5582d527a9f0 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cc49aa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca400d320 con 0x5582d529b390 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cc49aa740 10 client.0 _ll_register_callbacks cb 0x7f3ca800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d5679d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d5679d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56616/0 (socket says 172.21.15.112:56616) 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cb77fe640 1 -- 172.21.15.112:0/1560190280 learned_addr learned my addr 172.21.15.112:0/1560190280 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d527a9f0 0x5582d5679850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cbc9d3640 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d56eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cb77fe640 1 -- 172.21.15.112:0/1560190280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 msgr2=0x5582d56eee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cb77fe640 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d56eee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cb77fe640 1 -- 172.21.15.112:0/1560190280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d527a9f0 msgr2=0x5582d5679850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cb77fe640 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d527a9f0 0x5582d5679850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cb77fe640 1 -- 172.21.15.112:0/1560190280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d567a630 con 0x5582d52931b0 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d527a9f0 0x5582d5679850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3cb77fe640 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d5679d90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3cac004700 tx=0x7f3cac01e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af97078d0ac106 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.609+0000 7f3c977fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.610+0000 7f3c977fe640 1 -- 172.21.15.112:0/1560190280 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac027030 con 0x5582d52931b0 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.610+0000 7f3c977fe640 1 -- 172.21.15.112:0/1560190280 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3cac020040 con 0x5582d52931b0 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.610+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1560190280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca80ad190 con 0x5582d52931b0 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.610+0000 7f3c977fe640 1 -- 172.21.15.112:0/1560190280 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3cac031050 con 0x5582d52931b0 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.610+0000 7f3cc49aa740 10 client.15750 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.610+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1560190280 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582d56f04a0 con 0x5582d52931b0 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.610+0000 7f3c977fe640 1 -- 172.21.15.112:0/1560190280 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca80ad190 con 0x5582d52931b0 2024-09-21T03:29:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.610+0000 7f3c977fe640 1 -- 172.21.15.112:0/1560190280 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3cac004b80 con 0x5582d52931b0 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.611+0000 7f3cc49aa740 10 client.15750 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.611+0000 7f3cc49aa740 10 client.15750 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.611+0000 7f3cc49aa740 20 client.15750 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.611+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5582d52bc070 0x5582d56f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.611+0000 7f3cc49aa740 4 client.15750 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.611+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1560190280 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582d55dbdf0 con 0x5582d52bc070 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.611+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5582d52bc070 0x5582d56f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.611+0000 7f3c977fe640 10 client.15750 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.611+0000 7f3cb7fff640 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5582d52bc070 0x5582d56f2ce0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d47b247a6f46a55a server_cookie=76fe0011df76fd81 in_seq=0 out_seq=0 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3c977fe640 1 -- 172.21.15.112:0/1560190280 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582d55dbdf0 con 0x5582d52bc070 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3c977fe640 10 client.15750 handle_command_reply: tid=0 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 1 client.15750 shutdown 2024-09-21T03:29:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1560190280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 msgr2=0x5582d5679d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d5679d90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3cac004700 tx=0x7f3cac01e960 comp rx=0 tx=0).stop 2024-09-21T03:29:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1560190280 shutdown_connections 2024-09-21T03:29:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5582d52bc070 0x5582d56f2ce0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d529b390 0x5582d56eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3c977fe640 0 client.15750 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:29:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582d52931b0 0x5582d5679d90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 1 --2- 172.21.15.112:0/1560190280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582d527a9f0 0x5582d5679850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1560190280 >> 172.21.15.112:0/1560190280 conn(0x5582d52b0460 msgr2=0x5582d56645f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1560190280 shutdown_connections 2024-09-21T03:29:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 1 -- 172.21.15.112:0/1560190280 wait complete. 2024-09-21T03:29:55.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.612+0000 7f3cc49aa740 20 client.15750 trim_cache size 0 max 16384 2024-09-21T03:29:55.620 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:29:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 -- 172.21.15.112:0/4110254812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3ca00b6700 msgr2=0x7f3ca00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3ca00b6700 0x7f3ca00b8c10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3cb004d000 tx=0x7f3cb004e000 comp rx=0 tx=0).stop 2024-09-21T03:29:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 -- 172.21.15.112:0/4110254812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8102270 msgr2=0x7f3cb819b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8102270 0x7f3cb819b2c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800e9f0 tx=0x7f3ca800edf0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 -- 172.21.15.112:0/4110254812 shutdown_connections 2024-09-21T03:29:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3ca00b6700 0x7f3ca00b8c10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cb81084b0 0x7f3cb819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb8107b60 0x7f3cb819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 --2- 172.21.15.112:0/4110254812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8102270 0x7f3cb819b2c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 -- 172.21.15.112:0/4110254812 >> 172.21.15.112:0/4110254812 conn(0x7f3cb8077cc0 msgr2=0x7f3cb8103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 -- 172.21.15.112:0/4110254812 shutdown_connections 2024-09-21T03:29:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:55.613+0000 7f3cbe436640 1 -- 172.21.15.112:0/4110254812 wait complete. 2024-09-21T03:29:56.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:29:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:56 smithi112 ceph-mon[31442]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-21T03:29:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:56 smithi203 ceph-mon[34723]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-21T03:29:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:56 smithi187 ceph-mon[34603]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-21T03:29:56.969 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:57.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.671+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/4201479925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d481083a0 msgr2=0x7f1d481087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.671+0000 7f1d4ca6b640 1 --2- 172.21.15.112:0/4201479925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d481083a0 0x7f1d481087b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009900 tx=0x7f1d3c02a410 comp rx=0 tx=0).stop 2024-09-21T03:29:57.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.671+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/4201479925 shutdown_connections 2024-09-21T03:29:57.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.671+0000 7f1d4ca6b640 1 --2- 172.21.15.112:0/4201479925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d48108cf0 0x7f1d4810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.671+0000 7f1d4ca6b640 1 --2- 172.21.15.112:0/4201479925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d481083a0 0x7f1d481087b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.671+0000 7f1d4ca6b640 1 --2- 172.21.15.112:0/4201479925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d48102a60 0x7f1d48102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.671+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/4201479925 >> 172.21.15.112:0/4201479925 conn(0x7f1d48079730 msgr2=0x7f1d48079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.671+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/4201479925 shutdown_connections 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.671+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/4201479925 wait complete. 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d4ca6b640 1 Processor -- start 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d4ca6b640 1 -- start start 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d4ca6b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d48102a60 0x7f1d4819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d4ca6b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d481083a0 0x7f1d4819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d4ca6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48108cf0 0x7f1d4819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d4ca6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d48113a80 con 0x7f1d48108cf0 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d4ca6b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1d48113900 con 0x7f1d48102a60 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d4ca6b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d48113c00 con 0x7f1d481083a0 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d46d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48108cf0 0x7f1d4819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d46d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48108cf0 0x7f1d4819fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56648/0 (socket says 172.21.15.112:56648) 2024-09-21T03:29:57.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d46d9d640 1 -- 172.21.15.112:0/345019307 learned_addr learned my addr 172.21.15.112:0/345019307 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.672+0000 7f1d4659c640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d48102a60 0x7f1d4819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d481083a0 0x7f1d4819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d46d9d640 1 -- 172.21.15.112:0/345019307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d481083a0 msgr2=0x7f1d4819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d46d9d640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d481083a0 0x7f1d4819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d46d9d640 1 -- 172.21.15.112:0/345019307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d48102a60 msgr2=0x7f1d4819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d46d9d640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d48102a60 0x7f1d4819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d46d9d640 1 -- 172.21.15.112:0/345019307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d481a04e0 con 0x7f1d48108cf0 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d46d9d640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48108cf0 0x7f1d4819fdb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000eaa0 tx=0x7f1d4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d377fe640 1 -- 172.21.15.112:0/345019307 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4000cdb0 con 0x7f1d48108cf0 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/345019307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d481a07d0 con 0x7f1d48108cf0 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d377fe640 1 -- 172.21.15.112:0/345019307 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d40005020 con 0x7f1d48108cf0 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d377fe640 1 -- 172.21.15.112:0/345019307 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d400116d0 con 0x7f1d48108cf0 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.673+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/345019307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d481a6f30 con 0x7f1d48108cf0 2024-09-21T03:29:57.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.676+0000 7f1d52fdf740 1 Processor -- start 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.676+0000 7f1d377fe640 1 -- 172.21.15.112:0/345019307 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f1d40011870 con 0x7f1d48108cf0 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.676+0000 7f1d52fdf740 1 -- start start 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df46a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df46a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df46ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e0df3e0ea0 con 0x55e0df0a2390 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e0df3d02d0 con 0x55e0df0c3070 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e0df3d40e0 con 0x55e0df09a1b0 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d45d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df46ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d45d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df46ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56658/0 (socket says 172.21.15.112:56658) 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d45d9b640 1 -- 172.21.15.112:0/2589900194 learned_addr learned my addr 172.21.15.112:0/2589900194 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/2589900194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df46a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d4659c640 1 --2- 172.21.15.112:0/2589900194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df46a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d377fe640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1d300b6680 0x7f1d300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d45d9b640 1 -- 172.21.15.112:0/2589900194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 msgr2=0x55e0df46a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d46d9d640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1d300b6680 0x7f1d300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/2589900194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df46a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d45d9b640 1 -- 172.21.15.112:0/2589900194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 msgr2=0x55e0df46a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/2589900194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df46a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d45d9b640 1 -- 172.21.15.112:0/2589900194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0df12e650 con 0x55e0df0a2390 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.677+0000 7f1d377fe640 1 -- 172.21.15.112:0/345019307 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f1d400e7820 con 0x7f1d48108cf0 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.678+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/2589900194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df46ad40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c005940 tx=0x7f1d3c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5af76cd2e85f566 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.678+0000 7f1d46d9d640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1d300b6680 0x7f1d300b8b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d400e5080 tx=0x7f1d4007a9b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.678+0000 7f1d34ff9640 1 -- 172.21.15.112:0/2589900194 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c035040 con 0x55e0df0a2390 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.678+0000 7f1d34ff9640 1 -- 172.21.15.112:0/2589900194 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c004030 con 0x55e0df0a2390 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.678+0000 7f1d34ff9640 1 -- 172.21.15.112:0/2589900194 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c003850 con 0x55e0df0a2390 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.679+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2589900194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 msgr2=0x55e0df46ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.679+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/2589900194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df46ad40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c005940 tx=0x7f1d3c0058b0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.679+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2589900194 shutdown_connections 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.679+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/2589900194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df46ad40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.679+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/2589900194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df46a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.679+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/2589900194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df46a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.679+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2589900194 >> 172.21.15.112:0/2589900194 conn(0x55e0df0b7460 msgr2=0x55e0df4681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.679+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2589900194 shutdown_connections 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.679+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2589900194 wait complete. 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.679+0000 7f1d52fdf740 1 -- start start 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d52fdf740 1 Processor -- start 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df479d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df47a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df4860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e0df231700 con 0x55e0df0a2390 2024-09-21T03:29:57.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e0df36b160 con 0x55e0df0c3070 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e0df47cd70 con 0x55e0df09a1b0 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d52fdf740 10 client.0 _ll_register_callbacks cb 0x55e0df3849e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d4659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df47a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d4659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df47a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56666/0 (socket says 172.21.15.112:56666) 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d4659c640 1 -- 172.21.15.112:0/2686742153 learned_addr learned my addr 172.21.15.112:0/2686742153 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df4860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df479d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d4659c640 1 -- 172.21.15.112:0/2686742153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 msgr2=0x55e0df479d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d4659c640 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df479d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d4659c640 1 -- 172.21.15.112:0/2686742153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 msgr2=0x55e0df4860d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d4659c640 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df4860d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.680+0000 7f1d4659c640 1 -- 172.21.15.112:0/2686742153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0df398c90 con 0x55e0df0a2390 2024-09-21T03:29:57.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d4659c640 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df47a260 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800bfc0 tx=0x7f1d380118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da267060baaee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d2dffb640 1 -- 172.21.15.112:0/2686742153 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38004020 con 0x55e0df0a2390 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d2dffb640 1 -- 172.21.15.112:0/2686742153 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f1d380041c0 con 0x55e0df0a2390 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2686742153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3c004030 con 0x55e0df0a2390 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d2dffb640 1 -- 172.21.15.112:0/2686742153 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1d380043c0 con 0x55e0df0a2390 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d52fdf740 10 client.15762 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2686742153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0df488a90 con 0x55e0df0a2390 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d2dffb640 1 -- 172.21.15.112:0/2686742153 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c004030 con 0x55e0df0a2390 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d2dffb640 1 -- 172.21.15.112:0/2686742153 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1d38003a60 con 0x55e0df0a2390 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d52fdf740 10 client.15762 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d52fdf740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d52fdf740 20 client.15762 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e0df0819f0 0x55e0df48c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d52fdf740 4 client.15762 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.681+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2686742153 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0df3e2df0 con 0x55e0df0819f0 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.682+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e0df0819f0 0x55e0df48c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.682+0000 7f1d2dffb640 10 client.15762 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.682+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e0df0819f0 0x55e0df48c080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb499070a85e37a server_cookie=5be22ba4cbb7718b in_seq=0 out_seq=0 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d2dffb640 1 -- 172.21.15.112:0/2686742153 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0df3e2df0 con 0x55e0df0819f0 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d2dffb640 10 client.15762 handle_command_reply: tid=0 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 1 client.15762 shutdown 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2686742153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 msgr2=0x55e0df47a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df47a260 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800bfc0 tx=0x7f1d380118d0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2686742153 shutdown_connections 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e0df0819f0 0x55e0df48c080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df4860d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d2dffb640 0 client.15762 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df47a260 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/2686742153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df479d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2686742153 >> 172.21.15.112:0/2686742153 conn(0x55e0df0b7460 msgr2=0x55e0df466750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2686742153 shutdown_connections 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 1 -- 172.21.15.112:0/2686742153 wait complete. 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.683+0000 7f1d52fdf740 20 client.15762 trim_cache size 0 max 16384 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.696+0000 7f1d52fdf740 1 Processor -- start 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.696+0000 7f1d52fdf740 1 -- start start 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.696+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df4681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.696+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df485fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df46f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d481afd00 con 0x55e0df0a2390 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1d481ad5d0 con 0x55e0df0c3070 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d481ab420 con 0x55e0df09a1b0 2024-09-21T03:29:57.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d4659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df485fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d4659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df485fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56670/0 (socket says 172.21.15.112:56670) 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d4659c640 1 -- 172.21.15.112:0/3471383774 learned_addr learned my addr 172.21.15.112:0/3471383774 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/3471383774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df4681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/3471383774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df46f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d4659c640 1 -- 172.21.15.112:0/3471383774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 msgr2=0x55e0df4681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d4659c640 1 --2- 172.21.15.112:0/3471383774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df4681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d4659c640 1 -- 172.21.15.112:0/3471383774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 msgr2=0x55e0df46f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d4659c640 1 --2- 172.21.15.112:0/3471383774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df46f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d4659c640 1 -- 172.21.15.112:0/3471383774 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0df46cb90 con 0x55e0df0a2390 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d4659c640 1 --2- 172.21.15.112:0/3471383774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df485fd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1d38004610 tx=0x7f1d38011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fc98f6294cce82 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d2ffff640 1 -- 172.21.15.112:0/3471383774 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38004130 con 0x55e0df0a2390 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d2ffff640 1 -- 172.21.15.112:0/3471383774 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d38003aa0 con 0x55e0df0a2390 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.697+0000 7f1d2ffff640 1 -- 172.21.15.112:0/3471383774 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38005650 con 0x55e0df0a2390 2024-09-21T03:29:57.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.698+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3471383774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 msgr2=0x55e0df485fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.698+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/3471383774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df485fd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1d38004610 tx=0x7f1d38011e50 comp rx=0 tx=0).stop 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.698+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3471383774 shutdown_connections 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.698+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/3471383774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df0c3070 0x55e0df46f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.698+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/3471383774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0a2390 0x55e0df485fd0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.698+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/3471383774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df09a1b0 0x55e0df4681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.698+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3471383774 >> 172.21.15.112:0/3471383774 conn(0x55e0df0b7460 msgr2=0x55e0df467ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.698+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3471383774 shutdown_connections 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.698+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3471383774 wait complete. 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d52fdf740 1 -- start start 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d52fdf740 1 Processor -- start 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df09a1b0 0x55e0df482a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df0a2390 0x55e0df47df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d52fdf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0c3070 0x55e0df47e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d481aba90 con 0x55e0df0c3070 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1d481acf00 con 0x55e0df09a1b0 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d52fdf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d481ac160 con 0x55e0df0a2390 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d52fdf740 10 client.0 _ll_register_callbacks cb 0x7f1d3800bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d45d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0c3070 0x55e0df47e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d45d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0c3070 0x55e0df47e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56678/0 (socket says 172.21.15.112:56678) 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d45d9b640 1 -- 172.21.15.112:0/3231253715 learned_addr learned my addr 172.21.15.112:0/3231253715 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.699+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df09a1b0 0x55e0df482a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d4659c640 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df0a2390 0x55e0df47df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d45d9b640 1 -- 172.21.15.112:0/3231253715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df0a2390 msgr2=0x55e0df47df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df0a2390 0x55e0df47df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d45d9b640 1 -- 172.21.15.112:0/3231253715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df09a1b0 msgr2=0x55e0df482a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df09a1b0 0x55e0df482a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d45d9b640 1 -- 172.21.15.112:0/3231253715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0df47abc0 con 0x55e0df0c3070 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d45d9b640 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0c3070 0x55e0df47e6a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c000c00 tx=0x7f1d3800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965155b612614e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:29:57.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d2dffb640 1 -- 172.21.15.112:0/3231253715 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c05b070 con 0x55e0df0c3070 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d2dffb640 1 -- 172.21.15.112:0/3231253715 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f1d3c031890 con 0x55e0df0c3070 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3231253715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d38003aa0 con 0x55e0df0c3070 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d2dffb640 1 -- 172.21.15.112:0/3231253715 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1d3c035040 con 0x55e0df0c3070 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d52fdf740 10 client.15768 fetch_fsmap learned FSMap version 25 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.700+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3231253715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0df4f6a00 con 0x55e0df0c3070 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.701+0000 7f1d2dffb640 1 -- 172.21.15.112:0/3231253715 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c004470 con 0x55e0df0c3070 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.701+0000 7f1d2dffb640 1 -- 172.21.15.112:0/3231253715 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1d3c03a040 con 0x55e0df0c3070 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.701+0000 7f1d52fdf740 10 client.15768 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.701+0000 7f1d52fdf740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.701+0000 7f1d52fdf740 20 client.15768 populate_metadata read hostname 'smithi112' 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.701+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e0df056da0 0x55e0df4f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.701+0000 7f1d52fdf740 4 client.15768 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:29:57.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.701+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3231253715 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1d38003aa0 con 0x55e0df056da0 2024-09-21T03:29:57.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.701+0000 7f1d4659c640 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e0df056da0 0x55e0df4f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:29:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d2dffb640 10 client.15768 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d4659c640 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e0df056da0 0x55e0df4f8d50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce846733f318d55b server_cookie=3df9634d07e7de20 in_seq=0 out_seq=0 2024-09-21T03:29:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d2dffb640 1 -- 172.21.15.112:0/3231253715 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1d38003aa0 con 0x55e0df056da0 2024-09-21T03:29:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d2dffb640 10 client.15768 handle_command_reply: tid=0 2024-09-21T03:29:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d52fdf740 1 client.15768 shutdown 2024-09-21T03:29:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3231253715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0c3070 msgr2=0x55e0df47e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0c3070 0x55e0df47e6a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c000c00 tx=0x7f1d3800bb40 comp rx=0 tx=0).stop 2024-09-21T03:29:57.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3231253715 shutdown_connections 2024-09-21T03:29:57.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e0df056da0 0x55e0df4f8d50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d2dffb640 0 client.15768 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:29:57.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0df0c3070 0x55e0df47e6a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e0df0a2390 0x55e0df47df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d52fdf740 1 --2- 172.21.15.112:0/3231253715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e0df09a1b0 0x55e0df482a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3231253715 >> 172.21.15.112:0/3231253715 conn(0x55e0df0b7460 msgr2=0x55e0df46f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:57.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3231253715 shutdown_connections 2024-09-21T03:29:57.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.702+0000 7f1d52fdf740 1 -- 172.21.15.112:0/3231253715 wait complete. 2024-09-21T03:29:57.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d52fdf740 20 client.15768 trim_cache size 0 max 16384 2024-09-21T03:29:57.720 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:29:57.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/345019307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1d300b6680 msgr2=0x7f1d300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1d300b6680 0x7f1d300b8b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d400e5080 tx=0x7f1d4007a9b0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/345019307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48108cf0 msgr2=0x7f1d4819fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:29:57.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48108cf0 0x7f1d4819fdb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000eaa0 tx=0x7f1d4000ef60 comp rx=0 tx=0).stop 2024-09-21T03:29:57.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/345019307 shutdown_connections 2024-09-21T03:29:57.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1d300b6680 0x7f1d300b8b90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48108cf0 0x7f1d4819fdb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d481083a0 0x7f1d4819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 --2- 172.21.15.112:0/345019307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d48102a60 0x7f1d4819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:29:57.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/345019307 >> 172.21.15.112:0/345019307 conn(0x7f1d48079730 msgr2=0x7f1d481053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:29:57.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/345019307 shutdown_connections 2024-09-21T03:29:57.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:29:57.703+0000 7f1d4ca6b640 1 -- 172.21.15.112:0/345019307 wait complete. 2024-09-21T03:29:58.387 INFO:tasks.workunit.client.0.smithi112.stderr:Updating files: 94% (12026/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T03:29:58.516 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:29:58 smithi112 ceph-mon[31442]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-21T03:29:58.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:29:58 smithi203 ceph-mon[34723]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-21T03:29:58.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:29:58 smithi187 ceph-mon[34603]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-21T03:29:59.049 INFO:tasks.workunit.client.0.smithi112.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T03:29:59.049 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-21T03:29:59.049 INFO:tasks.workunit.client.0.smithi112.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr: git switch -c 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr:Or undo this operation with: 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr: git switch - 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-21T03:29:59.050 INFO:tasks.workunit.client.0.smithi112.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T03:29:59.051 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-09-21T03:29:59.051 INFO:tasks.workunit.client.0.smithi112.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:29:59.054 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T03:29:59.078 INFO:tasks.workunit.client.0.smithi112.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T03:29:59.080 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:29:59.080 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T03:29:59.518 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:29:59.518 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:29:59.771 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T03:29:59.773 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:29:59.905 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T03:29:59.944 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:29:59.946 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:29:59.946 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T03:29:59.985 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:29:59.988 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:29:59.988 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T03:30:00.004 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-21T03:30:00.006 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-21T03:30:00.007 DEBUG:teuthology.orchestra.run.smithi112:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-21T03:30:00.069 INFO:tasks.workunit.client.0.smithi112.stderr:+ dbench 1 2024-09-21T03:30:00.097 INFO:tasks.workunit.client.0.smithi112.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-21T03:30:00.097 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-09-21T03:30:00.097 INFO:tasks.workunit.client.0.smithi112.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-21T03:30:00.109 INFO:tasks.workunit.client.0.smithi112.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-21T03:30:00.151 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:00 smithi112 ceph-mon[31442]: overall HEALTH_OK 2024-09-21T03:30:00.210 INFO:tasks.workunit.client.0.smithi112.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-21T03:30:00.210 INFO:tasks.workunit.client.0.smithi112.stdout:releasing clients 2024-09-21T03:30:00.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.341+0000 7fe2532ae640 1 -- 172.21.15.112:0/2526979372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24c103150 msgr2=0x7fe24c103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:00.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.342+0000 7fe2532ae640 1 --2- 172.21.15.112:0/2526979372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24c103150 0x7fe24c103560 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe248009900 tx=0x7fe24802a410 comp rx=0 tx=0).stop 2024-09-21T03:30:00.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.342+0000 7fe2532ae640 1 -- 172.21.15.112:0/2526979372 shutdown_connections 2024-09-21T03:30:00.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.342+0000 7fe2532ae640 1 --2- 172.21.15.112:0/2526979372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe24c105b20 0x7fe24c10f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.342+0000 7fe2532ae640 1 --2- 172.21.15.112:0/2526979372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe24c1051d0 0x7fe24c1055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.342+0000 7fe2532ae640 1 --2- 172.21.15.112:0/2526979372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24c103150 0x7fe24c103560 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.342+0000 7fe2532ae640 1 -- 172.21.15.112:0/2526979372 >> 172.21.15.112:0/2526979372 conn(0x7fe24c079f90 msgr2=0x7fe24c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:00.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.342+0000 7fe2532ae640 1 -- 172.21.15.112:0/2526979372 shutdown_connections 2024-09-21T03:30:00.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.342+0000 7fe2532ae640 1 -- 172.21.15.112:0/2526979372 wait complete. 2024-09-21T03:30:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.342+0000 7fe2532ae640 1 Processor -- start 2024-09-21T03:30:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.342+0000 7fe2532ae640 1 -- start start 2024-09-21T03:30:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe2532ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24c103150 0x7fe24c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe2532ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe24c1051d0 0x7fe24c19b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe2532ae640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe24c105b20 0x7fe24c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe2532ae640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c112a10 con 0x7fe24c103150 2024-09-21T03:30:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe2532ae640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c112890 con 0x7fe24c105b20 2024-09-21T03:30:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe2532ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c112b90 con 0x7fe24c1051d0 2024-09-21T03:30:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe25104a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24c103150 0x7fe24c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe25104a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24c103150 0x7fe24c19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56694/0 (socket says 172.21.15.112:56694) 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe25104a640 1 -- 172.21.15.112:0/2252822204 learned_addr learned my addr 172.21.15.112:0/2252822204 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe250849640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe24c1051d0 0x7fe24c19b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe25184b640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe24c105b20 0x7fe24c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe25104a640 1 -- 172.21.15.112:0/2252822204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe24c1051d0 msgr2=0x7fe24c19b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe25104a640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe24c1051d0 0x7fe24c19b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe25104a640 1 -- 172.21.15.112:0/2252822204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe24c105b20 msgr2=0x7fe24c19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe25104a640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe24c105b20 0x7fe24c19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe25104a640 1 -- 172.21.15.112:0/2252822204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c1a65f0 con 0x7fe24c103150 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.343+0000 7fe250849640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe24c1051d0 0x7fe24c19b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.344+0000 7fe25104a640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24c103150 0x7fe24c19b360 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe248005940 tx=0x7fe248004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.344+0000 7fe2427fc640 1 -- 172.21.15.112:0/2252822204 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24803f030 con 0x7fe24c103150 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.344+0000 7fe2532ae640 1 -- 172.21.15.112:0/2252822204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24c1a6880 con 0x7fe24c103150 2024-09-21T03:30:00.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.344+0000 7fe2427fc640 1 -- 172.21.15.112:0/2252822204 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe248004440 con 0x7fe24c103150 2024-09-21T03:30:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.344+0000 7fe2427fc640 1 -- 172.21.15.112:0/2252822204 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248035070 con 0x7fe24c103150 2024-09-21T03:30:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.344+0000 7fe2532ae640 1 -- 172.21.15.112:0/2252822204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24c1a6d60 con 0x7fe24c103150 2024-09-21T03:30:00.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.347+0000 7fe259822740 1 Processor -- start 2024-09-21T03:30:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.347+0000 7fe259822740 1 -- start start 2024-09-21T03:30:00.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.347+0000 7fe2427fc640 1 -- 172.21.15.112:0/2252822204 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe248042030 con 0x7fe24c103150 2024-09-21T03:30:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.347+0000 7fe259822740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 0x562700f92430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.347+0000 7fe259822740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700f92970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.347+0000 7fe259822740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700bca670 0x562700f92eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.347+0000 7fe259822740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562700f08e60 con 0x562700bc2490 2024-09-21T03:30:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.347+0000 7fe259822740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562700ef8290 con 0x562700bca670 2024-09-21T03:30:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.347+0000 7fe259822740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562700efc0a0 con 0x562700beb350 2024-09-21T03:30:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe25184b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 0x562700f92430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe25184b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 0x562700f92430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43870/0 (socket says 172.21.15.112:43870) 2024-09-21T03:30:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe25104a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700f92970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe25184b640 1 -- 172.21.15.112:0/343309051 learned_addr learned my addr 172.21.15.112:0/343309051 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe250849640 1 --2- 172.21.15.112:0/343309051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700bca670 0x562700f92eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe25184b640 1 -- 172.21.15.112:0/343309051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700bca670 msgr2=0x562700f92eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe25184b640 1 --2- 172.21.15.112:0/343309051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700bca670 0x562700f92eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe25184b640 1 -- 172.21.15.112:0/343309051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 msgr2=0x562700f92970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe25184b640 1 --2- 172.21.15.112:0/343309051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700f92970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe25184b640 1 -- 172.21.15.112:0/343309051 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562700c56930 con 0x562700beb350 2024-09-21T03:30:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe25104a640 1 --2- 172.21.15.112:0/343309051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700f92970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe25184b640 1 --2- 172.21.15.112:0/343309051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 0x562700f92430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe244009fd0 tx=0x7fe2440118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a821caa3e5be64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe2427fc640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe2380b66b0 0x7fe2380b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.348+0000 7fe2377fe640 1 -- 172.21.15.112:0/343309051 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244004d20 con 0x562700beb350 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.349+0000 7fe25104a640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe2380b66b0 0x7fe2380b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.349+0000 7fe2377fe640 1 -- 172.21.15.112:0/343309051 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe244004050 con 0x562700beb350 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.349+0000 7fe2377fe640 1 -- 172.21.15.112:0/343309051 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244005280 con 0x562700beb350 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.349+0000 7fe2427fc640 1 -- 172.21.15.112:0/2252822204 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fe2481079d0 con 0x7fe24c103150 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe259822740 1 -- 172.21.15.112:0/343309051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 msgr2=0x562700f92430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe25104a640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe2380b66b0 0x7fe2380b8bc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe248105230 tx=0x7fe248000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe259822740 1 --2- 172.21.15.112:0/343309051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 0x562700f92430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe244009fd0 tx=0x7fe2440118d0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe259822740 1 -- 172.21.15.112:0/343309051 shutdown_connections 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe259822740 1 --2- 172.21.15.112:0/343309051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700bca670 0x562700f92eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe259822740 1 --2- 172.21.15.112:0/343309051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700f92970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe259822740 1 --2- 172.21.15.112:0/343309051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 0x562700f92430 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe259822740 1 -- 172.21.15.112:0/343309051 >> 172.21.15.112:0/343309051 conn(0x562700bdf740 msgr2=0x562700f90320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:00.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe259822740 1 -- 172.21.15.112:0/343309051 shutdown_connections 2024-09-21T03:30:00.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe259822740 1 -- 172.21.15.112:0/343309051 wait complete. 2024-09-21T03:30:00.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe259822740 1 -- start start 2024-09-21T03:30:00.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.350+0000 7fe259822740 1 Processor -- start 2024-09-21T03:30:00.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe259822740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700f9e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe259822740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700bca670 0x562700f9eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe259822740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 0x562700fa3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe259822740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562700d59890 con 0x562700bc2490 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe259822740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562700e92d40 con 0x562700bca670 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe259822740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562700f989a0 con 0x562700beb350 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe259822740 10 client.0 _ll_register_callbacks cb 0x562700ef6c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe25184b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700f9e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe25184b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700f9e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56708/0 (socket says 172.21.15.112:56708) 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe25184b640 1 -- 172.21.15.112:0/1153784708 learned_addr learned my addr 172.21.15.112:0/1153784708 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe250849640 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700bca670 0x562700f9eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe25184b640 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 0x562700fa3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe250849640 1 -- 172.21.15.112:0/1153784708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 msgr2=0x562700fa3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe250849640 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 0x562700fa3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe250849640 1 -- 172.21.15.112:0/1153784708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 msgr2=0x562700f9e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:00.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe250849640 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700f9e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe250849640 1 -- 172.21.15.112:0/1153784708 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562700fa41d0 con 0x562700bca670 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.351+0000 7fe25184b640 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700f9e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.352+0000 7fe250849640 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700bca670 0x562700f9eb20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe24c06a530 tx=0x7fe23c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b5b2241eaf8da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.352+0000 7fe234ff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.352+0000 7fe234ff9640 1 -- 172.21.15.112:0/1153784708 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c012110 con 0x562700bca670 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.352+0000 7fe259822740 1 -- 172.21.15.112:0/1153784708 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe244004050 con 0x562700bca670 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.352+0000 7fe234ff9640 1 -- 172.21.15.112:0/1153784708 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fe23c004050 con 0x562700bca670 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.352+0000 7fe234ff9640 1 -- 172.21.15.112:0/1153784708 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe23c0122b0 con 0x562700bca670 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.352+0000 7fe259822740 10 client.25361 fetch_fsmap learned FSMap version 25 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.352+0000 7fe259822740 1 -- 172.21.15.112:0/1153784708 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562700facdf0 con 0x562700bca670 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.352+0000 7fe234ff9640 1 -- 172.21.15.112:0/1153784708 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe244004050 con 0x562700bca670 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.353+0000 7fe234ff9640 1 -- 172.21.15.112:0/1153784708 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe23c026470 con 0x562700bca670 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.353+0000 7fe259822740 10 client.25361 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.353+0000 7fe259822740 10 client.25361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.353+0000 7fe259822740 20 client.25361 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.353+0000 7fe259822740 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562700ba9cd0 0x562700fb0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.353+0000 7fe259822740 4 client.25361 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.353+0000 7fe259822740 1 -- 172.21.15.112:0/1153784708 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562700f0adb0 con 0x562700ba9cd0 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.353+0000 7fe25184b640 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562700ba9cd0 0x562700fb0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.353+0000 7fe25184b640 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562700ba9cd0 0x562700fb0230 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ccafe0e72b0a3f1 server_cookie=2b38dd4a97b0159 in_seq=0 out_seq=0 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.353+0000 7fe234ff9640 10 client.25361 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe234ff9640 1 -- 172.21.15.112:0/1153784708 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562700f0adb0 con 0x562700ba9cd0 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe234ff9640 10 client.25361 handle_command_reply: tid=0 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 1 client.25361 shutdown 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 1 -- 172.21.15.112:0/1153784708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700bca670 msgr2=0x562700f9eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700bca670 0x562700f9eb20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe24c06a530 tx=0x7fe23c00f790 comp rx=0 tx=0).stop 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 1 -- 172.21.15.112:0/1153784708 shutdown_connections 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562700ba9cd0 0x562700fb0230 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700beb350 0x562700fa3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe234ff9640 0 client.25361 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:00.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700bca670 0x562700f9eb20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 1 --2- 172.21.15.112:0/1153784708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700f9e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 1 -- 172.21.15.112:0/1153784708 >> 172.21.15.112:0/1153784708 conn(0x562700bdf740 msgr2=0x562700f93e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:00.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 1 -- 172.21.15.112:0/1153784708 shutdown_connections 2024-09-21T03:30:00.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 1 -- 172.21.15.112:0/1153784708 wait complete. 2024-09-21T03:30:00.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.354+0000 7fe259822740 20 client.25361 trim_cache size 0 max 16384 2024-09-21T03:30:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.372+0000 7fe259822740 1 Processor -- start 2024-09-21T03:30:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe259822740 1 -- start start 2024-09-21T03:30:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe259822740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700bc2490 0x562700f9cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe259822740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bca670 0x562700f9d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe259822740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700beb350 0x562700f8fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe259822740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c1afdb0 con 0x562700bca670 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe259822740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c1ad680 con 0x562700beb350 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe259822740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c1ab4d0 con 0x562700bc2490 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe250849640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bca670 0x562700f9d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe250849640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bca670 0x562700f9d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56712/0 (socket says 172.21.15.112:56712) 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe250849640 1 -- 172.21.15.112:0/2741127606 learned_addr learned my addr 172.21.15.112:0/2741127606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe25184b640 1 --2- 172.21.15.112:0/2741127606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700beb350 0x562700f8fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe25184b640 1 --2- 172.21.15.112:0/2741127606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700bc2490 0x562700f9cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe250849640 1 -- 172.21.15.112:0/2741127606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700bc2490 msgr2=0x562700f9cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe250849640 1 --2- 172.21.15.112:0/2741127606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700bc2490 0x562700f9cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe250849640 1 -- 172.21.15.112:0/2741127606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700beb350 msgr2=0x562700f8fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe250849640 1 --2- 172.21.15.112:0/2741127606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700beb350 0x562700f8fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe250849640 1 -- 172.21.15.112:0/2741127606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562700f94950 con 0x562700bca670 2024-09-21T03:30:00.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.373+0000 7fe250849640 1 --2- 172.21.15.112:0/2741127606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bca670 0x562700f9d120 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe23c026810 tx=0x7fe23c012830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfb7629182b3b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:00.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.374+0000 7fe236ffd640 1 -- 172.21.15.112:0/2741127606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c026470 con 0x562700bca670 2024-09-21T03:30:00.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.374+0000 7fe236ffd640 1 -- 172.21.15.112:0/2741127606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe23c02b070 con 0x562700bca670 2024-09-21T03:30:00.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.374+0000 7fe236ffd640 1 -- 172.21.15.112:0/2741127606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c003850 con 0x562700bca670 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.374+0000 7fe259822740 1 -- 172.21.15.112:0/2741127606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bca670 msgr2=0x562700f9d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.374+0000 7fe259822740 1 --2- 172.21.15.112:0/2741127606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bca670 0x562700f9d120 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe23c026810 tx=0x7fe23c012830 comp rx=0 tx=0).stop 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.375+0000 7fe259822740 1 -- 172.21.15.112:0/2741127606 shutdown_connections 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.375+0000 7fe259822740 1 --2- 172.21.15.112:0/2741127606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700beb350 0x562700f8fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.375+0000 7fe259822740 1 --2- 172.21.15.112:0/2741127606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bca670 0x562700f9d120 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.375+0000 7fe259822740 1 --2- 172.21.15.112:0/2741127606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700bc2490 0x562700f9cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.375+0000 7fe259822740 1 -- 172.21.15.112:0/2741127606 >> 172.21.15.112:0/2741127606 conn(0x562700bdf740 msgr2=0x562700f994e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.375+0000 7fe259822740 1 -- 172.21.15.112:0/2741127606 shutdown_connections 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.375+0000 7fe259822740 1 -- 172.21.15.112:0/2741127606 wait complete. 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.375+0000 7fe259822740 1 -- start start 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.375+0000 7fe259822740 1 Processor -- start 2024-09-21T03:30:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe259822740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700fa9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe259822740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700bca670 0x562700faa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe259822740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700beb350 0x562700fa4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe259822740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c1abb40 con 0x562700bc2490 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe259822740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c1acfb0 con 0x562700beb350 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe259822740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c1ac210 con 0x562700bca670 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe259822740 10 client.0 _ll_register_callbacks cb 0x7fe23c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe25184b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700fa9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe25184b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700fa9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56716/0 (socket says 172.21.15.112:56716) 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe25184b640 1 -- 172.21.15.112:0/3527272238 learned_addr learned my addr 172.21.15.112:0/3527272238 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe250849640 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700bca670 0x562700faa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe25184b640 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700beb350 0x562700fa4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe25184b640 1 -- 172.21.15.112:0/3527272238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700bca670 msgr2=0x562700faa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe25184b640 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700bca670 0x562700faa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe25184b640 1 -- 172.21.15.112:0/3527272238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700beb350 msgr2=0x562700fa4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:00.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe25184b640 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700beb350 0x562700fa4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe25184b640 1 -- 172.21.15.112:0/3527272238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562700fa5340 con 0x562700bc2490 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe25184b640 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700fa9f20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe244014040 tx=0x7fe23c0128a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f7d0cf577efb23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe234ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe234ff9640 1 -- 172.21.15.112:0/3527272238 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244004700 con 0x562700bc2490 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe234ff9640 1 -- 172.21.15.112:0/3527272238 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fe24401f040 con 0x562700bc2490 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.376+0000 7fe259822740 1 -- 172.21.15.112:0/3527272238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe23c02b070 con 0x562700bc2490 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe234ff9640 1 -- 172.21.15.112:0/3527272238 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe244004900 con 0x562700bc2490 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe259822740 10 client.15786 fetch_fsmap learned FSMap version 25 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe259822740 1 -- 172.21.15.112:0/3527272238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56270101b4a0 con 0x562700bc2490 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe234ff9640 1 -- 172.21.15.112:0/3527272238 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe23c02b070 con 0x562700bc2490 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe234ff9640 1 -- 172.21.15.112:0/3527272238 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe244025040 con 0x562700bc2490 2024-09-21T03:30:00.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe259822740 10 client.15786 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:30:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe259822740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe259822740 20 client.15786 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe259822740 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562700b7f080 0x56270101daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe259822740 4 client.15786 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:30:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe259822740 1 -- 172.21.15.112:0/3527272238 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562700f06860 con 0x562700b7f080 2024-09-21T03:30:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.377+0000 7fe250849640 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562700b7f080 0x56270101daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.378+0000 7fe234ff9640 10 client.15786 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.378+0000 7fe250849640 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562700b7f080 0x56270101daa0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afcef67439f35b0e server_cookie=72cbe04402b35da7 in_seq=0 out_seq=0 2024-09-21T03:30:00.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.379+0000 7fe234ff9640 1 -- 172.21.15.112:0/3527272238 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562700f06860 con 0x562700b7f080 2024-09-21T03:30:00.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.379+0000 7fe234ff9640 10 client.15786 handle_command_reply: tid=0 2024-09-21T03:30:00.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.379+0000 7fe259822740 1 client.15786 shutdown 2024-09-21T03:30:00.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe259822740 1 -- 172.21.15.112:0/3527272238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 msgr2=0x562700fa9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:00.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe259822740 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700fa9f20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe244014040 tx=0x7fe23c0128a0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe259822740 1 -- 172.21.15.112:0/3527272238 shutdown_connections 2024-09-21T03:30:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe259822740 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562700b7f080 0x56270101daa0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe259822740 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562700beb350 0x562700fa4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe259822740 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562700bca670 0x562700faa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe234ff9640 0 client.15786 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe259822740 1 --2- 172.21.15.112:0/3527272238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562700bc2490 0x562700fa9f20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe259822740 1 -- 172.21.15.112:0/3527272238 >> 172.21.15.112:0/3527272238 conn(0x562700bdf740 msgr2=0x562700f994e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe259822740 1 -- 172.21.15.112:0/3527272238 shutdown_connections 2024-09-21T03:30:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe259822740 1 -- 172.21.15.112:0/3527272238 wait complete. 2024-09-21T03:30:00.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe259822740 20 client.15786 trim_cache size 0 max 16384 2024-09-21T03:30:00.385 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:30:00.385 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:30:00.386 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "1e6e5cf7-2802-4874-abd6-2efed3be7f30", 2024-09-21T03:30:00.386 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:30:00.386 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:30:00.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.380+0000 7fe2532ae640 1 -- 172.21.15.112:0/2252822204 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe2380b66b0 msgr2=0x7fe2380b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.381+0000 7fe2532ae640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe2380b66b0 0x7fe2380b8bc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe248105230 tx=0x7fe248000bc0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.381+0000 7fe2532ae640 1 -- 172.21.15.112:0/2252822204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24c103150 msgr2=0x7fe24c19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.381+0000 7fe2532ae640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24c103150 0x7fe24c19b360 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe248005940 tx=0x7fe248004040 comp rx=0 tx=0).stop 2024-09-21T03:30:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.381+0000 7fe2532ae640 1 -- 172.21.15.112:0/2252822204 shutdown_connections 2024-09-21T03:30:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.381+0000 7fe2532ae640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe2380b66b0 0x7fe2380b8bc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.381+0000 7fe2532ae640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe24c105b20 0x7fe24c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.381+0000 7fe2532ae640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe24c1051d0 0x7fe24c19b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.381+0000 7fe2532ae640 1 --2- 172.21.15.112:0/2252822204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe24c103150 0x7fe24c19b360 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.381+0000 7fe2532ae640 1 -- 172.21.15.112:0/2252822204 >> 172.21.15.112:0/2252822204 conn(0x7fe24c079f90 msgr2=0x7fe24c106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.381+0000 7fe2532ae640 1 -- 172.21.15.112:0/2252822204 shutdown_connections 2024-09-21T03:30:00.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:00.381+0000 7fe2532ae640 1 -- 172.21.15.112:0/2252822204 wait complete. 2024-09-21T03:30:00.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:00 smithi203 ceph-mon[34723]: overall HEALTH_OK 2024-09-21T03:30:00.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:00 smithi187 ceph-mon[34603]: overall HEALTH_OK 2024-09-21T03:30:01.209 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:30:01.210 INFO:tasks.workunit.client.0.smithi112.stdout: 1 3243 34.58 MB/sec warmup 1 sec latency 26.206 ms 2024-09-21T03:30:01.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:01 smithi112 ceph-mon[31442]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-21T03:30:01.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:01 smithi203 ceph-mon[34723]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-21T03:30:01.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:01 smithi187 ceph-mon[34603]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-09-21T03:30:01.471 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.978+0000 7ff948eb1640 1 -- 172.21.15.112:0/3653235003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9441083c0 msgr2=0x7ff9441087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.978+0000 7ff948eb1640 1 --2- 172.21.15.112:0/3653235003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9441083c0 0x7ff9441087d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff938009900 tx=0x7ff93802a410 comp rx=0 tx=0).stop 2024-09-21T03:30:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.979+0000 7ff948eb1640 1 -- 172.21.15.112:0/3653235003 shutdown_connections 2024-09-21T03:30:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.979+0000 7ff948eb1640 1 --2- 172.21.15.112:0/3653235003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944108d10 0x7ff94410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.979+0000 7ff948eb1640 1 --2- 172.21.15.112:0/3653235003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9441083c0 0x7ff9441087d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.979+0000 7ff948eb1640 1 --2- 172.21.15.112:0/3653235003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff944102a80 0x7ff944102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.979+0000 7ff948eb1640 1 -- 172.21.15.112:0/3653235003 >> 172.21.15.112:0/3653235003 conn(0x7ff944079740 msgr2=0x7ff944079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.979+0000 7ff948eb1640 1 -- 172.21.15.112:0/3653235003 shutdown_connections 2024-09-21T03:30:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.979+0000 7ff948eb1640 1 -- 172.21.15.112:0/3653235003 wait complete. 2024-09-21T03:30:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff948eb1640 1 Processor -- start 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff948eb1640 1 -- start start 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff948eb1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944102a80 0x7ff9440ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff948eb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9441083c0 0x7ff9440ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff948eb1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff944108d10 0x7ff944100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff948eb1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff944112980 con 0x7ff9441083c0 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff948eb1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff944112800 con 0x7ff944102a80 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff948eb1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff944112b00 con 0x7ff944108d10 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff941d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9441083c0 0x7ff9440ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff94259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944102a80 0x7ff9440ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff942d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff944108d10 0x7ff944100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff94259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944102a80 0x7ff9440ff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:38292/0 (socket says 172.21.15.112:38292) 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff941d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9441083c0 0x7ff9440ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56732/0 (socket says 172.21.15.112:56732) 2024-09-21T03:30:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff94259c640 1 -- 172.21.15.112:0/872529109 learned_addr learned my addr 172.21.15.112:0/872529109 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff941d9b640 1 -- 172.21.15.112:0/872529109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff944108d10 msgr2=0x7ff944100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff941d9b640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff944108d10 0x7ff944100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff941d9b640 1 -- 172.21.15.112:0/872529109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944102a80 msgr2=0x7ff9440ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff941d9b640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944102a80 0x7ff9440ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff941d9b640 1 -- 172.21.15.112:0/872529109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9441a8720 con 0x7ff9441083c0 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.980+0000 7ff942d9d640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff944108d10 0x7ff944100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.981+0000 7ff941d9b640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9441083c0 0x7ff9440ffbd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff93802a930 tx=0x7ff938031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.981+0000 7ff9337fe640 1 -- 172.21.15.112:0/872529109 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93803f070 con 0x7ff9441083c0 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.981+0000 7ff948eb1640 1 -- 172.21.15.112:0/872529109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9441a89b0 con 0x7ff9441083c0 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.981+0000 7ff9337fe640 1 -- 172.21.15.112:0/872529109 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff938004490 con 0x7ff9441083c0 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.981+0000 7ff9337fe640 1 -- 172.21.15.112:0/872529109 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938035070 con 0x7ff9441083c0 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.981+0000 7ff948eb1640 1 -- 172.21.15.112:0/872529109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9441a8ee0 con 0x7ff9441083c0 2024-09-21T03:30:01.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.983+0000 7ff94f425740 1 Processor -- start 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.983+0000 7ff9337fe640 1 -- 172.21.15.112:0/872529109 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff938043400 con 0x7ff9441083c0 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff94f425740 1 -- start start 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff94f425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 0x55e046e90270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff94f425740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046ac01b0 0x55e046e907b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff94f425740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e90cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff94f425740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e046e06e50 con 0x55e046ac8390 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff94f425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e046df6280 con 0x55e046ae9070 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff94f425740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e046dfa090 con 0x55e046ac01b0 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff941d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e90cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff942d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 0x55e046e90270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff94259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046ac01b0 0x55e046e907b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff941d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e90cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56734/0 (socket says 172.21.15.112:56734) 2024-09-21T03:30:01.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff941d9b640 1 -- 172.21.15.112:0/752787467 learned_addr learned my addr 172.21.15.112:0/752787467 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff942d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 0x55e046e90270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:38298/0 (socket says 172.21.15.112:38298) 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.984+0000 7ff9337fe640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff92c0b6680 0x7ff92c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff942d9d640 1 -- 172.21.15.112:0/752787467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046ac01b0 msgr2=0x55e046e907b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff942d9d640 1 --2- 172.21.15.112:0/752787467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046ac01b0 0x55e046e907b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff942d9d640 1 -- 172.21.15.112:0/752787467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 msgr2=0x55e046e90cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff942d9d640 1 --2- 172.21.15.112:0/752787467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e90cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff942d9d640 1 -- 172.21.15.112:0/752787467 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e046b54650 con 0x55e046ae9070 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff941d9b640 1 --2- 172.21.15.112:0/752787467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e90cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff942d9d640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff92c0b6680 0x7ff92c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff9337fe640 1 -- 172.21.15.112:0/872529109 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff938107b70 con 0x7ff9441083c0 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff942d9d640 1 --2- 172.21.15.112:0/752787467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 0x55e046e90270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff93c005fd0 tx=0x7ff93c011d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ca7cef5476ae984 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff930ff9640 1 -- 172.21.15.112:0/752787467 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c017070 con 0x55e046ae9070 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff930ff9640 1 -- 172.21.15.112:0/752787467 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff93c01c430 con 0x55e046ae9070 2024-09-21T03:30:01.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff930ff9640 1 -- 172.21.15.112:0/752787467 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c00e610 con 0x55e046ae9070 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.985+0000 7ff942d9d640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff92c0b6680 0x7ff92c0b8b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff93c023000 tx=0x7ff93c024000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.986+0000 7ff94f425740 1 -- 172.21.15.112:0/752787467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 msgr2=0x55e046e90270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.986+0000 7ff94f425740 1 --2- 172.21.15.112:0/752787467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 0x55e046e90270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff93c005fd0 tx=0x7ff93c011d50 comp rx=0 tx=0).stop 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.986+0000 7ff94f425740 1 -- 172.21.15.112:0/752787467 shutdown_connections 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.986+0000 7ff94f425740 1 --2- 172.21.15.112:0/752787467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e90cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.986+0000 7ff94f425740 1 --2- 172.21.15.112:0/752787467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046ac01b0 0x55e046e907b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.986+0000 7ff94f425740 1 --2- 172.21.15.112:0/752787467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 0x55e046e90270 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.986+0000 7ff94f425740 1 -- 172.21.15.112:0/752787467 >> 172.21.15.112:0/752787467 conn(0x55e046add460 msgr2=0x55e046e8e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.986+0000 7ff94f425740 1 -- 172.21.15.112:0/752787467 shutdown_connections 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.986+0000 7ff94f425740 1 -- 172.21.15.112:0/752787467 wait complete. 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff94f425740 1 -- start start 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff94f425740 1 Processor -- start 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff94f425740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046ac01b0 0x55e046e9b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff94f425740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e9be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff94f425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 0x55e046ea7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff94f425740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e046c57700 con 0x55e046ac8390 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff94f425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e046d91110 con 0x55e046ae9070 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff94f425740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e046e9ea50 con 0x55e046ac01b0 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff94f425740 10 client.0 _ll_register_callbacks cb 0x55e046daa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff941d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e9be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff94259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046ac01b0 0x55e046e9b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff941d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e9be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56736/0 (socket says 172.21.15.112:56736) 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff942d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 0x55e046ea7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff94259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046ac01b0 0x55e046e9b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43944/0 (socket says 172.21.15.112:43944) 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.987+0000 7ff941d9b640 1 -- 172.21.15.112:0/2176015150 learned_addr learned my addr 172.21.15.112:0/2176015150 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff942d9d640 1 -- 172.21.15.112:0/2176015150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046ac01b0 msgr2=0x55e046e9b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff942d9d640 1 --2- 172.21.15.112:0/2176015150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046ac01b0 0x55e046e9b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff942d9d640 1 -- 172.21.15.112:0/2176015150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 msgr2=0x55e046e9be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff942d9d640 1 --2- 172.21.15.112:0/2176015150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e9be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff942d9d640 1 -- 172.21.15.112:0/2176015150 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e046dbec40 con 0x55e046ae9070 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff942d9d640 1 --2- 172.21.15.112:0/2176015150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 0x55e046ea7d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009fd0 tx=0x7ff93c01cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86eeb8cf2995aec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff929ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff929ffb640 1 -- 172.21.15.112:0/2176015150 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c017040 con 0x55e046ae9070 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff94f425740 1 -- 172.21.15.112:0/2176015150 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93c01c430 con 0x55e046ae9070 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff929ffb640 1 -- 172.21.15.112:0/2176015150 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff93c003e00 con 0x55e046ae9070 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff929ffb640 1 -- 172.21.15.112:0/2176015150 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff93c00fb40 con 0x55e046ae9070 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff94f425740 10 client.25379 fetch_fsmap learned FSMap version 25 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff94f425740 1 -- 172.21.15.112:0/2176015150 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e046eaa770 con 0x55e046ae9070 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.988+0000 7ff929ffb640 1 -- 172.21.15.112:0/2176015150 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff93c01c430 con 0x55e046ae9070 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.989+0000 7ff929ffb640 1 -- 172.21.15.112:0/2176015150 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff93c01b070 con 0x55e046ae9070 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.989+0000 7ff94f425740 10 client.25379 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:30:01.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.989+0000 7ff94f425740 10 client.25379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:01.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.989+0000 7ff94f425740 20 client.25379 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:01.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.989+0000 7ff94f425740 1 --2- 172.21.15.112:0/2176015150 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e046aa79f0 0x55e046eadb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:01.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.989+0000 7ff94f425740 4 client.25379 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:01.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.989+0000 7ff94f425740 1 -- 172.21.15.112:0/2176015150 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e046e08da0 con 0x55e046aa79f0 2024-09-21T03:30:01.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.989+0000 7ff94259c640 1 --2- 172.21.15.112:0/2176015150 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e046aa79f0 0x55e046eadb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:01.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.989+0000 7ff94259c640 1 --2- 172.21.15.112:0/2176015150 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e046aa79f0 0x55e046eadb90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e82ae83714ceaa server_cookie=ad81af0d84134191 in_seq=0 out_seq=0 2024-09-21T03:30:01.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.989+0000 7ff929ffb640 10 client.25379 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:02.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff929ffb640 1 -- 172.21.15.112:0/2176015150 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e046e08da0 con 0x55e046aa79f0 2024-09-21T03:30:02.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff929ffb640 10 client.25379 handle_command_reply: tid=0 2024-09-21T03:30:02.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff94f425740 1 client.25379 shutdown 2024-09-21T03:30:02.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff94f425740 1 -- 172.21.15.112:0/2176015150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 msgr2=0x55e046ea7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff94f425740 1 --2- 172.21.15.112:0/2176015150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 0x55e046ea7d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009fd0 tx=0x7ff93c01cf90 comp rx=0 tx=0).stop 2024-09-21T03:30:02.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff94f425740 1 -- 172.21.15.112:0/2176015150 shutdown_connections 2024-09-21T03:30:02.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff94f425740 1 --2- 172.21.15.112:0/2176015150 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e046aa79f0 0x55e046eadb90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff94f425740 1 --2- 172.21.15.112:0/2176015150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ae9070 0x55e046ea7d40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff94f425740 1 --2- 172.21.15.112:0/2176015150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e9be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff929ffb640 0 client.25379 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:02.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff94f425740 1 --2- 172.21.15.112:0/2176015150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046ac01b0 0x55e046e9b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.990+0000 7ff94f425740 1 -- 172.21.15.112:0/2176015150 >> 172.21.15.112:0/2176015150 conn(0x55e046add460 msgr2=0x55e046e93450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:02.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.991+0000 7ff94f425740 1 -- 172.21.15.112:0/2176015150 shutdown_connections 2024-09-21T03:30:02.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.991+0000 7ff94f425740 1 -- 172.21.15.112:0/2176015150 wait complete. 2024-09-21T03:30:02.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:01.991+0000 7ff94f425740 20 client.25379 trim_cache size 0 max 16384 2024-09-21T03:30:02.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff94f425740 1 Processor -- start 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff94f425740 1 -- start start 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff94f425740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046add460 0x55e046e9d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff94f425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ac01b0 0x55e046e912a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff94f425740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046ea7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff94f425740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9441b3fd0 con 0x55e046ac8390 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff94f425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff9441b18a0 con 0x55e046ac01b0 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff94f425740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9441af6f0 con 0x55e046add460 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff942d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046ea7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff941d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ac01b0 0x55e046e912a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff942d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046ea7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56748/0 (socket says 172.21.15.112:56748) 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff94259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046add460 0x55e046e9d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.009+0000 7ff942d9d640 1 -- 172.21.15.112:0/1232195410 learned_addr learned my addr 172.21.15.112:0/1232195410 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:02.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.010+0000 7ff94259c640 1 -- 172.21.15.112:0/1232195410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ac01b0 msgr2=0x55e046e912a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:02.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.010+0000 7ff94259c640 1 --2- 172.21.15.112:0/1232195410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ac01b0 0x55e046e912a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.010+0000 7ff94259c640 1 -- 172.21.15.112:0/1232195410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 msgr2=0x55e046ea7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:02.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.010+0000 7ff94259c640 1 --2- 172.21.15.112:0/1232195410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046ea7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.010+0000 7ff94259c640 1 -- 172.21.15.112:0/1232195410 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e046e89580 con 0x55e046add460 2024-09-21T03:30:02.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.010+0000 7ff942d9d640 1 --2- 172.21.15.112:0/1232195410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046ea7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:02.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.010+0000 7ff94259c640 1 --2- 172.21.15.112:0/1232195410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046add460 0x55e046e9d820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55e046eae270 tx=0x7ff934005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc721d697e2e773 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:02.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.010+0000 7ff92bfff640 1 -- 172.21.15.112:0/1232195410 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934005020 con 0x55e046add460 2024-09-21T03:30:02.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.010+0000 7ff92bfff640 1 -- 172.21.15.112:0/1232195410 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9340047a0 con 0x55e046add460 2024-09-21T03:30:02.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.010+0000 7ff92bfff640 1 -- 172.21.15.112:0/1232195410 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93401f710 con 0x55e046add460 2024-09-21T03:30:02.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.011+0000 7ff94f425740 1 -- 172.21.15.112:0/1232195410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046add460 msgr2=0x55e046e9d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.011+0000 7ff94f425740 1 --2- 172.21.15.112:0/1232195410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046add460 0x55e046e9d820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55e046eae270 tx=0x7ff934005f90 comp rx=0 tx=0).stop 2024-09-21T03:30:02.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.011+0000 7ff94f425740 1 -- 172.21.15.112:0/1232195410 shutdown_connections 2024-09-21T03:30:02.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.011+0000 7ff94f425740 1 --2- 172.21.15.112:0/1232195410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046ea7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.011+0000 7ff94f425740 1 --2- 172.21.15.112:0/1232195410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ac01b0 0x55e046e912a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.011+0000 7ff94f425740 1 --2- 172.21.15.112:0/1232195410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046add460 0x55e046e9d820 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.011+0000 7ff94f425740 1 -- 172.21.15.112:0/1232195410 >> 172.21.15.112:0/1232195410 conn(0x55e046e9c000 msgr2=0x55e046e95380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:02.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.011+0000 7ff94f425740 1 -- 172.21.15.112:0/1232195410 shutdown_connections 2024-09-21T03:30:02.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.011+0000 7ff94f425740 1 -- 172.21.15.112:0/1232195410 wait complete. 2024-09-21T03:30:02.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.011+0000 7ff94f425740 1 -- start start 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff94f425740 1 Processor -- start 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff94f425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ac01b0 0x55e046ea4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff94f425740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e9fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff94f425740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046add460 0x55e046ea0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff94f425740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9441afd60 con 0x55e046ac8390 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff94f425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff9441b11d0 con 0x55e046ac01b0 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff94f425740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9441b0430 con 0x55e046add460 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff94f425740 10 client.0 _ll_register_callbacks cb 0x7ff93810b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff941d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e9fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff941d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e9fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56760/0 (socket says 172.21.15.112:56760) 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff94259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ac01b0 0x55e046ea4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff941d9b640 1 -- 172.21.15.112:0/2580881409 learned_addr learned my addr 172.21.15.112:0/2580881409 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff942d9d640 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046add460 0x55e046ea0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff941d9b640 1 -- 172.21.15.112:0/2580881409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046add460 msgr2=0x55e046ea0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff941d9b640 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046add460 0x55e046ea0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff941d9b640 1 -- 172.21.15.112:0/2580881409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ac01b0 msgr2=0x55e046ea4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff941d9b640 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ac01b0 0x55e046ea4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.012+0000 7ff941d9b640 1 -- 172.21.15.112:0/2580881409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e046ea07a0 con 0x55e046ac8390 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff941d9b640 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e9fa30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff938049040 tx=0x7ff93810b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f42fb26f0189f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff929ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff929ffb640 1 -- 172.21.15.112:0/2580881409 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938048040 con 0x55e046ac8390 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff929ffb640 1 -- 172.21.15.112:0/2580881409 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff93810c040 con 0x55e046ac8390 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff94f425740 1 -- 172.21.15.112:0/2580881409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9340047a0 con 0x55e046ac8390 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff929ffb640 1 -- 172.21.15.112:0/2580881409 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9380cd8a0 con 0x55e046ac8390 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff94f425740 10 client.15804 fetch_fsmap learned FSMap version 25 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff94f425740 1 -- 172.21.15.112:0/2580881409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e046f18550 con 0x55e046ac8390 2024-09-21T03:30:02.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff929ffb640 1 -- 172.21.15.112:0/2580881409 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9340047a0 con 0x55e046ac8390 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff929ffb640 1 -- 172.21.15.112:0/2580881409 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff93804b060 con 0x55e046ac8390 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff94f425740 10 client.15804 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff94f425740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.013+0000 7ff94f425740 20 client.15804 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.014+0000 7ff94f425740 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e046ae9070 0x55e046f1aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.014+0000 7ff94f425740 4 client.15804 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.014+0000 7ff94f425740 1 -- 172.21.15.112:0/2580881409 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e046e04850 con 0x55e046ae9070 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.014+0000 7ff94259c640 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e046ae9070 0x55e046f1aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.014+0000 7ff929ffb640 10 client.15804 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.014+0000 7ff94259c640 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e046ae9070 0x55e046f1aca0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10552fabd0051e20 server_cookie=2d29e4d6fe36dcc6 in_seq=0 out_seq=0 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.014+0000 7ff929ffb640 1 -- 172.21.15.112:0/2580881409 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x55e046e04850 con 0x55e046ae9070 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.014+0000 7ff929ffb640 10 client.15804 handle_command_reply: tid=0 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.014+0000 7ff94f425740 1 client.15804 shutdown 2024-09-21T03:30:02.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff94f425740 1 -- 172.21.15.112:0/2580881409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 msgr2=0x55e046e9fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff94f425740 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e9fa30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff938049040 tx=0x7ff93810b290 comp rx=0 tx=0).stop 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff94f425740 1 -- 172.21.15.112:0/2580881409 shutdown_connections 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff94f425740 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e046ae9070 0x55e046f1aca0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff94f425740 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e046add460 0x55e046ea0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff94f425740 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e046ac8390 0x55e046e9fa30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff94f425740 1 --2- 172.21.15.112:0/2580881409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e046ac01b0 0x55e046ea4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff94f425740 1 -- 172.21.15.112:0/2580881409 >> 172.21.15.112:0/2580881409 conn(0x55e046e9c000 msgr2=0x55e046e95380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff929ffb640 0 client.15804 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff94f425740 1 -- 172.21.15.112:0/2580881409 shutdown_connections 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff94f425740 1 -- 172.21.15.112:0/2580881409 wait complete. 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff94f425740 20 client.15804 trim_cache size 0 max 16384 2024-09-21T03:30:02.021 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "1e6e5cf7-2802-4874-abd6-2efed3be7f30", 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:30:02.022 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:30:02.023 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:30:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:30:02.025 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:30:02.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff948eb1640 1 -- 172.21.15.112:0/872529109 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff92c0b6680 msgr2=0x7ff92c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff948eb1640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff92c0b6680 0x7ff92c0b8b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff93c023000 tx=0x7ff93c024000 comp rx=0 tx=0).stop 2024-09-21T03:30:02.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff948eb1640 1 -- 172.21.15.112:0/872529109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9441083c0 msgr2=0x7ff9440ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:02.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.015+0000 7ff948eb1640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9441083c0 0x7ff9440ffbd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff93802a930 tx=0x7ff938031a20 comp rx=0 tx=0).stop 2024-09-21T03:30:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.016+0000 7ff948eb1640 1 -- 172.21.15.112:0/872529109 shutdown_connections 2024-09-21T03:30:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.016+0000 7ff948eb1640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff92c0b6680 0x7ff92c0b8b90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.016+0000 7ff948eb1640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff944108d10 0x7ff944100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.016+0000 7ff948eb1640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9441083c0 0x7ff9440ffbd0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.016+0000 7ff948eb1640 1 --2- 172.21.15.112:0/872529109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff944102a80 0x7ff9440ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.016+0000 7ff948eb1640 1 -- 172.21.15.112:0/872529109 >> 172.21.15.112:0/872529109 conn(0x7ff944079740 msgr2=0x7ff944101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.016+0000 7ff948eb1640 1 -- 172.21.15.112:0/872529109 shutdown_connections 2024-09-21T03:30:02.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:02.016+0000 7ff948eb1640 1 -- 172.21.15.112:0/872529109 wait complete. 2024-09-21T03:30:02.210 INFO:tasks.workunit.client.0.smithi112.stdout: 1 5245 22.67 MB/sec warmup 2 sec latency 42.328 ms 2024-09-21T03:30:02.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:02 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:30:02.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:02 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:02.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:02 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:30:02.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:02 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:02.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:02 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:30:02.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:02 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:02.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:02 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:30:02.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:02 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:02.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:02 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:30:02.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:02 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:02.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:02 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:30:02.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:02 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:02.697 INFO:tasks.cephfs.filesystem:scrub status for tag:1e6e5cf7-2802-4874-abd6-2efed3be7f30 - {'path': '/', 'tag': '1e6e5cf7-2802-4874-abd6-2efed3be7f30', 'options': 'recursive,force'} 2024-09-21T03:30:03.210 INFO:tasks.workunit.client.0.smithi112.stdout: 1 8322 20.43 MB/sec warmup 3 sec latency 13.203 ms 2024-09-21T03:30:03.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:03 smithi203 ceph-mon[34723]: pgmap v349: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 50 KiB/s wr, 9 op/s 2024-09-21T03:30:03.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:03 smithi187 ceph-mon[34603]: pgmap v349: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 50 KiB/s wr, 9 op/s 2024-09-21T03:30:03.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:03 smithi112 ceph-mon[31442]: pgmap v349: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 50 KiB/s wr, 9 op/s 2024-09-21T03:30:04.210 INFO:tasks.workunit.client.0.smithi112.stdout: 1 11638 19.81 MB/sec warmup 4 sec latency 15.825 ms 2024-09-21T03:30:04.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:04 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:30:04.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:04 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:30:04.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:04 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:30:05.210 INFO:tasks.workunit.client.0.smithi112.stdout: 1 14918 19.54 MB/sec warmup 5 sec latency 13.274 ms 2024-09-21T03:30:05.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:05 smithi203 ceph-mon[34723]: pgmap v350: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 345 KiB/s wr, 56 op/s 2024-09-21T03:30:05.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:05 smithi187 ceph-mon[34603]: pgmap v350: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 345 KiB/s wr, 56 op/s 2024-09-21T03:30:05.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:05 smithi112 ceph-mon[31442]: pgmap v350: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 345 KiB/s wr, 56 op/s 2024-09-21T03:30:06.210 INFO:tasks.workunit.client.0.smithi112.stdout: 1 18300 19.43 MB/sec warmup 6 sec latency 8.104 ms 2024-09-21T03:30:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:06 smithi112 ceph-mon[31442]: pgmap v351: 97 pgs: 97 active+clean; 8.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 746 KiB/s wr, 110 op/s 2024-09-21T03:30:06.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:06 smithi203 ceph-mon[34723]: pgmap v351: 97 pgs: 97 active+clean; 8.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 746 KiB/s wr, 110 op/s 2024-09-21T03:30:06.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:06 smithi187 ceph-mon[34603]: pgmap v351: 97 pgs: 97 active+clean; 8.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 746 KiB/s wr, 110 op/s 2024-09-21T03:30:07.210 INFO:tasks.workunit.client.0.smithi112.stdout: 1 21640 19.34 MB/sec warmup 7 sec latency 16.217 ms 2024-09-21T03:30:08.210 INFO:tasks.workunit.client.0.smithi112.stdout: 1 25030 19.19 MB/sec warmup 8 sec latency 14.546 ms 2024-09-21T03:30:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:08 smithi112 ceph-mon[31442]: pgmap v352: 97 pgs: 97 active+clean; 9.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 823 KiB/s wr, 113 op/s 2024-09-21T03:30:08.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:08 smithi203 ceph-mon[34723]: pgmap v352: 97 pgs: 97 active+clean; 9.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 823 KiB/s wr, 113 op/s 2024-09-21T03:30:08.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:08 smithi187 ceph-mon[34603]: pgmap v352: 97 pgs: 97 active+clean; 9.6 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 823 KiB/s wr, 113 op/s 2024-09-21T03:30:09.210 INFO:tasks.workunit.client.0.smithi112.stdout: 1 28231 18.82 MB/sec warmup 9 sec latency 13.246 ms 2024-09-21T03:30:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:09 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:09 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:09.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:09 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:30:10.210 INFO:tasks.workunit.client.0.smithi112.stdout: 1 30842 18.11 MB/sec warmup 10 sec latency 19.050 ms 2024-09-21T03:30:11.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:11.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:11.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:10 smithi112 ceph-mon[31442]: pgmap v353: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-21T03:30:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:10 smithi203 ceph-mon[34723]: pgmap v353: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-21T03:30:11.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:30:11.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:30:11.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:30:11.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:10 smithi187 ceph-mon[34603]: pgmap v353: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-21T03:30:11.210 INFO:tasks.workunit.client.0.smithi112.stdout: 1 33523 18.02 MB/sec warmup 11 sec latency 13.760 ms 2024-09-21T03:30:12.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 36663 17.85 MB/sec warmup 12 sec latency 13.090 ms 2024-09-21T03:30:12.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:12 smithi112 ceph-mon[31442]: pgmap v354: 97 pgs: 97 active+clean; 19 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-09-21T03:30:12.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:12 smithi203 ceph-mon[34723]: pgmap v354: 97 pgs: 97 active+clean; 19 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-09-21T03:30:12.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:12 smithi187 ceph-mon[34603]: pgmap v354: 97 pgs: 97 active+clean; 19 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-09-21T03:30:13.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 39722 17.80 MB/sec warmup 13 sec latency 25.519 ms 2024-09-21T03:30:14.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 42939 17.89 MB/sec warmup 14 sec latency 14.353 ms 2024-09-21T03:30:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:14 smithi112 ceph-mon[31442]: pgmap v355: 97 pgs: 97 active+clean; 19 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-21T03:30:14.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:14 smithi203 ceph-mon[34723]: pgmap v355: 97 pgs: 97 active+clean; 19 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-21T03:30:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:14 smithi187 ceph-mon[34603]: pgmap v355: 97 pgs: 97 active+clean; 19 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-21T03:30:15.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 46119 17.86 MB/sec warmup 15 sec latency 25.231 ms 2024-09-21T03:30:16.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 48923 17.54 MB/sec warmup 16 sec latency 14.470 ms 2024-09-21T03:30:16.745 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:16 smithi187 ceph-mon[34603]: pgmap v356: 97 pgs: 97 active+clean; 26 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-09-21T03:30:16.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:16 smithi112 ceph-mon[31442]: pgmap v356: 97 pgs: 97 active+clean; 26 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-09-21T03:30:16.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:16 smithi203 ceph-mon[34723]: pgmap v356: 97 pgs: 97 active+clean; 26 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-09-21T03:30:17.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 51951 17.32 MB/sec warmup 17 sec latency 12.967 ms 2024-09-21T03:30:18.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 54421 17.22 MB/sec warmup 18 sec latency 13.931 ms 2024-09-21T03:30:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:18 smithi112 ceph-mon[31442]: pgmap v357: 97 pgs: 97 active+clean; 26 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-21T03:30:18.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:18 smithi203 ceph-mon[34723]: pgmap v357: 97 pgs: 97 active+clean; 26 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-21T03:30:18.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:18 smithi187 ceph-mon[34603]: pgmap v357: 97 pgs: 97 active+clean; 26 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-21T03:30:19.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 57258 17.21 MB/sec warmup 19 sec latency 24.354 ms 2024-09-21T03:30:20.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60412 17.23 MB/sec warmup 20 sec latency 13.942 ms 2024-09-21T03:30:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:20 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:30:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:20 smithi112 ceph-mon[31442]: pgmap v358: 97 pgs: 97 active+clean; 32 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 258 op/s 2024-09-21T03:30:20.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:20 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:30:20.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:20 smithi203 ceph-mon[34723]: pgmap v358: 97 pgs: 97 active+clean; 32 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 258 op/s 2024-09-21T03:30:20.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:20 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:30:20.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:20 smithi187 ceph-mon[34603]: pgmap v358: 97 pgs: 97 active+clean; 32 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 258 op/s 2024-09-21T03:30:21.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60412 16.41 MB/sec warmup 21 sec latency 1001.411 ms 2024-09-21T03:30:22.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60412 15.66 MB/sec warmup 22 sec latency 2001.490 ms 2024-09-21T03:30:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:22 smithi112 ceph-mon[31442]: pgmap v359: 97 pgs: 97 active+clean; 34 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-09-21T03:30:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:22 smithi203 ceph-mon[34723]: pgmap v359: 97 pgs: 97 active+clean; 34 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-09-21T03:30:22.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:22 smithi187 ceph-mon[34603]: pgmap v359: 97 pgs: 97 active+clean; 34 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-09-21T03:30:23.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60412 14.98 MB/sec warmup 23 sec latency 3001.588 ms 2024-09-21T03:30:24.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60441 14.36 MB/sec warmup 24 sec latency 3831.493 ms 2024-09-21T03:30:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:24 smithi112 ceph-mon[31442]: pgmap v360: 97 pgs: 97 active+clean; 34 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-09-21T03:30:24.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:24 smithi203 ceph-mon[34723]: pgmap v360: 97 pgs: 97 active+clean; 34 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-09-21T03:30:24.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:24 smithi187 ceph-mon[34603]: pgmap v360: 97 pgs: 97 active+clean; 34 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-09-21T03:30:25.212 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60441 13.79 MB/sec warmup 25 sec latency 1150.486 ms 2024-09-21T03:30:26.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60441 13.26 MB/sec warmup 26 sec latency 2150.561 ms 2024-09-21T03:30:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:26 smithi112 ceph-mon[31442]: pgmap v361: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-21T03:30:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:26 smithi203 ceph-mon[34723]: pgmap v361: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-21T03:30:26.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:26 smithi187 ceph-mon[34603]: pgmap v361: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-21T03:30:27.211 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60441 12.77 MB/sec warmup 27 sec latency 3150.694 ms 2024-09-21T03:30:28.212 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60441 12.31 MB/sec warmup 28 sec latency 4150.832 ms 2024-09-21T03:30:28.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:28 smithi112 ceph-mon[31442]: pgmap v362: 97 pgs: 97 active+clean; 53 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-09-21T03:30:28.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:28 smithi203 ceph-mon[34723]: pgmap v362: 97 pgs: 97 active+clean; 53 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-09-21T03:30:28.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:28 smithi187 ceph-mon[34603]: pgmap v362: 97 pgs: 97 active+clean; 53 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-09-21T03:30:29.212 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60463 11.89 MB/sec warmup 29 sec latency 5021.643 ms 2024-09-21T03:30:30.212 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60463 11.49 MB/sec warmup 30 sec latency 1115.315 ms 2024-09-21T03:30:30.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:30 smithi112 ceph-mon[31442]: pgmap v363: 97 pgs: 97 active+clean; 58 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 4.7 MiB/s wr, 191 op/s 2024-09-21T03:30:30.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:30 smithi203 ceph-mon[34723]: pgmap v363: 97 pgs: 97 active+clean; 58 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 4.7 MiB/s wr, 191 op/s 2024-09-21T03:30:30.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:30 smithi187 ceph-mon[34603]: pgmap v363: 97 pgs: 97 active+clean; 58 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 4.7 MiB/s wr, 191 op/s 2024-09-21T03:30:31.212 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60463 11.12 MB/sec warmup 31 sec latency 2115.420 ms 2024-09-21T03:30:32.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60463 10.77 MB/sec warmup 32 sec latency 3115.559 ms 2024-09-21T03:30:32.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:30:32.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:32 smithi112 ceph-mon[31442]: pgmap v364: 97 pgs: 97 active+clean; 49 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 6.0 MiB/s wr, 195 op/s 2024-09-21T03:30:32.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:32 smithi203 ceph-mon[34723]: pgmap v364: 97 pgs: 97 active+clean; 49 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 6.0 MiB/s wr, 195 op/s 2024-09-21T03:30:33.112 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:33.113 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:32 smithi187 ceph-mon[34603]: pgmap v364: 97 pgs: 97 active+clean; 49 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 6.0 MiB/s wr, 195 op/s 2024-09-21T03:30:33.212 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60463 10.45 MB/sec warmup 33 sec latency 4115.610 ms 2024-09-21T03:30:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.409+0000 7f4e2a926640 1 -- 172.21.15.112:0/4258673088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e24108420 msgr2=0x7f4e24108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.409+0000 7f4e2a926640 1 --2- 172.21.15.112:0/4258673088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e24108420 0x7f4e24108830 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009900 tx=0x7f4e1802a410 comp rx=0 tx=0).stop 2024-09-21T03:30:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.409+0000 7f4e2a926640 1 -- 172.21.15.112:0/4258673088 shutdown_connections 2024-09-21T03:30:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.409+0000 7f4e2a926640 1 --2- 172.21.15.112:0/4258673088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e24108d70 0x7f4e2410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.409+0000 7f4e2a926640 1 --2- 172.21.15.112:0/4258673088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e24108420 0x7f4e24108830 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.409+0000 7f4e2a926640 1 --2- 172.21.15.112:0/4258673088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e24102ae0 0x7f4e24102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.409+0000 7f4e2a926640 1 -- 172.21.15.112:0/4258673088 >> 172.21.15.112:0/4258673088 conn(0x7f4e24079770 msgr2=0x7f4e24079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.409+0000 7f4e2a926640 1 -- 172.21.15.112:0/4258673088 shutdown_connections 2024-09-21T03:30:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.409+0000 7f4e2a926640 1 -- 172.21.15.112:0/4258673088 wait complete. 2024-09-21T03:30:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.410+0000 7f4e2a926640 1 Processor -- start 2024-09-21T03:30:33.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.410+0000 7f4e2a926640 1 -- start start 2024-09-21T03:30:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e2a926640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e24102ae0 0x7f4e2419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e2a926640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e24108420 0x7f4e2419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e2a926640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e24108d70 0x7f4e2419fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e2a926640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24112a50 con 0x7f4e24102ae0 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e2a926640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241128d0 con 0x7f4e24108d70 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e2a926640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24112bd0 con 0x7f4e24108420 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e23fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e24102ae0 0x7f4e2419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e23fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e24102ae0 0x7f4e2419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49840/0 (socket says 172.21.15.112:49840) 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e23fff640 1 -- 172.21.15.112:0/1105398160 learned_addr learned my addr 172.21.15.112:0/1105398160 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e237fe640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e24108420 0x7f4e2419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e24108d70 0x7f4e2419fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e23fff640 1 -- 172.21.15.112:0/1105398160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e24108420 msgr2=0x7f4e2419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e23fff640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e24108420 0x7f4e2419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e23fff640 1 -- 172.21.15.112:0/1105398160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e24108d70 msgr2=0x7f4e2419fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e23fff640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e24108d70 0x7f4e2419fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e23fff640 1 -- 172.21.15.112:0/1105398160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e241a05a0 con 0x7f4e24102ae0 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e23fff640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e24102ae0 0x7f4e2419a630 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400b4a0 tx=0x7f4e1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:33.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.411+0000 7f4e217fa640 1 -- 172.21.15.112:0/1105398160 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14014070 con 0x7f4e24102ae0 2024-09-21T03:30:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.412+0000 7f4e217fa640 1 -- 172.21.15.112:0/1105398160 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e14010ac0 con 0x7f4e24102ae0 2024-09-21T03:30:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.412+0000 7f4e2a926640 1 -- 172.21.15.112:0/1105398160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e241a0890 con 0x7f4e24102ae0 2024-09-21T03:30:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.412+0000 7f4e217fa640 1 -- 172.21.15.112:0/1105398160 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14017c30 con 0x7f4e24102ae0 2024-09-21T03:30:33.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.412+0000 7f4e2a926640 1 -- 172.21.15.112:0/1105398160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e241a6ff0 con 0x7f4e24102ae0 2024-09-21T03:30:33.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.414+0000 7f4e30e9a740 1 Processor -- start 2024-09-21T03:30:33.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.414+0000 7f4e217fa640 1 -- 172.21.15.112:0/1105398160 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f4e14017070 con 0x7f4e24102ae0 2024-09-21T03:30:33.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.414+0000 7f4e30e9a740 1 -- start start 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.414+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317605650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.414+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573172351b0 0x557317605b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.414+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 0x5573176060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55731757be50 con 0x55731725e070 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55731756b280 con 0x5573172351b0 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55731756f090 con 0x55731723d390 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e28ec3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317605650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e28ec3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317605650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49852/0 (socket says 172.21.15.112:49852) 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e28ec3640 1 -- 172.21.15.112:0/2768403554 learned_addr learned my addr 172.21.15.112:0/2768403554 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e23fff640 1 --2- 172.21.15.112:0/2768403554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573172351b0 0x557317605b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e237fe640 1 --2- 172.21.15.112:0/2768403554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 0x5573176060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e28ec3640 1 -- 172.21.15.112:0/2768403554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 msgr2=0x5573176060d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:33.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/2768403554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 0x5573176060d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e28ec3640 1 -- 172.21.15.112:0/2768403554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573172351b0 msgr2=0x557317605b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/2768403554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573172351b0 0x557317605b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e28ec3640 1 -- 172.21.15.112:0/2768403554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573172c9650 con 0x55731725e070 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e23fff640 1 --2- 172.21.15.112:0/2768403554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573172351b0 0x557317605b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e217fa640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e0c0b6680 0x7f4e0c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/2768403554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317605650 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00ae90 tx=0x7f4e1c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7db28c5ab3992f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e217fa640 1 -- 172.21.15.112:0/1105398160 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f4e140e6c20 con 0x7f4e24102ae0 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.415+0000 7f4e0a7fc640 1 -- 172.21.15.112:0/2768403554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c003b70 con 0x55731725e070 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.416+0000 7f4e0a7fc640 1 -- 172.21.15.112:0/2768403554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e1c003d10 con 0x55731725e070 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.416+0000 7f4e0a7fc640 1 -- 172.21.15.112:0/2768403554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c010ba0 con 0x55731725e070 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.416+0000 7f4e23fff640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e0c0b6680 0x7f4e0c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.416+0000 7f4e30e9a740 1 -- 172.21.15.112:0/2768403554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 msgr2=0x557317605650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.416+0000 7f4e23fff640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e0c0b6680 0x7f4e0c0b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e140e4480 tx=0x7f4e14010f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.416+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/2768403554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317605650 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00ae90 tx=0x7f4e1c00f790 comp rx=0 tx=0).stop 2024-09-21T03:30:33.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.416+0000 7f4e30e9a740 1 -- 172.21.15.112:0/2768403554 shutdown_connections 2024-09-21T03:30:33.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.416+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/2768403554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 0x5573176060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.416+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/2768403554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573172351b0 0x557317605b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.416+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/2768403554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317605650 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.416+0000 7f4e30e9a740 1 -- 172.21.15.112:0/2768403554 >> 172.21.15.112:0/2768403554 conn(0x557317252460 msgr2=0x557317603540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:33.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.417+0000 7f4e30e9a740 1 -- 172.21.15.112:0/2768403554 shutdown_connections 2024-09-21T03:30:33.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.417+0000 7f4e30e9a740 1 -- 172.21.15.112:0/2768403554 wait complete. 2024-09-21T03:30:33.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.417+0000 7f4e30e9a740 1 -- start start 2024-09-21T03:30:33.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.417+0000 7f4e30e9a740 1 Processor -- start 2024-09-21T03:30:33.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.417+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573172351b0 0x5573176150b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.417+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 0x5573176155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317615b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5573173cc700 con 0x55731725e070 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557317506110 con 0x5573172351b0 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557317618190 con 0x55731723d390 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e30e9a740 10 client.0 _ll_register_callbacks cb 0x55731751f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e28ec3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317615b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e28ec3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317615b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49864/0 (socket says 172.21.15.112:49864) 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3881728222 learned_addr learned my addr 172.21.15.112:0/3881728222 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e237fe640 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 0x5573176155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573172351b0 0x5573176150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3881728222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 msgr2=0x5573176155f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:33.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 0x5573176155f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3881728222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573172351b0 msgr2=0x5573176150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573172351b0 0x5573176150b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3881728222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557317533c40 con 0x55731725e070 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317615b50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00ac20 tx=0x7f4e1c010780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676d6282df08f38e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e037fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e037fe640 1 -- 172.21.15.112:0/3881728222 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c003ec0 con 0x55731725e070 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e037fe640 1 -- 172.21.15.112:0/3881728222 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f4e1c004060 con 0x55731725e070 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.418+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3881728222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e1c003d10 con 0x55731725e070 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e037fe640 1 -- 172.21.15.112:0/3881728222 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4e1c004260 con 0x55731725e070 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e30e9a740 10 client.15816 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3881728222 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557317623e50 con 0x55731725e070 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e037fe640 1 -- 172.21.15.112:0/3881728222 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e1c003d10 con 0x55731725e070 2024-09-21T03:30:33.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e037fe640 1 -- 172.21.15.112:0/3881728222 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4e1c023040 con 0x55731725e070 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e30e9a740 10 client.15816 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e30e9a740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e30e9a740 20 client.15816 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55731721c9f0 0x557317627260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e30e9a740 4 client.15816 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3881728222 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55731757dda0 con 0x55731721c9f0 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.419+0000 7f4e237fe640 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55731721c9f0 0x557317627260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.420+0000 7f4e037fe640 10 client.15816 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.420+0000 7f4e237fe640 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55731721c9f0 0x557317627260 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e21222caa124f29 server_cookie=66cc559946dfbaea in_seq=0 out_seq=0 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e037fe640 1 -- 172.21.15.112:0/3881728222 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55731757dda0 con 0x55731721c9f0 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e037fe640 10 client.15816 handle_command_reply: tid=0 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 1 client.15816 shutdown 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3881728222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 msgr2=0x557317615b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317615b50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00ac20 tx=0x7f4e1c010780 comp rx=0 tx=0).stop 2024-09-21T03:30:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3881728222 shutdown_connections 2024-09-21T03:30:33.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55731721c9f0 0x557317627260 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317615b50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 0x5573176155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3881728222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573172351b0 0x5573176150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e037fe640 0 client.15816 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:33.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3881728222 >> 172.21.15.112:0/3881728222 conn(0x557317252460 msgr2=0x557317601b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:33.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3881728222 shutdown_connections 2024-09-21T03:30:33.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3881728222 wait complete. 2024-09-21T03:30:33.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.421+0000 7f4e30e9a740 20 client.15816 trim_cache size 0 max 16384 2024-09-21T03:30:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.438+0000 7f4e30e9a740 1 Processor -- start 2024-09-21T03:30:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.438+0000 7f4e30e9a740 1 -- start start 2024-09-21T03:30:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5573172351b0 0x557317603500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 0x557317616ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55731725e070 0x5573176122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241afe50 con 0x5573172351b0 2024-09-21T03:30:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241ad720 con 0x55731725e070 2024-09-21T03:30:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241ab570 con 0x55731723d390 2024-09-21T03:30:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e28ec3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5573172351b0 0x557317603500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e28ec3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5573172351b0 0x557317603500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49878/0 (socket says 172.21.15.112:49878) 2024-09-21T03:30:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3320449796 learned_addr learned my addr 172.21.15.112:0/3320449796 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3320449796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 msgr2=0x557317616ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/3320449796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 0x557317616ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3320449796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55731725e070 msgr2=0x5573176122a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/3320449796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55731725e070 0x5573176122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3320449796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557317614010 con 0x5573172351b0 2024-09-21T03:30:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/3320449796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5573172351b0 0x557317603500 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c010750 tx=0x7f4e1c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51db1586c5843a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e09ffb640 1 -- 172.21.15.112:0/3320449796 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0107f0 con 0x5573172351b0 2024-09-21T03:30:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e09ffb640 1 -- 172.21.15.112:0/3320449796 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e1c010cb0 con 0x5573172351b0 2024-09-21T03:30:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.439+0000 7f4e09ffb640 1 -- 172.21.15.112:0/3320449796 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c004a00 con 0x5573172351b0 2024-09-21T03:30:33.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.440+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3320449796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5573172351b0 msgr2=0x557317603500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:33.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.440+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3320449796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5573172351b0 0x557317603500 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c010750 tx=0x7f4e1c005b50 comp rx=0 tx=0).stop 2024-09-21T03:30:33.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.440+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3320449796 shutdown_connections 2024-09-21T03:30:33.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.440+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3320449796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55731725e070 0x5573176122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.440+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3320449796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55731723d390 0x557317616ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.440+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3320449796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5573172351b0 0x557317603500 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.440+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3320449796 >> 172.21.15.112:0/3320449796 conn(0x557317252460 msgr2=0x5573176165d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.440+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3320449796 shutdown_connections 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.440+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3320449796 wait complete. 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e30e9a740 1 -- start start 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e30e9a740 1 Processor -- start 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573172351b0 0x55731761dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55731723d390 0x557317619240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e30e9a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317619890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241abbe0 con 0x55731725e070 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241ad050 con 0x55731723d390 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e30e9a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241ac2b0 con 0x5573172351b0 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e30e9a740 10 client.0 _ll_register_callbacks cb 0x7f4e1c00ace0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e28ec3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317619890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e28ec3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317619890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49892/0 (socket says 172.21.15.112:49892) 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3494715692 learned_addr learned my addr 172.21.15.112:0/3494715692 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.441+0000 7f4e237fe640 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55731723d390 0x557317619240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573172351b0 0x55731761dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3494715692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573172351b0 msgr2=0x55731761dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573172351b0 0x55731761dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3494715692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55731723d390 msgr2=0x557317619240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55731723d390 0x557317619240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e28ec3640 1 -- 172.21.15.112:0/3494715692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55731761a000 con 0x55731725e070 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e28ec3640 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317619890 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x55731760fdd0 tx=0x7f4e1c00a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28daba61bcd5aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e037fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e037fe640 1 -- 172.21.15.112:0/3494715692 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c004e30 con 0x55731725e070 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e037fe640 1 -- 172.21.15.112:0/3494715692 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f4e1c004400 con 0x55731725e070 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3494715692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e1c010cb0 con 0x55731725e070 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e037fe640 1 -- 172.21.15.112:0/3494715692 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4e1c004680 con 0x55731725e070 2024-09-21T03:30:33.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e30e9a740 10 client.15822 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3494715692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557317691a50 con 0x55731725e070 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.442+0000 7f4e037fe640 1 -- 172.21.15.112:0/3494715692 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e1c010cb0 con 0x55731725e070 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.443+0000 7f4e037fe640 1 -- 172.21.15.112:0/3494715692 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4e1c042020 con 0x55731725e070 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.443+0000 7f4e30e9a740 10 client.15822 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.443+0000 7f4e30e9a740 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.443+0000 7f4e30e9a740 20 client.15822 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.443+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5573171f1da0 0x557317694100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.443+0000 7f4e30e9a740 4 client.15822 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.443+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3494715692 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557317579850 con 0x5573171f1da0 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.443+0000 7f4e237fe640 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5573171f1da0 0x557317694100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.443+0000 7f4e237fe640 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5573171f1da0 0x557317694100 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d05433734ec4af8 server_cookie=fc8e4df053086e37 in_seq=0 out_seq=0 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.443+0000 7f4e037fe640 10 client.15822 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e037fe640 1 -- 172.21.15.112:0/3494715692 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x557317579850 con 0x5573171f1da0 2024-09-21T03:30:33.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e037fe640 10 client.15822 handle_command_reply: tid=0 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 1 client.15822 shutdown 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3494715692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 msgr2=0x557317619890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317619890 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x55731760fdd0 tx=0x7f4e1c00a950 comp rx=0 tx=0).stop 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3494715692 shutdown_connections 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5573171f1da0 0x557317694100 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55731725e070 0x557317619890 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55731723d390 0x557317619240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 1 --2- 172.21.15.112:0/3494715692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573172351b0 0x55731761dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e037fe640 0 client.15822 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3494715692 >> 172.21.15.112:0/3494715692 conn(0x557317252460 msgr2=0x557317607000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3494715692 shutdown_connections 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 1 -- 172.21.15.112:0/3494715692 wait complete. 2024-09-21T03:30:33.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.444+0000 7f4e30e9a740 20 client.15822 trim_cache size 0 max 16384 2024-09-21T03:30:33.452 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:30:33.452 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:30:33.452 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:30:33.452 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:30:33.452 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:30:33.452 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:30:33.452 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:33.452 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:33.452 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:33.452 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:33.452 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:33.453 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:30:33.454 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:30:33.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 -- 172.21.15.112:0/1105398160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e0c0b6680 msgr2=0x7f4e0c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:33.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e0c0b6680 0x7f4e0c0b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e140e4480 tx=0x7f4e14010f90 comp rx=0 tx=0).stop 2024-09-21T03:30:33.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 -- 172.21.15.112:0/1105398160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e24102ae0 msgr2=0x7f4e2419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:33.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e24102ae0 0x7f4e2419a630 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400b4a0 tx=0x7f4e1400b960 comp rx=0 tx=0).stop 2024-09-21T03:30:33.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 -- 172.21.15.112:0/1105398160 shutdown_connections 2024-09-21T03:30:33.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e0c0b6680 0x7f4e0c0b8b90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e24108d70 0x7f4e2419fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e24108420 0x7f4e2419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 --2- 172.21.15.112:0/1105398160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e24102ae0 0x7f4e2419a630 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:33.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 -- 172.21.15.112:0/1105398160 >> 172.21.15.112:0/1105398160 conn(0x7f4e24079770 msgr2=0x7f4e24101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:33.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 -- 172.21.15.112:0/1105398160 shutdown_connections 2024-09-21T03:30:33.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:33.445+0000 7f4e2a926640 1 -- 172.21.15.112:0/1105398160 wait complete. 2024-09-21T03:30:34.212 INFO:tasks.workunit.client.0.smithi112.stdout: 1 60611 10.17 MB/sec warmup 34 sec latency 4941.678 ms 2024-09-21T03:30:34.355 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:30:34.355 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:30:34.619 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:34.740 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:34 smithi112 ceph-mon[31442]: pgmap v365: 97 pgs: 97 active+clean; 49 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 5.7 MiB/s wr, 171 op/s 2024-09-21T03:30:34.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:34 smithi203 ceph-mon[34723]: pgmap v365: 97 pgs: 97 active+clean; 49 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 5.7 MiB/s wr, 171 op/s 2024-09-21T03:30:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:34 smithi187 ceph-mon[34603]: pgmap v365: 97 pgs: 97 active+clean; 49 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 5.7 MiB/s wr, 171 op/s 2024-09-21T03:30:35.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.133+0000 7fbcca1c8640 1 -- 172.21.15.112:0/2345612340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc4108d10 msgr2=0x7fbcc410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:35.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.133+0000 7fbcca1c8640 1 --2- 172.21.15.112:0/2345612340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc4108d10 0x7fbcc410f540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009ca0 tx=0x7fbcbc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.133+0000 7fbcca1c8640 1 -- 172.21.15.112:0/2345612340 shutdown_connections 2024-09-21T03:30:35.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.133+0000 7fbcca1c8640 1 --2- 172.21.15.112:0/2345612340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc4108d10 0x7fbcc410f540 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.133+0000 7fbcca1c8640 1 --2- 172.21.15.112:0/2345612340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc41083c0 0x7fbcc41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.133+0000 7fbcca1c8640 1 --2- 172.21.15.112:0/2345612340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcc4102a80 0x7fbcc4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.133+0000 7fbcca1c8640 1 -- 172.21.15.112:0/2345612340 >> 172.21.15.112:0/2345612340 conn(0x7fbcc4079740 msgr2=0x7fbcc4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.133+0000 7fbcca1c8640 1 -- 172.21.15.112:0/2345612340 shutdown_connections 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.133+0000 7fbcca1c8640 1 -- 172.21.15.112:0/2345612340 wait complete. 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.134+0000 7fbcca1c8640 1 Processor -- start 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcca1c8640 1 -- start start 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcca1c8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcc4102a80 0x7fbcc406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcca1c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc41083c0 0x7fbcc406c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcca1c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc4108d10 0x7fbcc419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcca1c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4112980 con 0x7fbcc41083c0 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcca1c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4112800 con 0x7fbcc4108d10 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcca1c8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4112b00 con 0x7fbcc4102a80 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcc2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc41083c0 0x7fbcc406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcc37fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcc4102a80 0x7fbcc406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcc3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc4108d10 0x7fbcc419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:35.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcc37fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcc4102a80 0x7fbcc406bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:56674/0 (socket says 172.21.15.112:56674) 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcc37fe640 1 -- 172.21.15.112:0/3433154805 learned_addr learned my addr 172.21.15.112:0/3433154805 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcc3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc4108d10 0x7fbcc419ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:46150/0 (socket says 172.21.15.112:46150) 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcc37fe640 1 -- 172.21.15.112:0/3433154805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc4108d10 msgr2=0x7fbcc419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcc37fe640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc4108d10 0x7fbcc419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcc37fe640 1 -- 172.21.15.112:0/3433154805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc41083c0 msgr2=0x7fbcc406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcc37fe640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc41083c0 0x7fbcc406c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.135+0000 7fbcc37fe640 1 -- 172.21.15.112:0/3433154805 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc419d140 con 0x7fbcc4102a80 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.136+0000 7fbcc37fe640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcc4102a80 0x7fbcc406bad0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400ea60 tx=0x7fbcb400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.136+0000 7fbcc0ff9640 1 -- 172.21.15.112:0/3433154805 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400cde0 con 0x7fbcc4102a80 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.136+0000 7fbcca1c8640 1 -- 172.21.15.112:0/3433154805 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc419d430 con 0x7fbcc4102a80 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.136+0000 7fbcc0ff9640 1 -- 172.21.15.112:0/3433154805 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcb40035f0 con 0x7fbcc4102a80 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.136+0000 7fbcc0ff9640 1 -- 172.21.15.112:0/3433154805 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4010690 con 0x7fbcc4102a80 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.136+0000 7fbcca1c8640 1 -- 172.21.15.112:0/3433154805 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc41a6dd0 con 0x7fbcc4102a80 2024-09-21T03:30:35.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.137+0000 7fbcca1c8640 1 -- 172.21.15.112:0/3433154805 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc4102e90 con 0x7fbcc4102a80 2024-09-21T03:30:35.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.138+0000 7fbcc0ff9640 1 -- 172.21.15.112:0/3433154805 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fbcb40037a0 con 0x7fbcc4102a80 2024-09-21T03:30:35.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.139+0000 7fbcc0ff9640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbcac0b6680 0x7fbcac0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:35.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.139+0000 7fbcc2ffd640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbcac0b6680 0x7fbcac0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:35.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.139+0000 7fbcc0ff9640 1 -- 172.21.15.112:0/3433154805 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fbcb40e7bc0 con 0x7fbcc4102a80 2024-09-21T03:30:35.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.140+0000 7fbcc2ffd640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbcac0b6680 0x7fbcac0b8b90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4105f80 tx=0x7fbcb8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:35.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.143+0000 7fbcc0ff9640 1 -- 172.21.15.112:0/3433154805 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbcb4056710 con 0x7fbcc4102a80 2024-09-21T03:30:35.212 INFO:tasks.workunit.client.0.smithi112.stdout: 1 62732 10.09 MB/sec warmup 35 sec latency 31.587 ms 2024-09-21T03:30:35.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.379+0000 7fbcca1c8640 1 -- 172.21.15.112:0/3433154805 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcc419b560 con 0x7fbcc4102a80 2024-09-21T03:30:35.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.380+0000 7fbcc0ff9640 1 -- 172.21.15.112:0/3433154805 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11932 (secure 0 0 0) 0x7fbcb40ae110 con 0x7fbcc4102a80 2024-09-21T03:30:35.382 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:30:35.382 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"btime":"2024-09-21T03:30:19:484252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:30:19.046905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:30:35.382 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 -- 172.21.15.112:0/3433154805 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbcac0b6680 msgr2=0x7fbcac0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbcac0b6680 0x7fbcac0b8b90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4105f80 tx=0x7fbcb8005f90 comp rx=0 tx=0).stop 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 -- 172.21.15.112:0/3433154805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcc4102a80 msgr2=0x7fbcc406bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcc4102a80 0x7fbcc406bad0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400ea60 tx=0x7fbcb400ef20 comp rx=0 tx=0).stop 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 -- 172.21.15.112:0/3433154805 shutdown_connections 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbcac0b6680 0x7fbcac0b8b90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc4108d10 0x7fbcc419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc41083c0 0x7fbcc406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 --2- 172.21.15.112:0/3433154805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcc4102a80 0x7fbcc406bad0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 -- 172.21.15.112:0/3433154805 >> 172.21.15.112:0/3433154805 conn(0x7fbcc4079740 msgr2=0x7fbcc41043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 -- 172.21.15.112:0/3433154805 shutdown_connections 2024-09-21T03:30:35.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:35.383+0000 7fbcca1c8640 1 -- 172.21.15.112:0/3433154805 wait complete. 2024-09-21T03:30:35.675 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:35 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3433154805' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:35.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:35 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3433154805' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:35.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:35 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3433154805' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:30:35.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:30:36.212 INFO:tasks.workunit.client.0.smithi112.stdout: 1 64660 10.15 MB/sec warmup 36 sec latency 12.717 ms 2024-09-21T03:30:36.246 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:36.655 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:36 smithi112 ceph-mon[31442]: pgmap v366: 97 pgs: 97 active+clean; 51 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 5.9 MiB/s wr, 187 op/s 2024-09-21T03:30:36.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.751+0000 7f32b1cfe640 1 -- 172.21.15.112:0/4291205262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ac105220 msgr2=0x7f32ac105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:36.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.751+0000 7f32b1cfe640 1 --2- 172.21.15.112:0/4291205262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ac105220 0x7f32ac105630 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009900 tx=0x7f32a002a410 comp rx=0 tx=0).stop 2024-09-21T03:30:36.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.751+0000 7f32b1cfe640 1 -- 172.21.15.112:0/4291205262 shutdown_connections 2024-09-21T03:30:36.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.751+0000 7f32b1cfe640 1 --2- 172.21.15.112:0/4291205262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ac105b70 0x7f32ac10f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.751+0000 7f32b1cfe640 1 --2- 172.21.15.112:0/4291205262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ac105220 0x7f32ac105630 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.751+0000 7f32b1cfe640 1 --2- 172.21.15.112:0/4291205262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ac103210 0x7f32ac103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.751+0000 7f32b1cfe640 1 -- 172.21.15.112:0/4291205262 >> 172.21.15.112:0/4291205262 conn(0x7f32ac079ec0 msgr2=0x7f32ac07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:36.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.751+0000 7f32b1cfe640 1 -- 172.21.15.112:0/4291205262 shutdown_connections 2024-09-21T03:30:36.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.751+0000 7f32b1cfe640 1 -- 172.21.15.112:0/4291205262 wait complete. 2024-09-21T03:30:36.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32b1cfe640 1 Processor -- start 2024-09-21T03:30:36.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32b1cfe640 1 -- start start 2024-09-21T03:30:36.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32b1cfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ac103210 0x7f32ac19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32b1cfe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ac105220 0x7f32ac19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32b1cfe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ac105b70 0x7f32ac19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32b1cfe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac112a70 con 0x7f32ac103210 2024-09-21T03:30:36.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32b1cfe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac1128f0 con 0x7f32ac105b70 2024-09-21T03:30:36.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32b1cfe640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac112bf0 con 0x7f32ac105220 2024-09-21T03:30:36.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32ab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ac103210 0x7f32ac19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32ab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ac103210 0x7f32ac19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44636/0 (socket says 172.21.15.112:44636) 2024-09-21T03:30:36.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32ab7fe640 1 -- 172.21.15.112:0/281068262 learned_addr learned my addr 172.21.15.112:0/281068262 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32abfff640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ac105b70 0x7f32ac19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.752+0000 7f32aaffd640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ac105220 0x7f32ac19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.753+0000 7f32ab7fe640 1 -- 172.21.15.112:0/281068262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ac105220 msgr2=0x7f32ac19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.753+0000 7f32ab7fe640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ac105220 0x7f32ac19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.753+0000 7f32ab7fe640 1 -- 172.21.15.112:0/281068262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ac105b70 msgr2=0x7f32ac19fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.753+0000 7f32ab7fe640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ac105b70 0x7f32ac19fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.753+0000 7f32ab7fe640 1 -- 172.21.15.112:0/281068262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac1a0500 con 0x7f32ac103210 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.753+0000 7f32ab7fe640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ac103210 0x7f32ac19a590 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e9f0 tx=0x7f329c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.753+0000 7f32a8ff9640 1 -- 172.21.15.112:0/281068262 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c00ccc0 con 0x7f32ac103210 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.753+0000 7f32a8ff9640 1 -- 172.21.15.112:0/281068262 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f329c00ce60 con 0x7f32ac103210 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.753+0000 7f32b1cfe640 1 -- 172.21.15.112:0/281068262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac1a07f0 con 0x7f32ac103210 2024-09-21T03:30:36.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.753+0000 7f32a8ff9640 1 -- 172.21.15.112:0/281068262 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c010430 con 0x7f32ac103210 2024-09-21T03:30:36.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.753+0000 7f32b1cfe640 1 -- 172.21.15.112:0/281068262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac1a6f50 con 0x7f32ac103210 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32b8272740 1 Processor -- start 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32b8272740 1 -- start start 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32a8ff9640 1 -- 172.21.15.112:0/281068262 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f329c0046c0 con 0x7f32ac103210 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32b8272740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab98070 0x56266af3f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32b8272740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab6f1b0 0x56266af3f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32b8272740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af3fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32b8272740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56266aeb5ea0 con 0x56266ab98070 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32b8272740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56266aea52d0 con 0x56266ab77390 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32b8272740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56266aea90e0 con 0x56266ab6f1b0 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab98070 0x56266af3f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab98070 0x56266af3f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44640/0 (socket says 172.21.15.112:44640) 2024-09-21T03:30:36.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.756+0000 7f32abfff640 1 -- 172.21.15.112:0/1504346500 learned_addr learned my addr 172.21.15.112:0/1504346500 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f32ab7fe640 1 --2- 172.21.15.112:0/1504346500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab6f1b0 0x56266af3f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f32abfff640 1 -- 172.21.15.112:0/1504346500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab6f1b0 msgr2=0x56266af3f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f32abfff640 1 --2- 172.21.15.112:0/1504346500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab6f1b0 0x56266af3f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f32abfff640 1 -- 172.21.15.112:0/1504346500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 msgr2=0x56266af3fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f32abfff640 1 --2- 172.21.15.112:0/1504346500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af3fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f32abfff640 1 -- 172.21.15.112:0/1504346500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56266ac03650 con 0x56266ab98070 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f32ab7fe640 1 --2- 172.21.15.112:0/1504346500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab6f1b0 0x56266af3f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f32abfff640 1 --2- 172.21.15.112:0/1504346500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab98070 0x56266af3f2c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f32a4011440 tx=0x7f32a4011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71803da5ee9f80a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f32a8ff9640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32940b6680 0x7f32940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f3291ffb640 1 -- 172.21.15.112:0/1504346500 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4003c30 con 0x56266ab98070 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f32ab7fe640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32940b6680 0x7f32940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f3291ffb640 1 -- 172.21.15.112:0/1504346500 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a4003dd0 con 0x56266ab98070 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f3291ffb640 1 -- 172.21.15.112:0/1504346500 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a400fa40 con 0x56266ab98070 2024-09-21T03:30:36.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.757+0000 7f32a8ff9640 1 -- 172.21.15.112:0/281068262 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f329c0e7ad0 con 0x7f32ac103210 2024-09-21T03:30:36.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.758+0000 7f32b8272740 1 -- 172.21.15.112:0/1504346500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab98070 msgr2=0x56266af3f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:36.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.758+0000 7f32ab7fe640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32940b6680 0x7f32940b8b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f329c0e5330 tx=0x7f329c0a9820 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:36.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.758+0000 7f32b8272740 1 --2- 172.21.15.112:0/1504346500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab98070 0x56266af3f2c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f32a4011440 tx=0x7f32a4011900 comp rx=0 tx=0).stop 2024-09-21T03:30:36.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.758+0000 7f32b8272740 1 -- 172.21.15.112:0/1504346500 shutdown_connections 2024-09-21T03:30:36.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.758+0000 7f32b8272740 1 --2- 172.21.15.112:0/1504346500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af3fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.758+0000 7f32b8272740 1 --2- 172.21.15.112:0/1504346500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab6f1b0 0x56266af3f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.758+0000 7f32b8272740 1 --2- 172.21.15.112:0/1504346500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab98070 0x56266af3f2c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.758+0000 7f32b8272740 1 -- 172.21.15.112:0/1504346500 >> 172.21.15.112:0/1504346500 conn(0x56266ab8c460 msgr2=0x56266af3d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:36.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.758+0000 7f32b8272740 1 -- 172.21.15.112:0/1504346500 shutdown_connections 2024-09-21T03:30:36.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.758+0000 7f32b8272740 1 -- 172.21.15.112:0/1504346500 wait complete. 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32b8272740 1 -- start start 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32b8272740 1 Processor -- start 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32b8272740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 0x56266af4a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32b8272740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af4aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32b8272740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266af56d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32b8272740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56266ad06700 con 0x56266ab6f1b0 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32b8272740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56266ae40160 con 0x56266ab77390 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32b8272740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56266af4daa0 con 0x56266ab98070 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32b8272740 10 client.0 _ll_register_callbacks cb 0x56266ae599e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 0x56266af4a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 0x56266af4a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44644/0 (socket says 172.21.15.112:44644) 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32aaffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af4aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.759+0000 7f32abfff640 1 -- 172.21.15.112:0/578569322 learned_addr learned my addr 172.21.15.112:0/578569322 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:36.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.760+0000 7f32abfff640 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266af56d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.760+0000 7f32aaffd640 1 -- 172.21.15.112:0/578569322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 msgr2=0x56266af56d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.760+0000 7f32aaffd640 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266af56d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.760+0000 7f32aaffd640 1 -- 172.21.15.112:0/578569322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 msgr2=0x56266af4a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.760+0000 7f32aaffd640 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 0x56266af4a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.760+0000 7f32aaffd640 1 -- 172.21.15.112:0/578569322 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56266ae6dc90 con 0x56266ab77390 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.760+0000 7f32aaffd640 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af4aef0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f32a002a930 tx=0x7f32a0032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432ab740462003fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.760+0000 7f328affd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.760+0000 7f328affd640 1 -- 172.21.15.112:0/578569322 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0040070 con 0x56266ab77390 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.760+0000 7f32b8272740 1 -- 172.21.15.112:0/578569322 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a4003dd0 con 0x56266ab77390 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.760+0000 7f328affd640 1 -- 172.21.15.112:0/578569322 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f32a00318b0 con 0x56266ab77390 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f328affd640 1 -- 172.21.15.112:0/578569322 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32a0035040 con 0x56266ab77390 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f32b8272740 10 client.25412 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f32b8272740 1 -- 172.21.15.112:0/578569322 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56266af597a0 con 0x56266ab77390 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f328affd640 1 -- 172.21.15.112:0/578569322 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a4003dd0 con 0x56266ab77390 2024-09-21T03:30:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f328affd640 1 -- 172.21.15.112:0/578569322 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f32a0004750 con 0x56266ab77390 2024-09-21T03:30:36.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f32b8272740 10 client.25412 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:36.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f32b8272740 10 client.25412 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:30:36.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f32b8272740 20 client.25412 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:36.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f32b8272740 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56266ab569f0 0x56266af5cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f32b8272740 4 client.25412 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:36.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f32b8272740 1 -- 172.21.15.112:0/578569322 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56266aeb7df0 con 0x56266ab569f0 2024-09-21T03:30:36.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.761+0000 7f32abfff640 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56266ab569f0 0x56266af5cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.762+0000 7f328affd640 10 client.25412 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:30:36.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.762+0000 7f32abfff640 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56266ab569f0 0x56266af5cbc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eeed5f5d886e30be server_cookie=13e91cf60b34972f in_seq=0 out_seq=0 2024-09-21T03:30:36.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f328affd640 1 -- 172.21.15.112:0/578569322 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56266aeb7df0 con 0x56266ab569f0 2024-09-21T03:30:36.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f328affd640 10 client.25412 handle_command_reply: tid=0 2024-09-21T03:30:36.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 1 client.25412 shutdown 2024-09-21T03:30:36.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 1 -- 172.21.15.112:0/578569322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 msgr2=0x56266af4aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:36.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af4aef0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f32a002a930 tx=0x7f32a0032d50 comp rx=0 tx=0).stop 2024-09-21T03:30:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 1 -- 172.21.15.112:0/578569322 shutdown_connections 2024-09-21T03:30:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56266ab569f0 0x56266af5cbc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266af56d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af4aef0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f328affd640 0 client.25412 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:30:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 1 --2- 172.21.15.112:0/578569322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 0x56266af4a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 1 -- 172.21.15.112:0/578569322 >> 172.21.15.112:0/578569322 conn(0x56266ab8c460 msgr2=0x56266af424a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 1 -- 172.21.15.112:0/578569322 shutdown_connections 2024-09-21T03:30:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 1 -- 172.21.15.112:0/578569322 wait complete. 2024-09-21T03:30:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.763+0000 7f32b8272740 20 client.25412 trim_cache size 0 max 16384 2024-09-21T03:30:36.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32b8272740 1 Processor -- start 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32b8272740 1 -- start start 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32b8272740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab6f1b0 0x56266af56c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32b8272740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab77390 0x56266af47ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32b8272740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266af480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32b8272740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac1b1f70 con 0x56266ab77390 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32b8272740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac1af840 con 0x56266ab6f1b0 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32b8272740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac1ad690 con 0x56266ab98070 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab77390 0x56266af47ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32aaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab77390 0x56266af47ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44646/0 (socket says 172.21.15.112:44646) 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32aaffd640 1 -- 172.21.15.112:0/3019187429 learned_addr learned my addr 172.21.15.112:0/3019187429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32abfff640 1 --2- 172.21.15.112:0/3019187429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab6f1b0 0x56266af56c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32abfff640 1 --2- 172.21.15.112:0/3019187429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266af480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32aaffd640 1 -- 172.21.15.112:0/3019187429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 msgr2=0x56266af480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:36.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.777+0000 7f32aaffd640 1 --2- 172.21.15.112:0/3019187429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266af480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.778+0000 7f32aaffd640 1 -- 172.21.15.112:0/3019187429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab6f1b0 msgr2=0x56266af56c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:36.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.778+0000 7f32aaffd640 1 --2- 172.21.15.112:0/3019187429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab6f1b0 0x56266af56c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.778+0000 7f32aaffd640 1 -- 172.21.15.112:0/3019187429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56266af417e0 con 0x56266ab77390 2024-09-21T03:30:36.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.778+0000 7f32aaffd640 1 --2- 172.21.15.112:0/3019187429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab77390 0x56266af47ba0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f32a0031440 tx=0x7f32a002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abae506f538f0c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:36.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.778+0000 7f32917fa640 1 -- 172.21.15.112:0/3019187429 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0040070 con 0x56266ab77390 2024-09-21T03:30:36.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.778+0000 7f32917fa640 1 -- 172.21.15.112:0/3019187429 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a0004750 con 0x56266ab77390 2024-09-21T03:30:36.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.778+0000 7f32917fa640 1 -- 172.21.15.112:0/3019187429 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00052a0 con 0x56266ab77390 2024-09-21T03:30:36.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.779+0000 7f32b8272740 1 -- 172.21.15.112:0/3019187429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab77390 msgr2=0x56266af47ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:36.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.779+0000 7f32b8272740 1 --2- 172.21.15.112:0/3019187429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab77390 0x56266af47ba0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f32a0031440 tx=0x7f32a002a410 comp rx=0 tx=0).stop 2024-09-21T03:30:36.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.779+0000 7f32b8272740 1 -- 172.21.15.112:0/3019187429 shutdown_connections 2024-09-21T03:30:36.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.779+0000 7f32b8272740 1 --2- 172.21.15.112:0/3019187429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266af480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.779+0000 7f32b8272740 1 --2- 172.21.15.112:0/3019187429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab77390 0x56266af47ba0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.779+0000 7f32b8272740 1 --2- 172.21.15.112:0/3019187429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab6f1b0 0x56266af56c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.779+0000 7f32b8272740 1 -- 172.21.15.112:0/3019187429 >> 172.21.15.112:0/3019187429 conn(0x56266ab8c460 msgr2=0x56266af49230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.779+0000 7f32b8272740 1 -- 172.21.15.112:0/3019187429 shutdown_connections 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.779+0000 7f32b8272740 1 -- 172.21.15.112:0/3019187429 wait complete. 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.779+0000 7f32b8272740 1 -- start start 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.779+0000 7f32b8272740 1 Processor -- start 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32b8272740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 0x56266af4f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32b8272740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af4f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32b8272740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266afca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32b8272740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac1add00 con 0x56266ab6f1b0 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32b8272740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac1af170 con 0x56266ab77390 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32b8272740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac1ae3d0 con 0x56266ab98070 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32b8272740 10 client.0 _ll_register_callbacks cb 0x7f329c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:36.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 0x56266af4f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 0x56266af4f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44650/0 (socket says 172.21.15.112:44650) 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32abfff640 1 -- 172.21.15.112:0/3843830471 learned_addr learned my addr 172.21.15.112:0/3843830471 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32aaffd640 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af4f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32abfff640 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266afca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32aaffd640 1 -- 172.21.15.112:0/3843830471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 msgr2=0x56266afca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32aaffd640 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266afca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32aaffd640 1 -- 172.21.15.112:0/3843830471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 msgr2=0x56266af4f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32aaffd640 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 0x56266af4f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.780+0000 7f32aaffd640 1 -- 172.21.15.112:0/3843830471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56266afcaed0 con 0x56266ab77390 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f32aaffd640 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af4f9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f32a0004b50 tx=0x7f32a0004d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0a5b364dab54c95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f328affd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f328affd640 1 -- 172.21.15.112:0/3843830471 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0040070 con 0x56266ab77390 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f328affd640 1 -- 172.21.15.112:0/3843830471 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f32a0004050 con 0x56266ab77390 2024-09-21T03:30:36.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f32b8272740 1 -- 172.21.15.112:0/3843830471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a0004750 con 0x56266ab77390 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f328affd640 1 -- 172.21.15.112:0/3843830471 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32a0068050 con 0x56266ab77390 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f32b8272740 10 client.25418 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f32b8272740 1 -- 172.21.15.112:0/3843830471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56266afcc150 con 0x56266ab77390 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f328affd640 1 -- 172.21.15.112:0/3843830471 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a005c020 con 0x56266ab77390 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f328affd640 1 -- 172.21.15.112:0/3843830471 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f32a0031740 con 0x56266ab77390 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f32b8272740 10 client.25418 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f32b8272740 10 client.25418 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f32b8272740 20 client.25418 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.781+0000 7f32b8272740 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56266ab2bda0 0x56266afce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.782+0000 7f32b8272740 4 client.25418 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.782+0000 7f32b8272740 1 -- 172.21.15.112:0/3843830471 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56266aeb38a0 con 0x56266ab2bda0 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.782+0000 7f32abfff640 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56266ab2bda0 0x56266afce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:36.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.782+0000 7f328affd640 10 client.25418 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.782+0000 7f32abfff640 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56266ab2bda0 0x56266afce810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5de2ae6c22c957c4 server_cookie=f35012ce4ec8e6e8 in_seq=0 out_seq=0 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.782+0000 7f328affd640 1 -- 172.21.15.112:0/3843830471 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56266aeb38a0 con 0x56266ab2bda0 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.782+0000 7f328affd640 10 client.25418 handle_command_reply: tid=0 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.782+0000 7f32b8272740 1 client.25418 shutdown 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b8272740 1 -- 172.21.15.112:0/3843830471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 msgr2=0x56266af4f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b8272740 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af4f9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f32a0004b50 tx=0x7f32a0004d40 comp rx=0 tx=0).stop 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b8272740 1 -- 172.21.15.112:0/3843830471 shutdown_connections 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b8272740 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56266ab2bda0 0x56266afce810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b8272740 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56266ab98070 0x56266afca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b8272740 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56266ab77390 0x56266af4f9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f328affd640 0 client.25418 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b8272740 1 --2- 172.21.15.112:0/3843830471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56266ab6f1b0 0x56266af4f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b8272740 1 -- 172.21.15.112:0/3843830471 >> 172.21.15.112:0/3843830471 conn(0x56266ab8c460 msgr2=0x56266af402f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:36.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b8272740 1 -- 172.21.15.112:0/3843830471 shutdown_connections 2024-09-21T03:30:36.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b8272740 1 -- 172.21.15.112:0/3843830471 wait complete. 2024-09-21T03:30:36.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b8272740 20 client.25418 trim_cache size 0 max 16384 2024-09-21T03:30:36.790 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:30:36.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b1cfe640 1 -- 172.21.15.112:0/281068262 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32940b6680 msgr2=0x7f32940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b1cfe640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32940b6680 0x7f32940b8b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f329c0e5330 tx=0x7f329c0a9820 comp rx=0 tx=0).stop 2024-09-21T03:30:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b1cfe640 1 -- 172.21.15.112:0/281068262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ac103210 msgr2=0x7f32ac19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b1cfe640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ac103210 0x7f32ac19a590 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e9f0 tx=0x7f329c00edf0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b1cfe640 1 -- 172.21.15.112:0/281068262 shutdown_connections 2024-09-21T03:30:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b1cfe640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f32940b6680 0x7f32940b8b90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b1cfe640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32ac105b70 0x7f32ac19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b1cfe640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32ac105220 0x7f32ac19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b1cfe640 1 --2- 172.21.15.112:0/281068262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32ac103210 0x7f32ac19a590 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.783+0000 7f32b1cfe640 1 -- 172.21.15.112:0/281068262 >> 172.21.15.112:0/281068262 conn(0x7f32ac079ec0 msgr2=0x7f32ac101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.784+0000 7f32b1cfe640 1 -- 172.21.15.112:0/281068262 shutdown_connections 2024-09-21T03:30:36.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:36.784+0000 7f32b1cfe640 1 -- 172.21.15.112:0/281068262 wait complete. 2024-09-21T03:30:36.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:36 smithi203 ceph-mon[34723]: pgmap v366: 97 pgs: 97 active+clean; 51 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 5.9 MiB/s wr, 187 op/s 2024-09-21T03:30:36.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:36 smithi187 ceph-mon[34603]: pgmap v366: 97 pgs: 97 active+clean; 51 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 5.9 MiB/s wr, 187 op/s 2024-09-21T03:30:37.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 67646 10.38 MB/sec warmup 37 sec latency 37.340 ms 2024-09-21T03:30:37.547 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:30:37.806 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:38.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 69312 10.26 MB/sec warmup 38 sec latency 11.549 ms 2024-09-21T03:30:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.283+0000 7fbe4870a640 1 -- 172.21.15.112:0/679573350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40102ae0 msgr2=0x7fbe40102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.283+0000 7fbe4870a640 1 --2- 172.21.15.112:0/679573350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40102ae0 0x7fbe40102ef0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009900 tx=0x7fbe3802a410 comp rx=0 tx=0).stop 2024-09-21T03:30:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.283+0000 7fbe4870a640 1 -- 172.21.15.112:0/679573350 shutdown_connections 2024-09-21T03:30:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.283+0000 7fbe4870a640 1 --2- 172.21.15.112:0/679573350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe40108d70 0x7fbe4010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.283+0000 7fbe4870a640 1 --2- 172.21.15.112:0/679573350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe40108420 0x7fbe40108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.283+0000 7fbe4870a640 1 --2- 172.21.15.112:0/679573350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40102ae0 0x7fbe40102ef0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.283+0000 7fbe4870a640 1 -- 172.21.15.112:0/679573350 >> 172.21.15.112:0/679573350 conn(0x7fbe400796b0 msgr2=0x7fbe40079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.283+0000 7fbe4870a640 1 -- 172.21.15.112:0/679573350 shutdown_connections 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.283+0000 7fbe4870a640 1 -- 172.21.15.112:0/679573350 wait complete. 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe4870a640 1 Processor -- start 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe4870a640 1 -- start start 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe4870a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe40102ae0 0x7fbe4019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe4870a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40108420 0x7fbe4019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe4870a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe40108d70 0x7fbe4019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe4870a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40113b50 con 0x7fbe40108420 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe4870a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401139d0 con 0x7fbe40102ae0 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe4870a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40113cd0 con 0x7fbe40108d70 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe464a6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe40102ae0 0x7fbe4019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe45ca5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40108420 0x7fbe4019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe464a6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe40102ae0 0x7fbe4019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42044/0 (socket says 172.21.15.112:42044) 2024-09-21T03:30:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe464a6640 1 -- 172.21.15.112:0/2800064156 learned_addr learned my addr 172.21.15.112:0/2800064156 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.284+0000 7fbe46ca7640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe40108d70 0x7fbe4019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe45ca5640 1 -- 172.21.15.112:0/2800064156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe40108d70 msgr2=0x7fbe4019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe45ca5640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe40108d70 0x7fbe4019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe45ca5640 1 -- 172.21.15.112:0/2800064156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe40102ae0 msgr2=0x7fbe4019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe45ca5640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe40102ae0 0x7fbe4019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe45ca5640 1 -- 172.21.15.112:0/2800064156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe401a0450 con 0x7fbe40108420 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe464a6640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe40102ae0 0x7fbe4019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe45ca5640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40108420 0x7fbe4019aa80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbe4006b670 tx=0x7fbe3400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe3f7fe640 1 -- 172.21.15.112:0/2800064156 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34017070 con 0x7fbe40108420 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe4870a640 1 -- 172.21.15.112:0/2800064156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe401a0740 con 0x7fbe40108420 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe3f7fe640 1 -- 172.21.15.112:0/2800064156 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe340099a0 con 0x7fbe40108420 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe3f7fe640 1 -- 172.21.15.112:0/2800064156 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe340047a0 con 0x7fbe40108420 2024-09-21T03:30:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.285+0000 7fbe4870a640 1 -- 172.21.15.112:0/2800064156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe401a6ea0 con 0x7fbe40108420 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.288+0000 7fbe4ec7e740 1 Processor -- start 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.288+0000 7fbe4ec7e740 1 -- start start 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.288+0000 7fbe3f7fe640 1 -- 172.21.15.112:0/2800064156 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fbe34004940 con 0x7fbe40108420 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.288+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.288+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dc91b0 0x55a515199800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.288+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dd1390 0x55a515199d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.288+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a51510fea0 con 0x55a514df2070 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.288+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a5150ff2d0 con 0x55a514dc91b0 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.288+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a5151030e0 con 0x55a514dd1390 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.288+0000 7fbe46ca7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe46ca7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44662/0 (socket says 172.21.15.112:44662) 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe46ca7640 1 -- 172.21.15.112:0/3021160840 learned_addr learned my addr 172.21.15.112:0/3021160840 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe464a6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dc91b0 0x55a515199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe45ca5640 1 --2- 172.21.15.112:0/3021160840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dd1390 0x55a515199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe464a6640 1 -- 172.21.15.112:0/3021160840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dd1390 msgr2=0x55a515199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe464a6640 1 --2- 172.21.15.112:0/3021160840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dd1390 0x55a515199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe464a6640 1 -- 172.21.15.112:0/3021160840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 msgr2=0x55a5151992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe464a6640 1 --2- 172.21.15.112:0/3021160840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe464a6640 1 -- 172.21.15.112:0/3021160840 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a514e5d650 con 0x55a514dc91b0 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe46ca7640 1 --2- 172.21.15.112:0/3021160840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe464a6640 1 --2- 172.21.15.112:0/3021160840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dc91b0 0x55a515199800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe38004770 tx=0x7fbe380314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e01da879f5f30ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe3f7fe640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbe280b6680 0x7fbe280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe46ca7640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbe280b6680 0x7fbe280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe3cff9640 1 -- 172.21.15.112:0/3021160840 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38035040 con 0x55a514dc91b0 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe3cff9640 1 -- 172.21.15.112:0/3021160840 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe38031890 con 0x55a514dc91b0 2024-09-21T03:30:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe3cff9640 1 -- 172.21.15.112:0/3021160840 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe380049c0 con 0x55a514dc91b0 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.289+0000 7fbe3f7fe640 1 -- 172.21.15.112:0/2800064156 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fbe340e72a0 con 0x7fbe40108420 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.290+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/3021160840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dc91b0 msgr2=0x55a515199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.290+0000 7fbe46ca7640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbe280b6680 0x7fbe280b8b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbe30011890 tx=0x7fbe30009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.290+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/3021160840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dc91b0 0x55a515199800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe38004770 tx=0x7fbe380314a0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.290+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/3021160840 shutdown_connections 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.290+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/3021160840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dd1390 0x55a515199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.290+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/3021160840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dc91b0 0x55a515199800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.290+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/3021160840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.290+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/3021160840 >> 172.21.15.112:0/3021160840 conn(0x55a514de6460 msgr2=0x55a5151971b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/3021160840 shutdown_connections 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/3021160840 wait complete. 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe4ec7e740 1 -- start start 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe4ec7e740 1 Processor -- start 2024-09-21T03:30:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dc91b0 0x55a5151a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dd1390 0x55a5151a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a514f60700 con 0x55a514df2070 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a51509a160 con 0x55a514dd1390 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a5151a7a40 con 0x55a514dc91b0 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe4ec7e740 10 client.0 _ll_register_callbacks cb 0x55a5150b39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe46ca7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe46ca7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151a5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44666/0 (socket says 172.21.15.112:44666) 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe464a6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dc91b0 0x55a5151a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe46ca7640 1 -- 172.21.15.112:0/2862043203 learned_addr learned my addr 172.21.15.112:0/2862043203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.291+0000 7fbe45ca5640 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dd1390 0x55a5151a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe464a6640 1 -- 172.21.15.112:0/2862043203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dd1390 msgr2=0x55a5151a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe464a6640 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dd1390 0x55a5151a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe464a6640 1 -- 172.21.15.112:0/2862043203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 msgr2=0x55a5151a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe464a6640 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe464a6640 1 -- 172.21.15.112:0/2862043203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5150c7c90 con 0x55a514dc91b0 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe46ca7640 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe464a6640 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dc91b0 0x55a5151a48b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbe38005940 tx=0x7fbe38009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96b65f2ae36dae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2862043203 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3804e070 con 0x55a514dc91b0 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2862043203 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fbe380035f0 con 0x55a514dc91b0 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2862043203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe38031890 con 0x55a514dc91b0 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.292+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2862043203 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe3805a050 con 0x55a514dc91b0 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.293+0000 7fbe4ec7e740 10 client.25426 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.293+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2862043203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a5151b3690 con 0x55a514dc91b0 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.293+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2862043203 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe38031890 con 0x55a514dc91b0 2024-09-21T03:30:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.293+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2862043203 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbe38046820 con 0x55a514dc91b0 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.293+0000 7fbe4ec7e740 10 client.25426 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.293+0000 7fbe4ec7e740 10 client.25426 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.293+0000 7fbe4ec7e740 20 client.25426 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.293+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a514db09f0 0x55a5151b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.293+0000 7fbe4ec7e740 4 client.25426 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.293+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2862043203 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a515111df0 con 0x55a514db09f0 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.293+0000 7fbe45ca5640 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a514db09f0 0x55a5151b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.294+0000 7fbe25ffb640 10 client.25426 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.294+0000 7fbe45ca5640 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a514db09f0 0x55a5151b69f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c7570b312a0364c server_cookie=32a102c206f4c94f in_seq=0 out_seq=0 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.294+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2862043203 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a515111df0 con 0x55a514db09f0 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.294+0000 7fbe25ffb640 10 client.25426 handle_command_reply: tid=0 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.294+0000 7fbe4ec7e740 1 client.25426 shutdown 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.294+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2862043203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dc91b0 msgr2=0x55a5151a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.294+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dc91b0 0x55a5151a48b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbe38005940 tx=0x7fbe38009260 comp rx=0 tx=0).stop 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.294+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2862043203 shutdown_connections 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.295+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a514db09f0 0x55a5151b69f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.295+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514df2070 0x55a5151a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.295+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dd1390 0x55a5151a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.295+0000 7fbe25ffb640 0 client.25426 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.295+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2862043203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dc91b0 0x55a5151a48b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.295+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2862043203 >> 172.21.15.112:0/2862043203 conn(0x55a514de6460 msgr2=0x55a51519c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.295+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2862043203 shutdown_connections 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.295+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2862043203 wait complete. 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.295+0000 7fbe4ec7e740 20 client.25426 trim_cache size 0 max 16384 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.309+0000 7fbe4ec7e740 1 Processor -- start 2024-09-21T03:30:38.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.309+0000 7fbe4ec7e740 1 -- start start 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.309+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a51519aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.309+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dd1390 0x55a5151a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.309+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514df2070 0x55a5151a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.309+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401b1e70 con 0x55a514dc91b0 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.309+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401af740 con 0x55a514dd1390 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.309+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401ad590 con 0x55a514df2070 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.309+0000 7fbe464a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a51519aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe464a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a51519aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44670/0 (socket says 172.21.15.112:44670) 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe464a6640 1 -- 172.21.15.112:0/488040021 learned_addr learned my addr 172.21.15.112:0/488040021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe45ca5640 1 --2- 172.21.15.112:0/488040021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dd1390 0x55a5151a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe46ca7640 1 --2- 172.21.15.112:0/488040021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514df2070 0x55a5151a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe464a6640 1 -- 172.21.15.112:0/488040021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514df2070 msgr2=0x55a5151a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe464a6640 1 --2- 172.21.15.112:0/488040021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514df2070 0x55a5151a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe464a6640 1 -- 172.21.15.112:0/488040021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dd1390 msgr2=0x55a5151a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe464a6640 1 --2- 172.21.15.112:0/488040021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dd1390 0x55a5151a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe464a6640 1 -- 172.21.15.112:0/488040021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5151a5b70 con 0x55a514dc91b0 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe464a6640 1 --2- 172.21.15.112:0/488040021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a51519aa00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbe38046d90 tx=0x7fbe3802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7a934340cc33f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe27fff640 1 -- 172.21.15.112:0/488040021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38063070 con 0x55a514dc91b0 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe27fff640 1 -- 172.21.15.112:0/488040021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe380035f0 con 0x55a514dc91b0 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.310+0000 7fbe27fff640 1 -- 172.21.15.112:0/488040021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3804e070 con 0x55a514dc91b0 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.311+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/488040021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 msgr2=0x55a51519aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.311+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/488040021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a51519aa00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbe38046d90 tx=0x7fbe3802a410 comp rx=0 tx=0).stop 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.311+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/488040021 shutdown_connections 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.311+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/488040021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514df2070 0x55a5151a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.311+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/488040021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514dd1390 0x55a5151a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.311+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/488040021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a51519aa00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.311+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/488040021 >> 172.21.15.112:0/488040021 conn(0x55a514de6460 msgr2=0x55a51519a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.311+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/488040021 shutdown_connections 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.311+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/488040021 wait complete. 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.311+0000 7fbe4ec7e740 1 -- start start 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe4ec7e740 1 Processor -- start 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a5151ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dd1390 0x55a5151b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe4ec7e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514df2070 0x55a5151b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401adc00 con 0x55a514dc91b0 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401af070 con 0x55a514df2070 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe4ec7e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401ae2d0 con 0x55a514dd1390 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe4ec7e740 10 client.0 _ll_register_callbacks cb 0x7fbe38040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe464a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a5151ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe464a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a5151ba340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44680/0 (socket says 172.21.15.112:44680) 2024-09-21T03:30:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe464a6640 1 -- 172.21.15.112:0/2581232002 learned_addr learned my addr 172.21.15.112:0/2581232002 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe45ca5640 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dd1390 0x55a5151b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe46ca7640 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514df2070 0x55a5151b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe464a6640 1 -- 172.21.15.112:0/2581232002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dd1390 msgr2=0x55a5151b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe464a6640 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dd1390 0x55a5151b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe464a6640 1 -- 172.21.15.112:0/2581232002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514df2070 msgr2=0x55a5151b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe464a6640 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514df2070 0x55a5151b7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe464a6640 1 -- 172.21.15.112:0/2581232002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5151b8620 con 0x55a514dc91b0 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe464a6640 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a5151ba340 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbe38046d10 tx=0x7fbe380314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfde3f7c30bde17d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.312+0000 7fbe25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.313+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2581232002 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38063070 con 0x55a514dc91b0 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.313+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2581232002 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fbe38046450 con 0x55a514dc91b0 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.313+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2581232002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe380035f0 con 0x55a514dc91b0 2024-09-21T03:30:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.313+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2581232002 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe3804e040 con 0x55a514dc91b0 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.313+0000 7fbe4ec7e740 10 client.15864 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.313+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2581232002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a5152211f0 con 0x55a514dc91b0 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.313+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2581232002 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe380035f0 con 0x55a514dc91b0 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.313+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2581232002 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbe3803c7e0 con 0x55a514dc91b0 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.314+0000 7fbe4ec7e740 10 client.15864 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.314+0000 7fbe4ec7e740 10 client.15864 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.314+0000 7fbe4ec7e740 20 client.15864 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.314+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a514d85da0 0x55a515223970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.314+0000 7fbe4ec7e740 4 client.15864 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.314+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2581232002 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a51510d8a0 con 0x55a514d85da0 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.314+0000 7fbe45ca5640 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a514d85da0 0x55a515223970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe45ca5640 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a514d85da0 0x55a515223970 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7bb463e3117b758 server_cookie=d7fcc3e6c75343d0 in_seq=0 out_seq=0 2024-09-21T03:30:38.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe25ffb640 10 client.15864 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe25ffb640 1 -- 172.21.15.112:0/2581232002 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a51510d8a0 con 0x55a514d85da0 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe25ffb640 10 client.15864 handle_command_reply: tid=0 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 1 client.15864 shutdown 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2581232002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 msgr2=0x55a5151ba340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a5151ba340 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbe38046d10 tx=0x7fbe380314a0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2581232002 shutdown_connections 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a514d85da0 0x55a515223970 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a514df2070 0x55a5151b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a514dd1390 0x55a5151b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe25ffb640 0 client.15864 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 1 --2- 172.21.15.112:0/2581232002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a514dc91b0 0x55a5151ba340 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2581232002 >> 172.21.15.112:0/2581232002 conn(0x55a514de6460 msgr2=0x55a5151a5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2581232002 shutdown_connections 2024-09-21T03:30:38.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 1 -- 172.21.15.112:0/2581232002 wait complete. 2024-09-21T03:30:38.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.315+0000 7fbe4ec7e740 20 client.15864 trim_cache size 0 max 16384 2024-09-21T03:30:38.333 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:30:38.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 -- 172.21.15.112:0/2800064156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbe280b6680 msgr2=0x7fbe280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:38.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbe280b6680 0x7fbe280b8b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbe30011890 tx=0x7fbe30009500 comp rx=0 tx=0).stop 2024-09-21T03:30:38.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 -- 172.21.15.112:0/2800064156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40108420 msgr2=0x7fbe4019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:38.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40108420 0x7fbe4019aa80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbe4006b670 tx=0x7fbe3400dbb0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 -- 172.21.15.112:0/2800064156 shutdown_connections 2024-09-21T03:30:38.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbe280b6680 0x7fbe280b8b90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe40108d70 0x7fbe4019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe40108420 0x7fbe4019aa80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 --2- 172.21.15.112:0/2800064156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe40102ae0 0x7fbe4019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:38.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 -- 172.21.15.112:0/2800064156 >> 172.21.15.112:0/2800064156 conn(0x7fbe400796b0 msgr2=0x7fbe40106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:38.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 -- 172.21.15.112:0/2800064156 shutdown_connections 2024-09-21T03:30:38.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:38.316+0000 7fbe4870a640 1 -- 172.21.15.112:0/2800064156 wait complete. 2024-09-21T03:30:38.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:38 smithi112 ceph-mon[31442]: pgmap v367: 97 pgs: 97 active+clean; 51 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 5.9 MiB/s wr, 185 op/s 2024-09-21T03:30:38.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:38 smithi203 ceph-mon[34723]: pgmap v367: 97 pgs: 97 active+clean; 51 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 5.9 MiB/s wr, 185 op/s 2024-09-21T03:30:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:38 smithi187 ceph-mon[34603]: pgmap v367: 97 pgs: 97 active+clean; 51 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 5.9 MiB/s wr, 185 op/s 2024-09-21T03:30:39.064 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:30:39.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 71556 10.34 MB/sec warmup 39 sec latency 10.835 ms 2024-09-21T03:30:39.317 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:39.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.790+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/3767948734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2bc1083d0 msgr2=0x7ff2bc1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.790+0000 7ff2c1e2e640 1 --2- 172.21.15.112:0/3767948734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2bc1083d0 0x7ff2bc1087e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009900 tx=0x7ff2b002a410 comp rx=0 tx=0).stop 2024-09-21T03:30:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.790+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/3767948734 shutdown_connections 2024-09-21T03:30:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.790+0000 7ff2c1e2e640 1 --2- 172.21.15.112:0/3767948734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2bc108d20 0x7ff2bc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.790+0000 7ff2c1e2e640 1 --2- 172.21.15.112:0/3767948734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2bc1083d0 0x7ff2bc1087e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.790+0000 7ff2c1e2e640 1 --2- 172.21.15.112:0/3767948734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2bc102ae0 0x7ff2bc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.790+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/3767948734 >> 172.21.15.112:0/3767948734 conn(0x7ff2bc0797b0 msgr2=0x7ff2bc079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.790+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/3767948734 shutdown_connections 2024-09-21T03:30:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.790+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/3767948734 wait complete. 2024-09-21T03:30:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.791+0000 7ff2c1e2e640 1 Processor -- start 2024-09-21T03:30:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.791+0000 7ff2c1e2e640 1 -- start start 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.791+0000 7ff2c1e2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2bc102ae0 0x7ff2bc199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.791+0000 7ff2c1e2e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2bc1083d0 0x7ff2bc199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.791+0000 7ff2c1e2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2bc108d20 0x7ff2bc19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.791+0000 7ff2c1e2e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1129e0 con 0x7ff2bc108d20 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.791+0000 7ff2c1e2e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc112860 con 0x7ff2bc1083d0 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.791+0000 7ff2c1e2e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc112b60 con 0x7ff2bc102ae0 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.791+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2bc102ae0 0x7ff2bc199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.791+0000 7ff2bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2bc108d20 0x7ff2bc19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2bc108d20 0x7ff2bc19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44698/0 (socket says 172.21.15.112:44698) 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2bc102ae0 0x7ff2bc199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60002/0 (socket says 172.21.15.112:60002) 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2bbfff640 1 -- 172.21.15.112:0/1413744279 learned_addr learned my addr 172.21.15.112:0/1413744279 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2baffd640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2bc1083d0 0x7ff2bc199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2bbfff640 1 -- 172.21.15.112:0/1413744279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2bc102ae0 msgr2=0x7ff2bc199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2bbfff640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2bc102ae0 0x7ff2bc199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2bbfff640 1 -- 172.21.15.112:0/1413744279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2bc1083d0 msgr2=0x7ff2bc199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2bbfff640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2bc1083d0 0x7ff2bc199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2bbfff640 1 -- 172.21.15.112:0/1413744279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc19f2e0 con 0x7ff2bc108d20 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2bc102ae0 0x7ff2bc199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2bbfff640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2bc108d20 0x7ff2bc19ebb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400eb10 tx=0x7ff2b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2b8ff9640 1 -- 172.21.15.112:0/1413744279 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4004b20 con 0x7ff2bc108d20 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/1413744279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2bc19f5d0 con 0x7ff2bc108d20 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2b8ff9640 1 -- 172.21.15.112:0/1413744279 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b4004cc0 con 0x7ff2bc108d20 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2b8ff9640 1 -- 172.21.15.112:0/1413744279 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4010670 con 0x7ff2bc108d20 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.792+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/1413744279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2bc1a5d30 con 0x7ff2bc108d20 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.795+0000 7ff2c83a2740 1 Processor -- start 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.795+0000 7ff2b8ff9640 1 -- 172.21.15.112:0/1413744279 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff2b4005020 con 0x7ff2bc108d20 2024-09-21T03:30:39.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.795+0000 7ff2c83a2740 1 -- start start 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.795+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d3c070 0x55f1d10e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.795+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 0x55f1d10e3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d1b390 0x55f1d10e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f1d1059ea0 con 0x55f1d0d131b0 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f1d10492d0 con 0x55f1d0d1b390 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f1d104d0e0 con 0x55f1d0d3c070 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 0x55f1d10e3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d3c070 0x55f1d10e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 0x55f1d10e3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44700/0 (socket says 172.21.15.112:44700) 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d3c070 0x55f1d10e36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60010/0 (socket says 172.21.15.112:60010) 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2bb7fe640 1 -- 172.21.15.112:0/1948676001 learned_addr learned my addr 172.21.15.112:0/1948676001 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2baffd640 1 --2- 172.21.15.112:0/1948676001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d1b390 0x55f1d10e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2bb7fe640 1 -- 172.21.15.112:0/1948676001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d3c070 msgr2=0x55f1d10e36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/1948676001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d3c070 0x55f1d10e36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2bb7fe640 1 -- 172.21.15.112:0/1948676001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d1b390 msgr2=0x55f1d10e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/1948676001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d1b390 0x55f1d10e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2bb7fe640 1 -- 172.21.15.112:0/1948676001 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d0da7650 con 0x55f1d0d131b0 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2baffd640 1 --2- 172.21.15.112:0/1948676001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d1b390 0x55f1d10e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2b8ff9640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff2a40b6680 0x7ff2a40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2baffd640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff2a40b6680 0x7ff2a40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2b8ff9640 1 -- 172.21.15.112:0/1413744279 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff2b40e7c90 con 0x7ff2bc108d20 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.796+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/1948676001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 0x55f1d10e3be0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0115f0 tx=0x7ff2ac011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f9d589708074e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.797+0000 7ff2a1ffb640 1 -- 172.21.15.112:0/1948676001 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac01e070 con 0x55f1d0d131b0 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.797+0000 7ff2a1ffb640 1 -- 172.21.15.112:0/1948676001 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2ac00faf0 con 0x55f1d0d131b0 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.797+0000 7ff2a1ffb640 1 -- 172.21.15.112:0/1948676001 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac019b60 con 0x55f1d0d131b0 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.798+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1948676001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 msgr2=0x55f1d10e3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.798+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1948676001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 0x55f1d10e3be0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0115f0 tx=0x7ff2ac011ab0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.798+0000 7ff2baffd640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff2a40b6680 0x7ff2a40b8b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009230 tx=0x7ff2b0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.798+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1948676001 shutdown_connections 2024-09-21T03:30:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.798+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1948676001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d1b390 0x55f1d10e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.798+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1948676001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 0x55f1d10e3be0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.798+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1948676001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d3c070 0x55f1d10e36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.798+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1948676001 >> 172.21.15.112:0/1948676001 conn(0x55f1d0d30460 msgr2=0x55f1d10e1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.798+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1948676001 shutdown_connections 2024-09-21T03:30:39.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.798+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1948676001 wait complete. 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.798+0000 7ff2c83a2740 1 -- start start 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2c83a2740 1 Processor -- start 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d131b0 0x55f1d10f0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 0x55f1d10f1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d3c070 0x55f1d10f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f1d0eaa700 con 0x55f1d0d3c070 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f1d0fe4160 con 0x55f1d0d131b0 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f1d10f3fe0 con 0x55f1d0d1b390 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2c83a2740 10 client.0 _ll_register_callbacks cb 0x55f1d0ffd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d3c070 0x55f1d10f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d3c070 0x55f1d10f19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44714/0 (socket says 172.21.15.112:44714) 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2bbfff640 1 -- 172.21.15.112:0/1725983928 learned_addr learned my addr 172.21.15.112:0/1725983928 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:39.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d131b0 0x55f1d10f0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2baffd640 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 0x55f1d10f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2bbfff640 1 -- 172.21.15.112:0/1725983928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 msgr2=0x55f1d10f1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2bbfff640 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 0x55f1d10f1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2bbfff640 1 -- 172.21.15.112:0/1725983928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d131b0 msgr2=0x55f1d10f0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.799+0000 7ff2bbfff640 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d131b0 0x55f1d10f0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff2bbfff640 1 -- 172.21.15.112:0/1725983928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d1011c90 con 0x55f1d0d3c070 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d131b0 0x55f1d10f0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff2bbfff640 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d3c070 0x55f1d10f19a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40ad840 tx=0x7ff2b40a9aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f5d79754644259 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff29affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff29affd640 1 -- 172.21.15.112:0/1725983928 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40ad650 con 0x55f1d0d3c070 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff29affd640 1 -- 172.21.15.112:0/1725983928 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff2b40b2180 con 0x55f1d0d3c070 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1725983928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2ac00faf0 con 0x55f1d0d3c070 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff29affd640 1 -- 172.21.15.112:0/1725983928 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2b40ae240 con 0x55f1d0d3c070 2024-09-21T03:30:39.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff2c83a2740 10 client.15876 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1725983928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1d10ffd30 con 0x55f1d0d3c070 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff29affd640 1 -- 172.21.15.112:0/1725983928 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2ac00faf0 con 0x55f1d0d3c070 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.800+0000 7ff29affd640 1 -- 172.21.15.112:0/1725983928 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff2b40e5b90 con 0x55f1d0d3c070 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.801+0000 7ff2c83a2740 10 client.15876 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.801+0000 7ff2c83a2740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.801+0000 7ff2c83a2740 20 client.15876 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.801+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f1d0cfa9f0 0x55f1d1103320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.801+0000 7ff2c83a2740 4 client.15876 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.801+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1725983928 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1d105bdf0 con 0x55f1d0cfa9f0 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.801+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f1d0cfa9f0 0x55f1d1103320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.801+0000 7ff29affd640 10 client.15876 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.801+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f1d0cfa9f0 0x55f1d1103320 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6629a44cd71ed80 server_cookie=eab08077a8c60be9 in_seq=0 out_seq=0 2024-09-21T03:30:39.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.802+0000 7ff29affd640 1 -- 172.21.15.112:0/1725983928 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1d105bdf0 con 0x55f1d0cfa9f0 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.802+0000 7ff29affd640 10 client.15876 handle_command_reply: tid=0 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.802+0000 7ff2c83a2740 1 client.15876 shutdown 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1725983928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d3c070 msgr2=0x55f1d10f19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d3c070 0x55f1d10f19a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40ad840 tx=0x7ff2b40a9aa0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1725983928 shutdown_connections 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f1d0cfa9f0 0x55f1d1103320 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff29affd640 0 client.15876 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d3c070 0x55f1d10f19a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 0x55f1d10f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1725983928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d131b0 0x55f1d10f0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1725983928 >> 172.21.15.112:0/1725983928 conn(0x55f1d0d30460 msgr2=0x55f1d10e5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1725983928 shutdown_connections 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1725983928 wait complete. 2024-09-21T03:30:39.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.803+0000 7ff2c83a2740 20 client.15876 trim_cache size 0 max 16384 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2c83a2740 1 Processor -- start 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2c83a2740 1 -- start start 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d131b0 0x55f1d10ee0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d1b390 0x55f1d10ee630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d3c070 0x55f1d1108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1afc50 con 0x55f1d0d1b390 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1ad520 con 0x55f1d0d3c070 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1ab390 con 0x55f1d0d131b0 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d1b390 0x55f1d10ee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d1b390 0x55f1d10ee630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44720/0 (socket says 172.21.15.112:44720) 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2baffd640 1 -- 172.21.15.112:0/2618729891 learned_addr learned my addr 172.21.15.112:0/2618729891 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:39.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/2618729891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d131b0 0x55f1d10ee0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.817+0000 7ff2bbfff640 1 --2- 172.21.15.112:0/2618729891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d3c070 0x55f1d1108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.818+0000 7ff2baffd640 1 -- 172.21.15.112:0/2618729891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d131b0 msgr2=0x55f1d10ee0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.818+0000 7ff2baffd640 1 --2- 172.21.15.112:0/2618729891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d131b0 0x55f1d10ee0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.818+0000 7ff2baffd640 1 -- 172.21.15.112:0/2618729891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d3c070 msgr2=0x55f1d1108ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.818+0000 7ff2baffd640 1 --2- 172.21.15.112:0/2618729891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d3c070 0x55f1d1108ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.818+0000 7ff2baffd640 1 -- 172.21.15.112:0/2618729891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d10ef400 con 0x55f1d0d1b390 2024-09-21T03:30:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.818+0000 7ff2bbfff640 1 --2- 172.21.15.112:0/2618729891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d3c070 0x55f1d1108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.818+0000 7ff2baffd640 1 --2- 172.21.15.112:0/2618729891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d1b390 0x55f1d10ee630 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009900 tx=0x7ff2b002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50211dd99bda4629 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.818+0000 7ff2a17fa640 1 -- 172.21.15.112:0/2618729891 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0035040 con 0x55f1d0d1b390 2024-09-21T03:30:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.818+0000 7ff2a17fa640 1 -- 172.21.15.112:0/2618729891 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b00054f0 con 0x55f1d0d1b390 2024-09-21T03:30:39.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.818+0000 7ff2a17fa640 1 -- 172.21.15.112:0/2618729891 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0004de0 con 0x55f1d0d1b390 2024-09-21T03:30:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.819+0000 7ff2c83a2740 1 -- 172.21.15.112:0/2618729891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d1b390 msgr2=0x55f1d10ee630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.819+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/2618729891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d1b390 0x55f1d10ee630 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009900 tx=0x7ff2b002a410 comp rx=0 tx=0).stop 2024-09-21T03:30:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.819+0000 7ff2c83a2740 1 -- 172.21.15.112:0/2618729891 shutdown_connections 2024-09-21T03:30:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.819+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/2618729891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d3c070 0x55f1d1108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.819+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/2618729891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d1b390 0x55f1d10ee630 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.819+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/2618729891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d131b0 0x55f1d10ee0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.819+0000 7ff2c83a2740 1 -- 172.21.15.112:0/2618729891 >> 172.21.15.112:0/2618729891 conn(0x55f1d0d30460 msgr2=0x55f1d10ef780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.819+0000 7ff2c83a2740 1 -- 172.21.15.112:0/2618729891 shutdown_connections 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.819+0000 7ff2c83a2740 1 -- 172.21.15.112:0/2618729891 wait complete. 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.819+0000 7ff2c83a2740 1 -- start start 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2c83a2740 1 Processor -- start 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 0x55f1d10f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 0x55f1d10f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2c83a2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d3c070 0x55f1d10f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1ab9e0 con 0x55f1d0d131b0 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1ace50 con 0x55f1d0d3c070 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2c83a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc1ac0b0 con 0x55f1d0d1b390 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2c83a2740 10 client.0 _ll_register_callbacks cb 0x7ff2b40a9fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 0x55f1d10f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2bb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 0x55f1d10f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 0x55f1d10f6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60032/0 (socket says 172.21.15.112:60032) 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2baffd640 1 -- 172.21.15.112:0/1282925748 learned_addr learned my addr 172.21.15.112:0/1282925748 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.820+0000 7ff2bbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d3c070 0x55f1d10f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff2baffd640 1 -- 172.21.15.112:0/1282925748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d3c070 msgr2=0x55f1d10f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff2baffd640 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d3c070 0x55f1d10f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff2baffd640 1 -- 172.21.15.112:0/1282925748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 msgr2=0x55f1d10f99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff2baffd640 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 0x55f1d10f99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff2baffd640 1 -- 172.21.15.112:0/1282925748 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d10f7ab0 con 0x55f1d0d1b390 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 0x55f1d10f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff2baffd640 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 0x55f1d10f6e80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005790 tx=0x7ff2b0004940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53fcc784a12bd68c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff29affd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff29affd640 1 -- 172.21.15.112:0/1282925748 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b005b070 con 0x55f1d0d1b390 2024-09-21T03:30:39.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1282925748 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2b00054f0 con 0x55f1d0d1b390 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff29affd640 1 -- 172.21.15.112:0/1282925748 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff2b0004b60 con 0x55f1d0d1b390 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff29affd640 1 -- 172.21.15.112:0/1282925748 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2b0067050 con 0x55f1d0d1b390 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff2c83a2740 10 client.25450 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.821+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1282925748 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1d116db50 con 0x55f1d0d1b390 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.822+0000 7ff29affd640 1 -- 172.21.15.112:0/1282925748 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b00054f0 con 0x55f1d0d1b390 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.822+0000 7ff29affd640 1 -- 172.21.15.112:0/1282925748 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff2b0003990 con 0x55f1d0d1b390 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.822+0000 7ff2c83a2740 10 client.25450 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.822+0000 7ff2c83a2740 10 client.25450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.822+0000 7ff2c83a2740 20 client.25450 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.822+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f1d0ccfda0 0x55f1d1170350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.822+0000 7ff2c83a2740 4 client.25450 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.822+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1282925748 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1d10578a0 con 0x55f1d0ccfda0 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.822+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f1d0ccfda0 0x55f1d1170350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff29affd640 10 client.25450 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:39.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2bb7fe640 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f1d0ccfda0 0x55f1d1170350 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f857090c52f1bad server_cookie=30a10b95777b8ee5 in_seq=0 out_seq=0 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff29affd640 1 -- 172.21.15.112:0/1282925748 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1d10578a0 con 0x55f1d0ccfda0 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff29affd640 10 client.25450 handle_command_reply: tid=0 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 1 client.25450 shutdown 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1282925748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 msgr2=0x55f1d10f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 0x55f1d10f6e80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005790 tx=0x7ff2b0004940 comp rx=0 tx=0).stop 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1282925748 shutdown_connections 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f1d0ccfda0 0x55f1d1170350 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1d0d3c070 0x55f1d10f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f1d0d1b390 0x55f1d10f6e80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 1 --2- 172.21.15.112:0/1282925748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1d0d131b0 0x55f1d10f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff29affd640 0 client.25450 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1282925748 >> 172.21.15.112:0/1282925748 conn(0x55f1d0d30460 msgr2=0x55f1d10e4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1282925748 shutdown_connections 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 1 -- 172.21.15.112:0/1282925748 wait complete. 2024-09-21T03:30:39.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.823+0000 7ff2c83a2740 20 client.25450 trim_cache size 0 max 16384 2024-09-21T03:30:39.830 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:30:39.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/1413744279 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff2a40b6680 msgr2=0x7ff2a40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff2a40b6680 0x7ff2a40b8b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009230 tx=0x7ff2b0031400 comp rx=0 tx=0).stop 2024-09-21T03:30:39.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/1413744279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2bc108d20 msgr2=0x7ff2bc19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2bc108d20 0x7ff2bc19ebb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400eb10 tx=0x7ff2b400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/1413744279 shutdown_connections 2024-09-21T03:30:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff2a40b6680 0x7ff2a40b8b90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2bc108d20 0x7ff2bc19ebb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2bc1083d0 0x7ff2bc199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 --2- 172.21.15.112:0/1413744279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2bc102ae0 0x7ff2bc199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/1413744279 >> 172.21.15.112:0/1413744279 conn(0x7ff2bc0797b0 msgr2=0x7ff2bc106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/1413744279 shutdown_connections 2024-09-21T03:30:39.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:39.824+0000 7ff2c1e2e640 1 -- 172.21.15.112:0/1413744279 wait complete. 2024-09-21T03:30:40.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 74548 10.52 MB/sec warmup 40 sec latency 42.070 ms 2024-09-21T03:30:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:40 smithi112 ceph-mon[31442]: pgmap v368: 97 pgs: 97 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 4.5 MiB/s wr, 186 op/s 2024-09-21T03:30:40.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:40 smithi203 ceph-mon[34723]: pgmap v368: 97 pgs: 97 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 4.5 MiB/s wr, 186 op/s 2024-09-21T03:30:40.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:40 smithi187 ceph-mon[34603]: pgmap v368: 97 pgs: 97 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 4.5 MiB/s wr, 186 op/s 2024-09-21T03:30:41.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 76183 10.43 MB/sec warmup 41 sec latency 21.505 ms 2024-09-21T03:30:41.524 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:30:41.524 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:30:41.782 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:42.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 78634 10.52 MB/sec warmup 42 sec latency 9.436 ms 2024-09-21T03:30:42.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.239+0000 7f57dd295640 1 -- 172.21.15.112:0/588796398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d8102650 msgr2=0x7f57d8102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.239+0000 7f57dd295640 1 --2- 172.21.15.112:0/588796398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d8102650 0x7f57d8102a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009900 tx=0x7f57c802a480 comp rx=0 tx=0).stop 2024-09-21T03:30:42.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.239+0000 7f57dd295640 1 -- 172.21.15.112:0/588796398 shutdown_connections 2024-09-21T03:30:42.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.239+0000 7f57dd295640 1 --2- 172.21.15.112:0/588796398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57d8110200 0x7f57d8112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.239+0000 7f57dd295640 1 --2- 172.21.15.112:0/588796398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57d8102fa0 0x7f57d810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.239+0000 7f57dd295640 1 --2- 172.21.15.112:0/588796398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d8102650 0x7f57d8102a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.239+0000 7f57dd295640 1 -- 172.21.15.112:0/588796398 >> 172.21.15.112:0/588796398 conn(0x7f57d8079730 msgr2=0x7f57d8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:42.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.239+0000 7f57dd295640 1 -- 172.21.15.112:0/588796398 shutdown_connections 2024-09-21T03:30:42.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.239+0000 7f57dd295640 1 -- 172.21.15.112:0/588796398 wait complete. 2024-09-21T03:30:42.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.240+0000 7f57dd295640 1 Processor -- start 2024-09-21T03:30:42.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.240+0000 7f57dd295640 1 -- start start 2024-09-21T03:30:42.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.240+0000 7f57dd295640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57d8102650 0x7f57d819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.240+0000 7f57dd295640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57d8102fa0 0x7f57d819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.240+0000 7f57dd295640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d8110200 0x7f57d81a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.240+0000 7f57dd295640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8115c70 con 0x7f57d8102650 2024-09-21T03:30:42.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.240+0000 7f57dd295640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8115af0 con 0x7f57d8102fa0 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.240+0000 7f57dd295640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8115df0 con 0x7f57d8110200 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.240+0000 7f57d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57d8102650 0x7f57d819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.240+0000 7f57d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57d8102650 0x7f57d819c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44740/0 (socket says 172.21.15.112:44740) 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.240+0000 7f57d6ffd640 1 -- 172.21.15.112:0/563627906 learned_addr learned my addr 172.21.15.112:0/563627906 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57d77fe640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d8110200 0x7f57d81a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57d67fc640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57d8102fa0 0x7f57d819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57d6ffd640 1 -- 172.21.15.112:0/563627906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d8110200 msgr2=0x7f57d81a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57d6ffd640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d8110200 0x7f57d81a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57d6ffd640 1 -- 172.21.15.112:0/563627906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57d8102fa0 msgr2=0x7f57d819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57d6ffd640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57d8102fa0 0x7f57d819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57d6ffd640 1 -- 172.21.15.112:0/563627906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d81a2650 con 0x7f57d8102650 2024-09-21T03:30:42.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57d67fc640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57d8102fa0 0x7f57d819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:42.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57d6ffd640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57d8102650 0x7f57d819c6e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f57c8005940 tx=0x7f57c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:42.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57bffff640 1 -- 172.21.15.112:0/563627906 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c803f030 con 0x7f57d8102650 2024-09-21T03:30:42.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57bffff640 1 -- 172.21.15.112:0/563627906 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57c8003aa0 con 0x7f57d8102650 2024-09-21T03:30:42.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57dd295640 1 -- 172.21.15.112:0/563627906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d81a28e0 con 0x7f57d8102650 2024-09-21T03:30:42.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57bffff640 1 -- 172.21.15.112:0/563627906 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8035070 con 0x7f57d8102650 2024-09-21T03:30:42.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.241+0000 7f57dd295640 1 -- 172.21.15.112:0/563627906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d81a90a0 con 0x7f57d8102650 2024-09-21T03:30:42.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57e3809740 1 Processor -- start 2024-09-21T03:30:42.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57e3809740 1 -- start start 2024-09-21T03:30:42.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57e3809740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57e3809740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 0x55605d3b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57e3809740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe9670 0x55605d3b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57e3809740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55605d327e60 con 0x55605cfe1490 2024-09-21T03:30:42.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57e3809740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55605d317290 con 0x55605cfe9670 2024-09-21T03:30:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57e3809740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55605d31b0a0 con 0x55605d00a350 2024-09-21T03:30:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57bffff640 1 -- 172.21.15.112:0/563627906 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f57c8042030 con 0x7f57d8102650 2024-09-21T03:30:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60044/0 (socket says 172.21.15.112:60044) 2024-09-21T03:30:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57d77fe640 1 -- 172.21.15.112:0/1400720697 learned_addr learned my addr 172.21.15.112:0/1400720697 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57d67fc640 1 --2- 172.21.15.112:0/1400720697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe9670 0x55605d3b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.245+0000 7f57d6ffd640 1 --2- 172.21.15.112:0/1400720697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 0x55605d3b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57d77fe640 1 -- 172.21.15.112:0/1400720697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe9670 msgr2=0x55605d3b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57d77fe640 1 --2- 172.21.15.112:0/1400720697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe9670 0x55605d3b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57d77fe640 1 -- 172.21.15.112:0/1400720697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 msgr2=0x55605d3b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:42.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57d77fe640 1 --2- 172.21.15.112:0/1400720697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 0x55605d3b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57d77fe640 1 -- 172.21.15.112:0/1400720697 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55605d075930 con 0x55605d00a350 2024-09-21T03:30:42.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57d67fc640 1 --2- 172.21.15.112:0/1400720697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe9670 0x55605d3b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:30:42.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57d77fe640 1 --2- 172.21.15.112:0/1400720697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3b1430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009f90 tx=0x7f57d0010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96eabac1bd95eb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:42.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57bd7fa640 1 -- 172.21.15.112:0/1400720697 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0011700 con 0x55605d00a350 2024-09-21T03:30:42.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57bffff640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f57c00b6680 0x7f57c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57bd7fa640 1 -- 172.21.15.112:0/1400720697 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d0011d50 con 0x55605d00a350 2024-09-21T03:30:42.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57bd7fa640 1 -- 172.21.15.112:0/1400720697 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0019d40 con 0x55605d00a350 2024-09-21T03:30:42.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57d6ffd640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f57c00b6680 0x7f57c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.246+0000 7f57bffff640 1 -- 172.21.15.112:0/563627906 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f57c8107e50 con 0x7f57d8102650 2024-09-21T03:30:42.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.247+0000 7f57e3809740 1 -- 172.21.15.112:0/1400720697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 msgr2=0x55605d3b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.247+0000 7f57d6ffd640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f57c00b6680 0x7f57c00b8b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f57c81056b0 tx=0x7f57c803e000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:42.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.247+0000 7f57e3809740 1 --2- 172.21.15.112:0/1400720697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3b1430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009f90 tx=0x7f57d0010860 comp rx=0 tx=0).stop 2024-09-21T03:30:42.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.247+0000 7f57e3809740 1 -- 172.21.15.112:0/1400720697 shutdown_connections 2024-09-21T03:30:42.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.247+0000 7f57e3809740 1 --2- 172.21.15.112:0/1400720697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe9670 0x55605d3b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.247+0000 7f57e3809740 1 --2- 172.21.15.112:0/1400720697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 0x55605d3b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.247+0000 7f57e3809740 1 --2- 172.21.15.112:0/1400720697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3b1430 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.247+0000 7f57e3809740 1 -- 172.21.15.112:0/1400720697 >> 172.21.15.112:0/1400720697 conn(0x55605cffe740 msgr2=0x55605d3af320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:42.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.247+0000 7f57e3809740 1 -- 172.21.15.112:0/1400720697 shutdown_connections 2024-09-21T03:30:42.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.247+0000 7f57e3809740 1 -- 172.21.15.112:0/1400720697 wait complete. 2024-09-21T03:30:42.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57e3809740 1 -- start start 2024-09-21T03:30:42.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57e3809740 1 Processor -- start 2024-09-21T03:30:42.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57e3809740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe1490 0x55605d3bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57e3809740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3bd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57e3809740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57e3809740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55605d178890 con 0x55605cfe9670 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57e3809740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55605d2b1d40 con 0x55605cfe1490 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57e3809740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55605d3bfc10 con 0x55605d00a350 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57e3809740 10 client.0 _ll_register_callbacks cb 0x55605d315c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57d67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3bd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57d67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3bd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44756/0 (socket says 172.21.15.112:44756) 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57d67fc640 1 -- 172.21.15.112:0/2278917535 learned_addr learned my addr 172.21.15.112:0/2278917535 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57d77fe640 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe1490 0x55605d3bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.248+0000 7f57d77fe640 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57d67fc640 1 -- 172.21.15.112:0/2278917535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 msgr2=0x55605d3c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57d67fc640 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57d67fc640 1 -- 172.21.15.112:0/2278917535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe1490 msgr2=0x55605d3bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57d67fc640 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe1490 0x55605d3bcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57d67fc640 1 -- 172.21.15.112:0/2278917535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55605d3c9440 con 0x55605cfe9670 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57d67fc640 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3bd480 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f57cc013790 tx=0x7f57cc013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c521bb633eb485 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57baff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57baff5640 1 -- 172.21.15.112:0/2278917535 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc020060 con 0x55605cfe9670 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57baff5640 1 -- 172.21.15.112:0/2278917535 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f57cc011dd0 con 0x55605cfe9670 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57e3809740 1 -- 172.21.15.112:0/2278917535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d0011d50 con 0x55605cfe9670 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57baff5640 1 -- 172.21.15.112:0/2278917535 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f57cc025450 con 0x55605cfe9670 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57e3809740 10 client.15894 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57e3809740 1 -- 172.21.15.112:0/2278917535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55605d3cbda0 con 0x55605cfe9670 2024-09-21T03:30:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.249+0000 7f57baff5640 1 -- 172.21.15.112:0/2278917535 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d0011d50 con 0x55605cfe9670 2024-09-21T03:30:42.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.250+0000 7f57baff5640 1 -- 172.21.15.112:0/2278917535 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f57cc01a030 con 0x55605cfe9670 2024-09-21T03:30:42.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.250+0000 7f57e3809740 10 client.15894 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:42.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.250+0000 7f57e3809740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:42.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.250+0000 7f57e3809740 20 client.15894 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:42.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.250+0000 7f57e3809740 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55605cfc8cd0 0x55605d3cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.250+0000 7f57e3809740 4 client.15894 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:42.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.250+0000 7f57e3809740 1 -- 172.21.15.112:0/2278917535 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55605d329db0 con 0x55605cfc8cd0 2024-09-21T03:30:42.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.250+0000 7f57d77fe640 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55605cfc8cd0 0x55605d3cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.250+0000 7f57baff5640 10 client.15894 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:42.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.250+0000 7f57d77fe640 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55605cfc8cd0 0x55605d3cf300 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a4810ce59ac1d6 server_cookie=89a00128a8da67eb in_seq=0 out_seq=0 2024-09-21T03:30:42.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57baff5640 1 -- 172.21.15.112:0/2278917535 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55605d329db0 con 0x55605cfc8cd0 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57baff5640 10 client.15894 handle_command_reply: tid=0 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 1 client.15894 shutdown 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 1 -- 172.21.15.112:0/2278917535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 msgr2=0x55605d3bd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3bd480 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f57cc013790 tx=0x7f57cc013c50 comp rx=0 tx=0).stop 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 1 -- 172.21.15.112:0/2278917535 shutdown_connections 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55605cfc8cd0 0x55605d3cf300 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3bd480 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 1 --2- 172.21.15.112:0/2278917535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe1490 0x55605d3bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 1 -- 172.21.15.112:0/2278917535 >> 172.21.15.112:0/2278917535 conn(0x55605cffe740 msgr2=0x55605d3b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57baff5640 0 client.15894 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 1 -- 172.21.15.112:0/2278917535 shutdown_connections 2024-09-21T03:30:42.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 1 -- 172.21.15.112:0/2278917535 wait complete. 2024-09-21T03:30:42.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.251+0000 7f57e3809740 20 client.15894 trim_cache size 0 max 16384 2024-09-21T03:30:42.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57e3809740 1 Processor -- start 2024-09-21T03:30:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57e3809740 1 -- start start 2024-09-21T03:30:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57e3809740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 0x55605d3c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57e3809740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605cfe9670 0x55605d3b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57e3809740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605d00a350 0x55605d3b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57e3809740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81b40d0 con 0x55605cfe1490 2024-09-21T03:30:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57e3809740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81b19a0 con 0x55605d00a350 2024-09-21T03:30:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57e3809740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81af7f0 con 0x55605cfe9670 2024-09-21T03:30:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 0x55605d3c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 0x55605d3c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44758/0 (socket says 172.21.15.112:44758) 2024-09-21T03:30:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57d77fe640 1 -- 172.21.15.112:0/48898098 learned_addr learned my addr 172.21.15.112:0/48898098 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57d67fc640 1 --2- 172.21.15.112:0/48898098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605cfe9670 0x55605d3b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57d77fe640 1 --2- 172.21.15.112:0/48898098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605d00a350 0x55605d3b9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57d77fe640 1 -- 172.21.15.112:0/48898098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605cfe9670 msgr2=0x55605d3b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57d77fe640 1 --2- 172.21.15.112:0/48898098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605cfe9670 0x55605d3b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57d77fe640 1 -- 172.21.15.112:0/48898098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605d00a350 msgr2=0x55605d3b9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57d77fe640 1 --2- 172.21.15.112:0/48898098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605d00a350 0x55605d3b9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.269+0000 7f57d77fe640 1 -- 172.21.15.112:0/48898098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55605d3b85d0 con 0x55605cfe1490 2024-09-21T03:30:42.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.270+0000 7f57d77fe640 1 --2- 172.21.15.112:0/48898098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 0x55605d3c8e00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f57d0013010 tx=0x7f57d00096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6627ef5df740181 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:42.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.270+0000 7f57bcff9640 1 -- 172.21.15.112:0/48898098 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0003970 con 0x55605cfe1490 2024-09-21T03:30:42.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.270+0000 7f57bcff9640 1 -- 172.21.15.112:0/48898098 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d0020040 con 0x55605cfe1490 2024-09-21T03:30:42.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.270+0000 7f57bcff9640 1 -- 172.21.15.112:0/48898098 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0005730 con 0x55605cfe1490 2024-09-21T03:30:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 -- 172.21.15.112:0/48898098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 msgr2=0x55605d3c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 --2- 172.21.15.112:0/48898098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 0x55605d3c8e00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f57d0013010 tx=0x7f57d00096a0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 -- 172.21.15.112:0/48898098 shutdown_connections 2024-09-21T03:30:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 --2- 172.21.15.112:0/48898098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605d00a350 0x55605d3b9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 --2- 172.21.15.112:0/48898098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605cfe9670 0x55605d3b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 --2- 172.21.15.112:0/48898098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe1490 0x55605d3c8e00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 -- 172.21.15.112:0/48898098 >> 172.21.15.112:0/48898098 conn(0x55605cffe740 msgr2=0x55605d3bb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 -- 172.21.15.112:0/48898098 shutdown_connections 2024-09-21T03:30:42.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 -- 172.21.15.112:0/48898098 wait complete. 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 -- start start 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 Processor -- start 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe1490 0x55605d3c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3c1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81afe60 con 0x55605cfe9670 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81b12d0 con 0x55605cfe1490 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81b0530 con 0x55605d00a350 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.271+0000 7f57e3809740 10 client.0 _ll_register_callbacks cb 0x7f57cc009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3c1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe1490 0x55605d3c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3c1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44768/0 (socket says 172.21.15.112:44768) 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe1490 0x55605d3c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42198/0 (socket says 172.21.15.112:42198) 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d67fc640 1 -- 172.21.15.112:0/1107113301 learned_addr learned my addr 172.21.15.112:0/1107113301 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d77fe640 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d67fc640 1 -- 172.21.15.112:0/1107113301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 msgr2=0x55605d3c1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d67fc640 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3c1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d67fc640 1 -- 172.21.15.112:0/1107113301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe1490 msgr2=0x55605d3c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d67fc640 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe1490 0x55605d3c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d67fc640 1 -- 172.21.15.112:0/1107113301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55605d3c22d0 con 0x55605cfe9670 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57d67fc640 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3c1850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00f860 tx=0x7f57cc011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35e2236e4ba7c04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57baff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57baff5640 1 -- 172.21.15.112:0/1107113301 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc00fe60 con 0x55605cfe9670 2024-09-21T03:30:42.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57baff5640 1 -- 172.21.15.112:0/1107113301 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f57cc0050c0 con 0x55605cfe9670 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57e3809740 1 -- 172.21.15.112:0/1107113301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d0020040 con 0x55605cfe9670 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57baff5640 1 -- 172.21.15.112:0/1107113301 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f57cc0117b0 con 0x55605cfe9670 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57e3809740 10 client.15900 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.272+0000 7f57e3809740 1 -- 172.21.15.112:0/1107113301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55605d43aa10 con 0x55605cfe9670 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.273+0000 7f57baff5640 1 -- 172.21.15.112:0/1107113301 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d0020040 con 0x55605cfe9670 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.273+0000 7f57baff5640 1 -- 172.21.15.112:0/1107113301 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f57cc01a040 con 0x55605cfe9670 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.273+0000 7f57e3809740 10 client.15900 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.273+0000 7f57e3809740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.273+0000 7f57e3809740 20 client.15900 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.273+0000 7f57e3809740 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55605cf9e080 0x55605d43d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.273+0000 7f57e3809740 4 client.15900 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:30:42.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.273+0000 7f57e3809740 1 -- 172.21.15.112:0/1107113301 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55605d325860 con 0x55605cf9e080 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.273+0000 7f57d77fe640 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55605cf9e080 0x55605d43d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.273+0000 7f57baff5640 10 client.15900 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.273+0000 7f57d77fe640 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55605cf9e080 0x55605d43d2b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cf9cc2333ad9953 server_cookie=a7dc0bc29fa2a718 in_seq=0 out_seq=0 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57baff5640 1 -- 172.21.15.112:0/1107113301 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55605d325860 con 0x55605cf9e080 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57baff5640 10 client.15900 handle_command_reply: tid=0 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57e3809740 1 client.15900 shutdown 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57e3809740 1 -- 172.21.15.112:0/1107113301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 msgr2=0x55605d3c1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57e3809740 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3c1850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00f860 tx=0x7f57cc011e00 comp rx=0 tx=0).stop 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57e3809740 1 -- 172.21.15.112:0/1107113301 shutdown_connections 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57e3809740 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55605cf9e080 0x55605d43d2b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57e3809740 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55605d00a350 0x55605d3c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57e3809740 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55605cfe9670 0x55605d3c1850 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57baff5640 0 client.15900 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:42.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57e3809740 1 --2- 172.21.15.112:0/1107113301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55605cfe1490 0x55605d3c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57e3809740 1 -- 172.21.15.112:0/1107113301 >> 172.21.15.112:0/1107113301 conn(0x55605cffe740 msgr2=0x55605d3b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:42.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57e3809740 1 -- 172.21.15.112:0/1107113301 shutdown_connections 2024-09-21T03:30:42.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.275+0000 7f57e3809740 1 -- 172.21.15.112:0/1107113301 wait complete. 2024-09-21T03:30:42.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57e3809740 20 client.15900 trim_cache size 0 max 16384 2024-09-21T03:30:42.283 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:30:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:30:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "576b14cd-2e49-4418-842f-8aaa89f9d08b", 2024-09-21T03:30:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:30:42.284 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:30:42.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 -- 172.21.15.112:0/563627906 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f57c00b6680 msgr2=0x7f57c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f57c00b6680 0x7f57c00b8b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f57c81056b0 tx=0x7f57c803e000 comp rx=0 tx=0).stop 2024-09-21T03:30:42.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 -- 172.21.15.112:0/563627906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57d8102650 msgr2=0x7f57d819c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:42.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57d8102650 0x7f57d819c6e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f57c8005940 tx=0x7f57c8004270 comp rx=0 tx=0).stop 2024-09-21T03:30:42.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 -- 172.21.15.112:0/563627906 shutdown_connections 2024-09-21T03:30:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f57c00b6680 0x7f57c00b8b90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57d8110200 0x7f57d81a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57d8102fa0 0x7f57d819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 --2- 172.21.15.112:0/563627906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57d8102650 0x7f57d819c6e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 -- 172.21.15.112:0/563627906 >> 172.21.15.112:0/563627906 conn(0x7f57d8079730 msgr2=0x7f57d8108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 -- 172.21.15.112:0/563627906 shutdown_connections 2024-09-21T03:30:42.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:42.276+0000 7f57dd295640 1 -- 172.21.15.112:0/563627906 wait complete. 2024-09-21T03:30:42.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:42 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:30:42.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:42 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:30:42.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:42 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:30:43.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:30:43.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 81295 10.58 MB/sec warmup 43 sec latency 29.522 ms 2024-09-21T03:30:43.283 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:30:43.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:43 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:30:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:43 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:43 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:30:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:43 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:30:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:43 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:43 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:43 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:43 smithi203 ceph-mon[34723]: pgmap v369: 97 pgs: 97 active+clean; 56 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-21T03:30:43.665 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:43 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:30:43.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:43 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:43 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:30:43.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:43 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:30:43.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:43 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:43 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:43 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.666 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:43 smithi112 ceph-mon[31442]: pgmap v369: 97 pgs: 97 active+clean; 56 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-21T03:30:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:43 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:30:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:43 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:43 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:30:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:43 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:30:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:43 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:43 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:43 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:30:43.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:43 smithi187 ceph-mon[34603]: pgmap v369: 97 pgs: 97 active+clean; 56 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-21T03:30:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.755+0000 7f88d463f640 1 -- 172.21.15.112:0/2462182645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88cc109e20 msgr2=0x7f88cc110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.755+0000 7f88d463f640 1 --2- 172.21.15.112:0/2462182645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88cc109e20 0x7f88cc110650 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009ca0 tx=0x7f88c402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.755+0000 7f88d463f640 1 -- 172.21.15.112:0/2462182645 shutdown_connections 2024-09-21T03:30:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.755+0000 7f88d463f640 1 --2- 172.21.15.112:0/2462182645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88cc109e20 0x7f88cc110650 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.755+0000 7f88d463f640 1 --2- 172.21.15.112:0/2462182645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88cc1094d0 0x7f88cc1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.755+0000 7f88d463f640 1 --2- 172.21.15.112:0/2462182645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88cc103b90 0x7f88cc103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.755+0000 7f88d463f640 1 -- 172.21.15.112:0/2462182645 >> 172.21.15.112:0/2462182645 conn(0x7f88cc0ff600 msgr2=0x7f88cc101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.755+0000 7f88d463f640 1 -- 172.21.15.112:0/2462182645 shutdown_connections 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.755+0000 7f88d463f640 1 -- 172.21.15.112:0/2462182645 wait complete. 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d463f640 1 Processor -- start 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d463f640 1 -- start start 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d463f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88cc103b90 0x7f88cc19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d463f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88cc1094d0 0x7f88cc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d463f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88cc109e20 0x7f88cc1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d463f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc076640 con 0x7f88cc1094d0 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d463f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc0764c0 con 0x7f88cc103b90 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d463f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc0767c0 con 0x7f88cc109e20 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d1bda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88cc1094d0 0x7f88cc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d1bda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88cc1094d0 0x7f88cc19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44774/0 (socket says 172.21.15.112:44774) 2024-09-21T03:30:43.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d1bda640 1 -- 172.21.15.112:0/3842555219 learned_addr learned my addr 172.21.15.112:0/3842555219 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88cc109e20 0x7f88cc1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.756+0000 7f88d23db640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88cc103b90 0x7f88cc19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.757+0000 7f88d1bda640 1 -- 172.21.15.112:0/3842555219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88cc109e20 msgr2=0x7f88cc1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.757+0000 7f88d1bda640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88cc109e20 0x7f88cc1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.757+0000 7f88d1bda640 1 -- 172.21.15.112:0/3842555219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88cc103b90 msgr2=0x7f88cc19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.757+0000 7f88d1bda640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88cc103b90 0x7f88cc19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.757+0000 7f88d1bda640 1 -- 172.21.15.112:0/3842555219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc1a2580 con 0x7f88cc1094d0 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.757+0000 7f88d1bda640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88cc1094d0 0x7f88cc19cc70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c8c0 tx=0x7f88c000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.757+0000 7f88cb7fe640 1 -- 172.21.15.112:0/3842555219 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0007bb0 con 0x7f88cc1094d0 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.757+0000 7f88d463f640 1 -- 172.21.15.112:0/3842555219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc1a2870 con 0x7f88cc1094d0 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.757+0000 7f88cb7fe640 1 -- 172.21.15.112:0/3842555219 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c0007d50 con 0x7f88cc1094d0 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.757+0000 7f88cb7fe640 1 -- 172.21.15.112:0/3842555219 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c00187c0 con 0x7f88cc1094d0 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.757+0000 7f88d463f640 1 -- 172.21.15.112:0/3842555219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc1a9090 con 0x7f88cc1094d0 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.760+0000 7f88dabb3740 1 Processor -- start 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.760+0000 7f88dabb3740 1 -- start start 2024-09-21T03:30:43.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8f16070 0x562fa92bd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8eed1b0 0x562fa92bdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 0x562fa92be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88dabb3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562fa9233e50 con 0x562fa8ef5390 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88dabb3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562fa9223280 con 0x562fa8f16070 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88dabb3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562fa9227090 con 0x562fa8eed1b0 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88cb7fe640 1 -- 172.21.15.112:0/3842555219 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f88c0018960 con 0x7f88cc1094d0 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88d2bdc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8f16070 0x562fa92bd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88d2bdc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8f16070 0x562fa92bd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42244/0 (socket says 172.21.15.112:42244) 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88d23db640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8eed1b0 0x562fa92bdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88d2bdc640 1 -- 172.21.15.112:0/566617834 learned_addr learned my addr 172.21.15.112:0/566617834 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88d1bda640 1 --2- 172.21.15.112:0/566617834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 0x562fa92be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88d23db640 1 -- 172.21.15.112:0/566617834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8f16070 msgr2=0x562fa92bd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88d23db640 1 --2- 172.21.15.112:0/566617834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8f16070 0x562fa92bd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88d23db640 1 -- 172.21.15.112:0/566617834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 msgr2=0x562fa92be0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88d23db640 1 --2- 172.21.15.112:0/566617834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 0x562fa92be0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88d23db640 1 -- 172.21.15.112:0/566617834 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa8f81650 con 0x562fa8eed1b0 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.761+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/566617834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8f16070 0x562fa92bd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.762+0000 7f88d23db640 1 --2- 172.21.15.112:0/566617834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8eed1b0 0x562fa92bdb90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009c00 tx=0x7f88bc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d395f73966df4ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.762+0000 7f88c8ff9640 1 -- 172.21.15.112:0/566617834 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc004230 con 0x562fa8eed1b0 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.762+0000 7f88c8ff9640 1 -- 172.21.15.112:0/566617834 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88bc0043d0 con 0x562fa8eed1b0 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.762+0000 7f88c8ff9640 1 -- 172.21.15.112:0/566617834 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc004c60 con 0x562fa8eed1b0 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.762+0000 7f88cb7fe640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88b40b66b0 0x7f88b40b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.762+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88b40b66b0 0x7f88b40b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.762+0000 7f88cb7fe640 1 -- 172.21.15.112:0/3842555219 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f88c00e7680 con 0x7f88cc1094d0 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.763+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88b40b66b0 0x7f88b40b8bc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f88c402f500 tx=0x7f88c4009860 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.763+0000 7f88dabb3740 1 -- 172.21.15.112:0/566617834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8eed1b0 msgr2=0x562fa92bdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.763+0000 7f88dabb3740 1 --2- 172.21.15.112:0/566617834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8eed1b0 0x562fa92bdb90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009c00 tx=0x7f88bc011810 comp rx=0 tx=0).stop 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.763+0000 7f88dabb3740 1 -- 172.21.15.112:0/566617834 shutdown_connections 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.763+0000 7f88dabb3740 1 --2- 172.21.15.112:0/566617834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 0x562fa92be0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.763+0000 7f88dabb3740 1 --2- 172.21.15.112:0/566617834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8eed1b0 0x562fa92bdb90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.763+0000 7f88dabb3740 1 --2- 172.21.15.112:0/566617834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8f16070 0x562fa92bd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.763+0000 7f88dabb3740 1 -- 172.21.15.112:0/566617834 >> 172.21.15.112:0/566617834 conn(0x562fa8f0a460 msgr2=0x562fa92bb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.763+0000 7f88dabb3740 1 -- 172.21.15.112:0/566617834 shutdown_connections 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.763+0000 7f88dabb3740 1 -- 172.21.15.112:0/566617834 wait complete. 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88dabb3740 1 -- start start 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88dabb3740 1 Processor -- start 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8eed1b0 0x562fa92c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8ef5390 0x562fa92c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88dabb3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562fa9084700 con 0x562fa8f16070 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88dabb3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562fa91be110 con 0x562fa8ef5390 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88dabb3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562fa92cbe30 con 0x562fa8eed1b0 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88dabb3740 10 client.0 _ll_register_callbacks cb 0x562fa91d7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88d2bdc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88d2bdc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44790/0 (socket says 172.21.15.112:44790) 2024-09-21T03:30:43.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88d2bdc640 1 -- 172.21.15.112:0/1094811242 learned_addr learned my addr 172.21.15.112:0/1094811242 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88d23db640 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8eed1b0 0x562fa92c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.764+0000 7f88d1bda640 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8ef5390 0x562fa92c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88d2bdc640 1 -- 172.21.15.112:0/1094811242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8eed1b0 msgr2=0x562fa92c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8eed1b0 0x562fa92c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88d2bdc640 1 -- 172.21.15.112:0/1094811242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8ef5390 msgr2=0x562fa92c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8ef5390 0x562fa92c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88d2bdc640 1 -- 172.21.15.112:0/1094811242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa91ebc40 con 0x562fa8f16070 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c9710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f88c4031a00 tx=0x7f88c402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4da9cb5456545bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88b1ffb640 1 -- 172.21.15.112:0/1094811242 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4004040 con 0x562fa8f16070 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88b1ffb640 1 -- 172.21.15.112:0/1094811242 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f88c40041e0 con 0x562fa8f16070 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88dabb3740 1 -- 172.21.15.112:0/1094811242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88bc0043d0 con 0x562fa8f16070 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88b1ffb640 1 -- 172.21.15.112:0/1094811242 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88c40043e0 con 0x562fa8f16070 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88dabb3740 10 client.15912 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88dabb3740 1 -- 172.21.15.112:0/1094811242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562fa92d7b10 con 0x562fa8f16070 2024-09-21T03:30:43.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.765+0000 7f88b1ffb640 1 -- 172.21.15.112:0/1094811242 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88bc0043d0 con 0x562fa8f16070 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.766+0000 7f88b1ffb640 1 -- 172.21.15.112:0/1094811242 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f88c4051070 con 0x562fa8f16070 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.766+0000 7f88dabb3740 10 client.15912 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.766+0000 7f88dabb3740 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.766+0000 7f88dabb3740 20 client.15912 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.766+0000 7f88dabb3740 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562fa8ed49f0 0x562fa92db080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.766+0000 7f88dabb3740 4 client.15912 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.766+0000 7f88dabb3740 1 -- 172.21.15.112:0/1094811242 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fa9235da0 con 0x562fa8ed49f0 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.766+0000 7f88d23db640 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562fa8ed49f0 0x562fa92db080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.766+0000 7f88d23db640 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562fa8ed49f0 0x562fa92db080 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=117f8496cfb5ec40 server_cookie=d9f824ee7ca42883 in_seq=0 out_seq=0 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.766+0000 7f88b1ffb640 10 client.15912 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.767+0000 7f88b1ffb640 1 -- 172.21.15.112:0/1094811242 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562fa9235da0 con 0x562fa8ed49f0 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.767+0000 7f88b1ffb640 10 client.15912 handle_command_reply: tid=0 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.767+0000 7f88dabb3740 1 client.15912 shutdown 2024-09-21T03:30:43.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88dabb3740 1 -- 172.21.15.112:0/1094811242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 msgr2=0x562fa92c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88dabb3740 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c9710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f88c4031a00 tx=0x7f88c402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88dabb3740 1 -- 172.21.15.112:0/1094811242 shutdown_connections 2024-09-21T03:30:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88dabb3740 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562fa8ed49f0 0x562fa92db080 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88dabb3740 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c9710 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88dabb3740 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8ef5390 0x562fa92c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88dabb3740 1 --2- 172.21.15.112:0/1094811242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8eed1b0 0x562fa92c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88dabb3740 1 -- 172.21.15.112:0/1094811242 >> 172.21.15.112:0/1094811242 conn(0x562fa8f0a460 msgr2=0x562fa92c07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88b1ffb640 0 client.15912 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88dabb3740 1 -- 172.21.15.112:0/1094811242 shutdown_connections 2024-09-21T03:30:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88dabb3740 1 -- 172.21.15.112:0/1094811242 wait complete. 2024-09-21T03:30:43.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.768+0000 7f88dabb3740 20 client.15912 trim_cache size 0 max 16384 2024-09-21T03:30:43.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88dabb3740 1 Processor -- start 2024-09-21T03:30:43.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88dabb3740 1 -- start start 2024-09-21T03:30:43.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 0x562fa92ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8ef5390 0x562fa92c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88dabb3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1b3ef0 con 0x562fa8f16070 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88dabb3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1b17c0 con 0x562fa8eed1b0 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88dabb3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1af610 con 0x562fa8ef5390 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d2bdc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d23db640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 0x562fa92ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d2bdc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44792/0 (socket says 172.21.15.112:44792) 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d1bda640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8ef5390 0x562fa92c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d2bdc640 1 -- 172.21.15.112:0/3121811145 learned_addr learned my addr 172.21.15.112:0/3121811145 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d23db640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 0x562fa92ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42266/0 (socket says 172.21.15.112:42266) 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d2bdc640 1 -- 172.21.15.112:0/3121811145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8ef5390 msgr2=0x562fa92c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/3121811145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8ef5390 0x562fa92c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d2bdc640 1 -- 172.21.15.112:0/3121811145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 msgr2=0x562fa92ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/3121811145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 0x562fa92ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d2bdc640 1 -- 172.21.15.112:0/3121811145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa92c9f30 con 0x562fa8f16070 2024-09-21T03:30:43.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.786+0000 7f88d23db640 1 --2- 172.21.15.112:0/3121811145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 0x562fa92ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:43.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.787+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/3121811145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c63a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f88c4031d30 tx=0x7f88c4005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b7324d11b39fc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:43.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.787+0000 7f88b3fff640 1 -- 172.21.15.112:0/3121811145 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4060070 con 0x562fa8f16070 2024-09-21T03:30:43.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.787+0000 7f88b3fff640 1 -- 172.21.15.112:0/3121811145 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c4036050 con 0x562fa8f16070 2024-09-21T03:30:43.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.787+0000 7f88b3fff640 1 -- 172.21.15.112:0/3121811145 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c402f8c0 con 0x562fa8f16070 2024-09-21T03:30:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.787+0000 7f88dabb3740 1 -- 172.21.15.112:0/3121811145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 msgr2=0x562fa92c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.787+0000 7f88dabb3740 1 --2- 172.21.15.112:0/3121811145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c63a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f88c4031d30 tx=0x7f88c4005cb0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 -- 172.21.15.112:0/3121811145 shutdown_connections 2024-09-21T03:30:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 --2- 172.21.15.112:0/3121811145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8f16070 0x562fa92c63a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 --2- 172.21.15.112:0/3121811145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8ef5390 0x562fa92c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 --2- 172.21.15.112:0/3121811145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 0x562fa92ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 -- 172.21.15.112:0/3121811145 >> 172.21.15.112:0/3121811145 conn(0x562fa8f0a460 msgr2=0x562fa92ca190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:43.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 -- 172.21.15.112:0/3121811145 shutdown_connections 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 -- 172.21.15.112:0/3121811145 wait complete. 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 -- start start 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 Processor -- start 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 0x562fa92cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 0x562fa92cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.788+0000 7f88dabb3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8f16070 0x562fa92cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88dabb3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1afc80 con 0x562fa8ef5390 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88dabb3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1b10f0 con 0x562fa8eed1b0 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88dabb3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1b0350 con 0x562fa8f16070 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88dabb3740 10 client.0 _ll_register_callbacks cb 0x7f88c402f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d1bda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 0x562fa92cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d1bda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 0x562fa92cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44802/0 (socket says 172.21.15.112:44802) 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d1bda640 1 -- 172.21.15.112:0/3780716559 learned_addr learned my addr 172.21.15.112:0/3780716559 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:30:43.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8f16070 0x562fa92cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d23db640 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 0x562fa92cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d1bda640 1 -- 172.21.15.112:0/3780716559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8f16070 msgr2=0x562fa92cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d1bda640 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8f16070 0x562fa92cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d1bda640 1 -- 172.21.15.112:0/3780716559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 msgr2=0x562fa92cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d1bda640 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 0x562fa92cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d1bda640 1 -- 172.21.15.112:0/3780716559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa9344ad0 con 0x562fa8ef5390 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d2bdc640 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8f16070 0x562fa92cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88d1bda640 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 0x562fa92cd640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f88c001b000 tx=0x7f88c00a9590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd1b9583c78efce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88b1ffb640 1 -- 172.21.15.112:0/3780716559 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c00ace80 con 0x562fa8ef5390 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88b1ffb640 1 -- 172.21.15.112:0/3780716559 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f88c00b6cc0 con 0x562fa8ef5390 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88dabb3740 1 -- 172.21.15.112:0/3780716559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c4036050 con 0x562fa8ef5390 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.789+0000 7f88b1ffb640 1 -- 172.21.15.112:0/3780716559 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88c0018e60 con 0x562fa8ef5390 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.790+0000 7f88dabb3740 10 client.15918 fetch_fsmap learned FSMap version 26 2024-09-21T03:30:43.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.790+0000 7f88dabb3740 1 -- 172.21.15.112:0/3780716559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562fa9345dc0 con 0x562fa8ef5390 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.790+0000 7f88b1ffb640 1 -- 172.21.15.112:0/3780716559 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c4036050 con 0x562fa8ef5390 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.790+0000 7f88b1ffb640 1 -- 172.21.15.112:0/3780716559 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f88c00eace0 con 0x562fa8ef5390 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.790+0000 7f88dabb3740 10 client.15918 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.790+0000 7f88dabb3740 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.790+0000 7f88dabb3740 20 client.15918 populate_metadata read hostname 'smithi112' 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.790+0000 7f88dabb3740 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562fa8ea9da0 0x562fa9348590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.790+0000 7f88dabb3740 4 client.15918 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.790+0000 7f88dabb3740 1 -- 172.21.15.112:0/3780716559 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562fa9231850 con 0x562fa8ea9da0 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.790+0000 7f88d23db640 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562fa8ea9da0 0x562fa9348590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88d23db640 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562fa8ea9da0 0x562fa9348590 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a68efc3b508b287 server_cookie=9aee4f363b6e33b8 in_seq=0 out_seq=0 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88b1ffb640 10 client.15918 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88b1ffb640 1 -- 172.21.15.112:0/3780716559 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x562fa9231850 con 0x562fa8ea9da0 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88b1ffb640 10 client.15918 handle_command_reply: tid=0 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 1 client.15918 shutdown 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 1 -- 172.21.15.112:0/3780716559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 msgr2=0x562fa92cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:43.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 0x562fa92cd640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f88c001b000 tx=0x7f88c00a9590 comp rx=0 tx=0).stop 2024-09-21T03:30:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 1 -- 172.21.15.112:0/3780716559 shutdown_connections 2024-09-21T03:30:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562fa8ea9da0 0x562fa9348590 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562fa8f16070 0x562fa92cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fa8ef5390 0x562fa92cd640 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 1 --2- 172.21.15.112:0/3780716559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562fa8eed1b0 0x562fa92cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 1 -- 172.21.15.112:0/3780716559 >> 172.21.15.112:0/3780716559 conn(0x562fa8f0a460 msgr2=0x562fa92be680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88b1ffb640 0 client.15918 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:30:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 1 -- 172.21.15.112:0/3780716559 shutdown_connections 2024-09-21T03:30:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 1 -- 172.21.15.112:0/3780716559 wait complete. 2024-09-21T03:30:43.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.791+0000 7f88dabb3740 20 client.15918 trim_cache size 0 max 16384 2024-09-21T03:30:43.797 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "576b14cd-2e49-4418-842f-8aaa89f9d08b", 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:43.798 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:43.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:30:43.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:30:43.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:30:43.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:30:43.801 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:30:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 -- 172.21.15.112:0/3842555219 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88b40b66b0 msgr2=0x7f88b40b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88b40b66b0 0x7f88b40b8bc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f88c402f500 tx=0x7f88c4009860 comp rx=0 tx=0).stop 2024-09-21T03:30:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 -- 172.21.15.112:0/3842555219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88cc1094d0 msgr2=0x7f88cc19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:30:43.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88cc1094d0 0x7f88cc19cc70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c8c0 tx=0x7f88c000cd80 comp rx=0 tx=0).stop 2024-09-21T03:30:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 -- 172.21.15.112:0/3842555219 shutdown_connections 2024-09-21T03:30:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88b40b66b0 0x7f88b40b8bc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88cc109e20 0x7f88cc1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88cc1094d0 0x7f88cc19cc70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 --2- 172.21.15.112:0/3842555219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88cc103b90 0x7f88cc19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:30:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 -- 172.21.15.112:0/3842555219 >> 172.21.15.112:0/3842555219 conn(0x7f88cc0ff600 msgr2=0x7f88cc1062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:30:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 -- 172.21.15.112:0/3842555219 shutdown_connections 2024-09-21T03:30:43.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:30:43.792+0000 7f88d463f640 1 -- 172.21.15.112:0/3842555219 wait complete. 2024-09-21T03:30:44.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 83020 10.59 MB/sec warmup 44 sec latency 29.712 ms 2024-09-21T03:30:44.531 INFO:tasks.cephfs.filesystem:scrub status for tag:576b14cd-2e49-4418-842f-8aaa89f9d08b - {'path': '/', 'tag': '576b14cd-2e49-4418-842f-8aaa89f9d08b', 'options': 'recursive,force'} 2024-09-21T03:30:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:44 smithi112 ceph-mon[31442]: pgmap v370: 97 pgs: 97 active+clean; 56 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 942 KiB/s wr, 146 op/s 2024-09-21T03:30:44.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:44 smithi203 ceph-mon[34723]: pgmap v370: 97 pgs: 97 active+clean; 56 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 942 KiB/s wr, 146 op/s 2024-09-21T03:30:44.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:44 smithi187 ceph-mon[34603]: pgmap v370: 97 pgs: 97 active+clean; 56 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 942 KiB/s wr, 146 op/s 2024-09-21T03:30:45.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 85548 10.67 MB/sec warmup 45 sec latency 48.191 ms 2024-09-21T03:30:46.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 87729 10.61 MB/sec warmup 46 sec latency 20.405 ms 2024-09-21T03:30:46.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:46 smithi112 ceph-mon[31442]: pgmap v371: 97 pgs: 97 active+clean; 61 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-09-21T03:30:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:46 smithi203 ceph-mon[34723]: pgmap v371: 97 pgs: 97 active+clean; 61 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-09-21T03:30:46.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:46 smithi187 ceph-mon[34603]: pgmap v371: 97 pgs: 97 active+clean; 61 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-09-21T03:30:47.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 89600 10.63 MB/sec warmup 47 sec latency 11.756 ms 2024-09-21T03:30:48.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 92508 10.79 MB/sec warmup 48 sec latency 11.339 ms 2024-09-21T03:30:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:48 smithi112 ceph-mon[31442]: pgmap v372: 97 pgs: 97 active+clean; 61 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-09-21T03:30:48.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:48 smithi203 ceph-mon[34723]: pgmap v372: 97 pgs: 97 active+clean; 61 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-09-21T03:30:48.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:48 smithi187 ceph-mon[34603]: pgmap v372: 97 pgs: 97 active+clean; 61 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-09-21T03:30:49.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 94095 10.69 MB/sec warmup 49 sec latency 16.838 ms 2024-09-21T03:30:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:49 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:30:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:49 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:30:49.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:49 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:30:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:49 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:30:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:49 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:30:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:49 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:30:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:49 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:30:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:49 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:30:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:49 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:30:50.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 96217 10.75 MB/sec warmup 50 sec latency 81.872 ms 2024-09-21T03:30:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:50 smithi112 ceph-mon[31442]: pgmap v373: 97 pgs: 97 active+clean; 63 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-09-21T03:30:50.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:50 smithi203 ceph-mon[34723]: pgmap v373: 97 pgs: 97 active+clean; 63 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-09-21T03:30:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:50 smithi187 ceph-mon[34603]: pgmap v373: 97 pgs: 97 active+clean; 63 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-09-21T03:30:51.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 98717 10.77 MB/sec warmup 51 sec latency 14.261 ms 2024-09-21T03:30:52.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 100525 10.74 MB/sec warmup 52 sec latency 11.723 ms 2024-09-21T03:30:52.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:52 smithi112 ceph-mon[31442]: pgmap v374: 97 pgs: 97 active+clean; 67 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 296 op/s 2024-09-21T03:30:52.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:52 smithi203 ceph-mon[34723]: pgmap v374: 97 pgs: 97 active+clean; 67 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 296 op/s 2024-09-21T03:30:52.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:52 smithi187 ceph-mon[34603]: pgmap v374: 97 pgs: 97 active+clean; 67 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 296 op/s 2024-09-21T03:30:53.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 103271 10.86 MB/sec warmup 53 sec latency 20.236 ms 2024-09-21T03:30:54.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 105512 10.81 MB/sec warmup 54 sec latency 14.863 ms 2024-09-21T03:30:54.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:54 smithi112 ceph-mon[31442]: pgmap v375: 97 pgs: 97 active+clean; 67 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-09-21T03:30:54.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:54 smithi203 ceph-mon[34723]: pgmap v375: 97 pgs: 97 active+clean; 67 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-09-21T03:30:54.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:54 smithi187 ceph-mon[34603]: pgmap v375: 97 pgs: 97 active+clean; 67 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-09-21T03:30:55.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 107363 10.83 MB/sec warmup 55 sec latency 13.679 ms 2024-09-21T03:30:56.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 110253 10.96 MB/sec warmup 56 sec latency 13.107 ms 2024-09-21T03:30:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:56 smithi112 ceph-mon[31442]: pgmap v376: 97 pgs: 97 active+clean; 73 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-09-21T03:30:56.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:56 smithi203 ceph-mon[34723]: pgmap v376: 97 pgs: 97 active+clean; 73 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-09-21T03:30:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:56 smithi187 ceph-mon[34603]: pgmap v376: 97 pgs: 97 active+clean; 73 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-09-21T03:30:57.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 112335 10.89 MB/sec warmup 57 sec latency 14.585 ms 2024-09-21T03:30:58.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 114254 10.92 MB/sec warmup 58 sec latency 14.659 ms 2024-09-21T03:30:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:30:58 smithi112 ceph-mon[31442]: pgmap v377: 97 pgs: 97 active+clean; 73 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-21T03:30:58.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:30:58 smithi203 ceph-mon[34723]: pgmap v377: 97 pgs: 97 active+clean; 73 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-21T03:30:58.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:30:58 smithi187 ceph-mon[34603]: pgmap v377: 97 pgs: 97 active+clean; 73 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-21T03:30:59.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 117205 11.03 MB/sec warmup 59 sec latency 13.879 ms 2024-09-21T03:31:00.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 118712 10.95 MB/sec warmup 60 sec latency 18.060 ms 2024-09-21T03:31:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:00 smithi112 ceph-mon[31442]: pgmap v378: 97 pgs: 97 active+clean; 76 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-09-21T03:31:00.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:00 smithi203 ceph-mon[34723]: pgmap v378: 97 pgs: 97 active+clean; 76 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-09-21T03:31:00.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:00 smithi187 ceph-mon[34603]: pgmap v378: 97 pgs: 97 active+clean; 76 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-09-21T03:31:01.213 INFO:tasks.workunit.client.0.smithi112.stdout: 1 120874 11.00 MB/sec warmup 61 sec latency 25.409 ms 2024-09-21T03:31:02.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 122724 10.92 MB/sec warmup 62 sec latency 11.773 ms 2024-09-21T03:31:02.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:02 smithi112 ceph-mon[31442]: pgmap v379: 97 pgs: 97 active+clean; 80 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 304 op/s 2024-09-21T03:31:02.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:02 smithi203 ceph-mon[34723]: pgmap v379: 97 pgs: 97 active+clean; 80 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 304 op/s 2024-09-21T03:31:02.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:02 smithi187 ceph-mon[34603]: pgmap v379: 97 pgs: 97 active+clean; 80 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 304 op/s 2024-09-21T03:31:03.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 124745 10.96 MB/sec warmup 63 sec latency 15.319 ms 2024-09-21T03:31:04.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 127523 11.01 MB/sec warmup 64 sec latency 12.374 ms 2024-09-21T03:31:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:04 smithi112 ceph-mon[31442]: pgmap v380: 97 pgs: 97 active+clean; 80 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-09-21T03:31:04.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:04 smithi203 ceph-mon[34723]: pgmap v380: 97 pgs: 97 active+clean; 80 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-09-21T03:31:04.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:04 smithi187 ceph-mon[34603]: pgmap v380: 97 pgs: 97 active+clean; 80 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-09-21T03:31:05.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 128979 10.95 MB/sec warmup 65 sec latency 14.154 ms 2024-09-21T03:31:06.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 131637 11.04 MB/sec warmup 66 sec latency 12.607 ms 2024-09-21T03:31:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:06 smithi112 ceph-mon[31442]: pgmap v381: 97 pgs: 97 active+clean; 83 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.7 MiB/s wr, 308 op/s 2024-09-21T03:31:06.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:06 smithi203 ceph-mon[34723]: pgmap v381: 97 pgs: 97 active+clean; 83 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.7 MiB/s wr, 308 op/s 2024-09-21T03:31:06.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:06 smithi187 ceph-mon[34603]: pgmap v381: 97 pgs: 97 active+clean; 83 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.7 MiB/s wr, 308 op/s 2024-09-21T03:31:07.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 133679 10.98 MB/sec warmup 67 sec latency 38.531 ms 2024-09-21T03:31:08.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 135553 11.00 MB/sec warmup 68 sec latency 17.948 ms 2024-09-21T03:31:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:08 smithi112 ceph-mon[31442]: pgmap v382: 97 pgs: 97 active+clean; 87 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2024-09-21T03:31:08.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:08 smithi203 ceph-mon[34723]: pgmap v382: 97 pgs: 97 active+clean; 87 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2024-09-21T03:31:08.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:08 smithi187 ceph-mon[34603]: pgmap v382: 97 pgs: 97 active+clean; 87 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2024-09-21T03:31:09.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 138529 11.09 MB/sec warmup 69 sec latency 17.108 ms 2024-09-21T03:31:10.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 140026 11.03 MB/sec warmup 70 sec latency 50.634 ms 2024-09-21T03:31:10.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:10 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:10.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:10 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:10.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:10 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:31:11.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 142368 11.07 MB/sec warmup 71 sec latency 10.234 ms 2024-09-21T03:31:11.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:11 smithi112 ceph-mon[31442]: pgmap v383: 97 pgs: 97 active+clean; 89 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-09-21T03:31:11.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:11.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:11.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:31:11.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:11 smithi203 ceph-mon[34723]: pgmap v383: 97 pgs: 97 active+clean; 89 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-09-21T03:31:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:11.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:31:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:11 smithi187 ceph-mon[34603]: pgmap v383: 97 pgs: 97 active+clean; 89 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-09-21T03:31:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:31:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:31:11.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:31:12.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 144824 11.09 MB/sec warmup 72 sec latency 31.804 ms 2024-09-21T03:31:12.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:12 smithi112 ceph-mon[31442]: pgmap v384: 97 pgs: 97 active+clean; 94 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 330 op/s 2024-09-21T03:31:12.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:12 smithi203 ceph-mon[34723]: pgmap v384: 97 pgs: 97 active+clean; 94 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 330 op/s 2024-09-21T03:31:12.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:12 smithi187 ceph-mon[34603]: pgmap v384: 97 pgs: 97 active+clean; 94 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 330 op/s 2024-09-21T03:31:13.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 146372 11.04 MB/sec warmup 73 sec latency 34.725 ms 2024-09-21T03:31:14.214 INFO:tasks.workunit.client.0.smithi112.stdout: 1 149161 11.12 MB/sec warmup 74 sec latency 60.404 ms 2024-09-21T03:31:14.532 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:31:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:14 smithi112 ceph-mon[31442]: pgmap v385: 97 pgs: 97 active+clean; 94 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-09-21T03:31:14.798 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:14.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:14 smithi203 ceph-mon[34723]: pgmap v385: 97 pgs: 97 active+clean; 94 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-09-21T03:31:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:14 smithi187 ceph-mon[34603]: pgmap v385: 97 pgs: 97 active+clean; 94 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-09-21T03:31:15.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 150668 11.06 MB/sec warmup 75 sec latency 15.632 ms 2024-09-21T03:31:15.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.285+0000 7f08454da640 1 -- 172.21.15.112:0/3753503819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0840101fc0 msgr2=0x7f08401023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.285+0000 7f08454da640 1 --2- 172.21.15.112:0/3753503819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0840101fc0 0x7f08401023d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0830009ca0 tx=0x7f083002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.285+0000 7f08454da640 1 -- 172.21.15.112:0/3753503819 shutdown_connections 2024-09-21T03:31:15.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.285+0000 7f08454da640 1 --2- 172.21.15.112:0/3753503819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840109e50 0x7f0840110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.285+0000 7f08454da640 1 --2- 172.21.15.112:0/3753503819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0840109500 0x7f0840109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.285+0000 7f08454da640 1 --2- 172.21.15.112:0/3753503819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0840101fc0 0x7f08401023d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.285+0000 7f08454da640 1 -- 172.21.15.112:0/3753503819 >> 172.21.15.112:0/3753503819 conn(0x7f0840079f90 msgr2=0x7f084007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.285+0000 7f08454da640 1 -- 172.21.15.112:0/3753503819 shutdown_connections 2024-09-21T03:31:15.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.285+0000 7f08454da640 1 -- 172.21.15.112:0/3753503819 wait complete. 2024-09-21T03:31:15.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.286+0000 7f08454da640 1 Processor -- start 2024-09-21T03:31:15.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.286+0000 7f08454da640 1 -- start start 2024-09-21T03:31:15.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.286+0000 7f08454da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840101fc0 0x7f084019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.286+0000 7f08454da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0840109500 0x7f084019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.286+0000 7f08454da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0840109e50 0x7f084019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.286+0000 7f08454da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0840113b10 con 0x7f0840109500 2024-09-21T03:31:15.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.286+0000 7f08454da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0840113990 con 0x7f0840109e50 2024-09-21T03:31:15.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.286+0000 7f08454da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0840113c90 con 0x7f0840101fc0 2024-09-21T03:31:15.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0840109500 0x7f084019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840101fc0 0x7f084019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0840109500 0x7f084019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40434/0 (socket says 172.21.15.112:40434) 2024-09-21T03:31:15.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083f7fe640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0840109e50 0x7f084019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083e7fc640 1 -- 172.21.15.112:0/3878174069 learned_addr learned my addr 172.21.15.112:0/3878174069 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:15.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083effd640 1 -- 172.21.15.112:0/3878174069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0840109e50 msgr2=0x7f084019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083effd640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0840109e50 0x7f084019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083effd640 1 -- 172.21.15.112:0/3878174069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0840109500 msgr2=0x7f084019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083effd640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0840109500 0x7f084019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083effd640 1 -- 172.21.15.112:0/3878174069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08401a04b0 con 0x7f0840101fc0 2024-09-21T03:31:15.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083f7fe640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0840109e50 0x7f084019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:15.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083e7fc640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0840109500 0x7f084019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:15.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.287+0000 7f083effd640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840101fc0 0x7f084019a540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f083002ade0 tx=0x7f0830031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.288+0000 7f0837fff640 1 -- 172.21.15.112:0/3878174069 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0830003b70 con 0x7f0840101fc0 2024-09-21T03:31:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.288+0000 7f08454da640 1 -- 172.21.15.112:0/3878174069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08401a0740 con 0x7f0840101fc0 2024-09-21T03:31:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.288+0000 7f0837fff640 1 -- 172.21.15.112:0/3878174069 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0830031910 con 0x7f0840101fc0 2024-09-21T03:31:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.288+0000 7f0837fff640 1 -- 172.21.15.112:0/3878174069 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083002f950 con 0x7f0840101fc0 2024-09-21T03:31:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.288+0000 7f08454da640 1 -- 172.21.15.112:0/3878174069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08401a6f00 con 0x7f0840101fc0 2024-09-21T03:31:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.290+0000 7f084ba4e740 1 Processor -- start 2024-09-21T03:31:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.290+0000 7f084ba4e740 1 -- start start 2024-09-21T03:31:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3ee1070 0x5633d4288650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3eb81b0 0x5633d4288b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d42890d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f084ba4e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5633d41fee50 con 0x5633d3eb81b0 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f084ba4e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5633d41ee280 con 0x5633d3ec0390 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f084ba4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5633d41f2090 con 0x5633d3ee1070 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f0837fff640 1 -- 172.21.15.112:0/3878174069 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f083002faf0 con 0x7f0840101fc0 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3eb81b0 0x5633d4288b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083e7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d42890d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3ee1070 0x5633d4288650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083e7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d42890d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:56934/0 (socket says 172.21.15.112:56934) 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083e7fc640 1 -- 172.21.15.112:0/4068085973 learned_addr learned my addr 172.21.15.112:0/4068085973 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3ee1070 0x5633d4288650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:41330/0 (socket says 172.21.15.112:41330) 2024-09-21T03:31:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083e7fc640 1 -- 172.21.15.112:0/4068085973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3ee1070 msgr2=0x5633d4288650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083e7fc640 1 --2- 172.21.15.112:0/4068085973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3ee1070 0x5633d4288650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083e7fc640 1 -- 172.21.15.112:0/4068085973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3eb81b0 msgr2=0x5633d4288b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083e7fc640 1 --2- 172.21.15.112:0/4068085973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3eb81b0 0x5633d4288b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083e7fc640 1 -- 172.21.15.112:0/4068085973 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d3f4c650 con 0x5633d3ec0390 2024-09-21T03:31:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.291+0000 7f083effd640 1 --2- 172.21.15.112:0/4068085973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3eb81b0 0x5633d4288b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.292+0000 7f0837fff640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f082c0b6700 0x7f082c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.292+0000 7f083e7fc640 1 --2- 172.21.15.112:0/4068085973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d42890d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0828011540 tx=0x7f0828011940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb739af0d7bedd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.292+0000 7f083f7fe640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f082c0b6700 0x7f082c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.292+0000 7f08357fa640 1 -- 172.21.15.112:0/4068085973 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082801e070 con 0x5633d3ec0390 2024-09-21T03:31:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.292+0000 7f08357fa640 1 -- 172.21.15.112:0/4068085973 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f082800fa70 con 0x5633d3ec0390 2024-09-21T03:31:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.292+0000 7f08357fa640 1 -- 172.21.15.112:0/4068085973 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828019bc0 con 0x5633d3ec0390 2024-09-21T03:31:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.292+0000 7f0837fff640 1 -- 172.21.15.112:0/3878174069 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f08301085f0 con 0x7f0840101fc0 2024-09-21T03:31:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f084ba4e740 1 -- 172.21.15.112:0/4068085973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 msgr2=0x5633d42890d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f083f7fe640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f082c0b6700 0x7f082c0b8c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0838012560 tx=0x7f0838005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f084ba4e740 1 --2- 172.21.15.112:0/4068085973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d42890d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0828011540 tx=0x7f0828011940 comp rx=0 tx=0).stop 2024-09-21T03:31:15.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f084ba4e740 1 -- 172.21.15.112:0/4068085973 shutdown_connections 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f084ba4e740 1 --2- 172.21.15.112:0/4068085973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d42890d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f084ba4e740 1 --2- 172.21.15.112:0/4068085973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3eb81b0 0x5633d4288b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f084ba4e740 1 --2- 172.21.15.112:0/4068085973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3ee1070 0x5633d4288650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f084ba4e740 1 -- 172.21.15.112:0/4068085973 >> 172.21.15.112:0/4068085973 conn(0x5633d3ed5460 msgr2=0x5633d4286540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f084ba4e740 1 -- 172.21.15.112:0/4068085973 shutdown_connections 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f084ba4e740 1 -- 172.21.15.112:0/4068085973 wait complete. 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f084ba4e740 1 -- start start 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.293+0000 7f084ba4e740 1 Processor -- start 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4293d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ec0390 0x5633d4294290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ee1070 0x5633d42a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f084ba4e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5633d404f700 con 0x5633d3ec0390 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f084ba4e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5633d4189110 con 0x5633d3ee1070 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f084ba4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5633d4296e30 con 0x5633d3eb81b0 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f084ba4e740 10 client.0 _ll_register_callbacks cb 0x5633d41a2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ec0390 0x5633d4294290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ec0390 0x5633d4294290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40442/0 (socket says 172.21.15.112:40442) 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083e7fc640 1 -- 172.21.15.112:0/169531542 learned_addr learned my addr 172.21.15.112:0/169531542 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083effd640 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4293d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083f7fe640 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ee1070 0x5633d42a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083e7fc640 1 -- 172.21.15.112:0/169531542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 msgr2=0x5633d4293d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083e7fc640 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4293d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083e7fc640 1 -- 172.21.15.112:0/169531542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ee1070 msgr2=0x5633d42a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083e7fc640 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ee1070 0x5633d42a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083e7fc640 1 -- 172.21.15.112:0/169531542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633d41b6c40 con 0x5633d3ec0390 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083effd640 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4293d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f083e7fc640 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ec0390 0x5633d4294290 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0828009760 tx=0x7f0828005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d79440832f9a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.294+0000 7f08267fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f08267fc640 1 -- 172.21.15.112:0/169531542 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082801e070 con 0x5633d3ec0390 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f08267fc640 1 -- 172.21.15.112:0/169531542 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f082800f7d0 con 0x5633d3ec0390 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f084ba4e740 1 -- 172.21.15.112:0/169531542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f082800fa70 con 0x5633d3ec0390 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f08267fc640 1 -- 172.21.15.112:0/169531542 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0828004040 con 0x5633d3ec0390 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f084ba4e740 10 client.15930 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f084ba4e740 1 -- 172.21.15.112:0/169531542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5633d42a2ba0 con 0x5633d3ec0390 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f08267fc640 1 -- 172.21.15.112:0/169531542 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f082800fa70 con 0x5633d3ec0390 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f08267fc640 1 -- 172.21.15.112:0/169531542 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f08280035f0 con 0x5633d3ec0390 2024-09-21T03:31:15.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f084ba4e740 10 client.15930 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f084ba4e740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:31:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f084ba4e740 20 client.15930 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f084ba4e740 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5633d3e9f9f0 0x5633d42a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f084ba4e740 4 client.15930 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.295+0000 7f084ba4e740 1 -- 172.21.15.112:0/169531542 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633d4200da0 con 0x5633d3e9f9f0 2024-09-21T03:31:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.296+0000 7f083effd640 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5633d3e9f9f0 0x5633d42a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.296+0000 7f08267fc640 10 client.15930 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.296+0000 7f083effd640 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5633d3e9f9f0 0x5633d42a5f60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b02c2d4707f32d6c server_cookie=8c32d81e8b4f838 in_seq=0 out_seq=0 2024-09-21T03:31:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f08267fc640 1 -- 172.21.15.112:0/169531542 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5633d4200da0 con 0x5633d3e9f9f0 2024-09-21T03:31:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f08267fc640 10 client.15930 handle_command_reply: tid=0 2024-09-21T03:31:15.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 1 client.15930 shutdown 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 1 -- 172.21.15.112:0/169531542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ec0390 msgr2=0x5633d4294290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ec0390 0x5633d4294290 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0828009760 tx=0x7f0828005760 comp rx=0 tx=0).stop 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 1 -- 172.21.15.112:0/169531542 shutdown_connections 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5633d3e9f9f0 0x5633d42a5f60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ee1070 0x5633d42a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ec0390 0x5633d4294290 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 1 --2- 172.21.15.112:0/169531542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4293d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 1 -- 172.21.15.112:0/169531542 >> 172.21.15.112:0/169531542 conn(0x5633d3ed5460 msgr2=0x5633d428b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f08267fc640 0 client.15930 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 1 -- 172.21.15.112:0/169531542 shutdown_connections 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 1 -- 172.21.15.112:0/169531542 wait complete. 2024-09-21T03:31:15.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.297+0000 7f084ba4e740 20 client.15930 trim_cache size 0 max 16384 2024-09-21T03:31:15.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.316+0000 7f084ba4e740 1 Processor -- start 2024-09-21T03:31:15.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.316+0000 7f084ba4e740 1 -- start start 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4294210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d42a0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d4285b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f084ba4e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08401b4150 con 0x5633d3ee1070 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f084ba4e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08401b1a20 con 0x5633d3ec0390 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f084ba4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f08401af870 con 0x5633d3eb81b0 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f083f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d4285b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f083f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d4285b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40448/0 (socket says 172.21.15.112:40448) 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f083effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4294210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f083f7fe640 1 -- 172.21.15.112:0/4088637424 learned_addr learned my addr 172.21.15.112:0/4088637424 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f083e7fc640 1 --2- 172.21.15.112:0/4088637424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d42a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f083f7fe640 1 -- 172.21.15.112:0/4088637424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 msgr2=0x5633d4294210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f083f7fe640 1 --2- 172.21.15.112:0/4088637424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4294210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f083f7fe640 1 -- 172.21.15.112:0/4088637424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 msgr2=0x5633d42a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f083f7fe640 1 --2- 172.21.15.112:0/4088637424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d42a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.317+0000 7f083f7fe640 1 -- 172.21.15.112:0/4088637424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d428e980 con 0x5633d3ee1070 2024-09-21T03:31:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.318+0000 7f083f7fe640 1 --2- 172.21.15.112:0/4088637424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d4285b70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f08380045d0 tx=0x7f083801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35a211f57042b4d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.318+0000 7f0834ff9640 1 -- 172.21.15.112:0/4088637424 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083800f600 con 0x5633d3ee1070 2024-09-21T03:31:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.318+0000 7f0834ff9640 1 -- 172.21.15.112:0/4088637424 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083800fc50 con 0x5633d3ee1070 2024-09-21T03:31:15.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.318+0000 7f0834ff9640 1 -- 172.21.15.112:0/4088637424 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083802acb0 con 0x5633d3ee1070 2024-09-21T03:31:15.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.319+0000 7f084ba4e740 1 -- 172.21.15.112:0/4088637424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 msgr2=0x5633d4285b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.319+0000 7f084ba4e740 1 --2- 172.21.15.112:0/4088637424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d4285b70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f08380045d0 tx=0x7f083801b000 comp rx=0 tx=0).stop 2024-09-21T03:31:15.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.319+0000 7f084ba4e740 1 -- 172.21.15.112:0/4088637424 shutdown_connections 2024-09-21T03:31:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.319+0000 7f084ba4e740 1 --2- 172.21.15.112:0/4088637424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d4285b70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.319+0000 7f084ba4e740 1 --2- 172.21.15.112:0/4088637424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d42a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.319+0000 7f084ba4e740 1 --2- 172.21.15.112:0/4088637424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4294210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.319+0000 7f084ba4e740 1 -- 172.21.15.112:0/4088637424 >> 172.21.15.112:0/4088637424 conn(0x5633d3ed5460 msgr2=0x5633d4293b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.319+0000 7f084ba4e740 1 -- 172.21.15.112:0/4088637424 shutdown_connections 2024-09-21T03:31:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.319+0000 7f084ba4e740 1 -- 172.21.15.112:0/4088637424 wait complete. 2024-09-21T03:31:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.319+0000 7f084ba4e740 1 -- start start 2024-09-21T03:31:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f084ba4e740 1 Processor -- start 2024-09-21T03:31:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4297850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d4297d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f084ba4e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d430e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f084ba4e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08401afee0 con 0x5633d3ee1070 2024-09-21T03:31:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f084ba4e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08401b1350 con 0x5633d3ec0390 2024-09-21T03:31:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f084ba4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f08401b05b0 con 0x5633d3eb81b0 2024-09-21T03:31:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f084ba4e740 10 client.0 _ll_register_callbacks cb 0x7f082801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f083effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4297850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f083f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d430e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f083e7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d4297d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f083effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4297850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:41360/0 (socket says 172.21.15.112:41360) 2024-09-21T03:31:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f083f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d430e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40454/0 (socket says 172.21.15.112:40454) 2024-09-21T03:31:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f083effd640 1 -- 172.21.15.112:0/118859150 learned_addr learned my addr 172.21.15.112:0/118859150 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:15.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.320+0000 7f083e7fc640 1 -- 172.21.15.112:0/118859150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 msgr2=0x5633d4297850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f083e7fc640 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4297850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f083e7fc640 1 -- 172.21.15.112:0/118859150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 msgr2=0x5633d430e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f083e7fc640 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d430e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f083e7fc640 1 -- 172.21.15.112:0/118859150 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633d430ede0 con 0x5633d3ec0390 2024-09-21T03:31:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f083f7fe640 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d430e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f083e7fc640 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d4297d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f082802e040 tx=0x7f0828009e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efece6b5948d5934 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f08267fc640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:31:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f08267fc640 1 -- 172.21.15.112:0/118859150 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082803a060 con 0x5633d3ec0390 2024-09-21T03:31:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f08267fc640 1 -- 172.21.15.112:0/118859150 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f08280043e0 con 0x5633d3ec0390 2024-09-21T03:31:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f084ba4e740 1 -- 172.21.15.112:0/118859150 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083800fc50 con 0x5633d3ec0390 2024-09-21T03:31:15.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f08267fc640 1 -- 172.21.15.112:0/118859150 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f082801e040 con 0x5633d3ec0390 2024-09-21T03:31:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f084ba4e740 10 client.25505 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.321+0000 7f084ba4e740 1 -- 172.21.15.112:0/118859150 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5633d43100d0 con 0x5633d3ec0390 2024-09-21T03:31:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.322+0000 7f08267fc640 1 -- 172.21.15.112:0/118859150 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083800fc50 con 0x5633d3ec0390 2024-09-21T03:31:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.322+0000 7f08267fc640 1 -- 172.21.15.112:0/118859150 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f08280198b0 con 0x5633d3ec0390 2024-09-21T03:31:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.322+0000 7f084ba4e740 10 client.25505 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.322+0000 7f084ba4e740 10 client.25505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:31:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.322+0000 7f084ba4e740 20 client.25505 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.322+0000 7f084ba4e740 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5633d3e74da0 0x5633d4312820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.322+0000 7f084ba4e740 4 client.25505 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:31:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.322+0000 7f084ba4e740 1 -- 172.21.15.112:0/118859150 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5633d41fc850 con 0x5633d3e74da0 2024-09-21T03:31:15.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.322+0000 7f083effd640 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5633d3e74da0 0x5633d4312820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.322+0000 7f08267fc640 10 client.25505 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.322+0000 7f083effd640 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5633d3e74da0 0x5633d4312820 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99cde87f5380dcaa server_cookie=ceedb699f52afb43 in_seq=0 out_seq=0 2024-09-21T03:31:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f08267fc640 1 -- 172.21.15.112:0/118859150 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x5633d41fc850 con 0x5633d3e74da0 2024-09-21T03:31:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f08267fc640 10 client.25505 handle_command_reply: tid=0 2024-09-21T03:31:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 1 client.25505 shutdown 2024-09-21T03:31:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 1 -- 172.21.15.112:0/118859150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 msgr2=0x5633d4297d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d4297d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f082802e040 tx=0x7f0828009e10 comp rx=0 tx=0).stop 2024-09-21T03:31:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 1 -- 172.21.15.112:0/118859150 shutdown_connections 2024-09-21T03:31:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5633d3e74da0 0x5633d4312820 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633d3ee1070 0x5633d430e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633d3ec0390 0x5633d4297d90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 1 --2- 172.21.15.112:0/118859150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633d3eb81b0 0x5633d4297850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f08267fc640 0 client.25505 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:15.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 1 -- 172.21.15.112:0/118859150 >> 172.21.15.112:0/118859150 conn(0x5633d3ed5460 msgr2=0x5633d4289680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 1 -- 172.21.15.112:0/118859150 shutdown_connections 2024-09-21T03:31:15.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 1 -- 172.21.15.112:0/118859150 wait complete. 2024-09-21T03:31:15.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.323+0000 7f084ba4e740 20 client.25505 trim_cache size 0 max 16384 2024-09-21T03:31:15.331 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:31:15.331 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:31:15.332 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:15.333 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:31:15.334 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:31:15.335 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:15.335 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:15.335 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:15.335 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:15.335 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:15.335 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:31:15.335 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:31:15.335 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:31:15.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 -- 172.21.15.112:0/3878174069 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f082c0b6700 msgr2=0x7f082c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f082c0b6700 0x7f082c0b8c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0838012560 tx=0x7f0838005f90 comp rx=0 tx=0).stop 2024-09-21T03:31:15.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 -- 172.21.15.112:0/3878174069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840101fc0 msgr2=0x7f084019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:15.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840101fc0 0x7f084019a540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f083002ade0 tx=0x7f0830031660 comp rx=0 tx=0).stop 2024-09-21T03:31:15.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 -- 172.21.15.112:0/3878174069 shutdown_connections 2024-09-21T03:31:15.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f082c0b6700 0x7f082c0b8c10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0840109e50 0x7f084019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0840109500 0x7f084019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 --2- 172.21.15.112:0/3878174069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840101fc0 0x7f084019a540 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:15.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 -- 172.21.15.112:0/3878174069 >> 172.21.15.112:0/3878174069 conn(0x7f0840079f90 msgr2=0x7f0840107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:15.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 -- 172.21.15.112:0/3878174069 shutdown_connections 2024-09-21T03:31:15.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:15.324+0000 7f08454da640 1 -- 172.21.15.112:0/3878174069 wait complete. 2024-09-21T03:31:16.117 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:31:16.118 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:31:16.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 153005 11.10 MB/sec warmup 76 sec latency 10.065 ms 2024-09-21T03:31:16.382 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:16.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:16 smithi112 ceph-mon[31442]: pgmap v386: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2024-09-21T03:31:16.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.900+0000 7f9cdd462640 1 -- 172.21.15.112:0/2226202296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8108d30 msgr2=0x7f9cd810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.901+0000 7f9cdd462640 1 --2- 172.21.15.112:0/2226202296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8108d30 0x7f9cd810f560 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009ca0 tx=0x7f9cd002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.901+0000 7f9cdd462640 1 -- 172.21.15.112:0/2226202296 shutdown_connections 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.901+0000 7f9cdd462640 1 --2- 172.21.15.112:0/2226202296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd8108d30 0x7f9cd810f560 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.901+0000 7f9cdd462640 1 --2- 172.21.15.112:0/2226202296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cd81083e0 0x7f9cd81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.901+0000 7f9cdd462640 1 --2- 172.21.15.112:0/2226202296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cd8102aa0 0x7f9cd8102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.901+0000 7f9cdd462640 1 -- 172.21.15.112:0/2226202296 >> 172.21.15.112:0/2226202296 conn(0x7f9cd80797e0 msgr2=0x7f9cd8079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.901+0000 7f9cdd462640 1 -- 172.21.15.112:0/2226202296 shutdown_connections 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.901+0000 7f9cdd462640 1 -- 172.21.15.112:0/2226202296 wait complete. 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.901+0000 7f9cdd462640 1 Processor -- start 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.901+0000 7f9cdd462640 1 -- start start 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cdd462640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cd8102aa0 0x7f9cd80ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:16.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cdd462640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd81083e0 0x7f9cd80ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cdd462640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cd8108d30 0x7f9cd81000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cdd462640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd81129a0 con 0x7f9cd81083e0 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cdd462640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd8112820 con 0x7f9cd8102aa0 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cdd462640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd8112b20 con 0x7f9cd8108d30 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cce7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd81083e0 0x7f9cd80ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cce7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd81083e0 0x7f9cd80ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40470/0 (socket says 172.21.15.112:40470) 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cce7fc640 1 -- 172.21.15.112:0/644478336 learned_addr learned my addr 172.21.15.112:0/644478336 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cd6ffd640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cd8102aa0 0x7f9cd80ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cd77fe640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cd8108d30 0x7f9cd81000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cce7fc640 1 -- 172.21.15.112:0/644478336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cd8108d30 msgr2=0x7f9cd81000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cce7fc640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cd8108d30 0x7f9cd81000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cce7fc640 1 -- 172.21.15.112:0/644478336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cd8102aa0 msgr2=0x7f9cd80ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:16.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cce7fc640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cd8102aa0 0x7f9cd80ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cce7fc640 1 -- 172.21.15.112:0/644478336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd81a8720 con 0x7f9cd81083e0 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.902+0000 7f9cce7fc640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd81083e0 0x7f9cd80ffbb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000e9f0 tx=0x7f9cc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.903+0000 7f9cd4ff9640 1 -- 172.21.15.112:0/644478336 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000ccc0 con 0x7f9cd81083e0 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.903+0000 7f9cdd462640 1 -- 172.21.15.112:0/644478336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd81a8a10 con 0x7f9cd81083e0 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.903+0000 7f9cd4ff9640 1 -- 172.21.15.112:0/644478336 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc00054d0 con 0x7f9cd81083e0 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.903+0000 7f9cd4ff9640 1 -- 172.21.15.112:0/644478336 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0010430 con 0x7f9cd81083e0 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.903+0000 7f9cdd462640 1 -- 172.21.15.112:0/644478336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd81a8f50 con 0x7f9cd81083e0 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.904+0000 7f9cdd462640 1 -- 172.21.15.112:0/644478336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd8102eb0 con 0x7f9cd81083e0 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.906+0000 7f9cd4ff9640 1 -- 172.21.15.112:0/644478336 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9cc00046c0 con 0x7f9cd81083e0 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.906+0000 7f9cd4ff9640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9cc40b6680 0x7f9cc40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.907+0000 7f9cd6ffd640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9cc40b6680 0x7f9cc40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.907+0000 7f9cd4ff9640 1 -- 172.21.15.112:0/644478336 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f9cc00e7c20 con 0x7f9cd81083e0 2024-09-21T03:31:16.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.907+0000 7f9cd6ffd640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9cc40b6680 0x7f9cc40b8b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9cd81010c0 tx=0x7f9cc8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:16.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:16 smithi203 ceph-mon[34723]: pgmap v386: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2024-09-21T03:31:16.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:16.910+0000 7f9cd4ff9640 1 -- 172.21.15.112:0/644478336 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9cc0056710 con 0x7f9cd81083e0 2024-09-21T03:31:16.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:16 smithi187 ceph-mon[34603]: pgmap v386: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2024-09-21T03:31:17.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.135+0000 7f9cdd462640 1 -- 172.21.15.112:0/644478336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cd8100970 con 0x7f9cd81083e0 2024-09-21T03:31:17.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.136+0000 7f9cd4ff9640 1 -- 172.21.15.112:0/644478336 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11932 (secure 0 0 0) 0x7f9cc00ae1d0 con 0x7f9cd81083e0 2024-09-21T03:31:17.138 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:31:17.138 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"btime":"2024-09-21T03:30:19:484252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:30:19.046905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:31:17.138 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 -- 172.21.15.112:0/644478336 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9cc40b6680 msgr2=0x7f9cc40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9cc40b6680 0x7f9cc40b8b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9cd81010c0 tx=0x7f9cc8005f90 comp rx=0 tx=0).stop 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 -- 172.21.15.112:0/644478336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd81083e0 msgr2=0x7f9cd80ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd81083e0 0x7f9cd80ffbb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000e9f0 tx=0x7f9cc000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 -- 172.21.15.112:0/644478336 shutdown_connections 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9cc40b6680 0x7f9cc40b8b90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cd8108d30 0x7f9cd81000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cd81083e0 0x7f9cd80ffbb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 --2- 172.21.15.112:0/644478336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cd8102aa0 0x7f9cd80ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 -- 172.21.15.112:0/644478336 >> 172.21.15.112:0/644478336 conn(0x7f9cd80797e0 msgr2=0x7f9cd806bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 -- 172.21.15.112:0/644478336 shutdown_connections 2024-09-21T03:31:17.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:17.139+0000 7f9cdd462640 1 -- 172.21.15.112:0/644478336 wait complete. 2024-09-21T03:31:17.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 155442 11.11 MB/sec warmup 77 sec latency 30.223 ms 2024-09-21T03:31:17.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:17 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/644478336' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:17.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:17 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/644478336' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:17.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:17 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/644478336' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:31:18.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 157000 11.07 MB/sec warmup 78 sec latency 27.650 ms 2024-09-21T03:31:18.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:31:18.545 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:18 smithi112 ceph-mon[31442]: pgmap v387: 97 pgs: 97 active+clean; 98 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 296 op/s 2024-09-21T03:31:18.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:18 smithi203 ceph-mon[34723]: pgmap v387: 97 pgs: 97 active+clean; 98 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 296 op/s 2024-09-21T03:31:18.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:18 smithi187 ceph-mon[34603]: pgmap v387: 97 pgs: 97 active+clean; 98 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 296 op/s 2024-09-21T03:31:19.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.046+0000 7f38b5140640 1 -- 172.21.15.112:0/2225901502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b010de30 msgr2=0x7f38b0110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:19.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.046+0000 7f38b5140640 1 --2- 172.21.15.112:0/2225901502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b010de30 0x7f38b0110320 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009ca0 tx=0x7f38a802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.046+0000 7f38b5140640 1 -- 172.21.15.112:0/2225901502 shutdown_connections 2024-09-21T03:31:19.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.046+0000 7f38b5140640 1 --2- 172.21.15.112:0/2225901502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b010de30 0x7f38b0110320 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.046+0000 7f38b5140640 1 --2- 172.21.15.112:0/2225901502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b0105320 0x7f38b010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.046+0000 7f38b5140640 1 --2- 172.21.15.112:0/2225901502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38b01049d0 0x7f38b0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.046+0000 7f38b5140640 1 -- 172.21.15.112:0/2225901502 >> 172.21.15.112:0/2225901502 conn(0x7f38b0079740 msgr2=0x7f38b0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.047+0000 7f38b5140640 1 -- 172.21.15.112:0/2225901502 shutdown_connections 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.047+0000 7f38b5140640 1 -- 172.21.15.112:0/2225901502 wait complete. 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.047+0000 7f38b5140640 1 Processor -- start 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38b5140640 1 -- start start 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38b5140640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b01049d0 0x7f38b019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38b5140640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38b0105320 0x7f38b019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38b5140640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b010de30 0x7f38b019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38b5140640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0113ab0 con 0x7f38b01049d0 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38b5140640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0113930 con 0x7f38b010de30 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38b5140640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0113c30 con 0x7f38b0105320 2024-09-21T03:31:19.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b01049d0 0x7f38b019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b01049d0 0x7f38b019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40478/0 (socket says 172.21.15.112:40478) 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38aed9d640 1 -- 172.21.15.112:0/652579785 learned_addr learned my addr 172.21.15.112:0/652579785 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38af59e640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b010de30 0x7f38b019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38ae59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38b0105320 0x7f38b019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38aed9d640 1 -- 172.21.15.112:0/652579785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38b0105320 msgr2=0x7f38b019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38aed9d640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38b0105320 0x7f38b019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38aed9d640 1 -- 172.21.15.112:0/652579785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b010de30 msgr2=0x7f38b019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38aed9d640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b010de30 0x7f38b019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38aed9d640 1 -- 172.21.15.112:0/652579785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b01a03f0 con 0x7f38b01049d0 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38af59e640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b010de30 0x7f38b019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.048+0000 7f38ae59c640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38b0105320 0x7f38b019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.049+0000 7f38aed9d640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b01049d0 0x7f38b019a480 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f38a000e9c0 tx=0x7f38a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:19.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.049+0000 7f3897fff640 1 -- 172.21.15.112:0/652579785 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a000cd80 con 0x7f38b01049d0 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.049+0000 7f38b5140640 1 -- 172.21.15.112:0/652579785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b01a06e0 con 0x7f38b01049d0 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.049+0000 7f3897fff640 1 -- 172.21.15.112:0/652579785 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a00035f0 con 0x7f38b01049d0 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.049+0000 7f3897fff640 1 -- 172.21.15.112:0/652579785 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0010690 con 0x7f38b01049d0 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.049+0000 7f38b5140640 1 -- 172.21.15.112:0/652579785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b01a6e40 con 0x7f38b01049d0 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.052+0000 7f38bb6b4740 1 Processor -- start 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.052+0000 7f38bb6b4740 1 -- start start 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.052+0000 7f3897fff640 1 -- 172.21.15.112:0/652579785 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f38a0010830 con 0x7f38b01049d0 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.052+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b76e070 0x564e0bb152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.052+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 0x564e0bb15800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.052+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b74d390 0x564e0bb15d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.052+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e0ba8bea0 con 0x564e0b7451b0 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.052+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564e0ba7b2d0 con 0x564e0b74d390 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.052+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564e0ba7f0e0 con 0x564e0b76e070 2024-09-21T03:31:19.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 0x564e0bb15800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 0x564e0bb15800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40484/0 (socket says 172.21.15.112:40484) 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38aed9d640 1 -- 172.21.15.112:0/2986785468 learned_addr learned my addr 172.21.15.112:0/2986785468 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38ae59c640 1 --2- 172.21.15.112:0/2986785468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b74d390 0x564e0bb15d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38af59e640 1 --2- 172.21.15.112:0/2986785468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b76e070 0x564e0bb152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38af59e640 1 -- 172.21.15.112:0/2986785468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b74d390 msgr2=0x564e0bb15d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38af59e640 1 --2- 172.21.15.112:0/2986785468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b74d390 0x564e0bb15d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38af59e640 1 -- 172.21.15.112:0/2986785468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 msgr2=0x564e0bb15800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38af59e640 1 --2- 172.21.15.112:0/2986785468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 0x564e0bb15800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38af59e640 1 -- 172.21.15.112:0/2986785468 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e0b7d9650 con 0x564e0b76e070 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38aed9d640 1 --2- 172.21.15.112:0/2986785468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 0x564e0bb15800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.053+0000 7f38ae59c640 1 --2- 172.21.15.112:0/2986785468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b74d390 0x564e0bb15d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.054+0000 7f38af59e640 1 --2- 172.21.15.112:0/2986785468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b76e070 0x564e0bb152c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38a802ade0 tx=0x7f38a802f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f13baa2a4356b95e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:19.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.054+0000 7f3897fff640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38980b6680 0x7f38980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.054+0000 7f38957fa640 1 -- 172.21.15.112:0/2986785468 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8031650 con 0x564e0b76e070 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.054+0000 7f38957fa640 1 -- 172.21.15.112:0/2986785468 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a8036040 con 0x564e0b76e070 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.054+0000 7f38957fa640 1 -- 172.21.15.112:0/2986785468 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8004360 con 0x564e0b76e070 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.054+0000 7f38aed9d640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38980b6680 0x7f38980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.054+0000 7f3897fff640 1 -- 172.21.15.112:0/652579785 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f38a00e7c20 con 0x7f38b01049d0 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.055+0000 7f38bb6b4740 1 -- 172.21.15.112:0/2986785468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b76e070 msgr2=0x564e0bb152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.055+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/2986785468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b76e070 0x564e0bb152c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38a802ade0 tx=0x7f38a802f310 comp rx=0 tx=0).stop 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.055+0000 7f38aed9d640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38980b6680 0x7f38980b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f38a0003e90 tx=0x7f38a00eb770 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.055+0000 7f38bb6b4740 1 -- 172.21.15.112:0/2986785468 shutdown_connections 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.055+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/2986785468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b74d390 0x564e0bb15d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.055+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/2986785468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 0x564e0bb15800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.055+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/2986785468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b76e070 0x564e0bb152c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.055+0000 7f38bb6b4740 1 -- 172.21.15.112:0/2986785468 >> 172.21.15.112:0/2986785468 conn(0x564e0b762460 msgr2=0x564e0bb131b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:19.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.055+0000 7f38bb6b4740 1 -- 172.21.15.112:0/2986785468 shutdown_connections 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.055+0000 7f38bb6b4740 1 -- 172.21.15.112:0/2986785468 wait complete. 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.056+0000 7f38bb6b4740 1 -- start start 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.056+0000 7f38bb6b4740 1 Processor -- start 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.056+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 0x564e0bb24da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.056+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 0x564e0bb252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.056+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b76e070 0x564e0bb30f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.056+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e0b8dc700 con 0x564e0b7451b0 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.056+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564e0ba16160 con 0x564e0b76e070 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564e0bb27bf0 con 0x564e0b74d390 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38af59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 0x564e0bb24da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38ae59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 0x564e0bb252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38ae59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 0x564e0bb252e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:41420/0 (socket says 172.21.15.112:41420) 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38ae59c640 1 -- 172.21.15.112:0/3883336004 learned_addr learned my addr 172.21.15.112:0/3883336004 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38bb6b4740 10 client.0 _ll_register_callbacks cb 0x564e0ba2f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38af59e640 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b76e070 0x564e0bb30f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38ae59c640 1 -- 172.21.15.112:0/3883336004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b76e070 msgr2=0x564e0bb30f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38ae59c640 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b76e070 0x564e0bb30f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38ae59c640 1 -- 172.21.15.112:0/3883336004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 msgr2=0x564e0bb24da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38ae59c640 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 0x564e0bb24da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38ae59c640 1 -- 172.21.15.112:0/3883336004 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e0ba43c90 con 0x564e0b74d390 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38af59e640 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b76e070 0x564e0bb30f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f38ae59c640 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 0x564e0bb252e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f38a4012d20 tx=0x7f38a400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba669bb063571ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f3892ff5640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.057+0000 7f3892ff5640 1 -- 172.21.15.112:0/3883336004 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a40120d0 con 0x564e0b74d390 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f38bb6b4740 1 -- 172.21.15.112:0/3883336004 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a8036040 con 0x564e0b74d390 2024-09-21T03:31:19.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f3892ff5640 1 -- 172.21.15.112:0/3883336004 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f38a4004030 con 0x564e0b74d390 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f3892ff5640 1 -- 172.21.15.112:0/3883336004 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38a4012270 con 0x564e0b74d390 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f38bb6b4740 10 client.25522 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f38bb6b4740 1 -- 172.21.15.112:0/3883336004 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e0bb339a0 con 0x564e0b74d390 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f3892ff5640 1 -- 172.21.15.112:0/3883336004 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a8036040 con 0x564e0b74d390 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f3892ff5640 1 -- 172.21.15.112:0/3883336004 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f38a4026470 con 0x564e0b74d390 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f38bb6b4740 10 client.25522 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f38bb6b4740 10 client.25522 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f38bb6b4740 20 client.25522 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x564e0b72c9f0 0x564e0bb36fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f38bb6b4740 4 client.25522 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.058+0000 7f38bb6b4740 1 -- 172.21.15.112:0/3883336004 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e0ba8ddf0 con 0x564e0b72c9f0 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.059+0000 7f38af59e640 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x564e0b72c9f0 0x564e0bb36fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.059+0000 7f3892ff5640 10 client.25522 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:31:19.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.059+0000 7f38af59e640 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x564e0b72c9f0 0x564e0bb36fd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2e24987810c5c59 server_cookie=78eb98dc035487df in_seq=0 out_seq=0 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f3892ff5640 1 -- 172.21.15.112:0/3883336004 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e0ba8ddf0 con 0x564e0b72c9f0 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f3892ff5640 10 client.25522 handle_command_reply: tid=0 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f38bb6b4740 1 client.25522 shutdown 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f38bb6b4740 1 -- 172.21.15.112:0/3883336004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 msgr2=0x564e0bb252e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 0x564e0bb252e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f38a4012d20 tx=0x7f38a400f790 comp rx=0 tx=0).stop 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f38bb6b4740 1 -- 172.21.15.112:0/3883336004 shutdown_connections 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x564e0b72c9f0 0x564e0bb36fd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b76e070 0x564e0bb30f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 0x564e0bb252e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/3883336004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b7451b0 0x564e0bb24da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f3892ff5640 0 client.25522 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:31:19.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f38bb6b4740 1 -- 172.21.15.112:0/3883336004 >> 172.21.15.112:0/3883336004 conn(0x564e0b762460 msgr2=0x564e0bb18450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:19.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f38bb6b4740 1 -- 172.21.15.112:0/3883336004 shutdown_connections 2024-09-21T03:31:19.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.060+0000 7f38bb6b4740 1 -- 172.21.15.112:0/3883336004 wait complete. 2024-09-21T03:31:19.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.061+0000 7f38bb6b4740 20 client.25522 trim_cache size 0 max 16384 2024-09-21T03:31:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38bb6b4740 1 Processor -- start 2024-09-21T03:31:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38bb6b4740 1 -- start start 2024-09-21T03:31:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b7451b0 0x564e0bb16a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b74d390 0x564e0bb21f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b76e070 0x564e0bb224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01afc50 con 0x564e0b74d390 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01ad520 con 0x564e0b76e070 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01ab390 con 0x564e0b7451b0 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b74d390 0x564e0bb21f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b74d390 0x564e0bb21f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40500/0 (socket says 172.21.15.112:40500) 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38ae59c640 1 -- 172.21.15.112:0/391559930 learned_addr learned my addr 172.21.15.112:0/391559930 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38af59e640 1 --2- 172.21.15.112:0/391559930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b76e070 0x564e0bb224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38af59e640 1 --2- 172.21.15.112:0/391559930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b7451b0 0x564e0bb16a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38ae59c640 1 -- 172.21.15.112:0/391559930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b7451b0 msgr2=0x564e0bb16a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38ae59c640 1 --2- 172.21.15.112:0/391559930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b7451b0 0x564e0bb16a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38ae59c640 1 -- 172.21.15.112:0/391559930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b76e070 msgr2=0x564e0bb224d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38ae59c640 1 --2- 172.21.15.112:0/391559930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b76e070 0x564e0bb224d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38ae59c640 1 -- 172.21.15.112:0/391559930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e0bb278d0 con 0x564e0b74d390 2024-09-21T03:31:19.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.079+0000 7f38ae59c640 1 --2- 172.21.15.112:0/391559930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b74d390 0x564e0bb21f90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f38a4012870 tx=0x7f38a400fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dccc778c746af53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:19.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.080+0000 7f3894ff9640 1 -- 172.21.15.112:0/391559930 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4026470 con 0x564e0b74d390 2024-09-21T03:31:19.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.080+0000 7f3894ff9640 1 -- 172.21.15.112:0/391559930 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a4026610 con 0x564e0b74d390 2024-09-21T03:31:19.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.080+0000 7f3894ff9640 1 -- 172.21.15.112:0/391559930 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a401c970 con 0x564e0b74d390 2024-09-21T03:31:19.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.081+0000 7f38bb6b4740 1 -- 172.21.15.112:0/391559930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b74d390 msgr2=0x564e0bb21f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:19.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.081+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/391559930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b74d390 0x564e0bb21f90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f38a4012870 tx=0x7f38a400fd30 comp rx=0 tx=0).stop 2024-09-21T03:31:19.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.081+0000 7f38bb6b4740 1 -- 172.21.15.112:0/391559930 shutdown_connections 2024-09-21T03:31:19.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.081+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/391559930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b76e070 0x564e0bb224d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.081+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/391559930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b74d390 0x564e0bb21f90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.081+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/391559930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b7451b0 0x564e0bb16a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.081+0000 7f38bb6b4740 1 -- 172.21.15.112:0/391559930 >> 172.21.15.112:0/391559930 conn(0x564e0b762460 msgr2=0x564e0bb162f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.081+0000 7f38bb6b4740 1 -- 172.21.15.112:0/391559930 shutdown_connections 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.081+0000 7f38bb6b4740 1 -- 172.21.15.112:0/391559930 wait complete. 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.081+0000 7f38bb6b4740 1 -- start start 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.081+0000 7f38bb6b4740 1 Processor -- start 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b7451b0 0x564e0bb2d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 0x564e0bb2acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38bb6b4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b76e070 0x564e0bb2b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01ab9e0 con 0x564e0b76e070 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01ace50 con 0x564e0b7451b0 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38bb6b4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01ac0b0 con 0x564e0b74d390 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38bb6b4740 10 client.0 _ll_register_callbacks cb 0x7f38a4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38af59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b76e070 0x564e0bb2b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38ae59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 0x564e0bb2acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38af59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b76e070 0x564e0bb2b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40504/0 (socket says 172.21.15.112:40504) 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38af59e640 1 -- 172.21.15.112:0/570944514 learned_addr learned my addr 172.21.15.112:0/570944514 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38af59e640 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b7451b0 0x564e0bb2d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38af59e640 1 -- 172.21.15.112:0/570944514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 msgr2=0x564e0bb2acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38af59e640 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 0x564e0bb2acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38af59e640 1 -- 172.21.15.112:0/570944514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b7451b0 msgr2=0x564e0bb2d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38af59e640 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b7451b0 0x564e0bb2d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38af59e640 1 -- 172.21.15.112:0/570944514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e0bb0d3b0 con 0x564e0b76e070 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38af59e640 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b76e070 0x564e0bb2b360 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f38a8005d50 tx=0x7f38a8054040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f90a95726affca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f3892ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f3892ff5640 1 -- 172.21.15.112:0/570944514 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a802f720 con 0x564e0b76e070 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f3892ff5640 1 -- 172.21.15.112:0/570944514 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f38a802f8c0 con 0x564e0b76e070 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.082+0000 7f38bb6b4740 1 -- 172.21.15.112:0/570944514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a4026610 con 0x564e0b76e070 2024-09-21T03:31:19.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f3892ff5640 1 -- 172.21.15.112:0/570944514 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38a8004b40 con 0x564e0b76e070 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f38bb6b4740 10 client.15960 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f38bb6b4740 1 -- 172.21.15.112:0/570944514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e0bba1870 con 0x564e0b76e070 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f3892ff5640 1 -- 172.21.15.112:0/570944514 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a4026610 con 0x564e0b76e070 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f3892ff5640 1 -- 172.21.15.112:0/570944514 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f38a8044070 con 0x564e0b76e070 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f38bb6b4740 10 client.15960 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f38bb6b4740 10 client.15960 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f38bb6b4740 20 client.15960 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x564e0b701da0 0x564e0bba3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f38bb6b4740 4 client.15960 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f38bb6b4740 1 -- 172.21.15.112:0/570944514 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e0ba898a0 con 0x564e0b701da0 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.083+0000 7f38ae59c640 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x564e0b701da0 0x564e0bba3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f3892ff5640 10 client.15960 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38ae59c640 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x564e0b701da0 0x564e0bba3db0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e609a7bb6fb8340 server_cookie=6eed4f3c80269704 in_seq=0 out_seq=0 2024-09-21T03:31:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f3892ff5640 1 -- 172.21.15.112:0/570944514 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e0ba898a0 con 0x564e0b701da0 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f3892ff5640 10 client.15960 handle_command_reply: tid=0 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 1 client.15960 shutdown 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 1 -- 172.21.15.112:0/570944514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b76e070 msgr2=0x564e0bb2b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b76e070 0x564e0bb2b360 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f38a8005d50 tx=0x7f38a8054040 comp rx=0 tx=0).stop 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 1 -- 172.21.15.112:0/570944514 shutdown_connections 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x564e0b701da0 0x564e0bba3db0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e0b76e070 0x564e0bb2b360 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564e0b74d390 0x564e0bb2acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f3892ff5640 0 client.15960 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 1 --2- 172.21.15.112:0/570944514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e0b7451b0 0x564e0bb2d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 1 -- 172.21.15.112:0/570944514 >> 172.21.15.112:0/570944514 conn(0x564e0b762460 msgr2=0x564e0bb261b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 1 -- 172.21.15.112:0/570944514 shutdown_connections 2024-09-21T03:31:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 1 -- 172.21.15.112:0/570944514 wait complete. 2024-09-21T03:31:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.084+0000 7f38bb6b4740 20 client.15960 trim_cache size 0 max 16384 2024-09-21T03:31:19.092 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:31:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 -- 172.21.15.112:0/652579785 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38980b6680 msgr2=0x7f38980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38980b6680 0x7f38980b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f38a0003e90 tx=0x7f38a00eb770 comp rx=0 tx=0).stop 2024-09-21T03:31:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 -- 172.21.15.112:0/652579785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b01049d0 msgr2=0x7f38b019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b01049d0 0x7f38b019a480 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f38a000e9c0 tx=0x7f38a000ee80 comp rx=0 tx=0).stop 2024-09-21T03:31:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 -- 172.21.15.112:0/652579785 shutdown_connections 2024-09-21T03:31:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f38980b6680 0x7f38980b8b90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b010de30 0x7f38b019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38b0105320 0x7f38b019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 --2- 172.21.15.112:0/652579785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b01049d0 0x7f38b019a480 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 -- 172.21.15.112:0/652579785 >> 172.21.15.112:0/652579785 conn(0x7f38b0079740 msgr2=0x7f38b00ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 -- 172.21.15.112:0/652579785 shutdown_connections 2024-09-21T03:31:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:19.085+0000 7f38b5140640 1 -- 172.21.15.112:0/652579785 wait complete. 2024-09-21T03:31:19.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 159849 11.15 MB/sec warmup 79 sec latency 39.077 ms 2024-09-21T03:31:19.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:31:20.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 161414 11.09 MB/sec warmup 80 sec latency 16.074 ms 2024-09-21T03:31:20.250 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:20.714 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:20 smithi112 ceph-mon[31442]: pgmap v388: 97 pgs: 97 active+clean; 100 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 298 op/s 2024-09-21T03:31:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.809+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2639720432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c01083a0 msgr2=0x7f96c01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.809+0000 7f96c4fbf640 1 --2- 172.21.15.112:0/2639720432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c01083a0 0x7f96c01087b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009900 tx=0x7f96b402a410 comp rx=0 tx=0).stop 2024-09-21T03:31:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.809+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2639720432 shutdown_connections 2024-09-21T03:31:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.809+0000 7f96c4fbf640 1 --2- 172.21.15.112:0/2639720432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96c0108cf0 0x7f96c010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.809+0000 7f96c4fbf640 1 --2- 172.21.15.112:0/2639720432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c01083a0 0x7f96c01087b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.809+0000 7f96c4fbf640 1 --2- 172.21.15.112:0/2639720432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c0102a60 0x7f96c0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.809+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2639720432 >> 172.21.15.112:0/2639720432 conn(0x7f96c0079730 msgr2=0x7f96c0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.809+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2639720432 shutdown_connections 2024-09-21T03:31:20.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.809+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2639720432 wait complete. 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96c4fbf640 1 Processor -- start 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96c4fbf640 1 -- start start 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96c4fbf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c0102a60 0x7f96c019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96c4fbf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c01083a0 0x7f96c019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96c4fbf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96c0108cf0 0x7f96c019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96c4fbf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0113a80 con 0x7f96c01083a0 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96c4fbf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0113900 con 0x7f96c0108cf0 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96c4fbf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96c0113c00 con 0x7f96c0102a60 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c01083a0 0x7f96c019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96be59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c0102a60 0x7f96c019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96be59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c0102a60 0x7f96c019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:41466/0 (socket says 172.21.15.112:41466) 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96be59c640 1 -- 172.21.15.112:0/2704447135 learned_addr learned my addr 172.21.15.112:0/2704447135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c01083a0 0x7f96c019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40534/0 (socket says 172.21.15.112:40534) 2024-09-21T03:31:20.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.810+0000 7f96bed9d640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96c0108cf0 0x7f96c019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96be59c640 1 -- 172.21.15.112:0/2704447135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96c0108cf0 msgr2=0x7f96c019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96be59c640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96c0108cf0 0x7f96c019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96be59c640 1 -- 172.21.15.112:0/2704447135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c01083a0 msgr2=0x7f96c019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96be59c640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c01083a0 0x7f96c019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96be59c640 1 -- 172.21.15.112:0/2704447135 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c01a0500 con 0x7f96c0102a60 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96bed9d640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96c0108cf0 0x7f96c019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96be59c640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c0102a60 0x7f96c019a590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f96b000e9f0 tx=0x7f96b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96af7fe640 1 -- 172.21.15.112:0/2704447135 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b000cd80 con 0x7f96c0102a60 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2704447135 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c01a07f0 con 0x7f96c0102a60 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96af7fe640 1 -- 172.21.15.112:0/2704447135 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96b00054d0 con 0x7f96c0102a60 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96af7fe640 1 -- 172.21.15.112:0/2704447135 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0010690 con 0x7f96c0102a60 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.811+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2704447135 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c01a6f50 con 0x7f96c0102a60 2024-09-21T03:31:20.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.814+0000 7f96cb533740 1 Processor -- start 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.814+0000 7f96cb533740 1 -- start start 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96cb533740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb172070 0x5558bb5196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96cb533740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb519be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96cb533740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 0x5558bb51a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96cb533740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558bb48fea0 con 0x5558bb151390 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96cb533740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5558bb47f2d0 con 0x5558bb1491b0 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96cb533740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5558bb4830e0 con 0x5558bb172070 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96af7fe640 1 -- 172.21.15.112:0/2704447135 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f96b0010830 con 0x7f96c0102a60 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 0x5558bb51a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb172070 0x5558bb5196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb172070 0x5558bb5196a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:41478/0 (socket says 172.21.15.112:41478) 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 0x5558bb51a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40542/0 (socket says 172.21.15.112:40542) 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96bed9d640 1 -- 172.21.15.112:0/4188669684 learned_addr learned my addr 172.21.15.112:0/4188669684 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:20.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96be59c640 1 --2- 172.21.15.112:0/4188669684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb519be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96bed9d640 1 -- 172.21.15.112:0/4188669684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 msgr2=0x5558bb519be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96bed9d640 1 --2- 172.21.15.112:0/4188669684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb519be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96bed9d640 1 -- 172.21.15.112:0/4188669684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 msgr2=0x5558bb51a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96bed9d640 1 --2- 172.21.15.112:0/4188669684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 0x5558bb51a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96bed9d640 1 -- 172.21.15.112:0/4188669684 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558bb1dd650 con 0x5558bb172070 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.815+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/4188669684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 0x5558bb51a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.816+0000 7f96bed9d640 1 --2- 172.21.15.112:0/4188669684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb172070 0x5558bb5196a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009fd0 tx=0x7f96b8010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f53e7544da1402a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.816+0000 7f96af7fe640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96a80b6680 0x7f96a80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.816+0000 7f96acff9640 1 -- 172.21.15.112:0/4188669684 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8011700 con 0x5558bb172070 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.816+0000 7f96be59c640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96a80b6680 0x7f96a80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.816+0000 7f96acff9640 1 -- 172.21.15.112:0/4188669684 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96b8011d50 con 0x5558bb172070 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.816+0000 7f96acff9640 1 -- 172.21.15.112:0/4188669684 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b800e3c0 con 0x5558bb172070 2024-09-21T03:31:20.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.816+0000 7f96af7fe640 1 -- 172.21.15.112:0/2704447135 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f96b00e7b70 con 0x7f96c0102a60 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.817+0000 7f96cb533740 1 -- 172.21.15.112:0/4188669684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb172070 msgr2=0x5558bb5196a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.817+0000 7f96be59c640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96a80b6680 0x7f96a80b8b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f96b00e53d0 tx=0x7f96b00eb6c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.817+0000 7f96cb533740 1 --2- 172.21.15.112:0/4188669684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb172070 0x5558bb5196a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009fd0 tx=0x7f96b8010860 comp rx=0 tx=0).stop 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.817+0000 7f96cb533740 1 -- 172.21.15.112:0/4188669684 shutdown_connections 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.817+0000 7f96cb533740 1 --2- 172.21.15.112:0/4188669684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 0x5558bb51a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.817+0000 7f96cb533740 1 --2- 172.21.15.112:0/4188669684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb519be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.817+0000 7f96cb533740 1 --2- 172.21.15.112:0/4188669684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb172070 0x5558bb5196a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.817+0000 7f96cb533740 1 -- 172.21.15.112:0/4188669684 >> 172.21.15.112:0/4188669684 conn(0x5558bb166460 msgr2=0x5558bb517590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.817+0000 7f96cb533740 1 -- 172.21.15.112:0/4188669684 shutdown_connections 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.817+0000 7f96cb533740 1 -- 172.21.15.112:0/4188669684 wait complete. 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.817+0000 7f96cb533740 1 -- start start 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96cb533740 1 Processor -- start 2024-09-21T03:31:20.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96cb533740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb524d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96cb533740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb151390 0x5558bb5252b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96cb533740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb172070 0x5558bb531170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96cb533740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558bb2e0700 con 0x5558bb172070 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96cb533740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5558bb41a160 con 0x5558bb1491b0 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96cb533740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5558bb527e80 con 0x5558bb151390 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96cb533740 10 client.0 _ll_register_callbacks cb 0x5558bb4339e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb524d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb524d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:57074/0 (socket says 172.21.15.112:57074) 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96bed9d640 1 -- 172.21.15.112:0/123915956 learned_addr learned my addr 172.21.15.112:0/123915956 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb151390 0x5558bb5252b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96bed9d640 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb172070 0x5558bb531170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96bdd9b640 1 -- 172.21.15.112:0/123915956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 msgr2=0x5558bb524d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:20.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb524d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96bdd9b640 1 -- 172.21.15.112:0/123915956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb172070 msgr2=0x5558bb531170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb172070 0x5558bb531170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.818+0000 7f96bdd9b640 1 -- 172.21.15.112:0/123915956 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558bb447c90 con 0x5558bb151390 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.819+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb151390 0x5558bb5252b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f96b4006010 tx=0x7f96b4032e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e107e44a161d945a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.819+0000 7f96a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.819+0000 7f96a5ffb640 1 -- 172.21.15.112:0/123915956 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4035040 con 0x5558bb151390 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.819+0000 7f96a5ffb640 1 -- 172.21.15.112:0/123915956 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f96b40316a0 con 0x5558bb151390 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.819+0000 7f96cb533740 1 -- 172.21.15.112:0/123915956 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b8011d50 con 0x5558bb151390 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.819+0000 7f96a5ffb640 1 -- 172.21.15.112:0/123915956 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96b40318a0 con 0x5558bb151390 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.819+0000 7f96cb533740 10 client.25540 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.819+0000 7f96cb533740 1 -- 172.21.15.112:0/123915956 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5558bb533ba0 con 0x5558bb151390 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.819+0000 7f96a5ffb640 1 -- 172.21.15.112:0/123915956 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96b8011d50 con 0x5558bb151390 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.820+0000 7f96a5ffb640 1 -- 172.21.15.112:0/123915956 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f96b4004750 con 0x5558bb151390 2024-09-21T03:31:20.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.820+0000 7f96cb533740 10 client.25540 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.820+0000 7f96cb533740 10 client.25540 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.820+0000 7f96cb533740 20 client.25540 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.820+0000 7f96cb533740 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5558bb1309f0 0x5558bb537010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.820+0000 7f96cb533740 4 client.25540 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.820+0000 7f96cb533740 1 -- 172.21.15.112:0/123915956 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558bb491df0 con 0x5558bb1309f0 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.820+0000 7f96bed9d640 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5558bb1309f0 0x5558bb537010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.820+0000 7f96bed9d640 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5558bb1309f0 0x5558bb537010 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa30c5e9a7626dc2 server_cookie=745c59c4e4a3818c in_seq=0 out_seq=0 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.820+0000 7f96a5ffb640 10 client.25540 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96a5ffb640 1 -- 172.21.15.112:0/123915956 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5558bb491df0 con 0x5558bb1309f0 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96a5ffb640 10 client.25540 handle_command_reply: tid=0 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 1 client.25540 shutdown 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 1 -- 172.21.15.112:0/123915956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb151390 msgr2=0x5558bb5252b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb151390 0x5558bb5252b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f96b4006010 tx=0x7f96b4032e70 comp rx=0 tx=0).stop 2024-09-21T03:31:20.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 1 -- 172.21.15.112:0/123915956 shutdown_connections 2024-09-21T03:31:20.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5558bb1309f0 0x5558bb537010 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96a5ffb640 0 client.25540 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:31:20.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb172070 0x5558bb531170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb151390 0x5558bb5252b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 1 --2- 172.21.15.112:0/123915956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb524d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 1 -- 172.21.15.112:0/123915956 >> 172.21.15.112:0/123915956 conn(0x5558bb166460 msgr2=0x5558bb51c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:20.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 1 -- 172.21.15.112:0/123915956 shutdown_connections 2024-09-21T03:31:20.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 1 -- 172.21.15.112:0/123915956 wait complete. 2024-09-21T03:31:20.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.821+0000 7f96cb533740 20 client.25540 trim_cache size 0 max 16384 2024-09-21T03:31:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.835+0000 7f96cb533740 1 Processor -- start 2024-09-21T03:31:20.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.835+0000 7f96cb533740 1 -- start start 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96cb533740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 0x5558bb5252f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96cb533740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb1491b0 0x5558bb5175c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96cb533740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb151390 0x5558bb531070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96cb533740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01b1ef0 con 0x5558bb1491b0 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96cb533740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01af7c0 con 0x5558bb151390 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96cb533740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01ad610 con 0x5558bb1309f0 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb1491b0 0x5558bb5175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb1491b0 0x5558bb5175c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40554/0 (socket says 172.21.15.112:40554) 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bdd9b640 1 -- 172.21.15.112:0/649468509 learned_addr learned my addr 172.21.15.112:0/649468509 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bed9d640 1 --2- 172.21.15.112:0/649468509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb151390 0x5558bb531070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bed9d640 1 --2- 172.21.15.112:0/649468509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 0x5558bb5252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bdd9b640 1 -- 172.21.15.112:0/649468509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 msgr2=0x5558bb5252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/649468509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 0x5558bb5252f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bdd9b640 1 -- 172.21.15.112:0/649468509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb151390 msgr2=0x5558bb531070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:20.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/649468509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb151390 0x5558bb531070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bdd9b640 1 -- 172.21.15.112:0/649468509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558bb523270 con 0x5558bb1491b0 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bed9d640 1 --2- 172.21.15.112:0/649468509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb151390 0x5558bb531070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.836+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/649468509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb1491b0 0x5558bb5175c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f96b4000c00 tx=0x7f96b4031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bcc97263aa37bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.837+0000 7f96a7fff640 1 -- 172.21.15.112:0/649468509 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4035070 con 0x5558bb1491b0 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.837+0000 7f96a7fff640 1 -- 172.21.15.112:0/649468509 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96b40048c0 con 0x5558bb1491b0 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.837+0000 7f96a7fff640 1 -- 172.21.15.112:0/649468509 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4005410 con 0x5558bb1491b0 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 -- 172.21.15.112:0/649468509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb1491b0 msgr2=0x5558bb5175c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 --2- 172.21.15.112:0/649468509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb1491b0 0x5558bb5175c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f96b4000c00 tx=0x7f96b4031630 comp rx=0 tx=0).stop 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 -- 172.21.15.112:0/649468509 shutdown_connections 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 --2- 172.21.15.112:0/649468509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb151390 0x5558bb531070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 --2- 172.21.15.112:0/649468509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb1491b0 0x5558bb5175c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 --2- 172.21.15.112:0/649468509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 0x5558bb5252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 -- 172.21.15.112:0/649468509 >> 172.21.15.112:0/649468509 conn(0x5558bb166460 msgr2=0x5558bb516eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 -- 172.21.15.112:0/649468509 shutdown_connections 2024-09-21T03:31:20.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 -- 172.21.15.112:0/649468509 wait complete. 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 -- start start 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 Processor -- start 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 0x5558bb52d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb528e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 0x5558bb529420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01adc80 con 0x5558bb151390 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01af0f0 con 0x5558bb1491b0 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.838+0000 7f96cb533740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96c01ae350 con 0x5558bb1309f0 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96cb533740 10 client.0 _ll_register_callbacks cb 0x7f96b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 0x5558bb52d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 0x5558bb529420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 0x5558bb52d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:41498/0 (socket says 172.21.15.112:41498) 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bed9d640 1 -- 172.21.15.112:0/3670555472 learned_addr learned my addr 172.21.15.112:0/3670555472 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb528e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bed9d640 1 -- 172.21.15.112:0/3670555472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 msgr2=0x5558bb528e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bed9d640 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb528e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bed9d640 1 -- 172.21.15.112:0/3670555472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 msgr2=0x5558bb529420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bed9d640 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 0x5558bb529420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bed9d640 1 -- 172.21.15.112:0/3670555472 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558bb529c00 con 0x5558bb1309f0 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb528e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96bed9d640 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 0x5558bb52d970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f96b801ba30 tx=0x7f96b4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d368d18f3f8c6b53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96a5ffb640 1 -- 172.21.15.112:0/3670555472 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8004c90 con 0x5558bb1309f0 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.839+0000 7f96cb533740 1 -- 172.21.15.112:0/3670555472 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b40048c0 con 0x5558bb1309f0 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96a5ffb640 1 -- 172.21.15.112:0/3670555472 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f96b80116b0 con 0x5558bb1309f0 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96a5ffb640 1 -- 172.21.15.112:0/3670555472 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96b80118b0 con 0x5558bb1309f0 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96cb533740 10 client.25546 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96cb533740 1 -- 172.21.15.112:0/3670555472 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5558bb5a19c0 con 0x5558bb1309f0 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96a5ffb640 1 -- 172.21.15.112:0/3670555472 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96b40048c0 con 0x5558bb1309f0 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96a5ffb640 1 -- 172.21.15.112:0/3670555472 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f96b8029040 con 0x5558bb1309f0 2024-09-21T03:31:20.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96cb533740 10 client.25546 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96cb533740 10 client.25546 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:31:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96cb533740 20 client.25546 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96cb533740 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5558bb172070 0x5558bb5a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96cb533740 4 client.25546 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:31:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96cb533740 1 -- 172.21.15.112:0/3670555472 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558bb48d8a0 con 0x5558bb172070 2024-09-21T03:31:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.840+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5558bb172070 0x5558bb5a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96bdd9b640 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5558bb172070 0x5558bb5a3e50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3499cfdf69d62e5c server_cookie=422e09b0b0483b3 in_seq=0 out_seq=0 2024-09-21T03:31:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96a5ffb640 10 client.25546 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:31:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96a5ffb640 1 -- 172.21.15.112:0/3670555472 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558bb48d8a0 con 0x5558bb172070 2024-09-21T03:31:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96a5ffb640 10 client.25546 handle_command_reply: tid=0 2024-09-21T03:31:20.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 1 client.25546 shutdown 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 1 -- 172.21.15.112:0/3670555472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 msgr2=0x5558bb52d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 0x5558bb52d970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f96b801ba30 tx=0x7f96b4005ef0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 1 -- 172.21.15.112:0/3670555472 shutdown_connections 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5558bb172070 0x5558bb5a3e50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558bb151390 0x5558bb529420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558bb1491b0 0x5558bb528e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96a5ffb640 0 client.25546 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 1 --2- 172.21.15.112:0/3670555472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558bb1309f0 0x5558bb52d970 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 1 -- 172.21.15.112:0/3670555472 >> 172.21.15.112:0/3670555472 conn(0x5558bb166460 msgr2=0x5558bb51a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 1 -- 172.21.15.112:0/3670555472 shutdown_connections 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 1 -- 172.21.15.112:0/3670555472 wait complete. 2024-09-21T03:31:20.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.841+0000 7f96cb533740 20 client.25546 trim_cache size 0 max 16384 2024-09-21T03:31:20.851 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:31:20.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2704447135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96a80b6680 msgr2=0x7f96a80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:20.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96a80b6680 0x7f96a80b8b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f96b00e53d0 tx=0x7f96b00eb6c0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2704447135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c0102a60 msgr2=0x7f96c019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:20.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c0102a60 0x7f96c019a590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f96b000e9f0 tx=0x7f96b000edf0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2704447135 shutdown_connections 2024-09-21T03:31:20.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96a80b6680 0x7f96a80b8b90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96c0108cf0 0x7f96c019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c01083a0 0x7f96c019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 --2- 172.21.15.112:0/2704447135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c0102a60 0x7f96c019a590 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:20.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2704447135 >> 172.21.15.112:0/2704447135 conn(0x7f96c0079730 msgr2=0x7f96c01044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:20.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2704447135 shutdown_connections 2024-09-21T03:31:20.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:20.842+0000 7f96c4fbf640 1 -- 172.21.15.112:0/2704447135 wait complete. 2024-09-21T03:31:20.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:20 smithi203 ceph-mon[34723]: pgmap v388: 97 pgs: 97 active+clean; 100 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 298 op/s 2024-09-21T03:31:20.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:20 smithi187 ceph-mon[34603]: pgmap v388: 97 pgs: 97 active+clean; 100 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 298 op/s 2024-09-21T03:31:21.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 163641 11.12 MB/sec warmup 81 sec latency 39.928 ms 2024-09-21T03:31:21.590 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:31:21.849 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:22.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 166081 11.13 MB/sec warmup 82 sec latency 36.895 ms 2024-09-21T03:31:22.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.350+0000 7f7371669640 1 -- 172.21.15.112:0/3521882965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736c102a60 msgr2=0x7f736c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.350+0000 7f7371669640 1 --2- 172.21.15.112:0/3521882965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736c102a60 0x7f736c102e70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f735c009900 tx=0x7f735c02a410 comp rx=0 tx=0).stop 2024-09-21T03:31:22.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.350+0000 7f7371669640 1 -- 172.21.15.112:0/3521882965 shutdown_connections 2024-09-21T03:31:22.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.350+0000 7f7371669640 1 --2- 172.21.15.112:0/3521882965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f736c108cf0 0x7f736c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.350+0000 7f7371669640 1 --2- 172.21.15.112:0/3521882965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f736c1083a0 0x7f736c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.350+0000 7f7371669640 1 --2- 172.21.15.112:0/3521882965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736c102a60 0x7f736c102e70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.350+0000 7f7371669640 1 -- 172.21.15.112:0/3521882965 >> 172.21.15.112:0/3521882965 conn(0x7f736c079730 msgr2=0x7f736c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:22.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.350+0000 7f7371669640 1 -- 172.21.15.112:0/3521882965 shutdown_connections 2024-09-21T03:31:22.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.350+0000 7f7371669640 1 -- 172.21.15.112:0/3521882965 wait complete. 2024-09-21T03:31:22.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.350+0000 7f7371669640 1 Processor -- start 2024-09-21T03:31:22.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f7371669640 1 -- start start 2024-09-21T03:31:22.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f7371669640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f736c102a60 0x7f736c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f7371669640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f736c1083a0 0x7f736c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f7371669640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736c108cf0 0x7f736c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f7371669640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f736c113a80 con 0x7f736c108cf0 2024-09-21T03:31:22.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f7371669640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f736c113900 con 0x7f736c1083a0 2024-09-21T03:31:22.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f7371669640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f736c113c00 con 0x7f736c102a60 2024-09-21T03:31:22.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f736affd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f736c102a60 0x7f736c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f736b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736c108cf0 0x7f736c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f736a7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f736c1083a0 0x7f736c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f736affd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f736c102a60 0x7f736c19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:41516/0 (socket says 172.21.15.112:41516) 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f736a7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f736c1083a0 0x7f736c19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:57132/0 (socket says 172.21.15.112:57132) 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f736affd640 1 -- 172.21.15.112:0/2604439567 learned_addr learned my addr 172.21.15.112:0/2604439567 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f736b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736c108cf0 0x7f736c19fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40576/0 (socket says 172.21.15.112:40576) 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f736a7fc640 1 -- 172.21.15.112:0/2604439567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f736c102a60 msgr2=0x7f736c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.351+0000 7f736a7fc640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f736c102a60 0x7f736c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.352+0000 7f736a7fc640 1 -- 172.21.15.112:0/2604439567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736c108cf0 msgr2=0x7f736c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.352+0000 7f736a7fc640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736c108cf0 0x7f736c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.352+0000 7f736a7fc640 1 -- 172.21.15.112:0/2604439567 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736c1a03a0 con 0x7f736c1083a0 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.352+0000 7f736b7fe640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736c108cf0 0x7f736c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.352+0000 7f736a7fc640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f736c1083a0 0x7f736c19a9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f736000e9c0 tx=0x7f736000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.352+0000 7f7353fff640 1 -- 172.21.15.112:0/2604439567 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736000ccc0 con 0x7f736c1083a0 2024-09-21T03:31:22.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.352+0000 7f7353fff640 1 -- 172.21.15.112:0/2604439567 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73600108e0 con 0x7f736c1083a0 2024-09-21T03:31:22.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.352+0000 7f7353fff640 1 -- 172.21.15.112:0/2604439567 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600187c0 con 0x7f736c1083a0 2024-09-21T03:31:22.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.352+0000 7f7371669640 1 -- 172.21.15.112:0/2604439567 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736c1a0690 con 0x7f736c1083a0 2024-09-21T03:31:22.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.352+0000 7f7371669640 1 -- 172.21.15.112:0/2604439567 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736c10fe90 con 0x7f736c1083a0 2024-09-21T03:31:22.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f7377bdd740 1 Processor -- start 2024-09-21T03:31:22.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f7377bdd740 1 -- start start 2024-09-21T03:31:22.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f7353fff640 1 -- 172.21.15.112:0/2604439567 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7360010430 con 0x7f736c1083a0 2024-09-21T03:31:22.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a99b070 0x55d67ad426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad42be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a97a390 0x55d67ad43120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f7377bdd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d67acb8ea0 con 0x55d67a9721b0 2024-09-21T03:31:22.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f7377bdd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d67aca82d0 con 0x55d67a97a390 2024-09-21T03:31:22.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f7377bdd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d67acac0e0 con 0x55d67a99b070 2024-09-21T03:31:22.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f736affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad42be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f736b7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a99b070 0x55d67ad426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f736affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad42be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40584/0 (socket says 172.21.15.112:40584) 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f736affd640 1 -- 172.21.15.112:0/730200552 learned_addr learned my addr 172.21.15.112:0/730200552 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f736a7fc640 1 --2- 172.21.15.112:0/730200552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a97a390 0x55d67ad43120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f736b7fe640 1 -- 172.21.15.112:0/730200552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a97a390 msgr2=0x55d67ad43120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.355+0000 7f736b7fe640 1 --2- 172.21.15.112:0/730200552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a97a390 0x55d67ad43120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.356+0000 7f736b7fe640 1 -- 172.21.15.112:0/730200552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 msgr2=0x55d67ad42be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.356+0000 7f736b7fe640 1 --2- 172.21.15.112:0/730200552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad42be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.356+0000 7f736b7fe640 1 -- 172.21.15.112:0/730200552 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d67aa06650 con 0x55d67a99b070 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.356+0000 7f736a7fc640 1 --2- 172.21.15.112:0/730200552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a97a390 0x55d67ad43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.356+0000 7f736affd640 1 --2- 172.21.15.112:0/730200552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad42be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.356+0000 7f7353fff640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73540b6680 0x7f73540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.356+0000 7f736b7fe640 1 --2- 172.21.15.112:0/730200552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a99b070 0x55d67ad426a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f73640106f0 tx=0x7f7364010bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96d0bcdee8ff1bb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:22.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.356+0000 7f736affd640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73540b6680 0x7f73540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.356+0000 7f7353fff640 1 -- 172.21.15.112:0/2604439567 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f73600e79c0 con 0x7f736c1083a0 2024-09-21T03:31:22.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.356+0000 7f73517fa640 1 -- 172.21.15.112:0/730200552 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364003b70 con 0x55d67a99b070 2024-09-21T03:31:22.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.356+0000 7f73517fa640 1 -- 172.21.15.112:0/730200552 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736400ede0 con 0x55d67a99b070 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.357+0000 7f73517fa640 1 -- 172.21.15.112:0/730200552 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364004670 con 0x55d67a99b070 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.357+0000 7f7377bdd740 1 -- 172.21.15.112:0/730200552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a99b070 msgr2=0x55d67ad426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.357+0000 7f7377bdd740 1 --2- 172.21.15.112:0/730200552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a99b070 0x55d67ad426a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f73640106f0 tx=0x7f7364010bb0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.357+0000 7f736affd640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73540b6680 0x7f73540b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f735c005f70 tx=0x7f735c03c040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.357+0000 7f7377bdd740 1 -- 172.21.15.112:0/730200552 shutdown_connections 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.357+0000 7f7377bdd740 1 --2- 172.21.15.112:0/730200552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a97a390 0x55d67ad43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.357+0000 7f7377bdd740 1 --2- 172.21.15.112:0/730200552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad42be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.357+0000 7f7377bdd740 1 --2- 172.21.15.112:0/730200552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a99b070 0x55d67ad426a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.357+0000 7f7377bdd740 1 -- 172.21.15.112:0/730200552 >> 172.21.15.112:0/730200552 conn(0x55d67a98f460 msgr2=0x55d67ad40590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.358+0000 7f7377bdd740 1 -- 172.21.15.112:0/730200552 shutdown_connections 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.358+0000 7f7377bdd740 1 -- 172.21.15.112:0/730200552 wait complete. 2024-09-21T03:31:22.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.358+0000 7f7377bdd740 1 -- start start 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.358+0000 7f7377bdd740 1 Processor -- start 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.358+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad4dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.358+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a97a390 0x55d67ad4e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.358+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a99b070 0x55d67ad5a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.358+0000 7f7377bdd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d67ab09700 con 0x55d67a9721b0 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.358+0000 7f7377bdd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d67ac43160 con 0x55d67a99b070 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.358+0000 7f7377bdd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d67ad50f20 con 0x55d67a97a390 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.358+0000 7f7377bdd740 10 client.0 _ll_register_callbacks cb 0x55d67ac5c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad4dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad4dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40590/0 (socket says 172.21.15.112:40590) 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736affd640 1 -- 172.21.15.112:0/2349637339 learned_addr learned my addr 172.21.15.112:0/2349637339 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736b7fe640 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a99b070 0x55d67ad5a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736a7fc640 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a97a390 0x55d67ad4e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736affd640 1 -- 172.21.15.112:0/2349637339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a97a390 msgr2=0x55d67ad4e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:22.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736affd640 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a97a390 0x55d67ad4e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736affd640 1 -- 172.21.15.112:0/2349637339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a99b070 msgr2=0x55d67ad5a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736affd640 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a99b070 0x55d67ad5a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736affd640 1 -- 172.21.15.112:0/2349637339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d67ac70c90 con 0x55d67a9721b0 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736a7fc640 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a97a390 0x55d67ad4e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f736affd640 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad4dd70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f735c009230 tx=0x7f735c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cf4fcdcfc9343 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f734eff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f734eff5640 1 -- 172.21.15.112:0/2349637339 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c056060 con 0x55d67a9721b0 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f734eff5640 1 -- 172.21.15.112:0/2349637339 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f735c03a670 con 0x55d67a9721b0 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f7377bdd740 1 -- 172.21.15.112:0/2349637339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736400ede0 con 0x55d67a9721b0 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.359+0000 7f734eff5640 1 -- 172.21.15.112:0/2349637339 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f735c035040 con 0x55d67a9721b0 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.360+0000 7f7377bdd740 10 client.15990 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.360+0000 7f7377bdd740 1 -- 172.21.15.112:0/2349637339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d67ad5cbb0 con 0x55d67a9721b0 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.360+0000 7f734eff5640 1 -- 172.21.15.112:0/2349637339 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736400ede0 con 0x55d67a9721b0 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.360+0000 7f734eff5640 1 -- 172.21.15.112:0/2349637339 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f735c003a80 con 0x55d67a9721b0 2024-09-21T03:31:22.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.360+0000 7f7377bdd740 10 client.15990 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:22.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.360+0000 7f7377bdd740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:31:22.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.360+0000 7f7377bdd740 20 client.15990 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:22.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.360+0000 7f7377bdd740 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d67a9599f0 0x55d67ad60110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.360+0000 7f7377bdd740 4 client.15990 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:22.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.360+0000 7f7377bdd740 1 -- 172.21.15.112:0/2349637339 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d67acbadf0 con 0x55d67a9599f0 2024-09-21T03:31:22.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.360+0000 7f736a7fc640 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d67a9599f0 0x55d67ad60110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.361+0000 7f734eff5640 10 client.15990 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:22.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.361+0000 7f736a7fc640 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d67a9599f0 0x55d67ad60110 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=865deda8e7c68339 server_cookie=a636d9039cea3d32 in_seq=0 out_seq=0 2024-09-21T03:31:22.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.361+0000 7f734eff5640 1 -- 172.21.15.112:0/2349637339 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d67acbadf0 con 0x55d67a9599f0 2024-09-21T03:31:22.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f734eff5640 10 client.15990 handle_command_reply: tid=0 2024-09-21T03:31:22.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 1 client.15990 shutdown 2024-09-21T03:31:22.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 1 -- 172.21.15.112:0/2349637339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 msgr2=0x55d67ad4dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad4dd70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f735c009230 tx=0x7f735c02a410 comp rx=0 tx=0).stop 2024-09-21T03:31:22.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 1 -- 172.21.15.112:0/2349637339 shutdown_connections 2024-09-21T03:31:22.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d67a9599f0 0x55d67ad60110 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a99b070 0x55d67ad5a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a97a390 0x55d67ad4e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 1 --2- 172.21.15.112:0/2349637339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a9721b0 0x55d67ad4dd70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f734eff5640 0 client.15990 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:22.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 1 -- 172.21.15.112:0/2349637339 >> 172.21.15.112:0/2349637339 conn(0x55d67a98f460 msgr2=0x55d67ad45830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:22.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 1 -- 172.21.15.112:0/2349637339 shutdown_connections 2024-09-21T03:31:22.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 1 -- 172.21.15.112:0/2349637339 wait complete. 2024-09-21T03:31:22.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.362+0000 7f7377bdd740 20 client.15990 trim_cache size 0 max 16384 2024-09-21T03:31:22.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f7377bdd740 1 Processor -- start 2024-09-21T03:31:22.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f7377bdd740 1 -- start start 2024-09-21T03:31:22.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 0x55d67ad4e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a97a390 0x55d67ad5a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a99b070 0x55d67ad4af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f7377bdd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f736c1afc00 con 0x55d67a99b070 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f7377bdd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f736c1ad4d0 con 0x55d67a97a390 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f7377bdd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f736c1ab320 con 0x55d67a9721b0 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f736b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a99b070 0x55d67ad4af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f736affd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 0x55d67ad4e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f736a7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a97a390 0x55d67ad5a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f736b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a99b070 0x55d67ad4af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40594/0 (socket says 172.21.15.112:40594) 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.376+0000 7f736b7fe640 1 -- 172.21.15.112:0/2287853639 learned_addr learned my addr 172.21.15.112:0/2287853639 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.377+0000 7f736b7fe640 1 -- 172.21.15.112:0/2287853639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 msgr2=0x55d67ad4e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:22.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.377+0000 7f736b7fe640 1 --2- 172.21.15.112:0/2287853639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 0x55d67ad4e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.377+0000 7f736b7fe640 1 -- 172.21.15.112:0/2287853639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a97a390 msgr2=0x55d67ad5a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:22.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.377+0000 7f736b7fe640 1 --2- 172.21.15.112:0/2287853639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a97a390 0x55d67ad5a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.377+0000 7f736b7fe640 1 -- 172.21.15.112:0/2287853639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d67ad4c270 con 0x55d67a99b070 2024-09-21T03:31:22.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.377+0000 7f736b7fe640 1 --2- 172.21.15.112:0/2287853639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a99b070 0x55d67ad4af60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f736400e720 tx=0x7f7364010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c14ba01f9625f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:22.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.377+0000 7f7350ff9640 1 -- 172.21.15.112:0/2287853639 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736400ec70 con 0x55d67a99b070 2024-09-21T03:31:22.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.377+0000 7f7350ff9640 1 -- 172.21.15.112:0/2287853639 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736400ee10 con 0x55d67a99b070 2024-09-21T03:31:22.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.377+0000 7f7350ff9640 1 -- 172.21.15.112:0/2287853639 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736401a470 con 0x55d67a99b070 2024-09-21T03:31:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.378+0000 7f7377bdd740 1 -- 172.21.15.112:0/2287853639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a99b070 msgr2=0x55d67ad4af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.378+0000 7f7377bdd740 1 --2- 172.21.15.112:0/2287853639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a99b070 0x55d67ad4af60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f736400e720 tx=0x7f7364010ec0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.378+0000 7f7377bdd740 1 -- 172.21.15.112:0/2287853639 shutdown_connections 2024-09-21T03:31:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.378+0000 7f7377bdd740 1 --2- 172.21.15.112:0/2287853639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a99b070 0x55d67ad4af60 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.378+0000 7f7377bdd740 1 --2- 172.21.15.112:0/2287853639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a97a390 0x55d67ad5a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.378+0000 7f7377bdd740 1 --2- 172.21.15.112:0/2287853639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 0x55d67ad4e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.378+0000 7f7377bdd740 1 -- 172.21.15.112:0/2287853639 >> 172.21.15.112:0/2287853639 conn(0x55d67a98f460 msgr2=0x55d67ad4db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.378+0000 7f7377bdd740 1 -- 172.21.15.112:0/2287853639 shutdown_connections 2024-09-21T03:31:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.378+0000 7f7377bdd740 1 -- 172.21.15.112:0/2287853639 wait complete. 2024-09-21T03:31:22.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.378+0000 7f7377bdd740 1 -- start start 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.378+0000 7f7377bdd740 1 Processor -- start 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 0x55d67ad52530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a97a390 0x55d67ad52a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f7377bdd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a99b070 0x55d67adc9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f7377bdd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f736c1ab990 con 0x55d67a97a390 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f7377bdd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f736c1ace00 con 0x55d67a99b070 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f7377bdd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f736c1ac060 con 0x55d67a9721b0 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f7377bdd740 10 client.0 _ll_register_callbacks cb 0x7f735c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a97a390 0x55d67ad52a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736affd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 0x55d67ad52530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a97a390 0x55d67ad52a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40600/0 (socket says 172.21.15.112:40600) 2024-09-21T03:31:22.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736affd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 0x55d67ad52530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:41536/0 (socket says 172.21.15.112:41536) 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736a7fc640 1 -- 172.21.15.112:0/3168872005 learned_addr learned my addr 172.21.15.112:0/3168872005 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736b7fe640 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a99b070 0x55d67adc9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736affd640 1 -- 172.21.15.112:0/3168872005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a99b070 msgr2=0x55d67adc9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736affd640 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a99b070 0x55d67adc9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736affd640 1 -- 172.21.15.112:0/3168872005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a97a390 msgr2=0x55d67ad52a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736affd640 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a97a390 0x55d67ad52a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736affd640 1 -- 172.21.15.112:0/3168872005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d67adca010 con 0x55d67a9721b0 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736a7fc640 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a97a390 0x55d67ad52a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.379+0000 7f736b7fe640 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a99b070 0x55d67adc9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.380+0000 7f736affd640 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 0x55d67ad52530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f735c009a20 tx=0x7f735c051770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97d5aa903a0da68f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.380+0000 7f734eff5640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.380+0000 7f734eff5640 1 -- 172.21.15.112:0/3168872005 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c056040 con 0x55d67a9721b0 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.380+0000 7f7377bdd740 1 -- 172.21.15.112:0/3168872005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736400ee10 con 0x55d67a9721b0 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.380+0000 7f734eff5640 1 -- 172.21.15.112:0/3168872005 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f735c003800 con 0x55d67a9721b0 2024-09-21T03:31:22.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.380+0000 7f734eff5640 1 -- 172.21.15.112:0/3168872005 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f735c06a050 con 0x55d67a9721b0 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.380+0000 7f7377bdd740 10 client.25564 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.380+0000 7f7377bdd740 1 -- 172.21.15.112:0/3168872005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d67adcb140 con 0x55d67a9721b0 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.380+0000 7f734eff5640 1 -- 172.21.15.112:0/3168872005 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736400ee10 con 0x55d67a9721b0 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.380+0000 7f734eff5640 1 -- 172.21.15.112:0/3168872005 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f735c003a00 con 0x55d67a9721b0 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.381+0000 7f7377bdd740 10 client.25564 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.381+0000 7f7377bdd740 10 client.25564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.381+0000 7f7377bdd740 20 client.25564 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.381+0000 7f7377bdd740 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d67a92eda0 0x55d67adcd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.381+0000 7f7377bdd740 4 client.25564 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.381+0000 7f7377bdd740 1 -- 172.21.15.112:0/3168872005 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d67acb68a0 con 0x55d67a92eda0 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.381+0000 7f736a7fc640 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d67a92eda0 0x55d67adcd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.381+0000 7f734eff5640 10 client.25564 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:22.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.381+0000 7f736a7fc640 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d67a92eda0 0x55d67adcd930 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71be37b937818364 server_cookie=b04326fa505e35e7 in_seq=0 out_seq=0 2024-09-21T03:31:22.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f734eff5640 1 -- 172.21.15.112:0/3168872005 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d67acb68a0 con 0x55d67a92eda0 2024-09-21T03:31:22.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f734eff5640 10 client.25564 handle_command_reply: tid=0 2024-09-21T03:31:22.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 1 client.25564 shutdown 2024-09-21T03:31:22.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 1 -- 172.21.15.112:0/3168872005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 msgr2=0x55d67ad52530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 0x55d67ad52530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f735c009a20 tx=0x7f735c051770 comp rx=0 tx=0).stop 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 1 -- 172.21.15.112:0/3168872005 shutdown_connections 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d67a92eda0 0x55d67adcd930 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d67a99b070 0x55d67adc9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d67a97a390 0x55d67ad52a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 1 --2- 172.21.15.112:0/3168872005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d67a9721b0 0x55d67ad52530 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f734eff5640 0 client.25564 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 1 -- 172.21.15.112:0/3168872005 >> 172.21.15.112:0/3168872005 conn(0x55d67a98f460 msgr2=0x55d67ad436d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 1 -- 172.21.15.112:0/3168872005 shutdown_connections 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 1 -- 172.21.15.112:0/3168872005 wait complete. 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.382+0000 7f7377bdd740 20 client.25564 trim_cache size 0 max 16384 2024-09-21T03:31:22.388 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:31:22.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 -- 172.21.15.112:0/2604439567 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73540b6680 msgr2=0x7f73540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73540b6680 0x7f73540b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f735c005f70 tx=0x7f735c03c040 comp rx=0 tx=0).stop 2024-09-21T03:31:22.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 -- 172.21.15.112:0/2604439567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f736c1083a0 msgr2=0x7f736c19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:22.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f736c1083a0 0x7f736c19a9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f736000e9c0 tx=0x7f736000ee80 comp rx=0 tx=0).stop 2024-09-21T03:31:22.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 -- 172.21.15.112:0/2604439567 shutdown_connections 2024-09-21T03:31:22.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73540b6680 0x7f73540b8b90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736c108cf0 0x7f736c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f736c1083a0 0x7f736c19a9d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 --2- 172.21.15.112:0/2604439567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f736c102a60 0x7f736c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:22.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 -- 172.21.15.112:0/2604439567 >> 172.21.15.112:0/2604439567 conn(0x7f736c079730 msgr2=0x7f736c101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:22.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 -- 172.21.15.112:0/2604439567 shutdown_connections 2024-09-21T03:31:22.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:22.383+0000 7f7371669640 1 -- 172.21.15.112:0/2604439567 wait complete. 2024-09-21T03:31:22.745 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:22 smithi112 ceph-mon[31442]: pgmap v389: 97 pgs: 97 active+clean; 104 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 360 op/s 2024-09-21T03:31:22.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:22 smithi203 ceph-mon[34723]: pgmap v389: 97 pgs: 97 active+clean; 104 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 360 op/s 2024-09-21T03:31:22.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:22 smithi187 ceph-mon[34603]: pgmap v389: 97 pgs: 97 active+clean; 104 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 360 op/s 2024-09-21T03:31:23.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 167522 11.09 MB/sec warmup 83 sec latency 36.437 ms 2024-09-21T03:31:24.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 170484 11.17 MB/sec warmup 84 sec latency 17.675 ms 2024-09-21T03:31:24.218 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:31:24.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:31:24.475 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:24.746 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:24 smithi112 ceph-mon[31442]: pgmap v390: 97 pgs: 97 active+clean; 104 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-09-21T03:31:24.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:24 smithi203 ceph-mon[34723]: pgmap v390: 97 pgs: 97 active+clean; 104 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-09-21T03:31:24.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:24 smithi187 ceph-mon[34603]: pgmap v390: 97 pgs: 97 active+clean; 104 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-09-21T03:31:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.962+0000 7ff44df20640 1 -- 172.21.15.112:0/1410641480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4481083c0 msgr2=0x7ff4481087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.962+0000 7ff44df20640 1 --2- 172.21.15.112:0/1410641480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4481083c0 0x7ff4481087d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009900 tx=0x7ff43c02a410 comp rx=0 tx=0).stop 2024-09-21T03:31:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.962+0000 7ff44df20640 1 -- 172.21.15.112:0/1410641480 shutdown_connections 2024-09-21T03:31:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.962+0000 7ff44df20640 1 --2- 172.21.15.112:0/1410641480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff448108d10 0x7ff44810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.962+0000 7ff44df20640 1 --2- 172.21.15.112:0/1410641480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4481083c0 0x7ff4481087d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.962+0000 7ff44df20640 1 --2- 172.21.15.112:0/1410641480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff448102a80 0x7ff448102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.962+0000 7ff44df20640 1 -- 172.21.15.112:0/1410641480 >> 172.21.15.112:0/1410641480 conn(0x7ff448079740 msgr2=0x7ff448079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:24.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.962+0000 7ff44df20640 1 -- 172.21.15.112:0/1410641480 shutdown_connections 2024-09-21T03:31:24.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.962+0000 7ff44df20640 1 -- 172.21.15.112:0/1410641480 wait complete. 2024-09-21T03:31:24.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff44df20640 1 Processor -- start 2024-09-21T03:31:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff44df20640 1 -- start start 2024-09-21T03:31:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff44df20640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff448102a80 0x7ff448199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff44df20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4481083c0 0x7ff448199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff44df20640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff448108d10 0x7ff44819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff44df20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff448112980 con 0x7ff4481083c0 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff44df20640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff448112800 con 0x7ff448108d10 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff44df20640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff448112b00 con 0x7ff448102a80 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4481083c0 0x7ff448199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff4477fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff448102a80 0x7ff448199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff447fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff448108d10 0x7ff44819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4481083c0 0x7ff448199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40624/0 (socket says 172.21.15.112:40624) 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.963+0000 7ff446ffd640 1 -- 172.21.15.112:0/1043393321 learned_addr learned my addr 172.21.15.112:0/1043393321 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.964+0000 7ff447fff640 1 -- 172.21.15.112:0/1043393321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff448102a80 msgr2=0x7ff448199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.964+0000 7ff447fff640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff448102a80 0x7ff448199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.964+0000 7ff447fff640 1 -- 172.21.15.112:0/1043393321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4481083c0 msgr2=0x7ff448199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.964+0000 7ff447fff640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4481083c0 0x7ff448199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.964+0000 7ff447fff640 1 -- 172.21.15.112:0/1043393321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44819f2e0 con 0x7ff448108d10 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.964+0000 7ff447fff640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff448108d10 0x7ff44819ebb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff440009fd0 tx=0x7ff44000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:24.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.964+0000 7ff444ff9640 1 -- 172.21.15.112:0/1043393321 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44000ce20 con 0x7ff448108d10 2024-09-21T03:31:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.964+0000 7ff44df20640 1 -- 172.21.15.112:0/1043393321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44819f5d0 con 0x7ff448108d10 2024-09-21T03:31:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.964+0000 7ff444ff9640 1 -- 172.21.15.112:0/1043393321 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff440005020 con 0x7ff448108d10 2024-09-21T03:31:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.964+0000 7ff444ff9640 1 -- 172.21.15.112:0/1043393321 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff440010690 con 0x7ff448108d10 2024-09-21T03:31:24.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.964+0000 7ff44df20640 1 -- 172.21.15.112:0/1043393321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4481a5d30 con 0x7ff448108d10 2024-09-21T03:31:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.967+0000 7ff444ff9640 1 -- 172.21.15.112:0/1043393321 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff440010830 con 0x7ff448108d10 2024-09-21T03:31:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.967+0000 7ff454494740 1 Processor -- start 2024-09-21T03:31:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.967+0000 7ff454494740 1 -- start start 2024-09-21T03:31:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.967+0000 7ff454494740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b048430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.967+0000 7ff454494740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b048970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.967+0000 7ff454494740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac80670 0x55871b048eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.967+0000 7ff454494740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55871afbee60 con 0x55871aca1350 2024-09-21T03:31:24.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.967+0000 7ff454494740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55871afae290 con 0x55871ac80670 2024-09-21T03:31:24.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.967+0000 7ff454494740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55871afb20a0 con 0x55871ac78490 2024-09-21T03:31:24.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.967+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b048430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.967+0000 7ff4477fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b048970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b048430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40628/0 (socket says 172.21.15.112:40628) 2024-09-21T03:31:24.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff446ffd640 1 -- 172.21.15.112:0/247715390 learned_addr learned my addr 172.21.15.112:0/247715390 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:24.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff446ffd640 1 --2- 172.21.15.112:0/247715390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac80670 0x55871b048eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff4477fe640 1 -- 172.21.15.112:0/247715390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac80670 msgr2=0x55871b048eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:24.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff4477fe640 1 --2- 172.21.15.112:0/247715390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac80670 0x55871b048eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff4477fe640 1 -- 172.21.15.112:0/247715390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 msgr2=0x55871b048430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff444ff9640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff4300b6680 0x7ff4300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff4477fe640 1 --2- 172.21.15.112:0/247715390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b048430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff4477fe640 1 -- 172.21.15.112:0/247715390 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871ad0c930 con 0x55871ac78490 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff446ffd640 1 --2- 172.21.15.112:0/247715390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b048430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff447fff640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff4300b6680 0x7ff4300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff4477fe640 1 --2- 172.21.15.112:0/247715390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b048970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff438009c20 tx=0x7ff438011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13aee16f2de94ced server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff444ff9640 1 -- 172.21.15.112:0/1043393321 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff4400e7db0 con 0x7ff448108d10 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff42dffb640 1 -- 172.21.15.112:0/247715390 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438004050 con 0x55871ac78490 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff42dffb640 1 -- 172.21.15.112:0/247715390 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4380054d0 con 0x55871ac78490 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.968+0000 7ff42dffb640 1 -- 172.21.15.112:0/247715390 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438004c20 con 0x55871ac78490 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.969+0000 7ff454494740 1 -- 172.21.15.112:0/247715390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 msgr2=0x55871b048970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.969+0000 7ff454494740 1 --2- 172.21.15.112:0/247715390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b048970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff438009c20 tx=0x7ff438011810 comp rx=0 tx=0).stop 2024-09-21T03:31:24.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.969+0000 7ff447fff640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff4300b6680 0x7ff4300b8b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff4400e5610 tx=0x7ff4400aef90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:24.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.969+0000 7ff454494740 1 -- 172.21.15.112:0/247715390 shutdown_connections 2024-09-21T03:31:24.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.969+0000 7ff454494740 1 --2- 172.21.15.112:0/247715390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac80670 0x55871b048eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.969+0000 7ff454494740 1 --2- 172.21.15.112:0/247715390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b048970 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.969+0000 7ff454494740 1 --2- 172.21.15.112:0/247715390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b048430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.969+0000 7ff454494740 1 -- 172.21.15.112:0/247715390 >> 172.21.15.112:0/247715390 conn(0x55871ac95740 msgr2=0x55871b046320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:24.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.969+0000 7ff454494740 1 -- 172.21.15.112:0/247715390 shutdown_connections 2024-09-21T03:31:24.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.969+0000 7ff454494740 1 -- 172.21.15.112:0/247715390 wait complete. 2024-09-21T03:31:24.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.970+0000 7ff454494740 1 -- start start 2024-09-21T03:31:24.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.970+0000 7ff454494740 1 Processor -- start 2024-09-21T03:31:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.970+0000 7ff454494740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b056210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.970+0000 7ff454494740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac80670 0x55871b056750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.970+0000 7ff454494740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b0620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.970+0000 7ff454494740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55871ae0f890 con 0x55871aca1350 2024-09-21T03:31:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.970+0000 7ff454494740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55871af48d40 con 0x55871ac80670 2024-09-21T03:31:24.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.970+0000 7ff454494740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55871b058d70 con 0x55871ac78490 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.970+0000 7ff454494740 10 client.0 _ll_register_callbacks cb 0x55871afacc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b056210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff4477fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac80670 0x55871b056750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b0620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b0620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40634/0 (socket says 172.21.15.112:40634) 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff446ffd640 1 -- 172.21.15.112:0/1207761278 learned_addr learned my addr 172.21.15.112:0/1207761278 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff4477fe640 1 -- 172.21.15.112:0/1207761278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 msgr2=0x55871b056210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff4477fe640 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b056210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff4477fe640 1 -- 172.21.15.112:0/1207761278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 msgr2=0x55871b0620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff4477fe640 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b0620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff4477fe640 1 -- 172.21.15.112:0/1207761278 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871b062610 con 0x55871ac80670 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff446ffd640 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b0620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff4477fe640 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac80670 0x55871b056750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff438000c00 tx=0x7ff438005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b8561411553b3da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:24.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff426ffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff426ffd640 1 -- 172.21.15.112:0/1207761278 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438029070 con 0x55871ac80670 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff426ffd640 1 -- 172.21.15.112:0/1207761278 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff438003a00 con 0x55871ac80670 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.971+0000 7ff454494740 1 -- 172.21.15.112:0/1207761278 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4380054d0 con 0x55871ac80670 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff426ffd640 1 -- 172.21.15.112:0/1207761278 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff438035050 con 0x55871ac80670 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff454494740 10 client.25571 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff454494740 1 -- 172.21.15.112:0/1207761278 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55871b064f20 con 0x55871ac80670 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff426ffd640 1 -- 172.21.15.112:0/1207761278 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4380054d0 con 0x55871ac80670 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff426ffd640 1 -- 172.21.15.112:0/1207761278 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff438004ff0 con 0x55871ac80670 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff454494740 10 client.25571 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff454494740 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff454494740 20 client.25571 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff454494740 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55871ac5fcd0 0x55871b068280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff454494740 4 client.25571 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:24.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff454494740 1 -- 172.21.15.112:0/1207761278 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55871afc0db0 con 0x55871ac5fcd0 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.972+0000 7ff446ffd640 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55871ac5fcd0 0x55871b068280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.973+0000 7ff426ffd640 10 client.25571 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.973+0000 7ff446ffd640 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55871ac5fcd0 0x55871b068280 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6ee00a6cb24c02b server_cookie=b1985dd74e132a27 in_seq=0 out_seq=0 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.973+0000 7ff426ffd640 1 -- 172.21.15.112:0/1207761278 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55871afc0db0 con 0x55871ac5fcd0 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.973+0000 7ff426ffd640 10 client.25571 handle_command_reply: tid=0 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 1 client.25571 shutdown 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 1 -- 172.21.15.112:0/1207761278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac80670 msgr2=0x55871b056750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac80670 0x55871b056750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff438000c00 tx=0x7ff438005680 comp rx=0 tx=0).stop 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 1 -- 172.21.15.112:0/1207761278 shutdown_connections 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55871ac5fcd0 0x55871b068280 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871aca1350 0x55871b0620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff426ffd640 0 client.25571 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac80670 0x55871b056750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 1 --2- 172.21.15.112:0/1207761278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b056210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 1 -- 172.21.15.112:0/1207761278 >> 172.21.15.112:0/1207761278 conn(0x55871ac95740 msgr2=0x55871b049e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:24.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 1 -- 172.21.15.112:0/1207761278 shutdown_connections 2024-09-21T03:31:24.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 1 -- 172.21.15.112:0/1207761278 wait complete. 2024-09-21T03:31:24.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.974+0000 7ff454494740 20 client.25571 trim_cache size 0 max 16384 2024-09-21T03:31:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff454494740 1 Processor -- start 2024-09-21T03:31:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff454494740 1 -- start start 2024-09-21T03:31:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff454494740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac5fcd0 0x55871b05e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff454494740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b05e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff454494740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac80670 0x55871b044850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff454494740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4481afc90 con 0x55871ac80670 2024-09-21T03:31:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff454494740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4481ad560 con 0x55871ac5fcd0 2024-09-21T03:31:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff454494740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4481ab3b0 con 0x55871ac78490 2024-09-21T03:31:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac80670 0x55871b044850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac80670 0x55871b044850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40636/0 (socket says 172.21.15.112:40636) 2024-09-21T03:31:24.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff446ffd640 1 -- 172.21.15.112:0/1712461877 learned_addr learned my addr 172.21.15.112:0/1712461877 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:24.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff4477fe640 1 --2- 172.21.15.112:0/1712461877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b05e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.993+0000 7ff446ffd640 1 --2- 172.21.15.112:0/1712461877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac5fcd0 0x55871b05e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.994+0000 7ff446ffd640 1 -- 172.21.15.112:0/1712461877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 msgr2=0x55871b05e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.994+0000 7ff446ffd640 1 --2- 172.21.15.112:0/1712461877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b05e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.994+0000 7ff446ffd640 1 -- 172.21.15.112:0/1712461877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac5fcd0 msgr2=0x55871b05e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.994+0000 7ff446ffd640 1 --2- 172.21.15.112:0/1712461877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac5fcd0 0x55871b05e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.994+0000 7ff446ffd640 1 -- 172.21.15.112:0/1712461877 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871b04e510 con 0x55871ac80670 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.994+0000 7ff446ffd640 1 --2- 172.21.15.112:0/1712461877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac80670 0x55871b044850 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff43c031440 tx=0x7ff43c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3d741e1c329eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.994+0000 7ff42d7fa640 1 -- 172.21.15.112:0/1712461877 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c035040 con 0x55871ac80670 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.994+0000 7ff42d7fa640 1 -- 172.21.15.112:0/1712461877 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff43c005500 con 0x55871ac80670 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.994+0000 7ff42d7fa640 1 -- 172.21.15.112:0/1712461877 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c03dcd0 con 0x55871ac80670 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.995+0000 7ff454494740 1 -- 172.21.15.112:0/1712461877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac80670 msgr2=0x55871b044850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.995+0000 7ff454494740 1 --2- 172.21.15.112:0/1712461877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac80670 0x55871b044850 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff43c031440 tx=0x7ff43c009260 comp rx=0 tx=0).stop 2024-09-21T03:31:24.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.995+0000 7ff454494740 1 -- 172.21.15.112:0/1712461877 shutdown_connections 2024-09-21T03:31:24.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.995+0000 7ff454494740 1 --2- 172.21.15.112:0/1712461877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac80670 0x55871b044850 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.995+0000 7ff454494740 1 --2- 172.21.15.112:0/1712461877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac78490 0x55871b05e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.995+0000 7ff454494740 1 --2- 172.21.15.112:0/1712461877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac5fcd0 0x55871b05e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:24.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.995+0000 7ff454494740 1 -- 172.21.15.112:0/1712461877 >> 172.21.15.112:0/1712461877 conn(0x55871ac95740 msgr2=0x55871b054890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:24.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.995+0000 7ff454494740 1 -- 172.21.15.112:0/1712461877 shutdown_connections 2024-09-21T03:31:24.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.995+0000 7ff454494740 1 -- 172.21.15.112:0/1712461877 wait complete. 2024-09-21T03:31:24.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.995+0000 7ff454494740 1 -- start start 2024-09-21T03:31:25.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff454494740 1 Processor -- start 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff454494740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac5fcd0 0x55871b0521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff454494740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac78490 0x55871b052730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff454494740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac80670 0x55871b052c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff454494740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4481aba20 con 0x55871ac5fcd0 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff454494740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4481ace90 con 0x55871ac78490 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff454494740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4481ac0f0 con 0x55871ac80670 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff454494740 10 client.0 _ll_register_callbacks cb 0x7ff4400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac5fcd0 0x55871b0521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff446ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac5fcd0 0x55871b0521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40642/0 (socket says 172.21.15.112:40642) 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff446ffd640 1 -- 172.21.15.112:0/4132202228 learned_addr learned my addr 172.21.15.112:0/4132202228 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff446ffd640 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac80670 0x55871b052c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff4477fe640 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac78490 0x55871b052730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:25.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff446ffd640 1 -- 172.21.15.112:0/4132202228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac80670 msgr2=0x55871b052c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff446ffd640 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac80670 0x55871b052c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff446ffd640 1 -- 172.21.15.112:0/4132202228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac78490 msgr2=0x55871b052730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff446ffd640 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac78490 0x55871b052730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff446ffd640 1 -- 172.21.15.112:0/4132202228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871b0d3d60 con 0x55871ac5fcd0 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.996+0000 7ff4477fe640 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac78490 0x55871b052730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff446ffd640 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac5fcd0 0x55871b0521f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff43c03d910 tx=0x7ff43c004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb621cc9ff3225e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff426ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff426ffd640 1 -- 172.21.15.112:0/4132202228 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c058070 con 0x55871ac5fcd0 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff426ffd640 1 -- 172.21.15.112:0/4132202228 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7ff43c03d970 con 0x55871ac5fcd0 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff454494740 1 -- 172.21.15.112:0/4132202228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff43c005500 con 0x55871ac5fcd0 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff426ffd640 1 -- 172.21.15.112:0/4132202228 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff43c06c050 con 0x55871ac5fcd0 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff454494740 10 client.16014 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff454494740 1 -- 172.21.15.112:0/4132202228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55871b0d5000 con 0x55871ac5fcd0 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff426ffd640 1 -- 172.21.15.112:0/4132202228 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff43c005500 con 0x55871ac5fcd0 2024-09-21T03:31:25.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff426ffd640 1 -- 172.21.15.112:0/4132202228 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff43c003aa0 con 0x55871ac5fcd0 2024-09-21T03:31:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff454494740 10 client.16014 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff454494740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:31:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.997+0000 7ff454494740 20 client.16014 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.998+0000 7ff454494740 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55871aca1350 0x55871b0d77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.998+0000 7ff454494740 4 client.16014 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:31:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.998+0000 7ff454494740 1 -- 172.21.15.112:0/4132202228 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55871afbc860 con 0x55871aca1350 2024-09-21T03:31:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.998+0000 7ff4477fe640 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55871aca1350 0x55871b0d77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.998+0000 7ff426ffd640 10 client.16014 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:25.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:24.998+0000 7ff4477fe640 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55871aca1350 0x55871b0d77a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a9d9f8eba4be1c server_cookie=fe18febcbd983069 in_seq=0 out_seq=0 2024-09-21T03:31:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.000+0000 7ff426ffd640 1 -- 172.21.15.112:0/4132202228 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55871afbc860 con 0x55871aca1350 2024-09-21T03:31:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.000+0000 7ff426ffd640 10 client.16014 handle_command_reply: tid=0 2024-09-21T03:31:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.000+0000 7ff454494740 1 client.16014 shutdown 2024-09-21T03:31:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff454494740 1 -- 172.21.15.112:0/4132202228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac5fcd0 msgr2=0x55871b0521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff454494740 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac5fcd0 0x55871b0521f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff43c03d910 tx=0x7ff43c004230 comp rx=0 tx=0).stop 2024-09-21T03:31:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff454494740 1 -- 172.21.15.112:0/4132202228 shutdown_connections 2024-09-21T03:31:25.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff454494740 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55871aca1350 0x55871b0d77a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff454494740 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55871ac80670 0x55871b052c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff426ffd640 0 client.16014 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff454494740 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55871ac78490 0x55871b052730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff454494740 1 --2- 172.21.15.112:0/4132202228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55871ac5fcd0 0x55871b0521f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff454494740 1 -- 172.21.15.112:0/4132202228 >> 172.21.15.112:0/4132202228 conn(0x55871ac95740 msgr2=0x55871b04d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff454494740 1 -- 172.21.15.112:0/4132202228 shutdown_connections 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff454494740 1 -- 172.21.15.112:0/4132202228 wait complete. 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff454494740 20 client.16014 trim_cache size 0 max 16384 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "d5e6af18-7419-4d89-a8cb-849ebeffa647", 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:31:25.005 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:31:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff44df20640 1 -- 172.21.15.112:0/1043393321 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff4300b6680 msgr2=0x7ff4300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff44df20640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff4300b6680 0x7ff4300b8b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff4400e5610 tx=0x7ff4400aef90 comp rx=0 tx=0).stop 2024-09-21T03:31:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff44df20640 1 -- 172.21.15.112:0/1043393321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff448108d10 msgr2=0x7ff44819ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.001+0000 7ff44df20640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff448108d10 0x7ff44819ebb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff440009fd0 tx=0x7ff44000ef10 comp rx=0 tx=0).stop 2024-09-21T03:31:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.002+0000 7ff44df20640 1 -- 172.21.15.112:0/1043393321 shutdown_connections 2024-09-21T03:31:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.002+0000 7ff44df20640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff4300b6680 0x7ff4300b8b90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:25.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.002+0000 7ff44df20640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff448108d10 0x7ff44819ebb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:25.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.002+0000 7ff44df20640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4481083c0 0x7ff448199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:25.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.002+0000 7ff44df20640 1 --2- 172.21.15.112:0/1043393321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff448102a80 0x7ff448199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:25.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.002+0000 7ff44df20640 1 -- 172.21.15.112:0/1043393321 >> 172.21.15.112:0/1043393321 conn(0x7ff448079740 msgr2=0x7ff448104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:25.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.002+0000 7ff44df20640 1 -- 172.21.15.112:0/1043393321 shutdown_connections 2024-09-21T03:31:25.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:25.002+0000 7ff44df20640 1 -- 172.21.15.112:0/1043393321 wait complete. 2024-09-21T03:31:25.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 171832 11.11 MB/sec warmup 85 sec latency 34.315 ms 2024-09-21T03:31:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:25 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:31:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:25 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:25 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:31:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:25 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:31:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:25 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:31:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:25 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:25 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:25.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:25 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:25.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:31:25.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:25 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:31:25.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:25 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:25.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:25 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:31:25.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:25 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:31:25.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:25 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:31:25.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:25 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:25.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:25 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:25.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:25 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:25.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:25 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:31:25.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:25 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:25.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:25 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:31:25.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:25 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:31:25.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:25 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:31:25.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:25 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:25.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:25 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:25.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:25 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:31:26.103 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:26.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 174289 11.14 MB/sec warmup 86 sec latency 11.062 ms 2024-09-21T03:31:26.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.655+0000 7f611b71c640 1 -- 172.21.15.112:0/3474745708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6114104880 msgr2=0x7f6114104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:26.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.655+0000 7f611b71c640 1 --2- 172.21.15.112:0/3474745708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6114104880 0x7f6114104c90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6110009900 tx=0x7f611002a410 comp rx=0 tx=0).stop 2024-09-21T03:31:26.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.655+0000 7f611b71c640 1 -- 172.21.15.112:0/3474745708 shutdown_connections 2024-09-21T03:31:26.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.655+0000 7f611b71c640 1 --2- 172.21.15.112:0/3474745708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f611410de30 0x7f6114110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.655+0000 7f611b71c640 1 --2- 172.21.15.112:0/3474745708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61141052b0 0x7f611410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.655+0000 7f611b71c640 1 --2- 172.21.15.112:0/3474745708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6114104880 0x7f6114104c90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.655+0000 7f611b71c640 1 -- 172.21.15.112:0/3474745708 >> 172.21.15.112:0/3474745708 conn(0x7f61140797e0 msgr2=0x7f6114079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:26.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.655+0000 7f611b71c640 1 -- 172.21.15.112:0/3474745708 shutdown_connections 2024-09-21T03:31:26.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.655+0000 7f611b71c640 1 -- 172.21.15.112:0/3474745708 wait complete. 2024-09-21T03:31:26.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.656+0000 7f611b71c640 1 Processor -- start 2024-09-21T03:31:26.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.656+0000 7f611b71c640 1 -- start start 2024-09-21T03:31:26.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.656+0000 7f611b71c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6114104880 0x7f611419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.656+0000 7f611b71c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61141052b0 0x7f611419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.656+0000 7f611b71c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f611410de30 0x7f611419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.656+0000 7f611b71c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6114113b10 con 0x7f6114104880 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.656+0000 7f611b71c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6114113990 con 0x7f611410de30 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.656+0000 7f611b71c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6114113c90 con 0x7f61141052b0 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f61194b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6114104880 0x7f611419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f61194b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6114104880 0x7f611419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44724/0 (socket says 172.21.15.112:44724) 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f6118cb7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61141052b0 0x7f611419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f61194b8640 1 -- 172.21.15.112:0/4187403256 learned_addr learned my addr 172.21.15.112:0/4187403256 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f6119cb9640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f611410de30 0x7f611419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f61194b8640 1 -- 172.21.15.112:0/4187403256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61141052b0 msgr2=0x7f611419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f61194b8640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61141052b0 0x7f611419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f61194b8640 1 -- 172.21.15.112:0/4187403256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f611410de30 msgr2=0x7f611419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:26.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f61194b8640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f611410de30 0x7f611419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f61194b8640 1 -- 172.21.15.112:0/4187403256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61141a04b0 con 0x7f6114104880 2024-09-21T03:31:26.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f61194b8640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6114104880 0x7f611419a540 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6110006010 tx=0x7f6110004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:26.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f610a7fc640 1 -- 172.21.15.112:0/4187403256 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611003f030 con 0x7f6114104880 2024-09-21T03:31:26.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f611b71c640 1 -- 172.21.15.112:0/4187403256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61141a0740 con 0x7f6114104880 2024-09-21T03:31:26.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f610a7fc640 1 -- 172.21.15.112:0/4187403256 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6110004490 con 0x7f6114104880 2024-09-21T03:31:26.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f610a7fc640 1 -- 172.21.15.112:0/4187403256 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110035070 con 0x7f6114104880 2024-09-21T03:31:26.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.657+0000 7f611b71c640 1 -- 172.21.15.112:0/4187403256 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61141a6f00 con 0x7f6114104880 2024-09-21T03:31:26.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.660+0000 7f610a7fc640 1 -- 172.21.15.112:0/4187403256 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f6110042030 con 0x7f6114104880 2024-09-21T03:31:26.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.660+0000 7f6121c90740 1 Processor -- start 2024-09-21T03:31:26.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.660+0000 7f6121c90740 1 -- start start 2024-09-21T03:31:26.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6121c90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd5f070 0x55cb3d106650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6121c90740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 0x55cb3d106b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6121c90740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd3e390 0x55cb3d1070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6121c90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cb3d07ce50 con 0x55cb3cd5f070 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6121c90740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cb3d06c280 con 0x55cb3cd3e390 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6121c90740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cb3d070090 con 0x55cb3cd361b0 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6119cb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd5f070 0x55cb3d106650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6119cb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd5f070 0x55cb3d106650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44728/0 (socket says 172.21.15.112:44728) 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6119cb9640 1 -- 172.21.15.112:0/550158425 learned_addr learned my addr 172.21.15.112:0/550158425 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f61194b8640 1 --2- 172.21.15.112:0/550158425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 0x55cb3d106b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6118cb7640 1 --2- 172.21.15.112:0/550158425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd3e390 0x55cb3d1070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6119cb9640 1 -- 172.21.15.112:0/550158425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 msgr2=0x55cb3d106b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6119cb9640 1 --2- 172.21.15.112:0/550158425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 0x55cb3d106b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6119cb9640 1 -- 172.21.15.112:0/550158425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd3e390 msgr2=0x55cb3d1070d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6119cb9640 1 --2- 172.21.15.112:0/550158425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd3e390 0x55cb3d1070d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6119cb9640 1 -- 172.21.15.112:0/550158425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb3cdca650 con 0x55cb3cd5f070 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f610a7fc640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61000b6680 0x7f61000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.661+0000 7f6118cb7640 1 --2- 172.21.15.112:0/550158425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd3e390 0x55cb3d1070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.662+0000 7f610a7fc640 1 -- 172.21.15.112:0/4187403256 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f6110107a40 con 0x7f6114104880 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.662+0000 7f6119cb9640 1 --2- 172.21.15.112:0/550158425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd5f070 0x55cb3d106650 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f610c009fd0 tx=0x7f610c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73386f18af2b6503 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.662+0000 7f60ff7fe640 1 -- 172.21.15.112:0/550158425 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c004d20 con 0x55cb3cd5f070 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.662+0000 7f60ff7fe640 1 -- 172.21.15.112:0/550158425 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610c004050 con 0x55cb3cd5f070 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.662+0000 7f60ff7fe640 1 -- 172.21.15.112:0/550158425 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c005260 con 0x55cb3cd5f070 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.662+0000 7f61194b8640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61000b6680 0x7f61000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.663+0000 7f6121c90740 1 -- 172.21.15.112:0/550158425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd5f070 msgr2=0x55cb3d106650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.663+0000 7f6121c90740 1 --2- 172.21.15.112:0/550158425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd5f070 0x55cb3d106650 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f610c009fd0 tx=0x7f610c0118d0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.663+0000 7f61194b8640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61000b6680 0x7f61000b8b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f61101052a0 tx=0x7f611010b590 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:26.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.663+0000 7f6121c90740 1 -- 172.21.15.112:0/550158425 shutdown_connections 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.663+0000 7f6121c90740 1 --2- 172.21.15.112:0/550158425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd3e390 0x55cb3d1070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.663+0000 7f6121c90740 1 --2- 172.21.15.112:0/550158425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 0x55cb3d106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.663+0000 7f6121c90740 1 --2- 172.21.15.112:0/550158425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd5f070 0x55cb3d106650 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.663+0000 7f6121c90740 1 -- 172.21.15.112:0/550158425 >> 172.21.15.112:0/550158425 conn(0x55cb3cd53460 msgr2=0x55cb3d104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.663+0000 7f6121c90740 1 -- 172.21.15.112:0/550158425 shutdown_connections 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.663+0000 7f6121c90740 1 -- 172.21.15.112:0/550158425 wait complete. 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6121c90740 1 -- start start 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6121c90740 1 Processor -- start 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6121c90740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 0x55cb3d111d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6121c90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 0x55cb3d112240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6121c90740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd5f070 0x55cb3d11e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6121c90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cb3cecd700 con 0x55cb3cd3e390 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6121c90740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cb3d007110 con 0x55cb3cd5f070 2024-09-21T03:31:26.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6121c90740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cb3d114e30 con 0x55cb3cd361b0 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6121c90740 10 client.0 _ll_register_callbacks cb 0x55cb3d020990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6119cb9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 0x55cb3d111d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6119cb9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 0x55cb3d111d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:44834/0 (socket says 172.21.15.112:44834) 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6119cb9640 1 -- 172.21.15.112:0/2582156703 learned_addr learned my addr 172.21.15.112:0/2582156703 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.664+0000 7f6118cb7640 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 0x55cb3d112240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f6119cb9640 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd5f070 0x55cb3d11e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f6119cb9640 1 -- 172.21.15.112:0/2582156703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd5f070 msgr2=0x55cb3d11e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f6119cb9640 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd5f070 0x55cb3d11e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f6119cb9640 1 -- 172.21.15.112:0/2582156703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 msgr2=0x55cb3d112240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f6119cb9640 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 0x55cb3d112240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f6119cb9640 1 -- 172.21.15.112:0/2582156703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3d034c40 con 0x55cb3cd361b0 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f6119cb9640 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 0x55cb3d111d00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f610c004510 tx=0x7f610c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a904ef9c62181a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f60fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:31:26.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f60fcff9640 1 -- 172.21.15.112:0/2582156703 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c02b020 con 0x55cb3cd361b0 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f60fcff9640 1 -- 172.21.15.112:0/2582156703 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f610c00f4e0 con 0x55cb3cd361b0 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f6121c90740 1 -- 172.21.15.112:0/2582156703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f610c004050 con 0x55cb3cd361b0 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f60fcff9640 1 -- 172.21.15.112:0/2582156703 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f610c036050 con 0x55cb3cd361b0 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f6121c90740 10 client.25591 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.665+0000 7f6121c90740 1 -- 172.21.15.112:0/2582156703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cb3d120b40 con 0x55cb3cd361b0 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.666+0000 7f60fcff9640 1 -- 172.21.15.112:0/2582156703 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610c004050 con 0x55cb3cd361b0 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.666+0000 7f60fcff9640 1 -- 172.21.15.112:0/2582156703 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f610c027740 con 0x55cb3cd361b0 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.666+0000 7f6121c90740 10 client.25591 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.666+0000 7f6121c90740 10 client.25591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.666+0000 7f6121c90740 20 client.25591 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.666+0000 7f6121c90740 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55cb3cd1d9f0 0x55cb3d123fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.666+0000 7f6121c90740 4 client.25591 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:26.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.666+0000 7f6121c90740 1 -- 172.21.15.112:0/2582156703 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb3d07eda0 con 0x55cb3cd1d9f0 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.666+0000 7f6118cb7640 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55cb3cd1d9f0 0x55cb3d123fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.667+0000 7f60fcff9640 10 client.25591 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.667+0000 7f6118cb7640 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55cb3cd1d9f0 0x55cb3d123fb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c153bc233b7da1e9 server_cookie=bb637ec894a64e82 in_seq=0 out_seq=0 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f60fcff9640 1 -- 172.21.15.112:0/2582156703 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb3d07eda0 con 0x55cb3cd1d9f0 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f60fcff9640 10 client.25591 handle_command_reply: tid=0 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 1 client.25591 shutdown 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 1 -- 172.21.15.112:0/2582156703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 msgr2=0x55cb3d111d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 0x55cb3d111d00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f610c004510 tx=0x7f610c005bd0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 1 -- 172.21.15.112:0/2582156703 shutdown_connections 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55cb3cd1d9f0 0x55cb3d123fb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd5f070 0x55cb3d11e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 0x55cb3d112240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 1 --2- 172.21.15.112:0/2582156703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd361b0 0x55cb3d111d00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 1 -- 172.21.15.112:0/2582156703 >> 172.21.15.112:0/2582156703 conn(0x55cb3cd53460 msgr2=0x55cb3d109830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f60fcff9640 0 client.25591 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 1 -- 172.21.15.112:0/2582156703 shutdown_connections 2024-09-21T03:31:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 1 -- 172.21.15.112:0/2582156703 wait complete. 2024-09-21T03:31:26.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.668+0000 7f6121c90740 20 client.25591 trim_cache size 0 max 16384 2024-09-21T03:31:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.686+0000 7f6121c90740 1 Processor -- start 2024-09-21T03:31:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.686+0000 7f6121c90740 1 -- start start 2024-09-21T03:31:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6121c90740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd361b0 0x55cb3d107d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6121c90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 0x55cb3d11e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6121c90740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd5f070 0x55cb3d10eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6121c90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61141b1ff0 con 0x55cb3cd3e390 2024-09-21T03:31:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6121c90740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61141af8c0 con 0x55cb3cd361b0 2024-09-21T03:31:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6121c90740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f61141ad710 con 0x55cb3cd5f070 2024-09-21T03:31:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6118cb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 0x55cb3d11e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6119cb9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd361b0 0x55cb3d107d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6119cb9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd361b0 0x55cb3d107d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:57756/0 (socket says 172.21.15.112:57756) 2024-09-21T03:31:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6118cb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 0x55cb3d11e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44746/0 (socket says 172.21.15.112:44746) 2024-09-21T03:31:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6119cb9640 1 -- 172.21.15.112:0/4283688266 learned_addr learned my addr 172.21.15.112:0/4283688266 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6119cb9640 1 --2- 172.21.15.112:0/4283688266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd5f070 0x55cb3d10eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6118cb7640 1 -- 172.21.15.112:0/4283688266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd5f070 msgr2=0x55cb3d10eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6118cb7640 1 --2- 172.21.15.112:0/4283688266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd5f070 0x55cb3d10eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6118cb7640 1 -- 172.21.15.112:0/4283688266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd361b0 msgr2=0x55cb3d107d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6118cb7640 1 --2- 172.21.15.112:0/4283688266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd361b0 0x55cb3d107d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6118cb7640 1 -- 172.21.15.112:0/4283688266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb3d108940 con 0x55cb3cd3e390 2024-09-21T03:31:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.687+0000 7f6119cb9640 1 --2- 172.21.15.112:0/4283688266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd361b0 0x55cb3d107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.688+0000 7f6118cb7640 1 --2- 172.21.15.112:0/4283688266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 0x55cb3d11e020 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x55cb3d1245a0 tx=0x7f61040047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae9bac442239c18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.688+0000 7f60feffd640 1 -- 172.21.15.112:0/4283688266 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610402b070 con 0x55cb3cd3e390 2024-09-21T03:31:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.688+0000 7f60feffd640 1 -- 172.21.15.112:0/4283688266 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610401cbc0 con 0x55cb3cd3e390 2024-09-21T03:31:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.688+0000 7f60feffd640 1 -- 172.21.15.112:0/4283688266 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104026660 con 0x55cb3cd3e390 2024-09-21T03:31:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.689+0000 7f6121c90740 1 -- 172.21.15.112:0/4283688266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 msgr2=0x55cb3d11e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.689+0000 7f6121c90740 1 --2- 172.21.15.112:0/4283688266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 0x55cb3d11e020 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x55cb3d1245a0 tx=0x7f61040047d0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.689+0000 7f6121c90740 1 -- 172.21.15.112:0/4283688266 shutdown_connections 2024-09-21T03:31:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.689+0000 7f6121c90740 1 --2- 172.21.15.112:0/4283688266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd5f070 0x55cb3d10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.689+0000 7f6121c90740 1 --2- 172.21.15.112:0/4283688266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd3e390 0x55cb3d11e020 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.689+0000 7f6121c90740 1 --2- 172.21.15.112:0/4283688266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd361b0 0x55cb3d107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.689+0000 7f6121c90740 1 -- 172.21.15.112:0/4283688266 >> 172.21.15.112:0/4283688266 conn(0x55cb3cd53460 msgr2=0x55cb3d107630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.689+0000 7f6121c90740 1 -- 172.21.15.112:0/4283688266 shutdown_connections 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.689+0000 7f6121c90740 1 -- 172.21.15.112:0/4283688266 wait complete. 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.690+0000 7f6121c90740 1 -- start start 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.690+0000 7f6121c90740 1 Processor -- start 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.690+0000 7f6121c90740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd361b0 0x55cb3d116370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.690+0000 7f6121c90740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd3e390 0x55cb3d1168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.690+0000 7f6121c90740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd5f070 0x55cb3d18d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.690+0000 7f6121c90740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61141add80 con 0x55cb3cd361b0 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.690+0000 7f6121c90740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61141af1f0 con 0x55cb3cd3e390 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.690+0000 7f6121c90740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f61141ae450 con 0x55cb3cd5f070 2024-09-21T03:31:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6121c90740 10 client.0 _ll_register_callbacks cb 0x7f61040093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6119cb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd361b0 0x55cb3d116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6119cb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd361b0 0x55cb3d116370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44760/0 (socket says 172.21.15.112:44760) 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6119cb9640 1 -- 172.21.15.112:0/744638523 learned_addr learned my addr 172.21.15.112:0/744638523 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6118cb7640 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd3e390 0x55cb3d1168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6119cb9640 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd5f070 0x55cb3d18d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6119cb9640 1 -- 172.21.15.112:0/744638523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd5f070 msgr2=0x55cb3d18d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6119cb9640 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd5f070 0x55cb3d18d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6119cb9640 1 -- 172.21.15.112:0/744638523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd3e390 msgr2=0x55cb3d1168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6119cb9640 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd3e390 0x55cb3d1168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6119cb9640 1 -- 172.21.15.112:0/744638523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3d117150 con 0x55cb3cd361b0 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6119cb9640 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd361b0 0x55cb3d116370 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f610c00fde0 tx=0x7f610c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6c4d730ef2a62f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f60fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f60fcff9640 1 -- 172.21.15.112:0/744638523 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c02b070 con 0x55cb3cd361b0 2024-09-21T03:31:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f60fcff9640 1 -- 172.21.15.112:0/744638523 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f610c027470 con 0x55cb3cd361b0 2024-09-21T03:31:26.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.691+0000 7f6121c90740 1 -- 172.21.15.112:0/744638523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f610401cbc0 con 0x55cb3cd361b0 2024-09-21T03:31:26.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f60fcff9640 1 -- 172.21.15.112:0/744638523 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f610c0040f0 con 0x55cb3cd361b0 2024-09-21T03:31:26.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f6121c90740 10 client.16032 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:26.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f6121c90740 1 -- 172.21.15.112:0/744638523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cb3d18ee00 con 0x55cb3cd361b0 2024-09-21T03:31:26.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f60fcff9640 1 -- 172.21.15.112:0/744638523 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610c00f990 con 0x55cb3cd361b0 2024-09-21T03:31:26.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f60fcff9640 1 -- 172.21.15.112:0/744638523 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f610c019470 con 0x55cb3cd361b0 2024-09-21T03:31:26.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f6121c90740 10 client.16032 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:26.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f6121c90740 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:31:26.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f6121c90740 20 client.16032 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:26.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f6121c90740 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55cb3ccf2da0 0x55cb3d191560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:26.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f6121c90740 4 client.16032 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f6121c90740 1 -- 172.21.15.112:0/744638523 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f610401cbc0 con 0x55cb3ccf2da0 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.692+0000 7f6118cb7640 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55cb3ccf2da0 0x55cb3d191560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f60fcff9640 10 client.16032 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f6118cb7640 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55cb3ccf2da0 0x55cb3d191560 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b1aa862da096f6 server_cookie=82aabc2623a4268f in_seq=0 out_seq=0 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f60fcff9640 1 -- 172.21.15.112:0/744638523 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x7f610401cbc0 con 0x55cb3ccf2da0 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f60fcff9640 10 client.16032 handle_command_reply: tid=0 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f6121c90740 1 client.16032 shutdown 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f6121c90740 1 -- 172.21.15.112:0/744638523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd361b0 msgr2=0x55cb3d116370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f6121c90740 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd361b0 0x55cb3d116370 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f610c00fde0 tx=0x7f610c000f40 comp rx=0 tx=0).stop 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f6121c90740 1 -- 172.21.15.112:0/744638523 shutdown_connections 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f6121c90740 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55cb3ccf2da0 0x55cb3d191560 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f6121c90740 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb3cd5f070 0x55cb3d18d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f6121c90740 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb3cd3e390 0x55cb3d1168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f6121c90740 1 --2- 172.21.15.112:0/744638523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb3cd361b0 0x55cb3d116370 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f6121c90740 1 -- 172.21.15.112:0/744638523 >> 172.21.15.112:0/744638523 conn(0x55cb3cd53460 msgr2=0x55cb3d1108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:26.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.693+0000 7f60fcff9640 0 client.16032 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:26.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f6121c90740 1 -- 172.21.15.112:0/744638523 shutdown_connections 2024-09-21T03:31:26.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f6121c90740 1 -- 172.21.15.112:0/744638523 wait complete. 2024-09-21T03:31:26.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f6121c90740 20 client.16032 trim_cache size 0 max 16384 2024-09-21T03:31:26.699 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:31:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:31:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:31:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:31:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "d5e6af18-7419-4d89-a8cb-849ebeffa647", 2024-09-21T03:31:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:31:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:31:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:31:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:31:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:31:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:31:26.703 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:31:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f611b71c640 1 -- 172.21.15.112:0/4187403256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61000b6680 msgr2=0x7f61000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f611b71c640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61000b6680 0x7f61000b8b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f61101052a0 tx=0x7f611010b590 comp rx=0 tx=0).stop 2024-09-21T03:31:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f611b71c640 1 -- 172.21.15.112:0/4187403256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6114104880 msgr2=0x7f611419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f611b71c640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6114104880 0x7f611419a540 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6110006010 tx=0x7f6110004060 comp rx=0 tx=0).stop 2024-09-21T03:31:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f611b71c640 1 -- 172.21.15.112:0/4187403256 shutdown_connections 2024-09-21T03:31:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f611b71c640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f61000b6680 0x7f61000b8b90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f611b71c640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f611410de30 0x7f611419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f611b71c640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61141052b0 0x7f611419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f611b71c640 1 --2- 172.21.15.112:0/4187403256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6114104880 0x7f611419a540 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.694+0000 7f611b71c640 1 -- 172.21.15.112:0/4187403256 >> 172.21.15.112:0/4187403256 conn(0x7f61140797e0 msgr2=0x7f61141079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.695+0000 7f611b71c640 1 -- 172.21.15.112:0/4187403256 shutdown_connections 2024-09-21T03:31:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:26.695+0000 7f611b71c640 1 -- 172.21.15.112:0/4187403256 wait complete. 2024-09-21T03:31:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:26 smithi112 ceph-mon[31442]: pgmap v391: 97 pgs: 97 active+clean; 106 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 327 op/s 2024-09-21T03:31:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:26 smithi203 ceph-mon[34723]: pgmap v391: 97 pgs: 97 active+clean; 106 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 327 op/s 2024-09-21T03:31:26.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:26 smithi187 ceph-mon[34603]: pgmap v391: 97 pgs: 97 active+clean; 106 MiB data, 976 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 327 op/s 2024-09-21T03:31:27.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 176540 11.14 MB/sec warmup 87 sec latency 76.162 ms 2024-09-21T03:31:27.445 INFO:tasks.cephfs.filesystem:scrub status for tag:d5e6af18-7419-4d89-a8cb-849ebeffa647 - {'path': '/', 'tag': 'd5e6af18-7419-4d89-a8cb-849ebeffa647', 'options': 'recursive,force'} 2024-09-21T03:31:28.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 178146 11.11 MB/sec warmup 88 sec latency 32.494 ms 2024-09-21T03:31:28.747 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:28 smithi112 ceph-mon[31442]: pgmap v392: 97 pgs: 97 active+clean; 108 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 290 op/s 2024-09-21T03:31:28.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:28 smithi203 ceph-mon[34723]: pgmap v392: 97 pgs: 97 active+clean; 108 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 290 op/s 2024-09-21T03:31:28.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:28 smithi187 ceph-mon[34603]: pgmap v392: 97 pgs: 97 active+clean; 108 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 290 op/s 2024-09-21T03:31:29.215 INFO:tasks.workunit.client.0.smithi112.stdout: 1 181127 11.19 MB/sec warmup 89 sec latency 33.147 ms 2024-09-21T03:31:30.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 182802 11.13 MB/sec warmup 90 sec latency 25.364 ms 2024-09-21T03:31:30.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:30 smithi112 ceph-mon[31442]: pgmap v393: 97 pgs: 97 active+clean; 113 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 350 op/s 2024-09-21T03:31:30.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:30 smithi203 ceph-mon[34723]: pgmap v393: 97 pgs: 97 active+clean; 113 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 350 op/s 2024-09-21T03:31:30.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:30 smithi187 ceph-mon[34603]: pgmap v393: 97 pgs: 97 active+clean; 113 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 350 op/s 2024-09-21T03:31:31.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 185044 11.16 MB/sec warmup 91 sec latency 10.498 ms 2024-09-21T03:31:32.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 187985 11.23 MB/sec warmup 92 sec latency 17.049 ms 2024-09-21T03:31:32.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:32 smithi112 ceph-mon[31442]: pgmap v394: 97 pgs: 97 active+clean; 114 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2024-09-21T03:31:32.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:32 smithi203 ceph-mon[34723]: pgmap v394: 97 pgs: 97 active+clean; 114 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2024-09-21T03:31:32.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:32 smithi187 ceph-mon[34603]: pgmap v394: 97 pgs: 97 active+clean; 114 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2024-09-21T03:31:33.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 189440 11.18 MB/sec warmup 93 sec latency 35.926 ms 2024-09-21T03:31:34.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 191930 11.21 MB/sec warmup 94 sec latency 10.691 ms 2024-09-21T03:31:34.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:34 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:31:34.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:34 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:31:34.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:34 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:31:34.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:34 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:31:34.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:34 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:31:34.536 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:34 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:31:35.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 194309 11.21 MB/sec warmup 95 sec latency 14.923 ms 2024-09-21T03:31:35.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:35 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:31:35.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:35 smithi203 ceph-mon[34723]: pgmap v395: 97 pgs: 97 active+clean; 115 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2024-09-21T03:31:35.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:35 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:31:35.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:35 smithi187 ceph-mon[34603]: pgmap v395: 97 pgs: 97 active+clean; 115 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2024-09-21T03:31:35.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:35 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:31:35.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:35 smithi112 ceph-mon[31442]: pgmap v395: 97 pgs: 97 active+clean; 115 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2024-09-21T03:31:36.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 195999 11.18 MB/sec warmup 96 sec latency 47.077 ms 2024-09-21T03:31:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:36 smithi112 ceph-mon[31442]: pgmap v396: 97 pgs: 97 active+clean; 120 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 361 op/s 2024-09-21T03:31:36.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:36 smithi203 ceph-mon[34723]: pgmap v396: 97 pgs: 97 active+clean; 120 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 361 op/s 2024-09-21T03:31:36.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:36 smithi187 ceph-mon[34603]: pgmap v396: 97 pgs: 97 active+clean; 120 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 361 op/s 2024-09-21T03:31:37.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 198893 11.25 MB/sec warmup 97 sec latency 15.963 ms 2024-09-21T03:31:38.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 200619 11.20 MB/sec warmup 98 sec latency 27.727 ms 2024-09-21T03:31:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:38 smithi112 ceph-mon[31442]: pgmap v397: 97 pgs: 97 active+clean; 120 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 304 op/s 2024-09-21T03:31:38.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:38 smithi203 ceph-mon[34723]: pgmap v397: 97 pgs: 97 active+clean; 120 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 304 op/s 2024-09-21T03:31:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:38 smithi187 ceph-mon[34603]: pgmap v397: 97 pgs: 97 active+clean; 120 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 304 op/s 2024-09-21T03:31:39.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 202779 11.22 MB/sec warmup 99 sec latency 18.890 ms 2024-09-21T03:31:40.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 205820 11.29 MB/sec warmup 100 sec latency 14.858 ms 2024-09-21T03:31:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:40 smithi112 ceph-mon[31442]: pgmap v398: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 368 op/s 2024-09-21T03:31:40.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:40 smithi203 ceph-mon[34723]: pgmap v398: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 368 op/s 2024-09-21T03:31:40.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:40 smithi187 ceph-mon[34603]: pgmap v398: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.2 MiB/s wr, 368 op/s 2024-09-21T03:31:41.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 207308 11.24 MB/sec warmup 101 sec latency 11.812 ms 2024-09-21T03:31:42.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 209713 11.27 MB/sec warmup 102 sec latency 12.744 ms 2024-09-21T03:31:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:42 smithi112 ceph-mon[31442]: pgmap v399: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 302 op/s 2024-09-21T03:31:42.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:42 smithi203 ceph-mon[34723]: pgmap v399: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 302 op/s 2024-09-21T03:31:42.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:42 smithi187 ceph-mon[34603]: pgmap v399: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 302 op/s 2024-09-21T03:31:43.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 212055 11.26 MB/sec warmup 103 sec latency 34.330 ms 2024-09-21T03:31:44.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 213645 11.24 MB/sec warmup 104 sec latency 33.304 ms 2024-09-21T03:31:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:44 smithi112 ceph-mon[31442]: pgmap v400: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-09-21T03:31:44.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:44 smithi203 ceph-mon[34723]: pgmap v400: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-09-21T03:31:44.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:44 smithi187 ceph-mon[34603]: pgmap v400: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-09-21T03:31:45.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 216639 11.31 MB/sec warmup 105 sec latency 11.074 ms 2024-09-21T03:31:46.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 218128 11.25 MB/sec warmup 106 sec latency 14.822 ms 2024-09-21T03:31:46.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:46 smithi112 ceph-mon[31442]: pgmap v401: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-09-21T03:31:46.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:46 smithi203 ceph-mon[34723]: pgmap v401: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-09-21T03:31:46.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:46 smithi187 ceph-mon[34603]: pgmap v401: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-09-21T03:31:47.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 220422 11.28 MB/sec warmup 107 sec latency 10.378 ms 2024-09-21T03:31:48.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 222880 11.29 MB/sec warmup 108 sec latency 29.945 ms 2024-09-21T03:31:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:48 smithi112 ceph-mon[31442]: pgmap v402: 97 pgs: 97 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-09-21T03:31:48.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:48 smithi203 ceph-mon[34723]: pgmap v402: 97 pgs: 97 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-09-21T03:31:48.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:48 smithi187 ceph-mon[34603]: pgmap v402: 97 pgs: 97 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-09-21T03:31:49.216 INFO:tasks.workunit.client.0.smithi112.stdout: 1 224456 11.25 MB/sec warmup 109 sec latency 30.082 ms 2024-09-21T03:31:50.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 227202 11.30 MB/sec warmup 110 sec latency 60.600 ms 2024-09-21T03:31:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:50 smithi112 ceph-mon[31442]: pgmap v403: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 350 op/s 2024-09-21T03:31:50.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:50 smithi203 ceph-mon[34723]: pgmap v403: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 350 op/s 2024-09-21T03:31:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:50 smithi187 ceph-mon[34603]: pgmap v403: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 350 op/s 2024-09-21T03:31:51.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 228742 11.26 MB/sec warmup 111 sec latency 16.612 ms 2024-09-21T03:31:52.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 231062 11.29 MB/sec warmup 112 sec latency 10.099 ms 2024-09-21T03:31:52.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:52 smithi112 ceph-mon[31442]: pgmap v404: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 288 op/s 2024-09-21T03:31:52.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:52 smithi203 ceph-mon[34723]: pgmap v404: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 288 op/s 2024-09-21T03:31:52.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:52 smithi187 ceph-mon[34603]: pgmap v404: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 288 op/s 2024-09-21T03:31:53.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 233627 11.30 MB/sec warmup 113 sec latency 41.782 ms 2024-09-21T03:31:54.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 235255 11.27 MB/sec warmup 114 sec latency 24.364 ms 2024-09-21T03:31:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:54 smithi112 ceph-mon[31442]: pgmap v405: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 279 op/s 2024-09-21T03:31:54.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:54 smithi203 ceph-mon[34723]: pgmap v405: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 279 op/s 2024-09-21T03:31:54.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:54 smithi187 ceph-mon[34603]: pgmap v405: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 279 op/s 2024-09-21T03:31:55.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 238076 11.32 MB/sec warmup 115 sec latency 13.467 ms 2024-09-21T03:31:56.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 240254 11.29 MB/sec warmup 116 sec latency 16.412 ms 2024-09-21T03:31:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:56 smithi112 ceph-mon[31442]: pgmap v406: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-09-21T03:31:56.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:56 smithi203 ceph-mon[34723]: pgmap v406: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-09-21T03:31:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:56 smithi187 ceph-mon[34603]: pgmap v406: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-09-21T03:31:57.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 242033 11.30 MB/sec warmup 117 sec latency 15.047 ms 2024-09-21T03:31:57.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:31:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:31:58.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 244855 11.35 MB/sec warmup 118 sec latency 47.830 ms 2024-09-21T03:31:58.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.402+0000 7fca3d78f640 1 -- 172.21.15.112:0/4020507305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca381094c0 msgr2=0x7fca381098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.402+0000 7fca3d78f640 1 --2- 172.21.15.112:0/4020507305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca381094c0 0x7fca381098d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009900 tx=0x7fca2c02a410 comp rx=0 tx=0).stop 2024-09-21T03:31:58.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.402+0000 7fca3d78f640 1 -- 172.21.15.112:0/4020507305 shutdown_connections 2024-09-21T03:31:58.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.402+0000 7fca3d78f640 1 --2- 172.21.15.112:0/4020507305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca38109e10 0x7fca38110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.402+0000 7fca3d78f640 1 --2- 172.21.15.112:0/4020507305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca381094c0 0x7fca381098d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.402+0000 7fca3d78f640 1 --2- 172.21.15.112:0/4020507305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca38103b80 0x7fca38103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.402+0000 7fca3d78f640 1 -- 172.21.15.112:0/4020507305 >> 172.21.15.112:0/4020507305 conn(0x7fca380ff610 msgr2=0x7fca38101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.402+0000 7fca3d78f640 1 -- 172.21.15.112:0/4020507305 shutdown_connections 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.402+0000 7fca3d78f640 1 -- 172.21.15.112:0/4020507305 wait complete. 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.402+0000 7fca3d78f640 1 Processor -- start 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca3d78f640 1 -- start start 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca3d78f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca38103b80 0x7fca3819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca3d78f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca381094c0 0x7fca3819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca3d78f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca38109e10 0x7fca3819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca3d78f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca38113ad0 con 0x7fca381094c0 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca3d78f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fca38113950 con 0x7fca38103b80 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca3d78f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fca38113c50 con 0x7fca38109e10 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca367fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca381094c0 0x7fca3819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca367fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca381094c0 0x7fca3819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47048/0 (socket says 172.21.15.112:47048) 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca36ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca38103b80 0x7fca3819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca367fc640 1 -- 172.21.15.112:0/3756435743 learned_addr learned my addr 172.21.15.112:0/3756435743 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca377fe640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca38109e10 0x7fca3819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca367fc640 1 -- 172.21.15.112:0/3756435743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca38109e10 msgr2=0x7fca3819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca367fc640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca38109e10 0x7fca3819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca367fc640 1 -- 172.21.15.112:0/3756435743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca38103b80 msgr2=0x7fca3819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca367fc640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca38103b80 0x7fca3819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca367fc640 1 -- 172.21.15.112:0/3756435743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca381a0410 con 0x7fca381094c0 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.403+0000 7fca36ffd640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca38103b80 0x7fca3819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.404+0000 7fca367fc640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca381094c0 0x7fca3819aa40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fca2c02a930 tx=0x7fca2c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.404+0000 7fca1ffff640 1 -- 172.21.15.112:0/3756435743 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c03f030 con 0x7fca381094c0 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.404+0000 7fca1ffff640 1 -- 172.21.15.112:0/3756435743 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca2c004490 con 0x7fca381094c0 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.404+0000 7fca3d78f640 1 -- 172.21.15.112:0/3756435743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca381a06a0 con 0x7fca381094c0 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.404+0000 7fca1ffff640 1 -- 172.21.15.112:0/3756435743 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c035070 con 0x7fca381094c0 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.404+0000 7fca3d78f640 1 -- 172.21.15.112:0/3756435743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca380755f0 con 0x7fca381094c0 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.407+0000 7fca43d03740 1 Processor -- start 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.407+0000 7fca43d03740 1 -- start start 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca43d03740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f80a01070 0x559f80da8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca43d03740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 0x559f80da8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca43d03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809e0390 0x559f80da90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca43d03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f80d1ee50 con 0x559f809e0390 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca43d03740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559f80d0e280 con 0x559f80a01070 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca43d03740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559f80d12090 con 0x559f809d81b0 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca377fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f80a01070 0x559f80da8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca36ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 0x559f80da8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca377fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f80a01070 0x559f80da8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41044/0 (socket says 172.21.15.112:41044) 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca36ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 0x559f80da8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60966/0 (socket says 172.21.15.112:60966) 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca377fe640 1 -- 172.21.15.112:0/2189888032 learned_addr learned my addr 172.21.15.112:0/2189888032 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca1ffff640 1 -- 172.21.15.112:0/3756435743 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fca2c042030 con 0x7fca381094c0 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca367fc640 1 --2- 172.21.15.112:0/2189888032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809e0390 0x559f80da90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca377fe640 1 -- 172.21.15.112:0/2189888032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 msgr2=0x559f80da8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca377fe640 1 --2- 172.21.15.112:0/2189888032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 0x559f80da8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca377fe640 1 -- 172.21.15.112:0/2189888032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809e0390 msgr2=0x559f80da90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca377fe640 1 --2- 172.21.15.112:0/2189888032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809e0390 0x559f80da90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca377fe640 1 -- 172.21.15.112:0/2189888032 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f80a6c650 con 0x559f80a01070 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca36ffd640 1 --2- 172.21.15.112:0/2189888032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 0x559f80da8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.408+0000 7fca367fc640 1 --2- 172.21.15.112:0/2189888032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809e0390 0x559f80da90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.409+0000 7fca377fe640 1 --2- 172.21.15.112:0/2189888032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f80a01070 0x559f80da8650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca30009fd0 tx=0x7fca300118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85a8f41697c066a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.409+0000 7fca1d7fa640 1 -- 172.21.15.112:0/2189888032 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30004d20 con 0x559f80a01070 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.409+0000 7fca1d7fa640 1 -- 172.21.15.112:0/2189888032 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca30004050 con 0x559f80a01070 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.409+0000 7fca1d7fa640 1 -- 172.21.15.112:0/2189888032 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30005280 con 0x559f80a01070 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.409+0000 7fca1ffff640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca200b6700 0x7fca200b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.409+0000 7fca36ffd640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca200b6700 0x7fca200b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.409+0000 7fca1ffff640 1 -- 172.21.15.112:0/3756435743 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fca2c107960 con 0x7fca381094c0 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca43d03740 1 -- 172.21.15.112:0/2189888032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f80a01070 msgr2=0x559f80da8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca36ffd640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca200b6700 0x7fca200b8c10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x559f80da72d0 tx=0x7fca28012040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca43d03740 1 --2- 172.21.15.112:0/2189888032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f80a01070 0x559f80da8650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca30009fd0 tx=0x7fca300118d0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca43d03740 1 -- 172.21.15.112:0/2189888032 shutdown_connections 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca43d03740 1 --2- 172.21.15.112:0/2189888032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809e0390 0x559f80da90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca43d03740 1 --2- 172.21.15.112:0/2189888032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 0x559f80da8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca43d03740 1 --2- 172.21.15.112:0/2189888032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f80a01070 0x559f80da8650 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca43d03740 1 -- 172.21.15.112:0/2189888032 >> 172.21.15.112:0/2189888032 conn(0x559f809f5460 msgr2=0x559f80da6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca43d03740 1 -- 172.21.15.112:0/2189888032 shutdown_connections 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca43d03740 1 -- 172.21.15.112:0/2189888032 wait complete. 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca43d03740 1 -- start start 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.410+0000 7fca43d03740 1 Processor -- start 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca43d03740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 0x559f80db3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca43d03740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809e0390 0x559f80db4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca43d03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f80a01070 0x559f80dc0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca43d03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f80b6f700 con 0x559f80a01070 2024-09-21T03:31:58.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca43d03740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559f80ca9110 con 0x559f809e0390 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca43d03740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559f80db6e30 con 0x559f809d81b0 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca43d03740 10 client.0 _ll_register_callbacks cb 0x559f80cc2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca377fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f80a01070 0x559f80dc0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca36ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 0x559f80db3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca377fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f80a01070 0x559f80dc0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47058/0 (socket says 172.21.15.112:47058) 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca377fe640 1 -- 172.21.15.112:0/4262921597 learned_addr learned my addr 172.21.15.112:0/4262921597 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca367fc640 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809e0390 0x559f80db4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca377fe640 1 -- 172.21.15.112:0/4262921597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 msgr2=0x559f80db3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca377fe640 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 0x559f80db3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca377fe640 1 -- 172.21.15.112:0/4262921597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809e0390 msgr2=0x559f80db4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca377fe640 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809e0390 0x559f80db4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca377fe640 1 -- 172.21.15.112:0/4262921597 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f80cd6c40 con 0x559f80a01070 2024-09-21T03:31:58.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca377fe640 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f80a01070 0x559f80dc0150 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fca30003fd0 tx=0x7fca300044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bcf612628c5df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.411+0000 7fca1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca1aff5640 1 -- 172.21.15.112:0/4262921597 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30028020 con 0x559f80a01070 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca1aff5640 1 -- 172.21.15.112:0/4262921597 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fca30004c50 con 0x559f80a01070 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca43d03740 1 -- 172.21.15.112:0/4262921597 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca30004050 con 0x559f80a01070 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca1aff5640 1 -- 172.21.15.112:0/4262921597 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca30033050 con 0x559f80a01070 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca43d03740 10 client.16041 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca43d03740 1 -- 172.21.15.112:0/4262921597 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f80dc2aa0 con 0x559f80a01070 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca1aff5640 1 -- 172.21.15.112:0/4262921597 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca30004050 con 0x559f80a01070 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca1aff5640 1 -- 172.21.15.112:0/4262921597 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca3000f4e0 con 0x559f80a01070 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca43d03740 10 client.16041 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca43d03740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca43d03740 20 client.16041 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca43d03740 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559f809bf9f0 0x559f80dc5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca43d03740 4 client.16041 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:31:58.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca43d03740 1 -- 172.21.15.112:0/4262921597 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f80d20da0 con 0x559f809bf9f0 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.412+0000 7fca36ffd640 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559f809bf9f0 0x559f80dc5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.413+0000 7fca1aff5640 10 client.16041 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.413+0000 7fca36ffd640 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559f809bf9f0 0x559f80dc5f10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae0218475f2e835 server_cookie=ad8b6ed40407447c in_seq=0 out_seq=0 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca1aff5640 1 -- 172.21.15.112:0/4262921597 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f80d20da0 con 0x559f809bf9f0 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca1aff5640 10 client.16041 handle_command_reply: tid=0 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 1 client.16041 shutdown 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 1 -- 172.21.15.112:0/4262921597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f80a01070 msgr2=0x559f80dc0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f80a01070 0x559f80dc0150 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fca30003fd0 tx=0x7fca300044b0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 1 -- 172.21.15.112:0/4262921597 shutdown_connections 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559f809bf9f0 0x559f80dc5f10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca1aff5640 0 client.16041 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f80a01070 0x559f80dc0150 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809e0390 0x559f80db4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 1 --2- 172.21.15.112:0/4262921597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809d81b0 0x559f80db3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 1 -- 172.21.15.112:0/4262921597 >> 172.21.15.112:0/4262921597 conn(0x559f809f5460 msgr2=0x559f80dab7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 1 -- 172.21.15.112:0/4262921597 shutdown_connections 2024-09-21T03:31:58.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 1 -- 172.21.15.112:0/4262921597 wait complete. 2024-09-21T03:31:58.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.414+0000 7fca43d03740 20 client.16041 trim_cache size 0 max 16384 2024-09-21T03:31:58.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.434+0000 7fca43d03740 1 Processor -- start 2024-09-21T03:31:58.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.434+0000 7fca43d03740 1 -- start start 2024-09-21T03:31:58.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.434+0000 7fca43d03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809bf9f0 0x559f80dc0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.434+0000 7fca43d03740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809d81b0 0x559f80db3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.434+0000 7fca43d03740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809e0390 0x559f80db3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.434+0000 7fca43d03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca381afbe0 con 0x559f809bf9f0 2024-09-21T03:31:58.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.434+0000 7fca43d03740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fca381ad4b0 con 0x559f809d81b0 2024-09-21T03:31:58.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.434+0000 7fca43d03740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fca381ab300 con 0x559f809e0390 2024-09-21T03:31:58.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.434+0000 7fca367fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809d81b0 0x559f80db3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.434+0000 7fca36ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809bf9f0 0x559f80dc0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca367fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809d81b0 0x559f80db3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41054/0 (socket says 172.21.15.112:41054) 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca367fc640 1 -- 172.21.15.112:0/986276443 learned_addr learned my addr 172.21.15.112:0/986276443 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca377fe640 1 --2- 172.21.15.112:0/986276443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809e0390 0x559f80db3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca36ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809bf9f0 0x559f80dc0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47066/0 (socket says 172.21.15.112:47066) 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca367fc640 1 -- 172.21.15.112:0/986276443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809e0390 msgr2=0x559f80db3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca367fc640 1 --2- 172.21.15.112:0/986276443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809e0390 0x559f80db3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca367fc640 1 -- 172.21.15.112:0/986276443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809bf9f0 msgr2=0x559f80dc0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca367fc640 1 --2- 172.21.15.112:0/986276443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809bf9f0 0x559f80dc0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca367fc640 1 -- 172.21.15.112:0/986276443 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f80db4f80 con 0x559f809d81b0 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca367fc640 1 --2- 172.21.15.112:0/986276443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809d81b0 0x559f80db3a70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0cab30 tx=0x7fca2c03b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a80633bdbe47a6d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca1cff9640 1 -- 172.21.15.112:0/986276443 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c049070 con 0x559f809d81b0 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca1cff9640 1 -- 172.21.15.112:0/986276443 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca2c0cd3d0 con 0x559f809d81b0 2024-09-21T03:31:58.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.435+0000 7fca1cff9640 1 -- 172.21.15.112:0/986276443 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0c9cc0 con 0x559f809d81b0 2024-09-21T03:31:58.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.436+0000 7fca43d03740 1 -- 172.21.15.112:0/986276443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809d81b0 msgr2=0x559f80db3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.436+0000 7fca43d03740 1 --2- 172.21.15.112:0/986276443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809d81b0 0x559f80db3a70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0cab30 tx=0x7fca2c03b560 comp rx=0 tx=0).stop 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.436+0000 7fca43d03740 1 -- 172.21.15.112:0/986276443 shutdown_connections 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.436+0000 7fca43d03740 1 --2- 172.21.15.112:0/986276443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809e0390 0x559f80db3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.436+0000 7fca43d03740 1 --2- 172.21.15.112:0/986276443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809d81b0 0x559f80db3a70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.436+0000 7fca43d03740 1 --2- 172.21.15.112:0/986276443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809bf9f0 0x559f80dc0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.436+0000 7fca43d03740 1 -- 172.21.15.112:0/986276443 >> 172.21.15.112:0/986276443 conn(0x559f809f5460 msgr2=0x559f80db2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.436+0000 7fca43d03740 1 -- 172.21.15.112:0/986276443 shutdown_connections 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.436+0000 7fca43d03740 1 -- 172.21.15.112:0/986276443 wait complete. 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca43d03740 1 -- start start 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca43d03740 1 Processor -- start 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca43d03740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809bf9f0 0x559f80dbca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca43d03740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809d81b0 0x559f80db7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca43d03740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809e0390 0x559f80db8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca43d03740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca381ab970 con 0x559f809d81b0 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca43d03740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fca381acde0 con 0x559f809bf9f0 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca43d03740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fca381ac040 con 0x559f809e0390 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca43d03740 10 client.0 _ll_register_callbacks cb 0x7fca30009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca367fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809d81b0 0x559f80db7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca367fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809d81b0 0x559f80db7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47070/0 (socket says 172.21.15.112:47070) 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.437+0000 7fca367fc640 1 -- 172.21.15.112:0/1709990355 learned_addr learned my addr 172.21.15.112:0/1709990355 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca377fe640 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809e0390 0x559f80db8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca36ffd640 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809bf9f0 0x559f80dbca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca377fe640 1 -- 172.21.15.112:0/1709990355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809bf9f0 msgr2=0x559f80dbca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca377fe640 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809bf9f0 0x559f80dbca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca377fe640 1 -- 172.21.15.112:0/1709990355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809d81b0 msgr2=0x559f80db7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca377fe640 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809d81b0 0x559f80db7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca377fe640 1 -- 172.21.15.112:0/1709990355 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f80e2f8a0 con 0x559f809e0390 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca377fe640 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809e0390 0x559f80db8630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca30002230 tx=0x7fca30005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c443817dc6ca0095 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:31:58.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca1aff5640 1 -- 172.21.15.112:0/1709990355 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3003b020 con 0x559f809e0390 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca1aff5640 1 -- 172.21.15.112:0/1709990355 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fca3000fb70 con 0x559f809e0390 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.438+0000 7fca43d03740 1 -- 172.21.15.112:0/1709990355 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2c0cd3d0 con 0x559f809e0390 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca1aff5640 1 -- 172.21.15.112:0/1709990355 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca30046050 con 0x559f809e0390 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca43d03740 10 client.25615 fetch_fsmap learned FSMap version 26 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca43d03740 1 -- 172.21.15.112:0/1709990355 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f80e309d0 con 0x559f809e0390 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca1aff5640 1 -- 172.21.15.112:0/1709990355 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca2c0cd3d0 con 0x559f809e0390 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca1aff5640 1 -- 172.21.15.112:0/1709990355 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca30019630 con 0x559f809e0390 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca43d03740 10 client.25615 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca43d03740 10 client.25615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca43d03740 20 client.25615 populate_metadata read hostname 'smithi112' 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca43d03740 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559f80a01070 0x559f80e331b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca43d03740 4 client.25615 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca43d03740 1 -- 172.21.15.112:0/1709990355 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f80d1c850 con 0x559f80a01070 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.439+0000 7fca36ffd640 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559f80a01070 0x559f80e331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:31:58.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca1aff5640 10 client.25615 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:58.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca36ffd640 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559f80a01070 0x559f80e331b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a35ac4862ad0157c server_cookie=80d82cbd5d63f556 in_seq=0 out_seq=0 2024-09-21T03:31:58.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca1aff5640 1 -- 172.21.15.112:0/1709990355 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x559f80d1c850 con 0x559f80a01070 2024-09-21T03:31:58.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca1aff5640 10 client.25615 handle_command_reply: tid=0 2024-09-21T03:31:58.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca43d03740 1 client.25615 shutdown 2024-09-21T03:31:58.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca43d03740 1 -- 172.21.15.112:0/1709990355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809e0390 msgr2=0x559f80db8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca43d03740 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809e0390 0x559f80db8630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca30002230 tx=0x7fca30005f10 comp rx=0 tx=0).stop 2024-09-21T03:31:58.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca43d03740 1 -- 172.21.15.112:0/1709990355 shutdown_connections 2024-09-21T03:31:58.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca43d03740 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559f80a01070 0x559f80e331b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca43d03740 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f809e0390 0x559f80db8630 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca43d03740 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f809d81b0 0x559f80db7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca1aff5640 0 client.25615 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:31:58.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.440+0000 7fca43d03740 1 --2- 172.21.15.112:0/1709990355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f809bf9f0 0x559f80dbca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca43d03740 1 -- 172.21.15.112:0/1709990355 >> 172.21.15.112:0/1709990355 conn(0x559f809f5460 msgr2=0x559f80db24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca43d03740 1 -- 172.21.15.112:0/1709990355 shutdown_connections 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca43d03740 1 -- 172.21.15.112:0/1709990355 wait complete. 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca43d03740 20 client.25615 trim_cache size 0 max 16384 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:31:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:31:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:31:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:31:58.451 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:31:58.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca3d78f640 1 -- 172.21.15.112:0/3756435743 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca200b6700 msgr2=0x7fca200b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca3d78f640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca200b6700 0x7fca200b8c10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x559f80da72d0 tx=0x7fca28012040 comp rx=0 tx=0).stop 2024-09-21T03:31:58.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca3d78f640 1 -- 172.21.15.112:0/3756435743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca381094c0 msgr2=0x7fca3819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:31:58.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca3d78f640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca381094c0 0x7fca3819aa40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fca2c02a930 tx=0x7fca2c031a20 comp rx=0 tx=0).stop 2024-09-21T03:31:58.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca3d78f640 1 -- 172.21.15.112:0/3756435743 shutdown_connections 2024-09-21T03:31:58.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca3d78f640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca200b6700 0x7fca200b8c10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca3d78f640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca38109e10 0x7fca3819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca3d78f640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca381094c0 0x7fca3819aa40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca3d78f640 1 --2- 172.21.15.112:0/3756435743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca38103b80 0x7fca3819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:31:58.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.441+0000 7fca3d78f640 1 -- 172.21.15.112:0/3756435743 >> 172.21.15.112:0/3756435743 conn(0x7fca380ff610 msgr2=0x7fca38101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:31:58.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.442+0000 7fca3d78f640 1 -- 172.21.15.112:0/3756435743 shutdown_connections 2024-09-21T03:31:58.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:31:58.442+0000 7fca3d78f640 1 -- 172.21.15.112:0/3756435743 wait complete. 2024-09-21T03:31:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:31:58 smithi112 ceph-mon[31442]: pgmap v407: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2024-09-21T03:31:58.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:31:58 smithi203 ceph-mon[34723]: pgmap v407: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2024-09-21T03:31:58.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:31:58 smithi187 ceph-mon[34603]: pgmap v407: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2024-09-21T03:31:59.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 246345 11.31 MB/sec warmup 119 sec latency 27.353 ms 2024-09-21T03:31:59.315 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:31:59.315 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:31:59.584 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.115+0000 7f2820649640 1 -- 172.21.15.112:0/4217006175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2818109e00 msgr2=0x7f2818110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.115+0000 7f2820649640 1 --2- 172.21.15.112:0/4217006175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2818109e00 0x7f2818110630 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2810009ca0 tx=0x7f281002ecd0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.115+0000 7f2820649640 1 -- 172.21.15.112:0/4217006175 shutdown_connections 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.115+0000 7f2820649640 1 --2- 172.21.15.112:0/4217006175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2818109e00 0x7f2818110630 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.115+0000 7f2820649640 1 --2- 172.21.15.112:0/4217006175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28181094b0 0x7f28181098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.115+0000 7f2820649640 1 --2- 172.21.15.112:0/4217006175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2818101f70 0x7f2818102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.115+0000 7f2820649640 1 -- 172.21.15.112:0/4217006175 >> 172.21.15.112:0/4217006175 conn(0x7f2818077c60 msgr2=0x7f28180780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.115+0000 7f2820649640 1 -- 172.21.15.112:0/4217006175 shutdown_connections 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.115+0000 7f2820649640 1 -- 172.21.15.112:0/4217006175 wait complete. 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f2820649640 1 Processor -- start 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f2820649640 1 -- start start 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f2820649640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2818101f70 0x7f2818198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f2820649640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28181094b0 0x7f28181987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f2820649640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2818109e00 0x7f281819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f2820649640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28181118a0 con 0x7f28181094b0 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f2820649640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2818111720 con 0x7f2818109e00 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f2820649640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2818111a20 con 0x7f2818101f70 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f281dbe4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28181094b0 0x7f28181987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f281ebe6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2818109e00 0x7f281819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f281dbe4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28181094b0 0x7f28181987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47094/0 (socket says 172.21.15.112:47094) 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.116+0000 7f281dbe4640 1 -- 172.21.15.112:0/2775630588 learned_addr learned my addr 172.21.15.112:0/2775630588 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f281e3e5640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2818101f70 0x7f2818198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f281dbe4640 1 -- 172.21.15.112:0/2775630588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2818101f70 msgr2=0x7f2818198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f281dbe4640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2818101f70 0x7f2818198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f281dbe4640 1 -- 172.21.15.112:0/2775630588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2818109e00 msgr2=0x7f281819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f281dbe4640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2818109e00 0x7f281819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f281dbe4640 1 -- 172.21.15.112:0/2775630588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281819e1a0 con 0x7f28181094b0 2024-09-21T03:32:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f281dbe4640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28181094b0 0x7f28181987d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f280800e9f0 tx=0x7f280800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f280f7fe640 1 -- 172.21.15.112:0/2775630588 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280800ccc0 con 0x7f28181094b0 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f280f7fe640 1 -- 172.21.15.112:0/2775630588 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28080108e0 con 0x7f28181094b0 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f280f7fe640 1 -- 172.21.15.112:0/2775630588 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808018860 con 0x7f28181094b0 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f2820649640 1 -- 172.21.15.112:0/2775630588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281819e490 con 0x7f28181094b0 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.117+0000 7f2820649640 1 -- 172.21.15.112:0/2775630588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181a4bf0 con 0x7f28181094b0 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.118+0000 7f2820649640 1 -- 172.21.15.112:0/2775630588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2818102380 con 0x7f28181094b0 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.121+0000 7f280f7fe640 1 -- 172.21.15.112:0/2775630588 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2808010430 con 0x7f28181094b0 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.122+0000 7f280f7fe640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28040b6700 0x7f28040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.122+0000 7f281e3e5640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28040b6700 0x7f28040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.122+0000 7f280f7fe640 1 -- 172.21.15.112:0/2775630588 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f28080e7920 con 0x7f28181094b0 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.122+0000 7f281e3e5640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28040b6700 0x7f28040b8c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2814009bd0 tx=0x7f2814005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:00.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.127+0000 7f280f7fe640 1 -- 172.21.15.112:0/2775630588 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f28080b2250 con 0x7f28181094b0 2024-09-21T03:32:00.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.364+0000 7f2820649640 1 -- 172.21.15.112:0/2775630588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f281802e640 con 0x7f28181094b0 2024-09-21T03:32:00.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.365+0000 7f280f7fe640 1 -- 172.21.15.112:0/2775630588 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11932 (secure 0 0 0) 0x7f28080b7190 con 0x7f28181094b0 2024-09-21T03:32:00.367 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:32:00.367 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":26,"btime":"2024-09-21T03:30:19:484252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:30:19.046905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:32:00.368 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 26 2024-09-21T03:32:00.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.367+0000 7f2820649640 1 -- 172.21.15.112:0/2775630588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28040b6700 msgr2=0x7f28040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.367+0000 7f2820649640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28040b6700 0x7f28040b8c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2814009bd0 tx=0x7f2814005f90 comp rx=0 tx=0).stop 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.367+0000 7f2820649640 1 -- 172.21.15.112:0/2775630588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28181094b0 msgr2=0x7f28181987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.367+0000 7f2820649640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28181094b0 0x7f28181987d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f280800e9f0 tx=0x7f280800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.368+0000 7f2820649640 1 -- 172.21.15.112:0/2775630588 shutdown_connections 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.368+0000 7f2820649640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f28040b6700 0x7f28040b8c10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.368+0000 7f2820649640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2818109e00 0x7f281819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.368+0000 7f2820649640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28181094b0 0x7f28181987d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.368+0000 7f2820649640 1 --2- 172.21.15.112:0/2775630588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2818101f70 0x7f2818198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.368+0000 7f2820649640 1 -- 172.21.15.112:0/2775630588 >> 172.21.15.112:0/2775630588 conn(0x7f2818077c60 msgr2=0x7f2818107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.368+0000 7f2820649640 1 -- 172.21.15.112:0/2775630588 shutdown_connections 2024-09-21T03:32:00.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:00.368+0000 7f2820649640 1 -- 172.21.15.112:0/2775630588 wait complete. 2024-09-21T03:32:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:00 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2775630588' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:00.737 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:00 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2775630588' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:00.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:00 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2775630588' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:01.175 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:32:01.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 249785 5.07 MB/sec execute 1 sec latency 102.046 ms 2024-09-21T03:32:01.436 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:01.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:01 smithi187 ceph-mon[34603]: pgmap v408: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-09-21T03:32:01.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:01 smithi112 ceph-mon[31442]: pgmap v408: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-09-21T03:32:01.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:01 smithi203 ceph-mon[34723]: pgmap v408: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-09-21T03:32:02.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.169+0000 7f1eae9a7640 1 -- 172.21.15.112:0/4078329130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea81083b0 msgr2=0x7f1ea81087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:02.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.169+0000 7f1eae9a7640 1 --2- 172.21.15.112:0/4078329130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea81083b0 0x7f1ea81087c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009900 tx=0x7f1e9002a410 comp rx=0 tx=0).stop 2024-09-21T03:32:02.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.169+0000 7f1eae9a7640 1 -- 172.21.15.112:0/4078329130 shutdown_connections 2024-09-21T03:32:02.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.169+0000 7f1eae9a7640 1 --2- 172.21.15.112:0/4078329130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ea8108d00 0x7f1ea810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.169+0000 7f1eae9a7640 1 --2- 172.21.15.112:0/4078329130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea81083b0 0x7f1ea81087c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.169+0000 7f1eae9a7640 1 --2- 172.21.15.112:0/4078329130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ea8102a70 0x7f1ea8102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.169+0000 7f1eae9a7640 1 -- 172.21.15.112:0/4078329130 >> 172.21.15.112:0/4078329130 conn(0x7f1ea8079730 msgr2=0x7f1ea8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:02.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.169+0000 7f1eae9a7640 1 -- 172.21.15.112:0/4078329130 shutdown_connections 2024-09-21T03:32:02.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.169+0000 7f1eae9a7640 1 -- 172.21.15.112:0/4078329130 wait complete. 2024-09-21T03:32:02.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1eae9a7640 1 Processor -- start 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1eae9a7640 1 -- start start 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1eae9a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea8102a70 0x7f1ea819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1eae9a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ea81083b0 0x7f1ea819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1eae9a7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ea8108d00 0x7f1ea819be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1eae9a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8112970 con 0x7f1ea8102a70 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1eae9a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81127f0 con 0x7f1ea81083b0 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1eae9a7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8112af0 con 0x7f1ea8108d00 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea8102a70 0x7f1ea819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea8102a70 0x7f1ea819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47102/0 (socket says 172.21.15.112:47102) 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1ea7fff640 1 -- 172.21.15.112:0/528703982 learned_addr learned my addr 172.21.15.112:0/528703982 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1e9ffff640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ea81083b0 0x7f1ea819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1eacf44640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ea8108d00 0x7f1ea819be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1ea7fff640 1 -- 172.21.15.112:0/528703982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ea8108d00 msgr2=0x7f1ea819be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1ea7fff640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ea8108d00 0x7f1ea819be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1ea7fff640 1 -- 172.21.15.112:0/528703982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ea81083b0 msgr2=0x7f1ea819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1ea7fff640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ea81083b0 0x7f1ea819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1ea7fff640 1 -- 172.21.15.112:0/528703982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea81a6690 con 0x7f1ea8102a70 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.170+0000 7f1e9ffff640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ea81083b0 0x7f1ea819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.171+0000 7f1ea7fff640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea8102a70 0x7f1ea819b380 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800e9f0 tx=0x7f1e9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.171+0000 7f1ea5ffb640 1 -- 172.21.15.112:0/528703982 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9800cd80 con 0x7f1ea8102a70 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.171+0000 7f1ea5ffb640 1 -- 172.21.15.112:0/528703982 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e980108e0 con 0x7f1ea8102a70 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.171+0000 7f1eae9a7640 1 -- 172.21.15.112:0/528703982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea81a6980 con 0x7f1ea8102a70 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.171+0000 7f1ea5ffb640 1 -- 172.21.15.112:0/528703982 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98018980 con 0x7f1ea8102a70 2024-09-21T03:32:02.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.171+0000 7f1eae9a7640 1 -- 172.21.15.112:0/528703982 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea81a6e90 con 0x7f1ea8102a70 2024-09-21T03:32:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1ea5ffb640 1 -- 172.21.15.112:0/528703982 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f1e98010430 con 0x7f1ea8102a70 2024-09-21T03:32:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1eb4f1b740 1 Processor -- start 2024-09-21T03:32:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1eb4f1b740 1 -- start start 2024-09-21T03:32:02.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620ca0d070 0x55620cdb46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620c9e41b0 0x55620cdb4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdb5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55620cd2aea0 con 0x55620c9e41b0 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55620cd1a2d0 con 0x55620ca0d070 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55620cd1e0e0 con 0x55620c9ec390 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1eacf44640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620ca0d070 0x55620cdb46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620c9e41b0 0x55620cdb4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1eacf44640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620ca0d070 0x55620cdb46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41110/0 (socket says 172.21.15.112:41110) 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1eacf44640 1 -- 172.21.15.112:0/2092680647 learned_addr learned my addr 172.21.15.112:0/2092680647 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.174+0000 7f1e9ffff640 1 --2- 172.21.15.112:0/2092680647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdb5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1ea5ffb640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1e940b6680 0x7f1e940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1eacf44640 1 -- 172.21.15.112:0/2092680647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 msgr2=0x55620cdb5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:02.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1eacf44640 1 --2- 172.21.15.112:0/2092680647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdb5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1eacf44640 1 -- 172.21.15.112:0/2092680647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620c9e41b0 msgr2=0x55620cdb4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1eacf44640 1 --2- 172.21.15.112:0/2092680647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620c9e41b0 0x55620cdb4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1eacf44640 1 -- 172.21.15.112:0/2092680647 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620ca78650 con 0x55620ca0d070 2024-09-21T03:32:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1e9ffff640 1 --2- 172.21.15.112:0/2092680647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdb5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1ea5ffb640 1 -- 172.21.15.112:0/528703982 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f1e980e79f0 con 0x7f1ea8102a70 2024-09-21T03:32:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1eacf44640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1e940b6680 0x7f1e940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1eacf44640 1 --2- 172.21.15.112:0/2092680647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620ca0d070 0x55620cdb46a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009f30 tx=0x7f1ea00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9cfb9e0622d5e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1e9e7fc640 1 -- 172.21.15.112:0/2092680647 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000f680 con 0x55620ca0d070 2024-09-21T03:32:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1e9e7fc640 1 -- 172.21.15.112:0/2092680647 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea000fcd0 con 0x55620ca0d070 2024-09-21T03:32:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.175+0000 7f1e9e7fc640 1 -- 172.21.15.112:0/2092680647 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea001ba30 con 0x55620ca0d070 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.176+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/2092680647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620ca0d070 msgr2=0x55620cdb46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.176+0000 7f1eacf44640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1e940b6680 0x7f1e940b8b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0020000 tx=0x7f1ea0021000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.176+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/2092680647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620ca0d070 0x55620cdb46a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009f30 tx=0x7f1ea00118d0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.176+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/2092680647 shutdown_connections 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.176+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/2092680647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdb5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.176+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/2092680647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620c9e41b0 0x55620cdb4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.176+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/2092680647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620ca0d070 0x55620cdb46a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.176+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/2092680647 >> 172.21.15.112:0/2092680647 conn(0x55620ca01460 msgr2=0x55620cdb2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.176+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/2092680647 shutdown_connections 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.177+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/2092680647 wait complete. 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.177+0000 7f1eb4f1b740 1 -- start start 2024-09-21T03:32:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.177+0000 7f1eb4f1b740 1 Processor -- start 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.177+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdc4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.177+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdc4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.177+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdd04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.177+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55620cb7b700 con 0x55620ca0d070 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.177+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55620ccb5160 con 0x55620c9e41b0 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.177+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55620cdc7180 con 0x55620c9ec390 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.177+0000 7f1eb4f1b740 10 client.0 _ll_register_callbacks cb 0x55620ccce9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdc4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdc4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41122/0 (socket says 172.21.15.112:41122) 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1e9ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdc4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1ea7fff640 1 -- 172.21.15.112:0/464043412 learned_addr learned my addr 172.21.15.112:0/464043412 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1eacf44640 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdd04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1ea7fff640 1 -- 172.21.15.112:0/464043412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 msgr2=0x55620cdc4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:02.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1ea7fff640 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdc4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1ea7fff640 1 -- 172.21.15.112:0/464043412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 msgr2=0x55620cdd04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1ea7fff640 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdd04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1ea7fff640 1 -- 172.21.15.112:0/464043412 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620cce2c90 con 0x55620c9e41b0 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1eacf44640 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdd04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1ea7fff640 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdc4150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800e9c0 tx=0x7f1e980eb090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff7a10714fbd586 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1e877fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1e877fe640 1 -- 172.21.15.112:0/464043412 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980ad720 con 0x55620c9e41b0 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.178+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/464043412 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea000fcd0 con 0x55620c9e41b0 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1e877fe640 1 -- 172.21.15.112:0/464043412 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f1e98018d20 con 0x55620c9e41b0 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1e877fe640 1 -- 172.21.15.112:0/464043412 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e980a90c0 con 0x55620c9e41b0 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1eb4f1b740 10 client.25631 fetch_fsmap learned FSMap version 26 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/464043412 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55620cdd2f40 con 0x55620c9e41b0 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1e877fe640 1 -- 172.21.15.112:0/464043412 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea000fcd0 con 0x55620c9e41b0 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1e877fe640 1 -- 172.21.15.112:0/464043412 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1e980ac620 con 0x55620c9e41b0 2024-09-21T03:32:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1eb4f1b740 10 client.25631 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1eb4f1b740 10 client.25631 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1eb4f1b740 20 client.25631 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55620c9cb9f0 0x55620cdd62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1eb4f1b740 4 client.25631 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/464043412 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55620cd2cdf0 con 0x55620c9cb9f0 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.179+0000 7f1e9ffff640 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55620c9cb9f0 0x55620cdd62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.180+0000 7f1e877fe640 10 client.25631 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.180+0000 7f1e9ffff640 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55620c9cb9f0 0x55620cdd62c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77669f51873284c7 server_cookie=d240a8e25d202121 in_seq=0 out_seq=0 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1e877fe640 1 -- 172.21.15.112:0/464043412 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55620cd2cdf0 con 0x55620c9cb9f0 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1e877fe640 10 client.25631 handle_command_reply: tid=0 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 1 client.25631 shutdown 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/464043412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 msgr2=0x55620cdc4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdc4150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800e9c0 tx=0x7f1e980eb090 comp rx=0 tx=0).stop 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/464043412 shutdown_connections 2024-09-21T03:32:02.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55620c9cb9f0 0x55620cdd62c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdd04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdc4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1e877fe640 0 client.25631 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:32:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/464043412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdc4150 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/464043412 >> 172.21.15.112:0/464043412 conn(0x55620ca01460 msgr2=0x55620cdb6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/464043412 shutdown_connections 2024-09-21T03:32:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/464043412 wait complete. 2024-09-21T03:32:02.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.181+0000 7f1eb4f1b740 20 client.25631 trim_cache size 0 max 16384 2024-09-21T03:32:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.194+0000 7f1eb4f1b740 1 Processor -- start 2024-09-21T03:32:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.194+0000 7f1eb4f1b740 1 -- start start 2024-09-21T03:32:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.194+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdbf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.194+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdc6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.194+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdb5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.194+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81afe00 con 0x55620ca0d070 2024-09-21T03:32:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.194+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81ad6d0 con 0x55620c9e41b0 2024-09-21T03:32:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.194+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81ab520 con 0x55620c9ec390 2024-09-21T03:32:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1eacf44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdb5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1eacf44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdb5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47136/0 (socket says 172.21.15.112:47136) 2024-09-21T03:32:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1eacf44640 1 -- 172.21.15.112:0/3122341649 learned_addr learned my addr 172.21.15.112:0/3122341649 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1ea7fff640 1 --2- 172.21.15.112:0/3122341649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdbf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1e9ffff640 1 --2- 172.21.15.112:0/3122341649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdc6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1eacf44640 1 -- 172.21.15.112:0/3122341649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 msgr2=0x55620cdc6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1eacf44640 1 --2- 172.21.15.112:0/3122341649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdc6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1eacf44640 1 -- 172.21.15.112:0/3122341649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 msgr2=0x55620cdbf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:02.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1eacf44640 1 --2- 172.21.15.112:0/3122341649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdbf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1eacf44640 1 -- 172.21.15.112:0/3122341649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620cdb6bc0 con 0x55620ca0d070 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1ea7fff640 1 --2- 172.21.15.112:0/3122341649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdbf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1eacf44640 1 --2- 172.21.15.112:0/3122341649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdb5de0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0003f20 tx=0x7f1ea0027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c325bc24b83268b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1e9dffb640 1 -- 172.21.15.112:0/3122341649 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0029040 con 0x55620ca0d070 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1e9dffb640 1 -- 172.21.15.112:0/3122341649 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea001bae0 con 0x55620ca0d070 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.195+0000 7f1e9dffb640 1 -- 172.21.15.112:0/3122341649 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000f6e0 con 0x55620ca0d070 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.196+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3122341649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 msgr2=0x55620cdb5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.196+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/3122341649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdb5de0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0003f20 tx=0x7f1ea0027000 comp rx=0 tx=0).stop 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.196+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3122341649 shutdown_connections 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.196+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/3122341649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdb5de0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.196+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/3122341649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdc6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.196+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/3122341649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdbf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.196+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3122341649 >> 172.21.15.112:0/3122341649 conn(0x55620ca01460 msgr2=0x55620cdb56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.196+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3122341649 shutdown_connections 2024-09-21T03:32:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.196+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3122341649 wait complete. 2024-09-21T03:32:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.196+0000 7f1eb4f1b740 1 -- start start 2024-09-21T03:32:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1eb4f1b740 1 Processor -- start 2024-09-21T03:32:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdcaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1eb4f1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdcaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81abb90 con 0x55620ca0d070 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81ad000 con 0x55620c9e41b0 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1eb4f1b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81ac260 con 0x55620c9ec390 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1eb4f1b740 10 client.0 _ll_register_callbacks cb 0x7f1e980eb230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1eacf44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdcaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1eacf44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdcaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47150/0 (socket says 172.21.15.112:47150) 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41140/0 (socket says 172.21.15.112:41140) 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1eacf44640 1 -- 172.21.15.112:0/3068325452 learned_addr learned my addr 172.21.15.112:0/3068325452 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.197+0000 7f1e9ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdcaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1ea7fff640 1 -- 172.21.15.112:0/3068325452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 msgr2=0x55620cdcaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1ea7fff640 1 --2- 172.21.15.112:0/3068325452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdcaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1ea7fff640 1 -- 172.21.15.112:0/3068325452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 msgr2=0x55620cdcaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1ea7fff640 1 --2- 172.21.15.112:0/3068325452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdcaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1ea7fff640 1 -- 172.21.15.112:0/3068325452 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620ce3fd60 con 0x55620c9e41b0 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1ea7fff640 1 --2- 172.21.15.112:0/3068325452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdca4f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e980aabc0 tx=0x7f1e980eb110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef8d1f13f5b5ffa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1e877fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1e877fe640 1 -- 172.21.15.112:0/3068325452 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980ad0c0 con 0x55620c9e41b0 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1e877fe640 1 -- 172.21.15.112:0/3068325452 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f1e980a92a0 con 0x55620c9e41b0 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3068325452 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea001bae0 con 0x55620c9e41b0 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1e877fe640 1 -- 172.21.15.112:0/3068325452 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e980ad2a0 con 0x55620c9e41b0 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1eb4f1b740 10 client.25637 fetch_fsmap learned FSMap version 26 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3068325452 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55620ce40fc0 con 0x55620c9e41b0 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1e877fe640 1 -- 172.21.15.112:0/3068325452 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea001bae0 con 0x55620c9e41b0 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.198+0000 7f1e877fe640 1 -- 172.21.15.112:0/3068325452 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1e980a94a0 con 0x55620c9e41b0 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1eb4f1b740 10 client.25637 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:32:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1eb4f1b740 10 client.25637 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1eb4f1b740 20 client.25637 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/3068325452 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55620c9a0da0 0x55620ce43620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1eb4f1b740 4 client.25637 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3068325452 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55620cd288a0 con 0x55620c9a0da0 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1e9ffff640 1 --2- 172.21.15.112:0/3068325452 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55620c9a0da0 0x55620ce43620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1e877fe640 10 client.25637 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1e9ffff640 1 --2- 172.21.15.112:0/3068325452 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55620c9a0da0 0x55620ce43620 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9cf3eaecafbb9c0c server_cookie=96a5694230afdacb in_seq=0 out_seq=0 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1e877fe640 1 -- 172.21.15.112:0/3068325452 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55620cd288a0 con 0x55620c9a0da0 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1e877fe640 10 client.25637 handle_command_reply: tid=0 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.199+0000 7f1eb4f1b740 1 client.25637 shutdown 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3068325452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 msgr2=0x55620cdca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/3068325452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdca4f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e980aabc0 tx=0x7f1e980eb110 comp rx=0 tx=0).stop 2024-09-21T03:32:02.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3068325452 shutdown_connections 2024-09-21T03:32:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/3068325452 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55620c9a0da0 0x55620ce43620 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/3068325452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55620ca0d070 0x55620cdcaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/3068325452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55620c9ec390 0x55620cdcaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eb4f1b740 1 --2- 172.21.15.112:0/3068325452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55620c9e41b0 0x55620cdca4f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1e877fe640 0 client.25637 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:32:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3068325452 >> 172.21.15.112:0/3068325452 conn(0x55620ca01460 msgr2=0x55620cdc5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3068325452 shutdown_connections 2024-09-21T03:32:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eb4f1b740 1 -- 172.21.15.112:0/3068325452 wait complete. 2024-09-21T03:32:02.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eb4f1b740 20 client.25637 trim_cache size 0 max 16384 2024-09-21T03:32:02.206 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:32:02.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eae9a7640 1 -- 172.21.15.112:0/528703982 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1e940b6680 msgr2=0x7f1e940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:02.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eae9a7640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1e940b6680 0x7f1e940b8b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0020000 tx=0x7f1ea0021000 comp rx=0 tx=0).stop 2024-09-21T03:32:02.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eae9a7640 1 -- 172.21.15.112:0/528703982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea8102a70 msgr2=0x7f1ea819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:02.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eae9a7640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea8102a70 0x7f1ea819b380 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800e9f0 tx=0x7f1e9800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eae9a7640 1 -- 172.21.15.112:0/528703982 shutdown_connections 2024-09-21T03:32:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eae9a7640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f1e940b6680 0x7f1e940b8b90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eae9a7640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ea8108d00 0x7f1ea819be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eae9a7640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ea81083b0 0x7f1ea819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eae9a7640 1 --2- 172.21.15.112:0/528703982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ea8102a70 0x7f1ea819b380 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.200+0000 7f1eae9a7640 1 -- 172.21.15.112:0/528703982 >> 172.21.15.112:0/528703982 conn(0x7f1ea8079730 msgr2=0x7f1ea8104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.201+0000 7f1eae9a7640 1 -- 172.21.15.112:0/528703982 shutdown_connections 2024-09-21T03:32:02.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:02.201+0000 7f1eae9a7640 1 -- 172.21.15.112:0/528703982 wait complete. 2024-09-21T03:32:02.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 252263 9.56 MB/sec execute 2 sec latency 9.798 ms 2024-09-21T03:32:03.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:32:03.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 254470 9.06 MB/sec execute 3 sec latency 11.605 ms 2024-09-21T03:32:03.356 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:03.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:03 smithi187 ceph-mon[34603]: pgmap v409: 97 pgs: 97 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 297 op/s 2024-09-21T03:32:03.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:03 smithi112 ceph-mon[31442]: pgmap v409: 97 pgs: 97 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 297 op/s 2024-09-21T03:32:03.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.835+0000 7f960869a640 1 -- 172.21.15.112:0/1441881479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9600109e00 msgr2=0x7f9600110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.835+0000 7f960869a640 1 --2- 172.21.15.112:0/1441881479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9600109e00 0x7f9600110630 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009ca0 tx=0x7f95f802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.836+0000 7f960869a640 1 -- 172.21.15.112:0/1441881479 shutdown_connections 2024-09-21T03:32:03.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.836+0000 7f960869a640 1 --2- 172.21.15.112:0/1441881479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9600109e00 0x7f9600110630 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.836+0000 7f960869a640 1 --2- 172.21.15.112:0/1441881479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96001094b0 0x7f96001098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.836+0000 7f960869a640 1 --2- 172.21.15.112:0/1441881479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9600103bc0 0x7f9600103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.836+0000 7f960869a640 1 -- 172.21.15.112:0/1441881479 >> 172.21.15.112:0/1441881479 conn(0x7f960007b910 msgr2=0x7f960007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.836+0000 7f960869a640 1 -- 172.21.15.112:0/1441881479 shutdown_connections 2024-09-21T03:32:03.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.836+0000 7f960869a640 1 -- 172.21.15.112:0/1441881479 wait complete. 2024-09-21T03:32:03.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.836+0000 7f960869a640 1 Processor -- start 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.836+0000 7f960869a640 1 -- start start 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f960869a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9600103bc0 0x7f9600075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f960869a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96001094b0 0x7f9600075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f960869a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9600109e00 0x7f9600076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f960869a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9600113ac0 con 0x7f9600103bc0 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f960869a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9600113940 con 0x7f96001094b0 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f960869a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9600113c40 con 0x7f9600109e00 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f9606436640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9600103bc0 0x7f9600075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f9606436640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9600103bc0 0x7f9600075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47156/0 (socket says 172.21.15.112:47156) 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f9606436640 1 -- 172.21.15.112:0/2041812035 learned_addr learned my addr 172.21.15.112:0/2041812035 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f9606c37640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9600109e00 0x7f9600076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f9605c35640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96001094b0 0x7f9600075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f9606436640 1 -- 172.21.15.112:0/2041812035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9600109e00 msgr2=0x7f9600076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f9606436640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9600109e00 0x7f9600076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f9606436640 1 -- 172.21.15.112:0/2041812035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96001094b0 msgr2=0x7f9600075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f9606436640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96001094b0 0x7f9600075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f9606436640 1 -- 172.21.15.112:0/2041812035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96001a87b0 con 0x7f9600103bc0 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f9606436640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9600103bc0 0x7f9600075710 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00e9f0 tx=0x7f95fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.837+0000 7f95f77fe640 1 -- 172.21.15.112:0/2041812035 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00cd80 con 0x7f9600103bc0 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.838+0000 7f960869a640 1 -- 172.21.15.112:0/2041812035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96001a8aa0 con 0x7f9600103bc0 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.838+0000 7f960869a640 1 -- 172.21.15.112:0/2041812035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96001a9030 con 0x7f9600103bc0 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.838+0000 7f95f77fe640 1 -- 172.21.15.112:0/2041812035 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95fc0108e0 con 0x7f9600103bc0 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.838+0000 7f95f77fe640 1 -- 172.21.15.112:0/2041812035 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0188e0 con 0x7f9600103bc0 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.840+0000 7f960ec0e740 1 Processor -- start 2024-09-21T03:32:03.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.840+0000 7f960ec0e740 1 -- start start 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f95f77fe640 1 -- 172.21.15.112:0/2041812035 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f95fc018a80 con 0x7f9600103bc0 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b92070 0x564fb8f392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b691b0 0x564fb8f39800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f39d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f960ec0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564fb8eafea0 con 0x564fb8b92070 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f960ec0e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564fb8e9f2d0 con 0x564fb8b691b0 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f960ec0e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564fb8ea30e0 con 0x564fb8b71390 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9606c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b92070 0x564fb8f392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9606436640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b691b0 0x564fb8f39800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9606c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b92070 0x564fb8f392c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47172/0 (socket says 172.21.15.112:47172) 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9606c37640 1 -- 172.21.15.112:0/104002653 learned_addr learned my addr 172.21.15.112:0/104002653 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9605c35640 1 --2- 172.21.15.112:0/104002653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f39d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9606c37640 1 -- 172.21.15.112:0/104002653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 msgr2=0x564fb8f39d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9606c37640 1 --2- 172.21.15.112:0/104002653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f39d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9606c37640 1 -- 172.21.15.112:0/104002653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b691b0 msgr2=0x564fb8f39800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9606c37640 1 --2- 172.21.15.112:0/104002653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b691b0 0x564fb8f39800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9606c37640 1 -- 172.21.15.112:0/104002653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb8bfd650 con 0x564fb8b92070 2024-09-21T03:32:03.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9606436640 1 --2- 172.21.15.112:0/104002653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b691b0 0x564fb8f39800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:03.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.841+0000 7f9606c37640 1 --2- 172.21.15.112:0/104002653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b92070 0x564fb8f392c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f95f802f410 tx=0x7f95f802f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbfef73110621b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.842+0000 7f95f4ff9640 1 -- 172.21.15.112:0/104002653 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8031cf0 con 0x564fb8b92070 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.842+0000 7f95f4ff9640 1 -- 172.21.15.112:0/104002653 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f8036030 con 0x564fb8b92070 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.842+0000 7f95f77fe640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95ec0b6680 0x7f95ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.842+0000 7f95f4ff9640 1 -- 172.21.15.112:0/104002653 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f80042a0 con 0x564fb8b92070 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.842+0000 7f95f77fe640 1 -- 172.21.15.112:0/2041812035 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f95fc0e7790 con 0x7f9600103bc0 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.843+0000 7f9606436640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95ec0b6680 0x7f95ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.843+0000 7f960ec0e740 1 -- 172.21.15.112:0/104002653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b92070 msgr2=0x564fb8f392c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.843+0000 7f960ec0e740 1 --2- 172.21.15.112:0/104002653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b92070 0x564fb8f392c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f95f802f410 tx=0x7f95f802f440 comp rx=0 tx=0).stop 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.843+0000 7f9606436640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95ec0b6680 0x7f95ec0b8b90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0e4ff0 tx=0x7f95fc0ed550 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.843+0000 7f960ec0e740 1 -- 172.21.15.112:0/104002653 shutdown_connections 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.843+0000 7f960ec0e740 1 --2- 172.21.15.112:0/104002653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f39d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.843+0000 7f960ec0e740 1 --2- 172.21.15.112:0/104002653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b691b0 0x564fb8f39800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.843+0000 7f960ec0e740 1 --2- 172.21.15.112:0/104002653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b92070 0x564fb8f392c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.843+0000 7f960ec0e740 1 -- 172.21.15.112:0/104002653 >> 172.21.15.112:0/104002653 conn(0x564fb8b86460 msgr2=0x564fb8f371b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.843+0000 7f960ec0e740 1 -- 172.21.15.112:0/104002653 shutdown_connections 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.843+0000 7f960ec0e740 1 -- 172.21.15.112:0/104002653 wait complete. 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f960ec0e740 1 -- start start 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f960ec0e740 1 Processor -- start 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f449b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f44ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8f50c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f960ec0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564fb8d00700 con 0x564fb8b691b0 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f960ec0e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564fb8e3a160 con 0x564fb8b92070 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f960ec0e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564fb8f478c0 con 0x564fb8b71390 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f960ec0e740 10 client.0 _ll_register_callbacks cb 0x564fb8e539e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f9606c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f9606c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47180/0 (socket says 172.21.15.112:47180) 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f9606c37640 1 -- 172.21.15.112:0/3378313940 learned_addr learned my addr 172.21.15.112:0/3378313940 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f9605c35640 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f44ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.844+0000 7f9606c37640 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8f50c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f9606c37640 1 -- 172.21.15.112:0/3378313940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 msgr2=0x564fb8f44ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f9606c37640 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f44ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f9606c37640 1 -- 172.21.15.112:0/3378313940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 msgr2=0x564fb8f50c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f9606c37640 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8f50c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f9606c37640 1 -- 172.21.15.112:0/3378313940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb8e67c90 con 0x564fb8b691b0 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f9606c37640 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f449b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009a60 tx=0x7f95f8003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837fe1087322c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f95e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f95e9ffb640 1 -- 172.21.15.112:0/3378313940 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8058060 con 0x564fb8b691b0 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f95e9ffb640 1 -- 172.21.15.112:0/3378313940 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f95f8031860 con 0x564fb8b691b0 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f960ec0e740 1 -- 172.21.15.112:0/3378313940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95f8036030 con 0x564fb8b691b0 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f95e9ffb640 1 -- 172.21.15.112:0/3378313940 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f95f802f660 con 0x564fb8b691b0 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f960ec0e740 10 client.16083 fetch_fsmap learned FSMap version 26 2024-09-21T03:32:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f960ec0e740 1 -- 172.21.15.112:0/3378313940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564fb8f53600 con 0x564fb8b691b0 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.845+0000 7f95e9ffb640 1 -- 172.21.15.112:0/3378313940 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f8036030 con 0x564fb8b691b0 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.846+0000 7f95e9ffb640 1 -- 172.21.15.112:0/3378313940 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f95f8043070 con 0x564fb8b691b0 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.846+0000 7f960ec0e740 10 client.16083 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.846+0000 7f960ec0e740 10 client.16083 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.846+0000 7f960ec0e740 20 client.16083 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.846+0000 7f960ec0e740 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x564fb8b509f0 0x564fb8f56960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.846+0000 7f960ec0e740 4 client.16083 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.846+0000 7f960ec0e740 1 -- 172.21.15.112:0/3378313940 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fb8eb1df0 con 0x564fb8b509f0 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.846+0000 7f9605c35640 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x564fb8b509f0 0x564fb8f56960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.846+0000 7f9605c35640 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x564fb8b509f0 0x564fb8f56960 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eca0b17021435f3b server_cookie=12c175b66b28bf25 in_seq=0 out_seq=0 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.846+0000 7f95e9ffb640 10 client.16083 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f95e9ffb640 1 -- 172.21.15.112:0/3378313940 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564fb8eb1df0 con 0x564fb8b509f0 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f95e9ffb640 10 client.16083 handle_command_reply: tid=0 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 1 client.16083 shutdown 2024-09-21T03:32:03.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 1 -- 172.21.15.112:0/3378313940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 msgr2=0x564fb8f449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f449b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009a60 tx=0x7f95f8003d70 comp rx=0 tx=0).stop 2024-09-21T03:32:03.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 1 -- 172.21.15.112:0/3378313940 shutdown_connections 2024-09-21T03:32:03.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x564fb8b509f0 0x564fb8f56960 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8f50c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f44ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 1 --2- 172.21.15.112:0/3378313940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f449b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 1 -- 172.21.15.112:0/3378313940 >> 172.21.15.112:0/3378313940 conn(0x564fb8b86460 msgr2=0x564fb8f3c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f95e9ffb640 0 client.16083 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:32:03.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 1 -- 172.21.15.112:0/3378313940 shutdown_connections 2024-09-21T03:32:03.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 1 -- 172.21.15.112:0/3378313940 wait complete. 2024-09-21T03:32:03.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.847+0000 7f960ec0e740 20 client.16083 trim_cache size 0 max 16384 2024-09-21T03:32:03.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f960ec0e740 1 Processor -- start 2024-09-21T03:32:03.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f960ec0e740 1 -- start start 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f46890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f3aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8f50b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f960ec0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4010f30 con 0x564fb8b691b0 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f960ec0e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f95e400e800 con 0x564fb8b92070 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f960ec0e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f95e400c650 con 0x564fb8b71390 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f9606c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f46890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f9606c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f46890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47194/0 (socket says 172.21.15.112:47194) 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f9605c35640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f3aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f9606c37640 1 -- 172.21.15.112:0/187369799 learned_addr learned my addr 172.21.15.112:0/187369799 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.861+0000 7f9606c37640 1 --2- 172.21.15.112:0/187369799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8f50b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.862+0000 7f9606c37640 1 -- 172.21.15.112:0/187369799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 msgr2=0x564fb8f3aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.862+0000 7f9606c37640 1 --2- 172.21.15.112:0/187369799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f3aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.862+0000 7f9606c37640 1 -- 172.21.15.112:0/187369799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 msgr2=0x564fb8f50b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.862+0000 7f9606c37640 1 --2- 172.21.15.112:0/187369799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8f50b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.862+0000 7f9606c37640 1 -- 172.21.15.112:0/187369799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb8f3b5b0 con 0x564fb8b691b0 2024-09-21T03:32:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.862+0000 7f9606c37640 1 --2- 172.21.15.112:0/187369799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f46890 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f95f8030bf0 tx=0x7f95f80021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9404d976c4b124c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.862+0000 7f95ebfff640 1 -- 172.21.15.112:0/187369799 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8058030 con 0x564fb8b691b0 2024-09-21T03:32:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.862+0000 7f95ebfff640 1 -- 172.21.15.112:0/187369799 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f8036070 con 0x564fb8b691b0 2024-09-21T03:32:03.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.862+0000 7f95ebfff640 1 -- 172.21.15.112:0/187369799 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f80318c0 con 0x564fb8b691b0 2024-09-21T03:32:03.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 -- 172.21.15.112:0/187369799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 msgr2=0x564fb8f46890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 --2- 172.21.15.112:0/187369799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f46890 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f95f8030bf0 tx=0x7f95f80021f0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 -- 172.21.15.112:0/187369799 shutdown_connections 2024-09-21T03:32:03.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 --2- 172.21.15.112:0/187369799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8f50b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 --2- 172.21.15.112:0/187369799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8f3aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 --2- 172.21.15.112:0/187369799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8f46890 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 -- 172.21.15.112:0/187369799 >> 172.21.15.112:0/187369799 conn(0x564fb8b86460 msgr2=0x564fb8f3a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 -- 172.21.15.112:0/187369799 shutdown_connections 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 -- 172.21.15.112:0/187369799 wait complete. 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 -- start start 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 Processor -- start 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8fb5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8fb5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8fc00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95e400ccc0 con 0x564fb8b691b0 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f95e400e130 con 0x564fb8b92070 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.863+0000 7f960ec0e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f95e400d390 con 0x564fb8b71390 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f960ec0e740 10 client.0 _ll_register_callbacks cb 0x7f95fc0ed420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9606c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8fb5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9606c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8fb5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47210/0 (socket says 172.21.15.112:47210) 2024-09-21T03:32:03.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9605c35640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8fb5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9606c37640 1 -- 172.21.15.112:0/1261449703 learned_addr learned my addr 172.21.15.112:0/1261449703 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9606c37640 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8fc00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9605c35640 1 -- 172.21.15.112:0/1261449703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 msgr2=0x564fb8fc00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9605c35640 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8fc00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9605c35640 1 -- 172.21.15.112:0/1261449703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 msgr2=0x564fb8fb5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9605c35640 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8fb5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9605c35640 1 -- 172.21.15.112:0/1261449703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb8fc0760 con 0x564fb8b71390 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9606c37640 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8fb5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f9605c35640 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8fb5830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x564fb8e9f3a0 tx=0x7f95f0012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=811f6b65e70c4bd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f95e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f95e9ffb640 1 -- 172.21.15.112:0/1261449703 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0004250 con 0x564fb8b71390 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f95e9ffb640 1 -- 172.21.15.112:0/1261449703 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f95f00043f0 con 0x564fb8b71390 2024-09-21T03:32:03.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.864+0000 7f960ec0e740 1 -- 172.21.15.112:0/1261449703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95f8036070 con 0x564fb8b71390 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f95e9ffb640 1 -- 172.21.15.112:0/1261449703 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f95f0005020 con 0x564fb8b71390 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f960ec0e740 10 client.25657 fetch_fsmap learned FSMap version 26 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f960ec0e740 1 -- 172.21.15.112:0/1261449703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564fb8fc1a20 con 0x564fb8b71390 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f95e9ffb640 1 -- 172.21.15.112:0/1261449703 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f8036070 con 0x564fb8b71390 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f95e9ffb640 1 -- 172.21.15.112:0/1261449703 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f95f00035f0 con 0x564fb8b71390 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f960ec0e740 10 client.25657 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f960ec0e740 10 client.25657 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f960ec0e740 20 client.25657 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f960ec0e740 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x564fb8b25da0 0x564fb8fc4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f960ec0e740 4 client.25657 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.865+0000 7f960ec0e740 1 -- 172.21.15.112:0/1261449703 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564fb8ead8a0 con 0x564fb8b25da0 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f9606c37640 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x564fb8b25da0 0x564fb8fc4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f95e9ffb640 10 client.25657 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:32:03.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f9606c37640 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x564fb8b25da0 0x564fb8fc4210 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f71b10f0478e15c server_cookie=7510be4ee83b3029 in_seq=0 out_seq=0 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f95e9ffb640 1 -- 172.21.15.112:0/1261449703 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fb8ead8a0 con 0x564fb8b25da0 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f95e9ffb640 10 client.25657 handle_command_reply: tid=0 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f960ec0e740 1 client.25657 shutdown 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f960ec0e740 1 -- 172.21.15.112:0/1261449703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 msgr2=0x564fb8fb5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f960ec0e740 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8fb5830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x564fb8e9f3a0 tx=0x7f95f0012820 comp rx=0 tx=0).stop 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f960ec0e740 1 -- 172.21.15.112:0/1261449703 shutdown_connections 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f960ec0e740 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x564fb8b25da0 0x564fb8fc4210 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f960ec0e740 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564fb8b92070 0x564fb8fc00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f960ec0e740 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564fb8b71390 0x564fb8fb5830 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f960ec0e740 1 --2- 172.21.15.112:0/1261449703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564fb8b691b0 0x564fb8fb5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f960ec0e740 1 -- 172.21.15.112:0/1261449703 >> 172.21.15.112:0/1261449703 conn(0x564fb8b86460 msgr2=0x564fb8f46790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.866+0000 7f95e9ffb640 0 client.25657 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960ec0e740 1 -- 172.21.15.112:0/1261449703 shutdown_connections 2024-09-21T03:32:03.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960ec0e740 1 -- 172.21.15.112:0/1261449703 wait complete. 2024-09-21T03:32:03.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960ec0e740 20 client.25657 trim_cache size 0 max 16384 2024-09-21T03:32:03.874 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:32:03.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 -- 172.21.15.112:0/2041812035 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95ec0b6680 msgr2=0x7f95ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95ec0b6680 0x7f95ec0b8b90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0e4ff0 tx=0x7f95fc0ed550 comp rx=0 tx=0).stop 2024-09-21T03:32:03.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 -- 172.21.15.112:0/2041812035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9600103bc0 msgr2=0x7f9600075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:03.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9600103bc0 0x7f9600075710 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00e9f0 tx=0x7f95fc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 -- 172.21.15.112:0/2041812035 shutdown_connections 2024-09-21T03:32:03.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95ec0b6680 0x7f95ec0b8b90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9600109e00 0x7f9600076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96001094b0 0x7f9600075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 --2- 172.21.15.112:0/2041812035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9600103bc0 0x7f9600075710 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:03.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 -- 172.21.15.112:0/2041812035 >> 172.21.15.112:0/2041812035 conn(0x7f960007b910 msgr2=0x7f9600107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:03.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 -- 172.21.15.112:0/2041812035 shutdown_connections 2024-09-21T03:32:03.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:03.867+0000 7f960869a640 1 -- 172.21.15.112:0/2041812035 wait complete. 2024-09-21T03:32:03.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:03 smithi203 ceph-mon[34723]: pgmap v409: 97 pgs: 97 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 297 op/s 2024-09-21T03:32:04.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 256040 9.57 MB/sec execute 4 sec latency 36.309 ms 2024-09-21T03:32:04.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:32:04.921 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:05.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 258974 11.14 MB/sec execute 5 sec latency 19.333 ms 2024-09-21T03:32:05.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.417+0000 7fd6b8983640 1 -- 172.21.15.112:0/1047303526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6b4105380 msgr2=0x7fd6b410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:05.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.417+0000 7fd6b8983640 1 --2- 172.21.15.112:0/1047303526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6b4105380 0x7fd6b410d8d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009900 tx=0x7fd69c02a410 comp rx=0 tx=0).stop 2024-09-21T03:32:05.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.417+0000 7fd6b8983640 1 -- 172.21.15.112:0/1047303526 shutdown_connections 2024-09-21T03:32:05.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.417+0000 7fd6b8983640 1 --2- 172.21.15.112:0/1047303526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b410de60 0x7fd6b4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.417+0000 7fd6b8983640 1 --2- 172.21.15.112:0/1047303526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6b4105380 0x7fd6b410d8d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.417+0000 7fd6b8983640 1 --2- 172.21.15.112:0/1047303526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6b4104a30 0x7fd6b4104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.417+0000 7fd6b8983640 1 -- 172.21.15.112:0/1047303526 >> 172.21.15.112:0/1047303526 conn(0x7fd6b4079760 msgr2=0x7fd6b4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:05.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.417+0000 7fd6b8983640 1 -- 172.21.15.112:0/1047303526 shutdown_connections 2024-09-21T03:32:05.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.417+0000 7fd6b8983640 1 -- 172.21.15.112:0/1047303526 wait complete. 2024-09-21T03:32:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.418+0000 7fd6b8983640 1 Processor -- start 2024-09-21T03:32:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b8983640 1 -- start start 2024-09-21T03:32:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b8983640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b4104a30 0x7fd6b419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b8983640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6b4105380 0x7fd6b419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b8983640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6b410de60 0x7fd6b419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b8983640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4113aa0 con 0x7fd6b4105380 2024-09-21T03:32:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b8983640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4113920 con 0x7fd6b410de60 2024-09-21T03:32:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b8983640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4113c20 con 0x7fd6b4104a30 2024-09-21T03:32:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6b4105380 0x7fd6b419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b4104a30 0x7fd6b419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6b4105380 0x7fd6b419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40840/0 (socket says 172.21.15.112:40840) 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b1d9b640 1 -- 172.21.15.112:0/3576832510 learned_addr learned my addr 172.21.15.112:0/3576832510 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b2d9d640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6b410de60 0x7fd6b419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.419+0000 7fd6b1d9b640 1 -- 172.21.15.112:0/3576832510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b4104a30 msgr2=0x7fd6b419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b4104a30 0x7fd6b419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6b1d9b640 1 -- 172.21.15.112:0/3576832510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6b410de60 msgr2=0x7fd6b419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6b410de60 0x7fd6b419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6b1d9b640 1 -- 172.21.15.112:0/3576832510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b41a0400 con 0x7fd6b4105380 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6b2d9d640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6b410de60 0x7fd6b419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6b259c640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b4104a30 0x7fd6b419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6b4105380 0x7fd6b419aa30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd69c02a930 tx=0x7fd69c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6ab7fe640 1 -- 172.21.15.112:0/3576832510 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c03f030 con 0x7fd6b4105380 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6b8983640 1 -- 172.21.15.112:0/3576832510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b41a0690 con 0x7fd6b4105380 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6ab7fe640 1 -- 172.21.15.112:0/3576832510 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd69c004490 con 0x7fd6b4105380 2024-09-21T03:32:05.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6ab7fe640 1 -- 172.21.15.112:0/3576832510 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c035070 con 0x7fd6b4105380 2024-09-21T03:32:05.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.420+0000 7fd6b8983640 1 -- 172.21.15.112:0/3576832510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b41a6e50 con 0x7fd6b4105380 2024-09-21T03:32:05.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.422+0000 7fd6beef7740 1 Processor -- start 2024-09-21T03:32:05.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6beef7740 1 -- start start 2024-09-21T03:32:05.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6ab7fe640 1 -- 172.21.15.112:0/3576832510 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fd69c042030 con 0x7fd6b4105380 2024-09-21T03:32:05.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 0x558bc457e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41ae1b0 0x558bc457ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc457f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6beef7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558bc44f4ea0 con 0x558bc41d7070 2024-09-21T03:32:05.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6beef7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558bc44e42d0 con 0x558bc41b6390 2024-09-21T03:32:05.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6beef7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558bc44e80e0 con 0x558bc41ae1b0 2024-09-21T03:32:05.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41ae1b0 0x558bc457ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6b1d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc457f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6b2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 0x558bc457e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41ae1b0 0x558bc457ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:57198/0 (socket says 172.21.15.112:57198) 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.423+0000 7fd6b259c640 1 -- 172.21.15.112:0/1391763160 learned_addr learned my addr 172.21.15.112:0/1391763160 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.424+0000 7fd6b259c640 1 -- 172.21.15.112:0/1391763160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 msgr2=0x558bc457f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.424+0000 7fd6b259c640 1 --2- 172.21.15.112:0/1391763160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc457f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.424+0000 7fd6b259c640 1 -- 172.21.15.112:0/1391763160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 msgr2=0x558bc457e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.424+0000 7fd6b259c640 1 --2- 172.21.15.112:0/1391763160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 0x558bc457e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.424+0000 7fd6b259c640 1 -- 172.21.15.112:0/1391763160 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc4242650 con 0x558bc41ae1b0 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.424+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/1391763160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc457f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.424+0000 7fd6b2d9d640 1 --2- 172.21.15.112:0/1391763160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 0x558bc457e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.424+0000 7fd6b259c640 1 --2- 172.21.15.112:0/1391763160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41ae1b0 0x558bc457ebe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0114c0 tx=0x7fd6ac011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acfdaae5cfc9d71f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.424+0000 7fd6ab7fe640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd6a00b6680 0x7fd6a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.424+0000 7fd6b2d9d640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd6a00b6680 0x7fd6a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.424+0000 7fd6ab7fe640 1 -- 172.21.15.112:0/3576832510 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd69c107980 con 0x7fd6b4105380 2024-09-21T03:32:05.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.425+0000 7fd6b2d9d640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd6a00b6680 0x7fd6a00b8b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009fd0 tx=0x7fd6a4009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:05.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.429+0000 7fd6a8ff9640 1 -- 172.21.15.112:0/1391763160 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac004020 con 0x558bc41ae1b0 2024-09-21T03:32:05.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.429+0000 7fd6a8ff9640 1 -- 172.21.15.112:0/1391763160 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6ac0054d0 con 0x558bc41ae1b0 2024-09-21T03:32:05.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.429+0000 7fd6a8ff9640 1 -- 172.21.15.112:0/1391763160 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac004ce0 con 0x558bc41ae1b0 2024-09-21T03:32:05.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.430+0000 7fd6beef7740 1 -- 172.21.15.112:0/1391763160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41ae1b0 msgr2=0x558bc457ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:05.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.430+0000 7fd6beef7740 1 --2- 172.21.15.112:0/1391763160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41ae1b0 0x558bc457ebe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0114c0 tx=0x7fd6ac011980 comp rx=0 tx=0).stop 2024-09-21T03:32:05.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.430+0000 7fd6beef7740 1 -- 172.21.15.112:0/1391763160 shutdown_connections 2024-09-21T03:32:05.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.430+0000 7fd6beef7740 1 --2- 172.21.15.112:0/1391763160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc457f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.430+0000 7fd6beef7740 1 --2- 172.21.15.112:0/1391763160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41ae1b0 0x558bc457ebe0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.430+0000 7fd6beef7740 1 --2- 172.21.15.112:0/1391763160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 0x558bc457e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.430+0000 7fd6beef7740 1 -- 172.21.15.112:0/1391763160 >> 172.21.15.112:0/1391763160 conn(0x558bc41cb460 msgr2=0x558bc457c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:05.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.430+0000 7fd6beef7740 1 -- 172.21.15.112:0/1391763160 shutdown_connections 2024-09-21T03:32:05.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.430+0000 7fd6beef7740 1 -- 172.21.15.112:0/1391763160 wait complete. 2024-09-21T03:32:05.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6beef7740 1 -- start start 2024-09-21T03:32:05.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6beef7740 1 Processor -- start 2024-09-21T03:32:05.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 0x558bc458e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc458e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41d7070 0x558bc458eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6beef7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558bc4345700 con 0x558bc41ae1b0 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6beef7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558bc447f160 con 0x558bc41b6390 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6beef7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558bc45911e0 con 0x558bc41d7070 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6beef7740 10 client.0 _ll_register_callbacks cb 0x558bc44989e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6b1d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc458e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6b1d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc458e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:55042/0 (socket says 172.21.15.112:55042) 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6b1d9b640 1 -- 172.21.15.112:0/3048926294 learned_addr learned my addr 172.21.15.112:0/3048926294 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.431+0000 7fd6b259c640 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 0x558bc458e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd6b2d9d640 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41d7070 0x558bc458eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd6b1d9b640 1 -- 172.21.15.112:0/3048926294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41d7070 msgr2=0x558bc458eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41d7070 0x558bc458eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd6b1d9b640 1 -- 172.21.15.112:0/3048926294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 msgr2=0x558bc458e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 0x558bc458e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd6b1d9b640 1 -- 172.21.15.112:0/3048926294 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc44acc90 con 0x558bc41b6390 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd6b2d9d640 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41d7070 0x558bc458eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc458e640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd69c006010 tx=0x7fd69c03b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78bb5446c248759c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd699ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd699ffb640 1 -- 172.21.15.112:0/3048926294 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c049070 con 0x558bc41b6390 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd699ffb640 1 -- 172.21.15.112:0/3048926294 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd69c0d6fc0 con 0x558bc41b6390 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd6beef7740 1 -- 172.21.15.112:0/3048926294 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6ac0054d0 con 0x558bc41b6390 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd699ffb640 1 -- 172.21.15.112:0/3048926294 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd69c0cd6b0 con 0x558bc41b6390 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd6beef7740 10 client.25667 fetch_fsmap learned FSMap version 26 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd6beef7740 1 -- 172.21.15.112:0/3048926294 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558bc459cf00 con 0x558bc41b6390 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.432+0000 7fd699ffb640 1 -- 172.21.15.112:0/3048926294 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6ac0054d0 con 0x558bc41b6390 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.433+0000 7fd699ffb640 1 -- 172.21.15.112:0/3048926294 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd69c0c9230 con 0x558bc41b6390 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.433+0000 7fd6beef7740 10 client.25667 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.433+0000 7fd6beef7740 10 client.25667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.433+0000 7fd6beef7740 20 client.25667 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.433+0000 7fd6beef7740 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558bc41959f0 0x558bc45a0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.433+0000 7fd6beef7740 4 client.25667 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.433+0000 7fd6beef7740 1 -- 172.21.15.112:0/3048926294 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bc44f6df0 con 0x558bc41959f0 2024-09-21T03:32:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.433+0000 7fd6b259c640 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558bc41959f0 0x558bc45a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.433+0000 7fd6b259c640 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558bc41959f0 0x558bc45a0490 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9158bbe66c0a1ca server_cookie=445c9c9f26ef86d in_seq=0 out_seq=0 2024-09-21T03:32:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.433+0000 7fd699ffb640 10 client.25667 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:05.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.434+0000 7fd699ffb640 1 -- 172.21.15.112:0/3048926294 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558bc44f6df0 con 0x558bc41959f0 2024-09-21T03:32:05.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.434+0000 7fd699ffb640 10 client.25667 handle_command_reply: tid=0 2024-09-21T03:32:05.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.434+0000 7fd6beef7740 1 client.25667 shutdown 2024-09-21T03:32:05.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd6beef7740 1 -- 172.21.15.112:0/3048926294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 msgr2=0x558bc458e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:05.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd6beef7740 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc458e640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd69c006010 tx=0x7fd69c03b560 comp rx=0 tx=0).stop 2024-09-21T03:32:05.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd6beef7740 1 -- 172.21.15.112:0/3048926294 shutdown_connections 2024-09-21T03:32:05.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd6beef7740 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558bc41959f0 0x558bc45a0490 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd6beef7740 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41d7070 0x558bc458eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd6beef7740 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc458e640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd6beef7740 1 --2- 172.21.15.112:0/3048926294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 0x558bc458e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd6beef7740 1 -- 172.21.15.112:0/3048926294 >> 172.21.15.112:0/3048926294 conn(0x558bc41cb460 msgr2=0x558bc4580090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:05.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd699ffb640 0 client.25667 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:05.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd6beef7740 1 -- 172.21.15.112:0/3048926294 shutdown_connections 2024-09-21T03:32:05.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd6beef7740 1 -- 172.21.15.112:0/3048926294 wait complete. 2024-09-21T03:32:05.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.435+0000 7fd6beef7740 20 client.25667 trim_cache size 0 max 16384 2024-09-21T03:32:05.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.448+0000 7fd6beef7740 1 Processor -- start 2024-09-21T03:32:05.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.448+0000 7fd6beef7740 1 -- start start 2024-09-21T03:32:05.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41ae1b0 0x558bc457beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41b6390 0x558bc457c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 0x558bc458b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6beef7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41afc70 con 0x558bc41d7070 2024-09-21T03:32:05.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6beef7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41ad540 con 0x558bc41ae1b0 2024-09-21T03:32:05.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6beef7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41ab390 con 0x558bc41b6390 2024-09-21T03:32:05.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 0x558bc458b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 0x558bc458b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40888/0 (socket says 172.21.15.112:40888) 2024-09-21T03:32:05.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b2d9d640 1 -- 172.21.15.112:0/1368147774 learned_addr learned my addr 172.21.15.112:0/1368147774 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:05.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/1368147774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41b6390 0x558bc457c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b259c640 1 --2- 172.21.15.112:0/1368147774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41ae1b0 0x558bc457beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b2d9d640 1 -- 172.21.15.112:0/1368147774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41b6390 msgr2=0x558bc457c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:05.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b2d9d640 1 --2- 172.21.15.112:0/1368147774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41b6390 0x558bc457c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b2d9d640 1 -- 172.21.15.112:0/1368147774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41ae1b0 msgr2=0x558bc457beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:05.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b2d9d640 1 --2- 172.21.15.112:0/1368147774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41ae1b0 0x558bc457beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b2d9d640 1 -- 172.21.15.112:0/1368147774 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc459a470 con 0x558bc41d7070 2024-09-21T03:32:05.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/1368147774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41b6390 0x558bc457c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:05.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.449+0000 7fd6b2d9d640 1 --2- 172.21.15.112:0/1368147774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 0x558bc458b2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40114a0 tx=0x7fd6a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb45e85368474ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.450+0000 7fd69bfff640 1 -- 172.21.15.112:0/1368147774 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4011df0 con 0x558bc41d7070 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.450+0000 7fd69bfff640 1 -- 172.21.15.112:0/1368147774 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a401e8e0 con 0x558bc41d7070 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.450+0000 7fd69bfff640 1 -- 172.21.15.112:0/1368147774 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4010650 con 0x558bc41d7070 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 -- 172.21.15.112:0/1368147774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 msgr2=0x558bc458b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 --2- 172.21.15.112:0/1368147774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 0x558bc458b2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40114a0 tx=0x7fd6a4016000 comp rx=0 tx=0).stop 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 -- 172.21.15.112:0/1368147774 shutdown_connections 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 --2- 172.21.15.112:0/1368147774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41d7070 0x558bc458b2f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 --2- 172.21.15.112:0/1368147774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41b6390 0x558bc457c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 --2- 172.21.15.112:0/1368147774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41ae1b0 0x558bc457beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 -- 172.21.15.112:0/1368147774 >> 172.21.15.112:0/1368147774 conn(0x558bc41cb460 msgr2=0x558bc458c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 -- 172.21.15.112:0/1368147774 shutdown_connections 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 -- 172.21.15.112:0/1368147774 wait complete. 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 -- start start 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 Processor -- start 2024-09-21T03:32:05.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 0x558bc4596c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc45920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41d7070 0x558bc4592620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41aba00 con 0x558bc41ae1b0 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41ace70 con 0x558bc41b6390 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41ac0d0 con 0x558bc41d7070 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.451+0000 7fd6beef7740 10 client.0 _ll_register_callbacks cb 0x7fd69c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 0x558bc4596c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b1d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc45920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 0x558bc4596c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40902/0 (socket says 172.21.15.112:40902) 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b259c640 1 -- 172.21.15.112:0/11194826 learned_addr learned my addr 172.21.15.112:0/11194826 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b1d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc45920e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:55066/0 (socket says 172.21.15.112:55066) 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b2d9d640 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41d7070 0x558bc4592620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b259c640 1 -- 172.21.15.112:0/11194826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41d7070 msgr2=0x558bc4592620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b259c640 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41d7070 0x558bc4592620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b259c640 1 -- 172.21.15.112:0/11194826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 msgr2=0x558bc45920e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b259c640 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc45920e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b259c640 1 -- 172.21.15.112:0/11194826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc4592d60 con 0x558bc41ae1b0 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc45920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6b259c640 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 0x558bc4596c30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac011e80 tx=0x7fd6ac0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16addf2b87761dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd699ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd699ffb640 1 -- 172.21.15.112:0/11194826 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac035060 con 0x558bc41ae1b0 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd699ffb640 1 -- 172.21.15.112:0/11194826 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd6ac016040 con 0x558bc41ae1b0 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6beef7740 1 -- 172.21.15.112:0/11194826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a401e8e0 con 0x558bc41ae1b0 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd699ffb640 1 -- 172.21.15.112:0/11194826 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd6ac01d040 con 0x558bc41ae1b0 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6beef7740 10 client.16107 fetch_fsmap learned FSMap version 26 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.452+0000 7fd6beef7740 1 -- 172.21.15.112:0/11194826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558bc460ad40 con 0x558bc41ae1b0 2024-09-21T03:32:05.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.453+0000 7fd699ffb640 1 -- 172.21.15.112:0/11194826 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a401e8e0 con 0x558bc41ae1b0 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.453+0000 7fd699ffb640 1 -- 172.21.15.112:0/11194826 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd6ac00fc60 con 0x558bc41ae1b0 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.453+0000 7fd6beef7740 10 client.16107 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.453+0000 7fd6beef7740 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.453+0000 7fd6beef7740 20 client.16107 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.453+0000 7fd6beef7740 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558bc416ada0 0x558bc460d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.453+0000 7fd6beef7740 4 client.16107 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.453+0000 7fd6beef7740 1 -- 172.21.15.112:0/11194826 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bc44f28a0 con 0x558bc416ada0 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.453+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558bc416ada0 0x558bc460d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.453+0000 7fd6b1d9b640 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558bc416ada0 0x558bc460d360 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6763b60568f92063 server_cookie=f5b8138239353968 in_seq=0 out_seq=0 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.453+0000 7fd699ffb640 10 client.16107 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd699ffb640 1 -- 172.21.15.112:0/11194826 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bc44f28a0 con 0x558bc416ada0 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd699ffb640 10 client.16107 handle_command_reply: tid=0 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 1 client.16107 shutdown 2024-09-21T03:32:05.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 1 -- 172.21.15.112:0/11194826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 msgr2=0x558bc4596c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 0x558bc4596c30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac011e80 tx=0x7fd6ac0047e0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 1 -- 172.21.15.112:0/11194826 shutdown_connections 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558bc416ada0 0x558bc460d360 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bc41d7070 0x558bc4592620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558bc41b6390 0x558bc45920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd699ffb640 0 client.16107 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 1 --2- 172.21.15.112:0/11194826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bc41ae1b0 0x558bc4596c30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 1 -- 172.21.15.112:0/11194826 >> 172.21.15.112:0/11194826 conn(0x558bc41cb460 msgr2=0x558bc457f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 1 -- 172.21.15.112:0/11194826 shutdown_connections 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 1 -- 172.21.15.112:0/11194826 wait complete. 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.454+0000 7fd6beef7740 20 client.16107 trim_cache size 0 max 16384 2024-09-21T03:32:05.460 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:32:05.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 -- 172.21.15.112:0/3576832510 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd6a00b6680 msgr2=0x7fd6a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:05.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd6a00b6680 0x7fd6a00b8b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009fd0 tx=0x7fd6a4009500 comp rx=0 tx=0).stop 2024-09-21T03:32:05.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 -- 172.21.15.112:0/3576832510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6b4105380 msgr2=0x7fd6b419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:05.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6b4105380 0x7fd6b419aa30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd69c02a930 tx=0x7fd69c031a20 comp rx=0 tx=0).stop 2024-09-21T03:32:05.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 -- 172.21.15.112:0/3576832510 shutdown_connections 2024-09-21T03:32:05.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd6a00b6680 0x7fd6a00b8b90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6b410de60 0x7fd6b419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6b4105380 0x7fd6b419aa30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 --2- 172.21.15.112:0/3576832510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b4104a30 0x7fd6b419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:05.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 -- 172.21.15.112:0/3576832510 >> 172.21.15.112:0/3576832510 conn(0x7fd6b4079760 msgr2=0x7fd6b41008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:05.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 -- 172.21.15.112:0/3576832510 shutdown_connections 2024-09-21T03:32:05.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:05.455+0000 7fd6b8983640 1 -- 172.21.15.112:0/3576832510 wait complete. 2024-09-21T03:32:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:05 smithi187 ceph-mon[34603]: pgmap v410: 97 pgs: 97 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-09-21T03:32:05.764 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:05 smithi112 ceph-mon[31442]: pgmap v410: 97 pgs: 97 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-09-21T03:32:05.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:05 smithi203 ceph-mon[34723]: pgmap v410: 97 pgs: 97 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-09-21T03:32:06.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 260187 9.91 MB/sec execute 6 sec latency 43.975 ms 2024-09-21T03:32:07.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 262767 10.88 MB/sec execute 7 sec latency 15.898 ms 2024-09-21T03:32:07.262 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:32:07.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:32:07.523 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:07.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:07 smithi112 ceph-mon[31442]: pgmap v411: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 375 op/s 2024-09-21T03:32:07.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:07 smithi187 ceph-mon[34603]: pgmap v411: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 375 op/s 2024-09-21T03:32:07.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:07 smithi203 ceph-mon[34723]: pgmap v411: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 375 op/s 2024-09-21T03:32:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.023+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/2708253508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d481083c0 msgr2=0x7f3d481087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.023+0000 7f3d4d8dc640 1 --2- 172.21.15.112:0/2708253508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d481083c0 0x7f3d481087d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009900 tx=0x7f3d3c02a410 comp rx=0 tx=0).stop 2024-09-21T03:32:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.024+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/2708253508 shutdown_connections 2024-09-21T03:32:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.024+0000 7f3d4d8dc640 1 --2- 172.21.15.112:0/2708253508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d48108d10 0x7f3d4810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.024+0000 7f3d4d8dc640 1 --2- 172.21.15.112:0/2708253508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d481083c0 0x7f3d481087d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.024+0000 7f3d4d8dc640 1 --2- 172.21.15.112:0/2708253508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d48102a80 0x7f3d48102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.024+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/2708253508 >> 172.21.15.112:0/2708253508 conn(0x7f3d48079740 msgr2=0x7f3d48079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:08.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.024+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/2708253508 shutdown_connections 2024-09-21T03:32:08.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.024+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/2708253508 wait complete. 2024-09-21T03:32:08.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.024+0000 7f3d4d8dc640 1 Processor -- start 2024-09-21T03:32:08.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d4d8dc640 1 -- start start 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d4d8dc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d48102a80 0x7f3d4819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d4d8dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d481083c0 0x7f3d4819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d4d8dc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d48108d10 0x7f3d4819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d4d8dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48112980 con 0x7f3d481083c0 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d4d8dc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48112800 con 0x7f3d48102a80 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d4d8dc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48112b00 con 0x7f3d48108d10 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d467fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d481083c0 0x7f3d4819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d467fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d481083c0 0x7f3d4819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40920/0 (socket says 172.21.15.112:40920) 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d46ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d48102a80 0x7f3d4819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d467fc640 1 -- 172.21.15.112:0/448677502 learned_addr learned my addr 172.21.15.112:0/448677502 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d477fe640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d48108d10 0x7f3d4819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d477fe640 1 -- 172.21.15.112:0/448677502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d48102a80 msgr2=0x7f3d4819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d477fe640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d48102a80 0x7f3d4819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d477fe640 1 -- 172.21.15.112:0/448677502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d481083c0 msgr2=0x7f3d4819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d477fe640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d481083c0 0x7f3d4819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d477fe640 1 -- 172.21.15.112:0/448677502 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d481a0450 con 0x7f3d48108d10 2024-09-21T03:32:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.025+0000 7f3d467fc640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d481083c0 0x7f3d4819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.026+0000 7f3d477fe640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d48108d10 0x7f3d4819fd20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000e970 tx=0x7f3d4000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.026+0000 7f3d4c8da640 1 -- 172.21.15.112:0/448677502 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4000ccc0 con 0x7f3d48108d10 2024-09-21T03:32:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.026+0000 7f3d4c8da640 1 -- 172.21.15.112:0/448677502 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d40005020 con 0x7f3d48108d10 2024-09-21T03:32:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.026+0000 7f3d4c8da640 1 -- 172.21.15.112:0/448677502 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40004a00 con 0x7f3d48108d10 2024-09-21T03:32:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.026+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/448677502 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d481a0740 con 0x7f3d48108d10 2024-09-21T03:32:08.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.026+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/448677502 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d480ff570 con 0x7f3d48108d10 2024-09-21T03:32:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d4c8da640 1 -- 172.21.15.112:0/448677502 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3d40004ba0 con 0x7f3d48108d10 2024-09-21T03:32:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d53e50740 1 Processor -- start 2024-09-21T03:32:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d53e50740 1 -- start start 2024-09-21T03:32:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aeea0350 0x5557af247430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af247970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 0x5557af247eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d53e50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5557af1bde60 con 0x5557aee7f670 2024-09-21T03:32:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d53e50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5557af1ad290 con 0x5557aeea0350 2024-09-21T03:32:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d53e50740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5557af1b10a0 con 0x5557aee77490 2024-09-21T03:32:08.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d467fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aeea0350 0x5557af247430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d46ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af247970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d467fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aeea0350 0x5557af247430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:55102/0 (socket says 172.21.15.112:55102) 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d467fc640 1 -- 172.21.15.112:0/3296812668 learned_addr learned my addr 172.21.15.112:0/3296812668 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d467fc640 1 --2- 172.21.15.112:0/3296812668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 0x5557af247eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d4c8da640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d300b6680 0x7f3d300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d46ffd640 1 -- 172.21.15.112:0/3296812668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aeea0350 msgr2=0x5557af247430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d46ffd640 1 --2- 172.21.15.112:0/3296812668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aeea0350 0x5557af247430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d46ffd640 1 -- 172.21.15.112:0/3296812668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 msgr2=0x5557af247eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d46ffd640 1 --2- 172.21.15.112:0/3296812668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 0x5557af247eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.029+0000 7f3d46ffd640 1 -- 172.21.15.112:0/3296812668 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557aef0b930 con 0x5557aee77490 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.030+0000 7f3d477fe640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d300b6680 0x7f3d300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.030+0000 7f3d4c8da640 1 -- 172.21.15.112:0/448677502 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3d400e7640 con 0x7f3d48108d10 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.030+0000 7f3d46ffd640 1 --2- 172.21.15.112:0/3296812668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af247970 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009f80 tx=0x7f3d380118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edcfd9a11a76119c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.030+0000 7f3d2dffb640 1 -- 172.21.15.112:0/3296812668 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38004d20 con 0x5557aee77490 2024-09-21T03:32:08.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.030+0000 7f3d477fe640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d300b6680 0x7f3d300b8b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3d400e4ea0 tx=0x7f3d4007a1a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.030+0000 7f3d2dffb640 1 -- 172.21.15.112:0/3296812668 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d38004050 con 0x5557aee77490 2024-09-21T03:32:08.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.030+0000 7f3d2dffb640 1 -- 172.21.15.112:0/3296812668 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38005260 con 0x5557aee77490 2024-09-21T03:32:08.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.031+0000 7f3d53e50740 1 -- 172.21.15.112:0/3296812668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 msgr2=0x5557af247970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:08.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.031+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3296812668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af247970 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009f80 tx=0x7f3d380118d0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.031+0000 7f3d53e50740 1 -- 172.21.15.112:0/3296812668 shutdown_connections 2024-09-21T03:32:08.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.031+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3296812668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 0x5557af247eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.031+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3296812668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af247970 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.031+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3296812668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aeea0350 0x5557af247430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.031+0000 7f3d53e50740 1 -- 172.21.15.112:0/3296812668 >> 172.21.15.112:0/3296812668 conn(0x5557aee94740 msgr2=0x5557af245320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.031+0000 7f3d53e50740 1 -- 172.21.15.112:0/3296812668 shutdown_connections 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.031+0000 7f3d53e50740 1 -- 172.21.15.112:0/3296812668 wait complete. 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d53e50740 1 -- start start 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d53e50740 1 Processor -- start 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af252f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee7f670 0x5557af253470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 0x5557af25ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d53e50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5557af00e890 con 0x5557aeea0350 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d53e50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5557af147d40 con 0x5557aee7f670 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d53e50740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5557af255c10 con 0x5557aee77490 2024-09-21T03:32:08.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d53e50740 10 client.0 _ll_register_callbacks cb 0x5557af1abc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d467fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af252f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d467fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 0x5557af25ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d46ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee7f670 0x5557af253470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d467fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 0x5557af25ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40940/0 (socket says 172.21.15.112:40940) 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.032+0000 7f3d467fc640 1 -- 172.21.15.112:0/2000531044 learned_addr learned my addr 172.21.15.112:0/2000531044 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d467fc640 1 -- 172.21.15.112:0/2000531044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 msgr2=0x5557af252f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d467fc640 1 --2- 172.21.15.112:0/2000531044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af252f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d467fc640 1 -- 172.21.15.112:0/2000531044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee7f670 msgr2=0x5557af253470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d467fc640 1 --2- 172.21.15.112:0/2000531044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee7f670 0x5557af253470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d467fc640 1 -- 172.21.15.112:0/2000531044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557af25f440 con 0x5557aeea0350 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d467fc640 1 --2- 172.21.15.112:0/2000531044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 0x5557af25ef00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c000c00 tx=0x7f3d3c032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4c03ac5ddcc241 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:08.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d26ffd640 1 -- 172.21.15.112:0/2000531044 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c035070 con 0x5557aeea0350 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d26ffd640 1 -- 172.21.15.112:0/2000531044 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3d3c005020 con 0x5557aeea0350 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d53e50740 1 -- 172.21.15.112:0/2000531044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d38004050 con 0x5557aeea0350 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d26ffd640 1 -- 172.21.15.112:0/2000531044 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d3c005220 con 0x5557aeea0350 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d53e50740 10 client.16119 fetch_fsmap learned FSMap version 26 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d53e50740 1 -- 172.21.15.112:0/2000531044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557af261d40 con 0x5557aeea0350 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.033+0000 7f3d26ffd640 1 -- 172.21.15.112:0/2000531044 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d38004050 con 0x5557aeea0350 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.034+0000 7f3d26ffd640 1 -- 172.21.15.112:0/2000531044 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3d3c004770 con 0x5557aeea0350 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.034+0000 7f3d53e50740 10 client.16119 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.034+0000 7f3d53e50740 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.034+0000 7f3d53e50740 20 client.16119 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.034+0000 7f3d53e50740 1 --2- 172.21.15.112:0/2000531044 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5557aee5ecd0 0x5557af265290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.034+0000 7f3d53e50740 4 client.16119 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.034+0000 7f3d53e50740 1 -- 172.21.15.112:0/2000531044 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557af1bfdb0 con 0x5557aee5ecd0 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.034+0000 7f3d46ffd640 1 --2- 172.21.15.112:0/2000531044 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5557aee5ecd0 0x5557af265290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.034+0000 7f3d26ffd640 10 client.16119 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:08.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.034+0000 7f3d46ffd640 1 --2- 172.21.15.112:0/2000531044 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5557aee5ecd0 0x5557af265290 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85427c578adb0c07 server_cookie=3d615e2d9157858f in_seq=0 out_seq=0 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.035+0000 7f3d26ffd640 1 -- 172.21.15.112:0/2000531044 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5557af1bfdb0 con 0x5557aee5ecd0 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.035+0000 7f3d26ffd640 10 client.16119 handle_command_reply: tid=0 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.035+0000 7f3d53e50740 1 client.16119 shutdown 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d53e50740 1 -- 172.21.15.112:0/2000531044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 msgr2=0x5557af25ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d53e50740 1 --2- 172.21.15.112:0/2000531044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 0x5557af25ef00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c000c00 tx=0x7f3d3c032ed0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d53e50740 1 -- 172.21.15.112:0/2000531044 shutdown_connections 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d53e50740 1 --2- 172.21.15.112:0/2000531044 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5557aee5ecd0 0x5557af265290 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d53e50740 1 --2- 172.21.15.112:0/2000531044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 0x5557af25ef00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d53e50740 1 --2- 172.21.15.112:0/2000531044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee7f670 0x5557af253470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d53e50740 1 --2- 172.21.15.112:0/2000531044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af252f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d53e50740 1 -- 172.21.15.112:0/2000531044 >> 172.21.15.112:0/2000531044 conn(0x5557aee94740 msgr2=0x5557af24a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d26ffd640 0 client.16119 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d53e50740 1 -- 172.21.15.112:0/2000531044 shutdown_connections 2024-09-21T03:32:08.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d53e50740 1 -- 172.21.15.112:0/2000531044 wait complete. 2024-09-21T03:32:08.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.036+0000 7f3d53e50740 20 client.16119 trim_cache size 0 max 16384 2024-09-21T03:32:08.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d53e50740 1 Processor -- start 2024-09-21T03:32:08.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d53e50740 1 -- start start 2024-09-21T03:32:08.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af25ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee7f670 0x5557af2515b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 0x5557af251af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d53e50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481b40f0 con 0x5557aeea0350 2024-09-21T03:32:08.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d53e50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481b19c0 con 0x5557aee7f670 2024-09-21T03:32:08.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d53e50740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481af810 con 0x5557aee77490 2024-09-21T03:32:08.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d467fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 0x5557af251af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d46ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee7f670 0x5557af2515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d467fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 0x5557af251af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40954/0 (socket says 172.21.15.112:40954) 2024-09-21T03:32:08.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.054+0000 7f3d467fc640 1 -- 172.21.15.112:0/3602734945 learned_addr learned my addr 172.21.15.112:0/3602734945 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:08.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.055+0000 7f3d467fc640 1 --2- 172.21.15.112:0/3602734945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af25ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.055+0000 7f3d46ffd640 1 -- 172.21.15.112:0/3602734945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 msgr2=0x5557af25ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.055+0000 7f3d46ffd640 1 --2- 172.21.15.112:0/3602734945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af25ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.055+0000 7f3d46ffd640 1 -- 172.21.15.112:0/3602734945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 msgr2=0x5557af251af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.055+0000 7f3d46ffd640 1 --2- 172.21.15.112:0/3602734945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 0x5557af251af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.055+0000 7f3d46ffd640 1 -- 172.21.15.112:0/3602734945 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557af244a70 con 0x5557aee7f670 2024-09-21T03:32:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.055+0000 7f3d46ffd640 1 --2- 172.21.15.112:0/3602734945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee7f670 0x5557af2515b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d38006010 tx=0x7f3d380096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f834b429b77162e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.055+0000 7f3d2d7fa640 1 -- 172.21.15.112:0/3602734945 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38032070 con 0x5557aee7f670 2024-09-21T03:32:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.055+0000 7f3d2d7fa640 1 -- 172.21.15.112:0/3602734945 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d38004050 con 0x5557aee7f670 2024-09-21T03:32:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.055+0000 7f3d2d7fa640 1 -- 172.21.15.112:0/3602734945 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38004de0 con 0x5557aee7f670 2024-09-21T03:32:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.056+0000 7f3d53e50740 1 -- 172.21.15.112:0/3602734945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee7f670 msgr2=0x5557af2515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.056+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3602734945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee7f670 0x5557af2515b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d38006010 tx=0x7f3d380096c0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.056+0000 7f3d53e50740 1 -- 172.21.15.112:0/3602734945 shutdown_connections 2024-09-21T03:32:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.056+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3602734945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aeea0350 0x5557af251af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.056+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3602734945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee7f670 0x5557af2515b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.056+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3602734945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aee77490 0x5557af25ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.056+0000 7f3d53e50740 1 -- 172.21.15.112:0/3602734945 >> 172.21.15.112:0/3602734945 conn(0x5557aee94740 msgr2=0x5557af24df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.056+0000 7f3d53e50740 1 -- 172.21.15.112:0/3602734945 shutdown_connections 2024-09-21T03:32:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.056+0000 7f3d53e50740 1 -- 172.21.15.112:0/3602734945 wait complete. 2024-09-21T03:32:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d53e50740 1 -- start start 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d53e50740 1 Processor -- start 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee77490 0x5557af257290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 0x5557af257850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d53e50740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aeea0350 0x5557af2cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d53e50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481afe80 con 0x5557aee7f670 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d53e50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481b12f0 con 0x5557aee77490 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d53e50740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481b0550 con 0x5557aeea0350 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d53e50740 10 client.0 _ll_register_callbacks cb 0x7f3d3c004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 0x5557af257850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d467fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee77490 0x5557af257290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d467fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee77490 0x5557af257290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:55128/0 (socket says 172.21.15.112:55128) 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 0x5557af257850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40962/0 (socket says 172.21.15.112:40962) 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.057+0000 7f3d467fc640 1 -- 172.21.15.112:0/3411002414 learned_addr learned my addr 172.21.15.112:0/3411002414 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d467fc640 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aeea0350 0x5557af2cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d46ffd640 1 -- 172.21.15.112:0/3411002414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aeea0350 msgr2=0x5557af2cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d46ffd640 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aeea0350 0x5557af2cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d46ffd640 1 -- 172.21.15.112:0/3411002414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee77490 msgr2=0x5557af257290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d46ffd640 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee77490 0x5557af257290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d46ffd640 1 -- 172.21.15.112:0/3411002414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557af2cf1d0 con 0x5557aee7f670 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d467fc640 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee77490 0x5557af257290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d46ffd640 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 0x5557af257850 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009d40 tx=0x7f3d38005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c130a397bbe1839 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d26ffd640 1 -- 172.21.15.112:0/3411002414 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38032040 con 0x5557aee7f670 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d26ffd640 1 -- 172.21.15.112:0/3411002414 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3d38004a80 con 0x5557aee7f670 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d53e50740 1 -- 172.21.15.112:0/3411002414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d38004050 con 0x5557aee7f670 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d26ffd640 1 -- 172.21.15.112:0/3411002414 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d38004c80 con 0x5557aee7f670 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d53e50740 10 client.16125 fetch_fsmap learned FSMap version 26 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.058+0000 7f3d53e50740 1 -- 172.21.15.112:0/3411002414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557af2d03a0 con 0x5557aee7f670 2024-09-21T03:32:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.059+0000 7f3d26ffd640 1 -- 172.21.15.112:0/3411002414 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d38004050 con 0x5557aee7f670 2024-09-21T03:32:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.059+0000 7f3d26ffd640 1 -- 172.21.15.112:0/3411002414 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3d3802d040 con 0x5557aee7f670 2024-09-21T03:32:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.059+0000 7f3d53e50740 10 client.16125 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:32:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.059+0000 7f3d53e50740 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.059+0000 7f3d53e50740 20 client.16125 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.059+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5557aee34080 0x5557af2d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.059+0000 7f3d53e50740 4 client.16125 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:32:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.059+0000 7f3d53e50740 1 -- 172.21.15.112:0/3411002414 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5557af1bb860 con 0x5557aee34080 2024-09-21T03:32:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.059+0000 7f3d467fc640 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5557aee34080 0x5557af2d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.059+0000 7f3d26ffd640 10 client.16125 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.059+0000 7f3d467fc640 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5557aee34080 0x5557af2d2880 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be50a389470172a7 server_cookie=14ca8491a4611d89 in_seq=0 out_seq=0 2024-09-21T03:32:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.061+0000 7f3d26ffd640 1 -- 172.21.15.112:0/3411002414 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5557af1bb860 con 0x5557aee34080 2024-09-21T03:32:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.061+0000 7f3d26ffd640 10 client.16125 handle_command_reply: tid=0 2024-09-21T03:32:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.061+0000 7f3d53e50740 1 client.16125 shutdown 2024-09-21T03:32:08.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d53e50740 1 -- 172.21.15.112:0/3411002414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 msgr2=0x5557af257850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 0x5557af257850 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009d40 tx=0x7f3d38005f10 comp rx=0 tx=0).stop 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d53e50740 1 -- 172.21.15.112:0/3411002414 shutdown_connections 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5557aee34080 0x5557af2d2880 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5557aeea0350 0x5557af2cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5557aee7f670 0x5557af257850 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d26ffd640 0 client.16125 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d53e50740 1 --2- 172.21.15.112:0/3411002414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557aee77490 0x5557af257290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d53e50740 1 -- 172.21.15.112:0/3411002414 >> 172.21.15.112:0/3411002414 conn(0x5557aee94740 msgr2=0x5557af24c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d53e50740 1 -- 172.21.15.112:0/3411002414 shutdown_connections 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d53e50740 1 -- 172.21.15.112:0/3411002414 wait complete. 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d53e50740 20 client.16125 trim_cache size 0 max 16384 2024-09-21T03:32:08.066 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:32:08.067 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:32:08.067 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "70d7715c-658d-43b2-ada8-48508980cc8e", 2024-09-21T03:32:08.067 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:32:08.067 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:32:08.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/448677502 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d300b6680 msgr2=0x7f3d300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d4d8dc640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d300b6680 0x7f3d300b8b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3d400e4ea0 tx=0x7f3d4007a1a0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/448677502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d48108d10 msgr2=0x7f3d4819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.062+0000 7f3d4d8dc640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d48108d10 0x7f3d4819fd20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000e970 tx=0x7f3d4000ee30 comp rx=0 tx=0).stop 2024-09-21T03:32:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.063+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/448677502 shutdown_connections 2024-09-21T03:32:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.063+0000 7f3d4d8dc640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d300b6680 0x7f3d300b8b90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.063+0000 7f3d4d8dc640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d48108d10 0x7f3d4819fd20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.063+0000 7f3d4d8dc640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d481083c0 0x7f3d4819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.063+0000 7f3d4d8dc640 1 --2- 172.21.15.112:0/448677502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d48102a80 0x7f3d4819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.063+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/448677502 >> 172.21.15.112:0/448677502 conn(0x7f3d48079740 msgr2=0x7f3d4806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.063+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/448677502 shutdown_connections 2024-09-21T03:32:08.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:08.063+0000 7f3d4d8dc640 1 -- 172.21.15.112:0/448677502 wait complete. 2024-09-21T03:32:08.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 264216 10.22 MB/sec execute 8 sec latency 27.906 ms 2024-09-21T03:32:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:08 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:32:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:08 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:08 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:32:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:08 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:32:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:08 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:08 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:32:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:08 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:08 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:32:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:08 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:32:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:08 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:08.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:32:08.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:08 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:32:08.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:08 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:08.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:08 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:32:08.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:08 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:32:08.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:08 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:09.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 266439 10.62 MB/sec execute 9 sec latency 58.689 ms 2024-09-21T03:32:09.645 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:09 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:32:09.645 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:09 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:09.645 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:09 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:09.646 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:09 smithi112 ceph-mon[31442]: pgmap v412: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 284 op/s 2024-09-21T03:32:09.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.738+0000 7f0b91a19640 1 -- 172.21.15.112:0/204251397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b8c105170 msgr2=0x7f0b8c105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.738+0000 7f0b91a19640 1 --2- 172.21.15.112:0/204251397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b8c105170 0x7f0b8c105580 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009900 tx=0x7f0b7402a410 comp rx=0 tx=0).stop 2024-09-21T03:32:09.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.738+0000 7f0b91a19640 1 -- 172.21.15.112:0/204251397 shutdown_connections 2024-09-21T03:32:09.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.738+0000 7f0b91a19640 1 --2- 172.21.15.112:0/204251397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8c105ac0 0x7f0b8c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.738+0000 7f0b91a19640 1 --2- 172.21.15.112:0/204251397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b8c105170 0x7f0b8c105580 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.738+0000 7f0b91a19640 1 --2- 172.21.15.112:0/204251397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b8c103160 0x7f0b8c103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.738+0000 7f0b91a19640 1 -- 172.21.15.112:0/204251397 >> 172.21.15.112:0/204251397 conn(0x7f0b8c077c60 msgr2=0x7f0b8c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.738+0000 7f0b91a19640 1 -- 172.21.15.112:0/204251397 shutdown_connections 2024-09-21T03:32:09.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.738+0000 7f0b91a19640 1 -- 172.21.15.112:0/204251397 wait complete. 2024-09-21T03:32:09.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b91a19640 1 Processor -- start 2024-09-21T03:32:09.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b91a19640 1 -- start start 2024-09-21T03:32:09.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b91a19640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8c103160 0x7f0b8c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b91a19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b8c105170 0x7f0b8c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b91a19640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b8c105ac0 0x7f0b8c1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b91a19640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1129c0 con 0x7f0b8c105170 2024-09-21T03:32:09.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b91a19640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112840 con 0x7f0b8c103160 2024-09-21T03:32:09.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b91a19640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112b40 con 0x7f0b8c105ac0 2024-09-21T03:32:09.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b8affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8c103160 0x7f0b8c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b8c105170 0x7f0b8c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b8c105170 0x7f0b8c19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40982/0 (socket says 172.21.15.112:40982) 2024-09-21T03:32:09.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b8affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8c103160 0x7f0b8c19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:55146/0 (socket says 172.21.15.112:55146) 2024-09-21T03:32:09.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b827fc640 1 -- 172.21.15.112:0/2277398104 learned_addr learned my addr 172.21.15.112:0/2277398104 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.739+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b8c105ac0 0x7f0b8c1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b8affd640 1 -- 172.21.15.112:0/2277398104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b8c105ac0 msgr2=0x7f0b8c1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b8affd640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b8c105ac0 0x7f0b8c1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b8affd640 1 -- 172.21.15.112:0/2277398104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b8c105170 msgr2=0x7f0b8c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b8affd640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b8c105170 0x7f0b8c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b8affd640 1 -- 172.21.15.112:0/2277398104 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c1a2650 con 0x7f0b8c103160 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b827fc640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b8c105170 0x7f0b8c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b8affd640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8c103160 0x7f0b8c19c6e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00e9f0 tx=0x7f0b7c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b90a17640 1 -- 172.21.15.112:0/2277398104 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c00ccc0 con 0x7f0b8c103160 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b91a19640 1 -- 172.21.15.112:0/2277398104 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c1a2940 con 0x7f0b8c103160 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b90a17640 1 -- 172.21.15.112:0/2277398104 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c005020 con 0x7f0b8c103160 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b91a19640 1 -- 172.21.15.112:0/2277398104 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c1a90a0 con 0x7f0b8c103160 2024-09-21T03:32:09.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.740+0000 7f0b90a17640 1 -- 172.21.15.112:0/2277398104 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004a00 con 0x7f0b8c103160 2024-09-21T03:32:09.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b97f8d740 1 Processor -- start 2024-09-21T03:32:09.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b97f8d740 1 -- start start 2024-09-21T03:32:09.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560501a1b070 0x560501dc2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dc2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019fa390 0x560501dc30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b90a17640 1 -- 172.21.15.112:0/2277398104 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0b7c004ba0 con 0x7f0b8c103160 2024-09-21T03:32:09.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560501d38e50 con 0x560501a1b070 2024-09-21T03:32:09.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560501d28280 con 0x5605019f21b0 2024-09-21T03:32:09.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560501d2c090 con 0x5605019fa390 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560501a1b070 0x560501dc2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560501a1b070 0x560501dc2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40992/0 (socket says 172.21.15.112:40992) 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/1938482624 learned_addr learned my addr 172.21.15.112:0/1938482624 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b827fc640 1 --2- 172.21.15.112:0/1938482624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019fa390 0x560501dc30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b8affd640 1 --2- 172.21.15.112:0/1938482624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dc2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/1938482624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019fa390 msgr2=0x560501dc30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/1938482624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019fa390 0x560501dc30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/1938482624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 msgr2=0x560501dc2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/1938482624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dc2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/1938482624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560501a86650 con 0x560501a1b070 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.743+0000 7f0b827fc640 1 --2- 172.21.15.112:0/1938482624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019fa390 0x560501dc30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.744+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/1938482624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560501a1b070 0x560501dc2650 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009fd0 tx=0x7f0b840118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fc22d0e2b513ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.744+0000 7f0b81ffb640 1 -- 172.21.15.112:0/1938482624 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84004d20 con 0x560501a1b070 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.744+0000 7f0b81ffb640 1 -- 172.21.15.112:0/1938482624 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b84004050 con 0x560501a1b070 2024-09-21T03:32:09.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.744+0000 7f0b81ffb640 1 -- 172.21.15.112:0/1938482624 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84005280 con 0x560501a1b070 2024-09-21T03:32:09.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.744+0000 7f0b90a17640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0b780b6680 0x7f0b780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.744+0000 7f0b8affd640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0b780b6680 0x7f0b780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.744+0000 7f0b90a17640 1 -- 172.21.15.112:0/2277398104 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f0b7c0e73f0 con 0x7f0b8c103160 2024-09-21T03:32:09.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.745+0000 7f0b97f8d740 1 -- 172.21.15.112:0/1938482624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560501a1b070 msgr2=0x560501dc2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.745+0000 7f0b8affd640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0b780b6680 0x7f0b780b8b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0e4c50 tx=0x7f0b7c0eaf40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.745+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/1938482624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560501a1b070 0x560501dc2650 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009fd0 tx=0x7f0b840118d0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.745+0000 7f0b97f8d740 1 -- 172.21.15.112:0/1938482624 shutdown_connections 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.745+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/1938482624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019fa390 0x560501dc30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.745+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/1938482624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dc2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.745+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/1938482624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560501a1b070 0x560501dc2650 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.745+0000 7f0b97f8d740 1 -- 172.21.15.112:0/1938482624 >> 172.21.15.112:0/1938482624 conn(0x560501a0f460 msgr2=0x560501dc0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.745+0000 7f0b97f8d740 1 -- 172.21.15.112:0/1938482624 shutdown_connections 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.745+0000 7f0b97f8d740 1 -- 172.21.15.112:0/1938482624 wait complete. 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.745+0000 7f0b97f8d740 1 -- start start 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b97f8d740 1 Processor -- start 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dcdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560501a1b070 0x560501dda180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560501b89700 con 0x5605019fa390 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560501cc3110 con 0x5605019f21b0 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560501dd0e30 con 0x560501a1b070 2024-09-21T03:32:09.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b97f8d740 10 client.0 _ll_register_callbacks cb 0x560501cdc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41000/0 (socket says 172.21.15.112:41000) 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b827fc640 1 -- 172.21.15.112:0/2938754504 learned_addr learned my addr 172.21.15.112:0/2938754504 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560501a1b070 0x560501dda180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dcdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/2938754504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 msgr2=0x560501dcdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dcdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/2938754504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 msgr2=0x560501dce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/2938754504 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560501cf0c40 con 0x560501a1b070 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.746+0000 7f0b827fc640 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560501a1b070 0x560501dda180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x560501db2760 tx=0x7f0b84003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32984be90edee38c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b6affd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b6affd640 1 -- 172.21.15.112:0/2938754504 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84003d20 con 0x560501a1b070 2024-09-21T03:32:09.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b6affd640 1 -- 172.21.15.112:0/2938754504 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f0b84004a40 con 0x560501a1b070 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b97f8d740 1 -- 172.21.15.112:0/2938754504 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b84004050 con 0x560501a1b070 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b6affd640 1 -- 172.21.15.112:0/2938754504 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b84004c40 con 0x560501a1b070 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b97f8d740 10 client.25705 fetch_fsmap learned FSMap version 26 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b97f8d740 1 -- 172.21.15.112:0/2938754504 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560501ddcb20 con 0x560501a1b070 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b6affd640 1 -- 172.21.15.112:0/2938754504 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b84004050 con 0x560501a1b070 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b6affd640 1 -- 172.21.15.112:0/2938754504 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0b8400f4e0 con 0x560501a1b070 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b97f8d740 10 client.25705 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b97f8d740 10 client.25705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b97f8d740 20 client.25705 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.747+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5605019d99f0 0x560501ddff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.748+0000 7f0b97f8d740 4 client.25705 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.748+0000 7f0b97f8d740 1 -- 172.21.15.112:0/2938754504 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560501d3ada0 con 0x5605019d99f0 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.748+0000 7f0b827fc640 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5605019d99f0 0x560501ddff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.748+0000 7f0b6affd640 10 client.25705 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.748+0000 7f0b827fc640 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5605019d99f0 0x560501ddff30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad044e2f3743653f server_cookie=c4d030c5ed26c59b in_seq=0 out_seq=0 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b6affd640 1 -- 172.21.15.112:0/2938754504 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560501d3ada0 con 0x5605019d99f0 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b6affd640 10 client.25705 handle_command_reply: tid=0 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 1 client.25705 shutdown 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 1 -- 172.21.15.112:0/2938754504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560501a1b070 msgr2=0x560501dda180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560501a1b070 0x560501dda180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x560501db2760 tx=0x7f0b84003f50 comp rx=0 tx=0).stop 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 1 -- 172.21.15.112:0/2938754504 shutdown_connections 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5605019d99f0 0x560501ddff30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560501a1b070 0x560501dda180 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b6affd640 0 client.25705 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/2938754504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dcdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 1 -- 172.21.15.112:0/2938754504 >> 172.21.15.112:0/2938754504 conn(0x560501a0f460 msgr2=0x560501dc57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 1 -- 172.21.15.112:0/2938754504 shutdown_connections 2024-09-21T03:32:09.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 1 -- 172.21.15.112:0/2938754504 wait complete. 2024-09-21T03:32:09.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.749+0000 7f0b97f8d740 20 client.25705 trim_cache size 0 max 16384 2024-09-21T03:32:09.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.767+0000 7f0b97f8d740 1 Processor -- start 2024-09-21T03:32:09.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.767+0000 7f0b97f8d740 1 -- start start 2024-09-21T03:32:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019d99f0 0x560501dc5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dc5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dcaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c010f30 con 0x5605019fa390 2024-09-21T03:32:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c00e800 con 0x5605019f21b0 2024-09-21T03:32:09.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c00c650 con 0x5605019d99f0 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dcaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dcaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41012/0 (socket says 172.21.15.112:41012) 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/3249025144 learned_addr learned my addr 172.21.15.112:0/3249025144 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/3249025144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019d99f0 0x560501dc5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b827fc640 1 --2- 172.21.15.112:0/3249025144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dc5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/3249025144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019d99f0 msgr2=0x560501dc5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/3249025144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019d99f0 0x560501dc5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/3249025144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 msgr2=0x560501dc5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/3249025144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dc5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/3249025144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560501dc54b0 con 0x5605019fa390 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/3249025144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dcaf00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0b84006010 tx=0x7f0b840118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ace0cbbd047580 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.768+0000 7f0b817fa640 1 -- 172.21.15.112:0/3249025144 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84035040 con 0x5605019fa390 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.769+0000 7f0b817fa640 1 -- 172.21.15.112:0/3249025144 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b84004940 con 0x5605019fa390 2024-09-21T03:32:09.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.769+0000 7f0b817fa640 1 -- 172.21.15.112:0/3249025144 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400fb20 con 0x5605019fa390 2024-09-21T03:32:09.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.769+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3249025144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 msgr2=0x560501dcaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.769+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/3249025144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dcaf00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0b84006010 tx=0x7f0b840118d0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.769+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3249025144 shutdown_connections 2024-09-21T03:32:09.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.769+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/3249025144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019fa390 0x560501dcaf00 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.769+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/3249025144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dc5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.769+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/3249025144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019d99f0 0x560501dc5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.769+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3249025144 >> 172.21.15.112:0/3249025144 conn(0x560501a0f460 msgr2=0x560501dcc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.769+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3249025144 shutdown_connections 2024-09-21T03:32:09.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.769+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3249025144 wait complete. 2024-09-21T03:32:09.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.770+0000 7f0b97f8d740 1 -- start start 2024-09-21T03:32:09.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.770+0000 7f0b97f8d740 1 Processor -- start 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.770+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019d99f0 0x560501dd2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.770+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dd27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.770+0000 7f0b97f8d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019fa390 0x560501dd2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.770+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c00ccc0 con 0x5605019d99f0 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.770+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c00e130 con 0x5605019f21b0 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.770+0000 7f0b97f8d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c00d390 con 0x5605019fa390 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b97f8d740 10 client.0 _ll_register_callbacks cb 0x7f0b7c0eae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019d99f0 0x560501dd2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019d99f0 0x560501dd2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41020/0 (socket says 172.21.15.112:41020) 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/3713993382 learned_addr learned my addr 172.21.15.112:0/3713993382 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b827fc640 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dd27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019fa390 0x560501dd2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/3713993382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019fa390 msgr2=0x560501dd2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019fa390 0x560501dd2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/3713993382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 msgr2=0x560501dd27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dd27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b8b7fe640 1 -- 172.21.15.112:0/3713993382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560501e49a30 con 0x5605019d99f0 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b827fc640 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dd27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b8b7fe640 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019d99f0 0x560501dd2290 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009ba0 tx=0x7f0b84004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88b0dbee03d7049 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b6affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b6affd640 1 -- 172.21.15.112:0/3713993382 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84035040 con 0x5605019d99f0 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3713993382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b84004940 con 0x5605019d99f0 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b6affd640 1 -- 172.21.15.112:0/3713993382 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f0b8400fd30 con 0x5605019d99f0 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.771+0000 7f0b6affd640 1 -- 172.21.15.112:0/3713993382 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b84003bc0 con 0x5605019d99f0 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.772+0000 7f0b97f8d740 10 client.16143 fetch_fsmap learned FSMap version 26 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.772+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3713993382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560501e4aca0 con 0x5605019d99f0 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.772+0000 7f0b6affd640 1 -- 172.21.15.112:0/3713993382 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b84004940 con 0x5605019d99f0 2024-09-21T03:32:09.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.772+0000 7f0b6affd640 1 -- 172.21.15.112:0/3713993382 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0b84048070 con 0x5605019d99f0 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.772+0000 7f0b97f8d740 10 client.16143 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.772+0000 7f0b97f8d740 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.772+0000 7f0b97f8d740 20 client.16143 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.772+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560501a1b070 0x560501e4d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.772+0000 7f0b97f8d740 4 client.16143 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.772+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3713993382 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560501d36850 con 0x560501a1b070 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.772+0000 7f0b827fc640 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560501a1b070 0x560501e4d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.773+0000 7f0b827fc640 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560501a1b070 0x560501e4d2f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a46c6eed841e4f00 server_cookie=b1550171c2538ab in_seq=0 out_seq=0 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.773+0000 7f0b6affd640 10 client.16143 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.773+0000 7f0b6affd640 1 -- 172.21.15.112:0/3713993382 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x560501d36850 con 0x560501a1b070 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.773+0000 7f0b6affd640 10 client.16143 handle_command_reply: tid=0 2024-09-21T03:32:09.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.773+0000 7f0b97f8d740 1 client.16143 shutdown 2024-09-21T03:32:09.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3713993382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019d99f0 msgr2=0x560501dd2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019d99f0 0x560501dd2290 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009ba0 tx=0x7f0b84004850 comp rx=0 tx=0).stop 2024-09-21T03:32:09.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3713993382 shutdown_connections 2024-09-21T03:32:09.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560501a1b070 0x560501e4d2f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605019fa390 0x560501dd2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605019f21b0 0x560501dd27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b6affd640 0 client.16143 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:09.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b97f8d740 1 --2- 172.21.15.112:0/3713993382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605019d99f0 0x560501dd2290 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3713993382 >> 172.21.15.112:0/3713993382 conn(0x560501a0f460 msgr2=0x560501dc5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3713993382 shutdown_connections 2024-09-21T03:32:09.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b97f8d740 1 -- 172.21.15.112:0/3713993382 wait complete. 2024-09-21T03:32:09.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b97f8d740 20 client.16143 trim_cache size 0 max 16384 2024-09-21T03:32:09.780 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "70d7715c-658d-43b2-ada8-48508980cc8e", 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:09.781 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:32:09.782 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:09.783 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:09.784 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:09.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:09.785 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:32:09.785 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:32:09.785 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:09.785 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:09.785 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:09.785 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:09.785 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:09.785 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:32:09.785 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:32:09.785 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:32:09.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b91a19640 1 -- 172.21.15.112:0/2277398104 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0b780b6680 msgr2=0x7f0b780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b91a19640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0b780b6680 0x7f0b780b8b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0e4c50 tx=0x7f0b7c0eaf40 comp rx=0 tx=0).stop 2024-09-21T03:32:09.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b91a19640 1 -- 172.21.15.112:0/2277398104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8c103160 msgr2=0x7f0b8c19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:09.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.774+0000 7f0b91a19640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8c103160 0x7f0b8c19c6e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00e9f0 tx=0x7f0b7c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.775+0000 7f0b91a19640 1 -- 172.21.15.112:0/2277398104 shutdown_connections 2024-09-21T03:32:09.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.775+0000 7f0b91a19640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0b780b6680 0x7f0b780b8b90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.775+0000 7f0b91a19640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b8c105ac0 0x7f0b8c1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.775+0000 7f0b91a19640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b8c105170 0x7f0b8c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.775+0000 7f0b91a19640 1 --2- 172.21.15.112:0/2277398104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b8c103160 0x7f0b8c19c6e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:09.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.775+0000 7f0b91a19640 1 -- 172.21.15.112:0/2277398104 >> 172.21.15.112:0/2277398104 conn(0x7f0b8c077c60 msgr2=0x7f0b8c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:09.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.775+0000 7f0b91a19640 1 -- 172.21.15.112:0/2277398104 shutdown_connections 2024-09-21T03:32:09.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:09.775+0000 7f0b91a19640 1 -- 172.21.15.112:0/2277398104 wait complete. 2024-09-21T03:32:09.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:09 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:32:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:09 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:09 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:09 smithi203 ceph-mon[34723]: pgmap v412: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 284 op/s 2024-09-21T03:32:09.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:09 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:32:09.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:09 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:09.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:09 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:09.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:09 smithi187 ceph-mon[34603]: pgmap v412: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 284 op/s 2024-09-21T03:32:10.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 268939 10.69 MB/sec execute 10 sec latency 18.239 ms 2024-09-21T03:32:10.620 INFO:tasks.cephfs.filesystem:scrub status for tag:70d7715c-658d-43b2-ada8-48508980cc8e - {'path': '/', 'tag': '70d7715c-658d-43b2-ada8-48508980cc8e', 'options': 'recursive,force'} 2024-09-21T03:32:11.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 270505 10.50 MB/sec execute 11 sec latency 11.433 ms 2024-09-21T03:32:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:11 smithi112 ceph-mon[31442]: pgmap v413: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 305 op/s 2024-09-21T03:32:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:32:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:11 smithi203 ceph-mon[34723]: pgmap v413: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 305 op/s 2024-09-21T03:32:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:32:11.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:11 smithi187 ceph-mon[34603]: pgmap v413: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 305 op/s 2024-09-21T03:32:11.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:32:11.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:32:11.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:32:11.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:32:12.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 273362 11.13 MB/sec execute 12 sec latency 11.840 ms 2024-09-21T03:32:13.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 274800 10.70 MB/sec execute 13 sec latency 18.036 ms 2024-09-21T03:32:13.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:13 smithi112 ceph-mon[31442]: pgmap v414: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 371 op/s 2024-09-21T03:32:13.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:13 smithi203 ceph-mon[34723]: pgmap v414: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 371 op/s 2024-09-21T03:32:13.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:13 smithi187 ceph-mon[34603]: pgmap v414: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 371 op/s 2024-09-21T03:32:14.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 277052 10.93 MB/sec execute 14 sec latency 26.533 ms 2024-09-21T03:32:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:14 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:32:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:14 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:32:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:14 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:32:14.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:14 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:32:14.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:14 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:32:14.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:14 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:32:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:14 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:32:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:14 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:32:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:14 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:32:15.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 279161 10.71 MB/sec execute 15 sec latency 30.281 ms 2024-09-21T03:32:15.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:15 smithi112 ceph-mon[31442]: pgmap v415: 97 pgs: 97 active+clean; 158 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 279 op/s 2024-09-21T03:32:15.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:15 smithi203 ceph-mon[34723]: pgmap v415: 97 pgs: 97 active+clean; 158 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 279 op/s 2024-09-21T03:32:15.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:15 smithi187 ceph-mon[34603]: pgmap v415: 97 pgs: 97 active+clean; 158 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 279 op/s 2024-09-21T03:32:16.217 INFO:tasks.workunit.client.0.smithi112.stdout: 1 281048 10.79 MB/sec execute 16 sec latency 13.257 ms 2024-09-21T03:32:17.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 283930 11.17 MB/sec execute 17 sec latency 15.713 ms 2024-09-21T03:32:17.712 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:17 smithi203 ceph-mon[34723]: pgmap v416: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 353 op/s 2024-09-21T03:32:17.731 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:17 smithi187 ceph-mon[34603]: pgmap v416: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 353 op/s 2024-09-21T03:32:17.786 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:17 smithi112 ceph-mon[31442]: pgmap v416: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 353 op/s 2024-09-21T03:32:18.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 285452 10.91 MB/sec execute 18 sec latency 17.830 ms 2024-09-21T03:32:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:18 smithi112 ceph-mon[31442]: pgmap v417: 97 pgs: 97 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 264 op/s 2024-09-21T03:32:18.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:18 smithi203 ceph-mon[34723]: pgmap v417: 97 pgs: 97 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 264 op/s 2024-09-21T03:32:18.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:18 smithi187 ceph-mon[34603]: pgmap v417: 97 pgs: 97 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 264 op/s 2024-09-21T03:32:19.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 287728 11.07 MB/sec execute 19 sec latency 29.669 ms 2024-09-21T03:32:20.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 289823 10.90 MB/sec execute 20 sec latency 17.675 ms 2024-09-21T03:32:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:20 smithi112 ceph-mon[31442]: pgmap v418: 97 pgs: 97 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 278 op/s 2024-09-21T03:32:20.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:20 smithi203 ceph-mon[34723]: pgmap v418: 97 pgs: 97 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 278 op/s 2024-09-21T03:32:20.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:20 smithi187 ceph-mon[34603]: pgmap v418: 97 pgs: 97 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 278 op/s 2024-09-21T03:32:21.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 291666 10.95 MB/sec execute 21 sec latency 16.077 ms 2024-09-21T03:32:22.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 294530 11.24 MB/sec execute 22 sec latency 12.695 ms 2024-09-21T03:32:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:22 smithi112 ceph-mon[31442]: pgmap v419: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 339 op/s 2024-09-21T03:32:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:22 smithi203 ceph-mon[34723]: pgmap v419: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 339 op/s 2024-09-21T03:32:22.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:22 smithi187 ceph-mon[34603]: pgmap v419: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 339 op/s 2024-09-21T03:32:23.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 295987 11.03 MB/sec execute 23 sec latency 13.268 ms 2024-09-21T03:32:24.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 298344 11.15 MB/sec execute 24 sec latency 23.311 ms 2024-09-21T03:32:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:24 smithi112 ceph-mon[31442]: pgmap v420: 97 pgs: 97 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-09-21T03:32:24.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:24 smithi203 ceph-mon[34723]: pgmap v420: 97 pgs: 97 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-09-21T03:32:24.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:24 smithi187 ceph-mon[34603]: pgmap v420: 97 pgs: 97 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-09-21T03:32:25.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 300443 11.01 MB/sec execute 25 sec latency 34.203 ms 2024-09-21T03:32:26.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 302349 11.05 MB/sec execute 26 sec latency 16.804 ms 2024-09-21T03:32:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:26 smithi112 ceph-mon[31442]: pgmap v421: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 347 op/s 2024-09-21T03:32:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:26 smithi203 ceph-mon[34723]: pgmap v421: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 347 op/s 2024-09-21T03:32:26.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:26 smithi187 ceph-mon[34603]: pgmap v421: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 347 op/s 2024-09-21T03:32:27.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 305305 11.32 MB/sec execute 27 sec latency 13.197 ms 2024-09-21T03:32:28.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 307033 11.12 MB/sec execute 28 sec latency 12.838 ms 2024-09-21T03:32:28.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:28 smithi112 ceph-mon[31442]: pgmap v422: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 256 op/s 2024-09-21T03:32:28.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:28 smithi203 ceph-mon[34723]: pgmap v422: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 256 op/s 2024-09-21T03:32:28.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:28 smithi187 ceph-mon[34603]: pgmap v422: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 256 op/s 2024-09-21T03:32:29.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 309076 11.21 MB/sec execute 29 sec latency 38.915 ms 2024-09-21T03:32:30.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 311951 11.32 MB/sec execute 30 sec latency 23.823 ms 2024-09-21T03:32:30.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:30 smithi112 ceph-mon[31442]: pgmap v423: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2024-09-21T03:32:30.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:30 smithi203 ceph-mon[34723]: pgmap v423: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2024-09-21T03:32:30.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:30 smithi187 ceph-mon[34603]: pgmap v423: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2024-09-21T03:32:31.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 313435 11.17 MB/sec execute 31 sec latency 18.478 ms 2024-09-21T03:32:32.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 316059 11.35 MB/sec execute 32 sec latency 13.914 ms 2024-09-21T03:32:32.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:32 smithi112 ceph-mon[31442]: pgmap v424: 97 pgs: 97 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 339 op/s 2024-09-21T03:32:32.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:32 smithi203 ceph-mon[34723]: pgmap v424: 97 pgs: 97 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 339 op/s 2024-09-21T03:32:32.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:32 smithi187 ceph-mon[34603]: pgmap v424: 97 pgs: 97 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 339 op/s 2024-09-21T03:32:33.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 318134 11.23 MB/sec execute 33 sec latency 12.228 ms 2024-09-21T03:32:34.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 319914 11.25 MB/sec execute 34 sec latency 32.322 ms 2024-09-21T03:32:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:34 smithi112 ceph-mon[31442]: pgmap v425: 97 pgs: 97 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-09-21T03:32:34.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:34 smithi203 ceph-mon[34723]: pgmap v425: 97 pgs: 97 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-09-21T03:32:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:34 smithi187 ceph-mon[34603]: pgmap v425: 97 pgs: 97 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-09-21T03:32:35.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 322854 11.43 MB/sec execute 35 sec latency 20.201 ms 2024-09-21T03:32:36.218 INFO:tasks.workunit.client.0.smithi112.stdout: 1 324444 11.30 MB/sec execute 36 sec latency 12.639 ms 2024-09-21T03:32:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:36 smithi112 ceph-mon[31442]: pgmap v426: 97 pgs: 97 active+clean; 189 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 316 op/s 2024-09-21T03:32:36.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:36 smithi203 ceph-mon[34723]: pgmap v426: 97 pgs: 97 active+clean; 189 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 316 op/s 2024-09-21T03:32:36.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:36 smithi187 ceph-mon[34603]: pgmap v426: 97 pgs: 97 active+clean; 189 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 316 op/s 2024-09-21T03:32:37.219 INFO:tasks.workunit.client.0.smithi112.stdout: 1 326794 11.37 MB/sec execute 37 sec latency 10.601 ms 2024-09-21T03:32:38.219 INFO:tasks.workunit.client.0.smithi112.stdout: 1 329098 11.36 MB/sec execute 38 sec latency 17.048 ms 2024-09-21T03:32:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:38 smithi112 ceph-mon[31442]: pgmap v427: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 258 op/s 2024-09-21T03:32:38.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:38 smithi203 ceph-mon[34723]: pgmap v427: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 258 op/s 2024-09-21T03:32:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:38 smithi187 ceph-mon[34603]: pgmap v427: 97 pgs: 97 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 258 op/s 2024-09-21T03:32:39.219 INFO:tasks.workunit.client.0.smithi112.stdout: 1 330675 11.29 MB/sec execute 39 sec latency 42.329 ms 2024-09-21T03:32:40.219 INFO:tasks.workunit.client.0.smithi112.stdout: 1 333632 11.46 MB/sec execute 40 sec latency 26.701 ms 2024-09-21T03:32:40.621 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:32:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:40 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:32:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:40 smithi112 ceph-mon[31442]: pgmap v428: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 279 op/s 2024-09-21T03:32:40.879 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:40.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:40 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:32:40.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:40 smithi203 ceph-mon[34723]: pgmap v428: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 279 op/s 2024-09-21T03:32:40.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:40 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:32:40.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:40 smithi187 ceph-mon[34603]: pgmap v428: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 279 op/s 2024-09-21T03:32:41.219 INFO:tasks.workunit.client.0.smithi112.stdout: 1 335204 11.32 MB/sec execute 41 sec latency 12.315 ms 2024-09-21T03:32:41.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.410+0000 7f49ffd7a640 1 -- 172.21.15.112:0/347904686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f8103bc0 msgr2=0x7f49f8103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:41.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.410+0000 7f49ffd7a640 1 --2- 172.21.15.112:0/347904686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f8103bc0 0x7f49f8103fd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009900 tx=0x7f49f402a410 comp rx=0 tx=0).stop 2024-09-21T03:32:41.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.410+0000 7f49ffd7a640 1 -- 172.21.15.112:0/347904686 shutdown_connections 2024-09-21T03:32:41.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.410+0000 7f49ffd7a640 1 --2- 172.21.15.112:0/347904686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f8109e00 0x7f49f8110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.410+0000 7f49ffd7a640 1 --2- 172.21.15.112:0/347904686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49f81094b0 0x7f49f81098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.410+0000 7f49ffd7a640 1 --2- 172.21.15.112:0/347904686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f8103bc0 0x7f49f8103fd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.410+0000 7f49ffd7a640 1 -- 172.21.15.112:0/347904686 >> 172.21.15.112:0/347904686 conn(0x7f49f807b910 msgr2=0x7f49f807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:41.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.410+0000 7f49ffd7a640 1 -- 172.21.15.112:0/347904686 shutdown_connections 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.410+0000 7f49ffd7a640 1 -- 172.21.15.112:0/347904686 wait complete. 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49ffd7a640 1 Processor -- start 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49ffd7a640 1 -- start start 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49ffd7a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49f8103bc0 0x7f49f819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49ffd7a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f81094b0 0x7f49f819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49ffd7a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f8109e00 0x7f49f81a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49ffd7a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8113ac0 con 0x7f49f8109e00 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49ffd7a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8113940 con 0x7f49f81094b0 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49ffd7a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8113c40 con 0x7f49f8103bc0 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49fd315640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f81094b0 0x7f49f819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49fe317640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f8109e00 0x7f49f81a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49fdb16640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49f8103bc0 0x7f49f819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49fd315640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f81094b0 0x7f49f819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:54910/0 (socket says 172.21.15.112:54910) 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.411+0000 7f49fd315640 1 -- 172.21.15.112:0/1770728973 learned_addr learned my addr 172.21.15.112:0/1770728973 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:41.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49fd315640 1 -- 172.21.15.112:0/1770728973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49f8103bc0 msgr2=0x7f49f819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49fd315640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49f8103bc0 0x7f49f819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49fd315640 1 -- 172.21.15.112:0/1770728973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f8109e00 msgr2=0x7f49f81a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49fd315640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f8109e00 0x7f49f81a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49fd315640 1 -- 172.21.15.112:0/1770728973 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f81a2600 con 0x7f49f81094b0 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49fe317640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f8109e00 0x7f49f81a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49fd315640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f81094b0 0x7f49f819cc30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49e800e9c0 tx=0x7f49e800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49eeffd640 1 -- 172.21.15.112:0/1770728973 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e800ccc0 con 0x7f49f81094b0 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49ffd7a640 1 -- 172.21.15.112:0/1770728973 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f81a28f0 con 0x7f49f81094b0 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49eeffd640 1 -- 172.21.15.112:0/1770728973 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e80054d0 con 0x7f49f81094b0 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49eeffd640 1 -- 172.21.15.112:0/1770728973 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8010430 con 0x7f49f81094b0 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.412+0000 7f49ffd7a640 1 -- 172.21.15.112:0/1770728973 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f81a9050 con 0x7f49f81094b0 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.415+0000 7f4a062ee740 1 Processor -- start 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f4a062ee740 1 -- start start 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f49eeffd640 1 -- 172.21.15.112:0/1770728973 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f49e8005020 con 0x7f49f81094b0 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c49070 0x560615ff0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 0x560615ff0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c28390 0x560615ff10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f4a062ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560615f66e50 con 0x560615c201b0 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f4a062ee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560615f56280 con 0x560615c28390 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f4a062ee740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560615f5a090 con 0x560615c49070 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f49fdb16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 0x560615ff0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f49fe317640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c49070 0x560615ff0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f49fd315640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c28390 0x560615ff10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f49fe317640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c49070 0x560615ff0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43198/0 (socket says 172.21.15.112:43198) 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f49fe317640 1 -- 172.21.15.112:0/100013873 learned_addr learned my addr 172.21.15.112:0/100013873 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f49fe317640 1 -- 172.21.15.112:0/100013873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c28390 msgr2=0x560615ff10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f49fe317640 1 --2- 172.21.15.112:0/100013873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c28390 0x560615ff10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f49fe317640 1 -- 172.21.15.112:0/100013873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 msgr2=0x560615ff0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f49fe317640 1 --2- 172.21.15.112:0/100013873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 0x560615ff0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.416+0000 7f49fe317640 1 -- 172.21.15.112:0/100013873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560615cb4650 con 0x560615c49070 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.417+0000 7f49fe317640 1 --2- 172.21.15.112:0/100013873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c49070 0x560615ff0650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f49f00117e0 tx=0x7f49f0011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e80bd5ae21c78e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.417+0000 7f49eeffd640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49e40b6680 0x7f49e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.417+0000 7f49e3fff640 1 -- 172.21.15.112:0/100013873 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0011310 con 0x560615c49070 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.417+0000 7f49fdb16640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49e40b6680 0x7f49e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.417+0000 7f49e3fff640 1 -- 172.21.15.112:0/100013873 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49f0003e20 con 0x560615c49070 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.417+0000 7f49e3fff640 1 -- 172.21.15.112:0/100013873 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f00044f0 con 0x560615c49070 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.417+0000 7f49eeffd640 1 -- 172.21.15.112:0/1770728973 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f49e80e7a40 con 0x7f49f81094b0 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.418+0000 7f4a062ee740 1 -- 172.21.15.112:0/100013873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c49070 msgr2=0x560615ff0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.418+0000 7f49fdb16640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49e40b6680 0x7f49e40b8b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f49f4006010 tx=0x7f49f4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.418+0000 7f4a062ee740 1 --2- 172.21.15.112:0/100013873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c49070 0x560615ff0650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f49f00117e0 tx=0x7f49f0011ca0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.418+0000 7f4a062ee740 1 -- 172.21.15.112:0/100013873 shutdown_connections 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.418+0000 7f4a062ee740 1 --2- 172.21.15.112:0/100013873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c28390 0x560615ff10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.418+0000 7f4a062ee740 1 --2- 172.21.15.112:0/100013873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 0x560615ff0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.418+0000 7f4a062ee740 1 --2- 172.21.15.112:0/100013873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c49070 0x560615ff0650 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.418+0000 7f4a062ee740 1 -- 172.21.15.112:0/100013873 >> 172.21.15.112:0/100013873 conn(0x560615c3d460 msgr2=0x560615fee540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.418+0000 7f4a062ee740 1 -- 172.21.15.112:0/100013873 shutdown_connections 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.418+0000 7f4a062ee740 1 -- 172.21.15.112:0/100013873 wait complete. 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.418+0000 7f4a062ee740 1 -- start start 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f4a062ee740 1 Processor -- start 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 0x560615ffbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 0x560615ffc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616008120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f4a062ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560615db7700 con 0x560615c28390 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f4a062ee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560615ef1110 con 0x560615c49070 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f4a062ee740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560615ffee30 con 0x560615c201b0 2024-09-21T03:32:41.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f4a062ee740 10 client.0 _ll_register_callbacks cb 0x560615f0a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f49fd315640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 0x560615ffc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f49fd315640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 0x560615ffc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36674/0 (socket says 172.21.15.112:36674) 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f49fd315640 1 -- 172.21.15.112:0/1341099503 learned_addr learned my addr 172.21.15.112:0/1341099503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f49fe317640 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616008120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.419+0000 7f49fdb16640 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 0x560615ffbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49fd315640 1 -- 172.21.15.112:0/1341099503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 msgr2=0x560615ffbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49fd315640 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 0x560615ffbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49fd315640 1 -- 172.21.15.112:0/1341099503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 msgr2=0x560616008120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49fd315640 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616008120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49fd315640 1 -- 172.21.15.112:0/1341099503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560615f1ec40 con 0x560615c28390 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49fe317640 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616008120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49fd315640 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 0x560615ffc2d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f49e80e5250 tx=0x7f49e80eb180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c56aa370a63b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:41.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49e17fa640 1 -- 172.21.15.112:0/1341099503 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80ee070 con 0x560615c28390 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49e17fa640 1 -- 172.21.15.112:0/1341099503 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f49e80a9240 con 0x560615c28390 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f4a062ee740 1 -- 172.21.15.112:0/1341099503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49f0003e20 con 0x560615c28390 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49e17fa640 1 -- 172.21.15.112:0/1341099503 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f49e80a9790 con 0x560615c28390 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f4a062ee740 10 client.16155 fetch_fsmap learned FSMap version 27 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f4a062ee740 1 -- 172.21.15.112:0/1341099503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56061600ac20 con 0x560615c28390 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.420+0000 7f49e17fa640 1 -- 172.21.15.112:0/1341099503 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49f0003e20 con 0x560615c28390 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.421+0000 7f49e17fa640 1 -- 172.21.15.112:0/1341099503 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f49e80a9440 con 0x560615c28390 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.421+0000 7f4a062ee740 10 client.16155 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.421+0000 7f4a062ee740 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.421+0000 7f4a062ee740 20 client.16155 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.421+0000 7f4a062ee740 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560615c079f0 0x56061600e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.421+0000 7f4a062ee740 4 client.16155 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.421+0000 7f4a062ee740 1 -- 172.21.15.112:0/1341099503 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560615f68da0 con 0x560615c079f0 2024-09-21T03:32:41.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.421+0000 7f49fdb16640 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560615c079f0 0x56061600e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.421+0000 7f49fdb16640 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560615c079f0 0x56061600e020 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cce98d14bed2953 server_cookie=d00d116653b1e107 in_seq=0 out_seq=0 2024-09-21T03:32:41.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.421+0000 7f49e17fa640 10 client.16155 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f49e17fa640 1 -- 172.21.15.112:0/1341099503 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560615f68da0 con 0x560615c079f0 2024-09-21T03:32:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f49e17fa640 10 client.16155 handle_command_reply: tid=0 2024-09-21T03:32:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 1 client.16155 shutdown 2024-09-21T03:32:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 1 -- 172.21.15.112:0/1341099503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 msgr2=0x560615ffc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:41.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 0x560615ffc2d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f49e80e5250 tx=0x7f49e80eb180 comp rx=0 tx=0).stop 2024-09-21T03:32:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 1 -- 172.21.15.112:0/1341099503 shutdown_connections 2024-09-21T03:32:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560615c079f0 0x56061600e020 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f49e17fa640 0 client.16155 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616008120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 0x560615ffc2d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 1 --2- 172.21.15.112:0/1341099503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 0x560615ffbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 1 -- 172.21.15.112:0/1341099503 >> 172.21.15.112:0/1341099503 conn(0x560615c3d460 msgr2=0x560615ff3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 1 -- 172.21.15.112:0/1341099503 shutdown_connections 2024-09-21T03:32:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 1 -- 172.21.15.112:0/1341099503 wait complete. 2024-09-21T03:32:41.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.424+0000 7f4a062ee740 20 client.16155 trim_cache size 0 max 16384 2024-09-21T03:32:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.442+0000 7f4a062ee740 1 Processor -- start 2024-09-21T03:32:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.442+0000 7f4a062ee740 1 -- start start 2024-09-21T03:32:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.442+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 0x560615ff2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c28390 0x560615ff3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616008020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f4a062ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81b4000 con 0x560615c201b0 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f4a062ee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81b18d0 con 0x560615c49070 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f4a062ee740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81af720 con 0x560615c28390 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49fdb16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 0x560615ff2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49fd315640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c28390 0x560615ff3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49fdb16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 0x560615ff2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36688/0 (socket says 172.21.15.112:36688) 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49fdb16640 1 -- 172.21.15.112:0/3229516293 learned_addr learned my addr 172.21.15.112:0/3229516293 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49fe317640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616008020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49fdb16640 1 -- 172.21.15.112:0/3229516293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c28390 msgr2=0x560615ff3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49fdb16640 1 --2- 172.21.15.112:0/3229516293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c28390 0x560615ff3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49fdb16640 1 -- 172.21.15.112:0/3229516293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 msgr2=0x560616008020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49fdb16640 1 --2- 172.21.15.112:0/3229516293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616008020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49fdb16640 1 -- 172.21.15.112:0/3229516293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56061600e3e0 con 0x560615c201b0 2024-09-21T03:32:41.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49fdb16640 1 --2- 172.21.15.112:0/3229516293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 0x560615ff2040 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009900 tx=0x7f49f403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd7ce0c02d4128f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:41.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.443+0000 7f49e37fe640 1 -- 172.21.15.112:0/3229516293 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4035040 con 0x560615c201b0 2024-09-21T03:32:41.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.444+0000 7f49e37fe640 1 -- 172.21.15.112:0/3229516293 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49f4005480 con 0x560615c201b0 2024-09-21T03:32:41.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.444+0000 7f49e37fe640 1 -- 172.21.15.112:0/3229516293 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4004ab0 con 0x560615c201b0 2024-09-21T03:32:41.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.444+0000 7f4a062ee740 1 -- 172.21.15.112:0/3229516293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 msgr2=0x560615ff2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.444+0000 7f4a062ee740 1 --2- 172.21.15.112:0/3229516293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 0x560615ff2040 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009900 tx=0x7f49f403f040 comp rx=0 tx=0).stop 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.444+0000 7f4a062ee740 1 -- 172.21.15.112:0/3229516293 shutdown_connections 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.445+0000 7f4a062ee740 1 --2- 172.21.15.112:0/3229516293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616008020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.445+0000 7f4a062ee740 1 --2- 172.21.15.112:0/3229516293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c28390 0x560615ff3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.445+0000 7f4a062ee740 1 --2- 172.21.15.112:0/3229516293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c201b0 0x560615ff2040 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.445+0000 7f4a062ee740 1 -- 172.21.15.112:0/3229516293 >> 172.21.15.112:0/3229516293 conn(0x560615c3d460 msgr2=0x560615ff3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.445+0000 7f4a062ee740 1 -- 172.21.15.112:0/3229516293 shutdown_connections 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.445+0000 7f4a062ee740 1 -- 172.21.15.112:0/3229516293 wait complete. 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.445+0000 7f4a062ee740 1 -- start start 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.445+0000 7f4a062ee740 1 Processor -- start 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.445+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 0x5606160049a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.445+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 0x560615fffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f4a062ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616000540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f4a062ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81afd90 con 0x560615c28390 2024-09-21T03:32:41.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f4a062ee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81b1200 con 0x560615c49070 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f4a062ee740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81b0460 con 0x560615c201b0 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f4a062ee740 10 client.0 _ll_register_callbacks cb 0x7f49e80eb280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fdb16640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 0x5606160049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fd315640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 0x560615fffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fdb16640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 0x5606160049a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43212/0 (socket says 172.21.15.112:43212) 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fdb16640 1 -- 172.21.15.112:0/3805973464 learned_addr learned my addr 172.21.15.112:0/3805973464 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fe317640 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616000540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fd315640 1 -- 172.21.15.112:0/3805973464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 msgr2=0x5606160049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fd315640 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 0x5606160049a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fd315640 1 -- 172.21.15.112:0/3805973464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 msgr2=0x560616000540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fd315640 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616000540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fd315640 1 -- 172.21.15.112:0/3805973464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560616000cb0 con 0x560615c28390 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fdb16640 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 0x5606160049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49fd315640 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 0x560615fffea0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f49e80ad5f0 tx=0x7f49e800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02fcf0084c26d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:41.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49e17fa640 1 -- 172.21.15.112:0/3805973464 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80ee070 con 0x560615c28390 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f49e17fa640 1 -- 172.21.15.112:0/3805973464 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f49e80ac5c0 con 0x560615c28390 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.446+0000 7f4a062ee740 1 -- 172.21.15.112:0/3805973464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49f4005480 con 0x560615c28390 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f49e17fa640 1 -- 172.21.15.112:0/3805973464 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f49e800ce60 con 0x560615c28390 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f4a062ee740 10 client.16161 fetch_fsmap learned FSMap version 27 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f4a062ee740 1 -- 172.21.15.112:0/3805973464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560616078ac0 con 0x560615c28390 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f49e17fa640 1 -- 172.21.15.112:0/3805973464 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e80b21c0 con 0x560615c28390 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f49e17fa640 1 -- 172.21.15.112:0/3805973464 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f49e80ac7c0 con 0x560615c28390 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f4a062ee740 10 client.16161 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f4a062ee740 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f4a062ee740 20 client.16161 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f4a062ee740 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560615bdcda0 0x56061607b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f4a062ee740 4 client.16161 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f4a062ee740 1 -- 172.21.15.112:0/3805973464 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f49f4005480 con 0x560615bdcda0 2024-09-21T03:32:41.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.447+0000 7f49fdb16640 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560615bdcda0 0x56061607b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f49e17fa640 10 client.16161 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f49fdb16640 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560615bdcda0 0x56061607b060 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbee1823493044c9 server_cookie=1081f19ec434878c in_seq=0 out_seq=0 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f49e17fa640 1 -- 172.21.15.112:0/3805973464 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x7f49f4005480 con 0x560615bdcda0 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f49e17fa640 10 client.16161 handle_command_reply: tid=0 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f4a062ee740 1 client.16161 shutdown 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f4a062ee740 1 -- 172.21.15.112:0/3805973464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 msgr2=0x560615fffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f4a062ee740 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 0x560615fffea0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f49e80ad5f0 tx=0x7f49e800e940 comp rx=0 tx=0).stop 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f4a062ee740 1 -- 172.21.15.112:0/3805973464 shutdown_connections 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f4a062ee740 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560615bdcda0 0x56061607b060 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f4a062ee740 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560615c49070 0x560616000540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f4a062ee740 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560615c28390 0x560615fffea0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f49e17fa640 0 client.16161 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f4a062ee740 1 --2- 172.21.15.112:0/3805973464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560615c201b0 0x5606160049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.448+0000 7f4a062ee740 1 -- 172.21.15.112:0/3805973464 >> 172.21.15.112:0/3805973464 conn(0x560615c3d460 msgr2=0x560615ff4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:41.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f4a062ee740 1 -- 172.21.15.112:0/3805973464 shutdown_connections 2024-09-21T03:32:41.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f4a062ee740 1 -- 172.21.15.112:0/3805973464 wait complete. 2024-09-21T03:32:41.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f4a062ee740 20 client.16161 trim_cache size 0 max 16384 2024-09-21T03:32:41.455 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:32:41.455 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:32:41.455 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:32:41.455 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:32:41.455 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:32:41.455 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:32:41.455 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:41.455 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:41.455 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:32:41.456 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:32:41.457 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:32:41.458 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:32:41.459 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:41.459 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:41.459 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:41.459 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:41.459 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:41.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:32:41.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:32:41.459 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f49ffd7a640 1 -- 172.21.15.112:0/1770728973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49e40b6680 msgr2=0x7f49e40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f49ffd7a640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49e40b6680 0x7f49e40b8b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f49f4006010 tx=0x7f49f4005a20 comp rx=0 tx=0).stop 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f49ffd7a640 1 -- 172.21.15.112:0/1770728973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f81094b0 msgr2=0x7f49f819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f49ffd7a640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f81094b0 0x7f49f819cc30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49e800e9c0 tx=0x7f49e800ee80 comp rx=0 tx=0).stop 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f49ffd7a640 1 -- 172.21.15.112:0/1770728973 shutdown_connections 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f49ffd7a640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49e40b6680 0x7f49e40b8b90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f49ffd7a640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f8109e00 0x7f49f81a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f49ffd7a640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f81094b0 0x7f49f819cc30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f49ffd7a640 1 --2- 172.21.15.112:0/1770728973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49f8103bc0 0x7f49f819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f49ffd7a640 1 -- 172.21.15.112:0/1770728973 >> 172.21.15.112:0/1770728973 conn(0x7f49f807b910 msgr2=0x7f49f81072c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.449+0000 7f49ffd7a640 1 -- 172.21.15.112:0/1770728973 shutdown_connections 2024-09-21T03:32:41.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:41.450+0000 7f49ffd7a640 1 -- 172.21.15.112:0/1770728973 wait complete. 2024-09-21T03:32:42.219 INFO:tasks.workunit.client.0.smithi112.stdout: 1 337501 11.38 MB/sec execute 42 sec latency 14.267 ms 2024-09-21T03:32:42.357 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:32:42.357 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:32:42.626 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:42.629 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:42 smithi112 ceph-mon[31442]: pgmap v429: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 357 op/s 2024-09-21T03:32:42.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:42 smithi203 ceph-mon[34723]: pgmap v429: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 357 op/s 2024-09-21T03:32:42.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:42 smithi187 ceph-mon[34603]: pgmap v429: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 357 op/s 2024-09-21T03:32:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.114+0000 7f5909997640 1 -- 172.21.15.112:0/2920144182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5904103bc0 msgr2=0x7f5904103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.114+0000 7f5909997640 1 --2- 172.21.15.112:0/2920144182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5904103bc0 0x7f5904103fd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009860 tx=0x7f58f402a340 comp rx=0 tx=0).stop 2024-09-21T03:32:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.115+0000 7f5909997640 1 -- 172.21.15.112:0/2920144182 shutdown_connections 2024-09-21T03:32:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.115+0000 7f5909997640 1 --2- 172.21.15.112:0/2920144182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5904109e00 0x7f5904110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.115+0000 7f5909997640 1 --2- 172.21.15.112:0/2920144182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59041094b0 0x7f59041098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.115+0000 7f5909997640 1 --2- 172.21.15.112:0/2920144182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5904103bc0 0x7f5904103fd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.115+0000 7f5909997640 1 -- 172.21.15.112:0/2920144182 >> 172.21.15.112:0/2920144182 conn(0x7f590407b910 msgr2=0x7f590407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.115+0000 7f5909997640 1 -- 172.21.15.112:0/2920144182 shutdown_connections 2024-09-21T03:32:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.115+0000 7f5909997640 1 -- 172.21.15.112:0/2920144182 wait complete. 2024-09-21T03:32:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.115+0000 7f5909997640 1 Processor -- start 2024-09-21T03:32:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f5909997640 1 -- start start 2024-09-21T03:32:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f5909997640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5904103bc0 0x7f590419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f5909997640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59041094b0 0x7f590419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f5909997640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5904109e00 0x7f590419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f5909997640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5904113b20 con 0x7f5904109e00 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f5909997640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59041139a0 con 0x7f59041094b0 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f5909997640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5904113ca0 con 0x7f5904103bc0 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f59037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5904109e00 0x7f590419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f5902ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5904103bc0 0x7f590419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f5902ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5904103bc0 0x7f590419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43220/0 (socket says 172.21.15.112:43220) 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f59037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5904109e00 0x7f590419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36722/0 (socket says 172.21.15.112:36722) 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f5902ffd640 1 -- 172.21.15.112:0/2061558593 learned_addr learned my addr 172.21.15.112:0/2061558593 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.116+0000 7f59027fc640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59041094b0 0x7f590419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f59037fe640 1 -- 172.21.15.112:0/2061558593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5904103bc0 msgr2=0x7f590419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f59037fe640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5904103bc0 0x7f590419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f59037fe640 1 -- 172.21.15.112:0/2061558593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59041094b0 msgr2=0x7f590419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f59037fe640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59041094b0 0x7f590419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f59037fe640 1 -- 172.21.15.112:0/2061558593 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59041a0440 con 0x7f5904109e00 2024-09-21T03:32:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f5902ffd640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5904103bc0 0x7f590419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f59037fe640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5904109e00 0x7f590419fd10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00be20 tx=0x7f58fc009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f5908995640 1 -- 172.21.15.112:0/2061558593 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc019070 con 0x7f5904109e00 2024-09-21T03:32:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f5909997640 1 -- 172.21.15.112:0/2061558593 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59041a0730 con 0x7f5904109e00 2024-09-21T03:32:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f5908995640 1 -- 172.21.15.112:0/2061558593 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58fc00cd70 con 0x7f5904109e00 2024-09-21T03:32:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f5908995640 1 -- 172.21.15.112:0/2061558593 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc012720 con 0x7f5904109e00 2024-09-21T03:32:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.117+0000 7f5909997640 1 -- 172.21.15.112:0/2061558593 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59041a6e90 con 0x7f5904109e00 2024-09-21T03:32:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.118+0000 7f5909997640 1 -- 172.21.15.112:0/2061558593 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5904103fd0 con 0x7f5904109e00 2024-09-21T03:32:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.120+0000 7f5908995640 1 -- 172.21.15.112:0/2061558593 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f58fc01c070 con 0x7f5904109e00 2024-09-21T03:32:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.120+0000 7f5908995640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58ec0b6680 0x7f58ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.120+0000 7f5902ffd640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58ec0b6680 0x7f58ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.120+0000 7f5908995640 1 -- 172.21.15.112:0/2061558593 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f58fc0e6ce0 con 0x7f5904109e00 2024-09-21T03:32:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.121+0000 7f5902ffd640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58ec0b6680 0x7f58ec0b8b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f58f40095f0 tx=0x7f58f4030340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.125+0000 7f5908995640 1 -- 172.21.15.112:0/2061558593 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f58fc0b1670 con 0x7f5904109e00 2024-09-21T03:32:43.219 INFO:tasks.workunit.client.0.smithi112.stdout: 1 340484 11.52 MB/sec execute 43 sec latency 13.282 ms 2024-09-21T03:32:43.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.349+0000 7f5909997640 1 -- 172.21.15.112:0/2061558593 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f590410bc30 con 0x7f5904109e00 2024-09-21T03:32:43.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.350+0000 7f5908995640 1 -- 172.21.15.112:0/2061558593 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7f58fc0b65b0 con 0x7f5904109e00 2024-09-21T03:32:43.352 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:32:43.352 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":27,"btime":"2024-09-21T03:32:39:500736+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:32:39.054230+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:32:43.352 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 27 2024-09-21T03:32:43.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 -- 172.21.15.112:0/2061558593 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58ec0b6680 msgr2=0x7f58ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58ec0b6680 0x7f58ec0b8b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f58f40095f0 tx=0x7f58f4030340 comp rx=0 tx=0).stop 2024-09-21T03:32:43.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 -- 172.21.15.112:0/2061558593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5904109e00 msgr2=0x7f590419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:43.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5904109e00 0x7f590419fd10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00be20 tx=0x7f58fc009ed0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 -- 172.21.15.112:0/2061558593 shutdown_connections 2024-09-21T03:32:43.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58ec0b6680 0x7f58ec0b8b90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5904109e00 0x7f590419fd10 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59041094b0 0x7f590419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 --2- 172.21.15.112:0/2061558593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5904103bc0 0x7f590419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:43.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 -- 172.21.15.112:0/2061558593 >> 172.21.15.112:0/2061558593 conn(0x7f590407b910 msgr2=0x7f5904107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:43.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 -- 172.21.15.112:0/2061558593 shutdown_connections 2024-09-21T03:32:43.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:43.352+0000 7f5909997640 1 -- 172.21.15.112:0/2061558593 wait complete. 2024-09-21T03:32:43.726 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:43 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2061558593' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:43.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:43 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2061558593' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:43.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:43 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2061558593' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:32:44.210 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:32:44.219 INFO:tasks.workunit.client.0.smithi112.stdout: 1 341943 11.41 MB/sec execute 44 sec latency 32.272 ms 2024-09-21T03:32:44.486 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:44.761 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:44 smithi112 ceph-mon[31442]: pgmap v430: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-09-21T03:32:44.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:44 smithi203 ceph-mon[34723]: pgmap v430: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-09-21T03:32:44.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:44 smithi187 ceph-mon[34603]: pgmap v430: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-09-21T03:32:44.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.996+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/2420789737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc10de00 msgr2=0x7fd1cc110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:44.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.996+0000 7fd1d1ebb640 1 --2- 172.21.15.112:0/2420789737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc10de00 0x7fd1cc110310 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009900 tx=0x7fd1c402a410 comp rx=0 tx=0).stop 2024-09-21T03:32:44.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.996+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/2420789737 shutdown_connections 2024-09-21T03:32:44.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.996+0000 7fd1d1ebb640 1 --2- 172.21.15.112:0/2420789737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc10de00 0x7fd1cc110310 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:44.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.996+0000 7fd1d1ebb640 1 --2- 172.21.15.112:0/2420789737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1cc105280 0x7fd1cc10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:44.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.996+0000 7fd1d1ebb640 1 --2- 172.21.15.112:0/2420789737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1cc104850 0x7fd1cc104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:44.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.996+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/2420789737 >> 172.21.15.112:0/2420789737 conn(0x7fd1cc079730 msgr2=0x7fd1cc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:44.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.996+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/2420789737 shutdown_connections 2024-09-21T03:32:44.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.996+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/2420789737 wait complete. 2024-09-21T03:32:45.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.997+0000 7fd1d1ebb640 1 Processor -- start 2024-09-21T03:32:45.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.997+0000 7fd1d1ebb640 1 -- start start 2024-09-21T03:32:45.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.997+0000 7fd1d1ebb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc104850 0x7fd1cc19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.997+0000 7fd1d1ebb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1cc105280 0x7fd1cc19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.997+0000 7fd1d1ebb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1cc10de00 0x7fd1cc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.997+0000 7fd1d1ebb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc113aa0 con 0x7fd1cc104850 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.997+0000 7fd1d1ebb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc113920 con 0x7fd1cc10de00 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.997+0000 7fd1d1ebb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc113c20 con 0x7fd1cc105280 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc104850 0x7fd1cc19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1caffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1cc105280 0x7fd1cc19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1caffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1cc105280 0x7fd1cc19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43238/0 (socket says 172.21.15.112:43238) 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1caffd640 1 -- 172.21.15.112:0/3610143581 learned_addr learned my addr 172.21.15.112:0/3610143581 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc104850 0x7fd1cc19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36738/0 (socket says 172.21.15.112:36738) 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1cbfff640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1cc10de00 0x7fd1cc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/3610143581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1cc105280 msgr2=0x7fd1cc19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1cb7fe640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1cc105280 0x7fd1cc19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/3610143581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1cc10de00 msgr2=0x7fd1cc19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1cb7fe640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1cc10de00 0x7fd1cc19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/3610143581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc1a03e0 con 0x7fd1cc104850 2024-09-21T03:32:45.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1caffd640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1cc105280 0x7fd1cc19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:45.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1cb7fe640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc104850 0x7fd1cc19a470 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00e9c0 tx=0x7fd1bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1c8ff9640 1 -- 172.21.15.112:0/3610143581 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00ccc0 con 0x7fd1cc104850 2024-09-21T03:32:45.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1c8ff9640 1 -- 172.21.15.112:0/3610143581 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1bc0035f0 con 0x7fd1cc104850 2024-09-21T03:32:45.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1c8ff9640 1 -- 172.21.15.112:0/3610143581 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00ce60 con 0x7fd1cc104850 2024-09-21T03:32:45.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.998+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/3610143581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc1a06d0 con 0x7fd1cc104850 2024-09-21T03:32:45.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:44.999+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/3610143581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc1a6e30 con 0x7fd1cc104850 2024-09-21T03:32:45.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.001+0000 7fd1c8ff9640 1 -- 172.21.15.112:0/3610143581 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fd1bc004000 con 0x7fd1cc104850 2024-09-21T03:32:45.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1d842f740 1 Processor -- start 2024-09-21T03:32:45.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1d842f740 1 -- start start 2024-09-21T03:32:45.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1c8ff9640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd1b40b6680 0x7fd1b40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173b2070 0x557d177592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173891b0 0x557d17759800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d17391390 0x557d17759d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1d842f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557d176cfea0 con 0x557d173b2070 2024-09-21T03:32:45.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1d842f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557d176bf2d0 con 0x557d173891b0 2024-09-21T03:32:45.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1d842f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557d176c30e0 con 0x557d17391390 2024-09-21T03:32:45.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173b2070 0x557d177592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1caffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173891b0 0x557d17759800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1cbfff640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd1b40b6680 0x7fd1b40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173b2070 0x557d177592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36744/0 (socket says 172.21.15.112:36744) 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/1451706558 learned_addr learned my addr 172.21.15.112:0/1451706558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1caffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173891b0 0x557d17759800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:55002/0 (socket says 172.21.15.112:55002) 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1cbfff640 1 --2- 172.21.15.112:0/1451706558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d17391390 0x557d17759d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.002+0000 7fd1c8ff9640 1 -- 172.21.15.112:0/3610143581 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd1bc0e7a20 con 0x7fd1cc104850 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.003+0000 7fd1caffd640 1 -- 172.21.15.112:0/1451706558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d17391390 msgr2=0x557d17759d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.003+0000 7fd1caffd640 1 --2- 172.21.15.112:0/1451706558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d17391390 0x557d17759d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.003+0000 7fd1caffd640 1 -- 172.21.15.112:0/1451706558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173b2070 msgr2=0x557d177592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.003+0000 7fd1caffd640 1 --2- 172.21.15.112:0/1451706558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173b2070 0x557d177592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.003+0000 7fd1caffd640 1 -- 172.21.15.112:0/1451706558 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1741d650 con 0x557d173891b0 2024-09-21T03:32:45.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.003+0000 7fd1cb7fe640 1 --2- 172.21.15.112:0/1451706558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173b2070 0x557d177592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:45.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.003+0000 7fd1caffd640 1 --2- 172.21.15.112:0/1451706558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173891b0 0x557d17759800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0011760 tx=0x7fd1c0011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e47ba3b7d087f60b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.003+0000 7fd1cbfff640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd1b40b6680 0x7fd1b40b8b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4006010 tx=0x7fd1c4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.003+0000 7fd1b1ffb640 1 -- 172.21.15.112:0/1451706558 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0003c30 con 0x557d173891b0 2024-09-21T03:32:45.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.003+0000 7fd1b1ffb640 1 -- 172.21.15.112:0/1451706558 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c000fe50 con 0x557d173891b0 2024-09-21T03:32:45.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.003+0000 7fd1b1ffb640 1 -- 172.21.15.112:0/1451706558 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c00193e0 con 0x557d173891b0 2024-09-21T03:32:45.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.004+0000 7fd1d842f740 1 -- 172.21.15.112:0/1451706558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173891b0 msgr2=0x557d17759800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.004+0000 7fd1d842f740 1 --2- 172.21.15.112:0/1451706558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173891b0 0x557d17759800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0011760 tx=0x7fd1c0011c20 comp rx=0 tx=0).stop 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.004+0000 7fd1d842f740 1 -- 172.21.15.112:0/1451706558 shutdown_connections 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.004+0000 7fd1d842f740 1 --2- 172.21.15.112:0/1451706558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d17391390 0x557d17759d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.004+0000 7fd1d842f740 1 --2- 172.21.15.112:0/1451706558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173891b0 0x557d17759800 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.004+0000 7fd1d842f740 1 --2- 172.21.15.112:0/1451706558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173b2070 0x557d177592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.004+0000 7fd1d842f740 1 -- 172.21.15.112:0/1451706558 >> 172.21.15.112:0/1451706558 conn(0x557d173a6460 msgr2=0x557d177571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.004+0000 7fd1d842f740 1 -- 172.21.15.112:0/1451706558 shutdown_connections 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.004+0000 7fd1d842f740 1 -- 172.21.15.112:0/1451706558 wait complete. 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.005+0000 7fd1d842f740 1 -- start start 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.005+0000 7fd1d842f740 1 Processor -- start 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.005+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d17768bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.005+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d17391390 0x557d17769110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.005+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 0x557d17769650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.005+0000 7fd1d842f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557d17520700 con 0x557d173891b0 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.005+0000 7fd1d842f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557d1765a160 con 0x557d173b2070 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.005+0000 7fd1d842f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557d1776bda0 con 0x557d17391390 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.005+0000 7fd1d842f740 10 client.0 _ll_register_callbacks cb 0x557d176739e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.005+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d17768bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d17768bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36754/0 (socket says 172.21.15.112:36754) 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/431634424 learned_addr learned my addr 172.21.15.112:0/431634424 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1caffd640 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d17391390 0x557d17769110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1cbfff640 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 0x557d17769650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/431634424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d17391390 msgr2=0x557d17769110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1cb7fe640 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d17391390 0x557d17769110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/431634424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 msgr2=0x557d17769650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1cb7fe640 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 0x557d17769650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/431634424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d17687c90 con 0x557d173891b0 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1caffd640 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d17391390 0x557d17769110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1cb7fe640 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d17768bd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00c6a0 tx=0x7fd1bc0763f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99d08ad82d91cd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1aaffd640 1 -- 172.21.15.112:0/431634424 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0ad3e0 con 0x557d173891b0 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1aaffd640 1 -- 172.21.15.112:0/431634424 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd1bc0d22c0 con 0x557d173891b0 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.006+0000 7fd1d842f740 1 -- 172.21.15.112:0/431634424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c000fe50 con 0x557d173891b0 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1aaffd640 1 -- 172.21.15.112:0/431634424 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1bc0adfd0 con 0x557d173891b0 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1d842f740 10 client.16179 fetch_fsmap learned FSMap version 28 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1d842f740 1 -- 172.21.15.112:0/431634424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d17777930 con 0x557d173891b0 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1aaffd640 1 -- 172.21.15.112:0/431634424 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c000fe50 con 0x557d173891b0 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1aaffd640 1 -- 172.21.15.112:0/431634424 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fd1bc0d4790 con 0x557d173891b0 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1d842f740 10 client.16179 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:32:45.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1d842f740 10 client.16179 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:32:45.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1d842f740 20 client.16179 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:45.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1d842f740 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x557d173709f0 0x557d1777ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1d842f740 4 client.16179 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:45.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1d842f740 1 -- 172.21.15.112:0/431634424 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d176d1df0 con 0x557d173709f0 2024-09-21T03:32:45.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.007+0000 7fd1caffd640 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x557d173709f0 0x557d1777ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.008+0000 7fd1aaffd640 10 client.16179 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:32:45.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.008+0000 7fd1caffd640 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x557d173709f0 0x557d1777ad40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe7c25c5173ad2c4 server_cookie=ce8e2c4e75caad7 in_seq=0 out_seq=0 2024-09-21T03:32:45.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.009+0000 7fd1aaffd640 1 -- 172.21.15.112:0/431634424 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557d176d1df0 con 0x557d173709f0 2024-09-21T03:32:45.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.009+0000 7fd1aaffd640 10 client.16179 handle_command_reply: tid=0 2024-09-21T03:32:45.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.009+0000 7fd1d842f740 1 client.16179 shutdown 2024-09-21T03:32:45.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.009+0000 7fd1d842f740 1 -- 172.21.15.112:0/431634424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 msgr2=0x557d17768bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.009+0000 7fd1d842f740 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d17768bd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00c6a0 tx=0x7fd1bc0763f0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.010+0000 7fd1d842f740 1 -- 172.21.15.112:0/431634424 shutdown_connections 2024-09-21T03:32:45.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.010+0000 7fd1d842f740 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x557d173709f0 0x557d1777ad40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.010+0000 7fd1d842f740 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 0x557d17769650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.010+0000 7fd1d842f740 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d17391390 0x557d17769110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.010+0000 7fd1d842f740 1 --2- 172.21.15.112:0/431634424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d17768bd0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.010+0000 7fd1d842f740 1 -- 172.21.15.112:0/431634424 >> 172.21.15.112:0/431634424 conn(0x557d173a6460 msgr2=0x557d17755750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:45.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.010+0000 7fd1aaffd640 0 client.16179 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:32:45.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.010+0000 7fd1d842f740 1 -- 172.21.15.112:0/431634424 shutdown_connections 2024-09-21T03:32:45.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.010+0000 7fd1d842f740 1 -- 172.21.15.112:0/431634424 wait complete. 2024-09-21T03:32:45.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.010+0000 7fd1d842f740 20 client.16179 trim_cache size 0 max 16384 2024-09-21T03:32:45.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1d842f740 1 Processor -- start 2024-09-21T03:32:45.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1d842f740 1 -- start start 2024-09-21T03:32:45.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d173891b0 0x557d177571e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d17391390 0x557d17765dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 0x557d17766300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1d842f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1afc70 con 0x557d17391390 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1d842f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1ad540 con 0x557d173b2070 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1d842f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1ab390 con 0x557d173891b0 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1caffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d17391390 0x557d17765dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d173891b0 0x557d177571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1cbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 0x557d17766300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d173891b0 0x557d177571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43256/0 (socket says 172.21.15.112:43256) 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1cbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 0x557d17766300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:55028/0 (socket says 172.21.15.112:55028) 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/2564349735 learned_addr learned my addr 172.21.15.112:0/2564349735 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1caffd640 1 -- 172.21.15.112:0/2564349735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d173891b0 msgr2=0x557d177571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1caffd640 1 --2- 172.21.15.112:0/2564349735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d173891b0 0x557d177571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1caffd640 1 -- 172.21.15.112:0/2564349735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 msgr2=0x557d17766300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1caffd640 1 --2- 172.21.15.112:0/2564349735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 0x557d17766300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1caffd640 1 -- 172.21.15.112:0/2564349735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1775bbb0 con 0x557d17391390 2024-09-21T03:32:45.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.024+0000 7fd1cbfff640 1 --2- 172.21.15.112:0/2564349735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 0x557d17766300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:45.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.025+0000 7fd1caffd640 1 --2- 172.21.15.112:0/2564349735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d17391390 0x557d17765dc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00044b0 tx=0x7fd1c000f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a72da0e2b0013c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.025+0000 7fd1b17fa640 1 -- 172.21.15.112:0/2564349735 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0011040 con 0x557d17391390 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.025+0000 7fd1b17fa640 1 -- 172.21.15.112:0/2564349735 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c00111e0 con 0x557d17391390 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.025+0000 7fd1b17fa640 1 -- 172.21.15.112:0/2564349735 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0004d60 con 0x557d17391390 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.026+0000 7fd1d842f740 1 -- 172.21.15.112:0/2564349735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d17391390 msgr2=0x557d17765dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.026+0000 7fd1d842f740 1 --2- 172.21.15.112:0/2564349735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d17391390 0x557d17765dc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd1c00044b0 tx=0x7fd1c000f8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.026+0000 7fd1d842f740 1 -- 172.21.15.112:0/2564349735 shutdown_connections 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.026+0000 7fd1d842f740 1 --2- 172.21.15.112:0/2564349735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d173b2070 0x557d17766300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.026+0000 7fd1d842f740 1 --2- 172.21.15.112:0/2564349735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d17391390 0x557d17765dc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.026+0000 7fd1d842f740 1 --2- 172.21.15.112:0/2564349735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d173891b0 0x557d177571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.026+0000 7fd1d842f740 1 -- 172.21.15.112:0/2564349735 >> 172.21.15.112:0/2564349735 conn(0x557d173a6460 msgr2=0x557d17756ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.026+0000 7fd1d842f740 1 -- 172.21.15.112:0/2564349735 shutdown_connections 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.026+0000 7fd1d842f740 1 -- 172.21.15.112:0/2564349735 wait complete. 2024-09-21T03:32:45.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.026+0000 7fd1d842f740 1 -- start start 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1d842f740 1 Processor -- start 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d1776c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d17391390 0x557d1776cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1d842f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d173b2070 0x557d177e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1d842f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1aba00 con 0x557d173891b0 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1d842f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1ace70 con 0x557d17391390 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1d842f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc1ac0d0 con 0x557d173b2070 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d1776c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1d842f740 10 client.0 _ll_register_callbacks cb 0x7fd1bc0a9050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d1776c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36764/0 (socket says 172.21.15.112:36764) 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/910027401 learned_addr learned my addr 172.21.15.112:0/910027401 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1cbfff640 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d173b2070 0x557d177e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1caffd640 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d17391390 0x557d1776cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/910027401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d173b2070 msgr2=0x557d177e36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1cb7fe640 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d173b2070 0x557d177e36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/910027401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d17391390 msgr2=0x557d1776cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1cb7fe640 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d17391390 0x557d1776cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.027+0000 7fd1cb7fe640 1 -- 172.21.15.112:0/910027401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d177e3c10 con 0x557d173891b0 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1cb7fe640 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d1776c830 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0e57d0 tx=0x7fd1bc0b23b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62efe35a5ab073be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1aaffd640 1 -- 172.21.15.112:0/910027401 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0b72f0 con 0x557d173891b0 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1aaffd640 1 -- 172.21.15.112:0/910027401 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fd1bc07a040 con 0x557d173891b0 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1d842f740 1 -- 172.21.15.112:0/910027401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c00111e0 con 0x557d173891b0 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1aaffd640 1 -- 172.21.15.112:0/910027401 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1bc07a280 con 0x557d173891b0 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1d842f740 10 client.16185 fetch_fsmap learned FSMap version 28 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1d842f740 1 -- 172.21.15.112:0/910027401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d177e4e30 con 0x557d173891b0 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1aaffd640 1 -- 172.21.15.112:0/910027401 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c00111e0 con 0x557d173891b0 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1aaffd640 1 -- 172.21.15.112:0/910027401 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fd1bc0d40d0 con 0x557d173891b0 2024-09-21T03:32:45.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1d842f740 10 client.16185 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1d842f740 10 client.16185 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1d842f740 20 client.16185 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1d842f740 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x557d17345da0 0x557d177e7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1d842f740 4 client.16185 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.028+0000 7fd1d842f740 1 -- 172.21.15.112:0/910027401 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d176cd8a0 con 0x557d17345da0 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.029+0000 7fd1caffd640 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x557d17345da0 0x557d177e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.029+0000 7fd1aaffd640 10 client.16185 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.029+0000 7fd1caffd640 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x557d17345da0 0x557d177e7670 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23fe102c69b7d8ef server_cookie=5eb49bddcbf90fea in_seq=0 out_seq=0 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.029+0000 7fd1aaffd640 1 -- 172.21.15.112:0/910027401 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d176cd8a0 con 0x557d17345da0 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.029+0000 7fd1aaffd640 10 client.16185 handle_command_reply: tid=0 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.029+0000 7fd1d842f740 1 client.16185 shutdown 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.029+0000 7fd1d842f740 1 -- 172.21.15.112:0/910027401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 msgr2=0x557d1776c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.029+0000 7fd1d842f740 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d1776c830 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc0e57d0 tx=0x7fd1bc0b23b0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d842f740 1 -- 172.21.15.112:0/910027401 shutdown_connections 2024-09-21T03:32:45.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d842f740 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x557d17345da0 0x557d177e7670 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d842f740 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557d173b2070 0x557d177e36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d842f740 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557d17391390 0x557d1776cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d842f740 1 --2- 172.21.15.112:0/910027401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557d173891b0 0x557d1776c830 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d842f740 1 -- 172.21.15.112:0/910027401 >> 172.21.15.112:0/910027401 conn(0x557d173a6460 msgr2=0x557d1775a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1aaffd640 0 client.16185 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:32:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d842f740 1 -- 172.21.15.112:0/910027401 shutdown_connections 2024-09-21T03:32:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d842f740 1 -- 172.21.15.112:0/910027401 wait complete. 2024-09-21T03:32:45.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d842f740 20 client.16185 trim_cache size 0 max 16384 2024-09-21T03:32:45.036 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:32:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/3610143581 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd1b40b6680 msgr2=0x7fd1b40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d1ebb640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd1b40b6680 0x7fd1b40b8b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4006010 tx=0x7fd1c4031400 comp rx=0 tx=0).stop 2024-09-21T03:32:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/3610143581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc104850 msgr2=0x7fd1cc19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d1ebb640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc104850 0x7fd1cc19a470 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00e9c0 tx=0x7fd1bc00ee80 comp rx=0 tx=0).stop 2024-09-21T03:32:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/3610143581 shutdown_connections 2024-09-21T03:32:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d1ebb640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd1b40b6680 0x7fd1b40b8b90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d1ebb640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1cc10de00 0x7fd1cc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d1ebb640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1cc105280 0x7fd1cc19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.030+0000 7fd1d1ebb640 1 --2- 172.21.15.112:0/3610143581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1cc104850 0x7fd1cc19a470 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:45.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.031+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/3610143581 >> 172.21.15.112:0/3610143581 conn(0x7fd1cc079730 msgr2=0x7fd1cc100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:45.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.031+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/3610143581 shutdown_connections 2024-09-21T03:32:45.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:45.031+0000 7fd1d1ebb640 1 -- 172.21.15.112:0/3610143581 wait complete. 2024-09-21T03:32:45.219 INFO:tasks.workunit.client.0.smithi112.stdout: 1 344386 11.47 MB/sec execute 45 sec latency 51.937 ms 2024-09-21T03:32:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:45 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:32:45.884 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:32:45.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:45 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:32:45.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:45 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:32:46.198 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:46.219 INFO:tasks.workunit.client.0.smithi112.stdout: 1 346435 11.37 MB/sec execute 46 sec latency 13.683 ms 2024-09-21T03:32:46.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:46 smithi112 ceph-mon[31442]: pgmap v431: 97 pgs: 97 active+clean; 203 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 297 op/s 2024-09-21T03:32:46.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.804+0000 7fb25728b640 1 -- 172.21.15.112:0/4092365268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb250108cf0 msgr2=0x7fb25010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.804+0000 7fb25728b640 1 --2- 172.21.15.112:0/4092365268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb250108cf0 0x7fb25010f520 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb248009ca0 tx=0x7fb24802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.804+0000 7fb25728b640 1 -- 172.21.15.112:0/4092365268 shutdown_connections 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.804+0000 7fb25728b640 1 --2- 172.21.15.112:0/4092365268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb250108cf0 0x7fb25010f520 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.804+0000 7fb25728b640 1 --2- 172.21.15.112:0/4092365268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2501083a0 0x7fb2501087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.804+0000 7fb25728b640 1 --2- 172.21.15.112:0/4092365268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb250102a60 0x7fb250102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.804+0000 7fb25728b640 1 -- 172.21.15.112:0/4092365268 >> 172.21.15.112:0/4092365268 conn(0x7fb250079730 msgr2=0x7fb250079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.805+0000 7fb25728b640 1 -- 172.21.15.112:0/4092365268 shutdown_connections 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.805+0000 7fb25728b640 1 -- 172.21.15.112:0/4092365268 wait complete. 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.805+0000 7fb25728b640 1 Processor -- start 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.805+0000 7fb25728b640 1 -- start start 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb25728b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb250102a60 0x7fb25019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb25728b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2501083a0 0x7fb25019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb25728b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb250108cf0 0x7fb25019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb25728b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb250112960 con 0x7fb250102a60 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb25728b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501127e0 con 0x7fb250108cf0 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb25728b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb250112ae0 con 0x7fb2501083a0 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb255027640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb250102a60 0x7fb25019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb255027640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb250102a60 0x7fb25019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53986/0 (socket says 172.21.15.112:53986) 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb255027640 1 -- 172.21.15.112:0/973737607 learned_addr learned my addr 172.21.15.112:0/973737607 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb254826640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2501083a0 0x7fb25019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb255828640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb250108cf0 0x7fb25019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb255027640 1 -- 172.21.15.112:0/973737607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2501083a0 msgr2=0x7fb25019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb255027640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2501083a0 0x7fb25019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb255027640 1 -- 172.21.15.112:0/973737607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb250108cf0 msgr2=0x7fb25019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb255027640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb250108cf0 0x7fb25019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb255027640 1 -- 172.21.15.112:0/973737607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2501a03d0 con 0x7fb250102a60 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.806+0000 7fb255027640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb250102a60 0x7fb25019a460 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb24400e9f0 tx=0x7fb24400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.807+0000 7fb2427fc640 1 -- 172.21.15.112:0/973737607 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24400cd80 con 0x7fb250102a60 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.807+0000 7fb2427fc640 1 -- 172.21.15.112:0/973737607 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2440108e0 con 0x7fb250102a60 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.807+0000 7fb25728b640 1 -- 172.21.15.112:0/973737607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2501a06c0 con 0x7fb250102a60 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.807+0000 7fb2427fc640 1 -- 172.21.15.112:0/973737607 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244018980 con 0x7fb250102a60 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.807+0000 7fb25728b640 1 -- 172.21.15.112:0/973737607 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2501a6e20 con 0x7fb250102a60 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.809+0000 7fb25d7ff740 1 Processor -- start 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.809+0000 7fb25d7ff740 1 -- start start 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb2427fc640 1 -- 172.21.15.112:0/973737607 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb244018b20 con 0x7fb250102a60 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 0x55cd71b5a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b5abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b5b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cd71ad0ea0 con 0x55cd717b3070 2024-09-21T03:32:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cd71ac02d0 con 0x55cd7178a1b0 2024-09-21T03:32:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cd71ac40e0 con 0x55cd71792390 2024-09-21T03:32:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb255828640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 0x55cd71b5a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb255828640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 0x55cd71b5a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53996/0 (socket says 172.21.15.112:53996) 2024-09-21T03:32:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb255828640 1 -- 172.21.15.112:0/1814200205 learned_addr learned my addr 172.21.15.112:0/1814200205 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb255027640 1 --2- 172.21.15.112:0/1814200205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b5abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb254826640 1 --2- 172.21.15.112:0/1814200205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b5b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb255828640 1 -- 172.21.15.112:0/1814200205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 msgr2=0x55cd71b5b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:46.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb255828640 1 --2- 172.21.15.112:0/1814200205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b5b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb255828640 1 -- 172.21.15.112:0/1814200205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 msgr2=0x55cd71b5abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb255828640 1 --2- 172.21.15.112:0/1814200205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b5abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb255828640 1 -- 172.21.15.112:0/1814200205 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd7181e650 con 0x55cd717b3070 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.810+0000 7fb255828640 1 --2- 172.21.15.112:0/1814200205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 0x55cd71b5a6a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb24802f410 tx=0x7fb24802f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ffb7a46ce49a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.811+0000 7fb2427fc640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb23c0b6680 0x7fb23c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.811+0000 7fb23b7fe640 1 -- 172.21.15.112:0/1814200205 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248031cf0 con 0x55cd717b3070 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.811+0000 7fb255027640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb23c0b6680 0x7fb23c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.811+0000 7fb23b7fe640 1 -- 172.21.15.112:0/1814200205 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb248036030 con 0x55cd717b3070 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.811+0000 7fb23b7fe640 1 -- 172.21.15.112:0/1814200205 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2480042a0 con 0x55cd717b3070 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.811+0000 7fb2427fc640 1 -- 172.21.15.112:0/973737607 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fb2440e7a20 con 0x7fb250102a60 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb25d7ff740 1 -- 172.21.15.112:0/1814200205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 msgr2=0x55cd71b5a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb255027640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb23c0b6680 0x7fb23c0b8b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb2440e5280 tx=0x7fb2440eb4c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/1814200205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 0x55cd71b5a6a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb24802f410 tx=0x7fb24802f440 comp rx=0 tx=0).stop 2024-09-21T03:32:46.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb25d7ff740 1 -- 172.21.15.112:0/1814200205 shutdown_connections 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/1814200205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b5b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/1814200205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b5abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/1814200205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 0x55cd71b5a6a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb25d7ff740 1 -- 172.21.15.112:0/1814200205 >> 172.21.15.112:0/1814200205 conn(0x55cd717a7460 msgr2=0x55cd71b58590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb25d7ff740 1 -- 172.21.15.112:0/1814200205 shutdown_connections 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb25d7ff740 1 -- 172.21.15.112:0/1814200205 wait complete. 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb25d7ff740 1 -- start start 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.812+0000 7fb25d7ff740 1 Processor -- start 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd7178a1b0 0x55cd71b6a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b6a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd717b3070 0x55cd71b6ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cd71921700 con 0x55cd7178a1b0 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cd71a5b160 con 0x55cd717b3070 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cd71b6d000 con 0x55cd71792390 2024-09-21T03:32:46.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb25d7ff740 10 client.0 _ll_register_callbacks cb 0x55cd71a749e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb255828640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd7178a1b0 0x55cd71b6a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb255828640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd7178a1b0 0x55cd71b6a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54004/0 (socket says 172.21.15.112:54004) 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb255828640 1 -- 172.21.15.112:0/3585685691 learned_addr learned my addr 172.21.15.112:0/3585685691 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb254826640 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b6a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb255828640 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd717b3070 0x55cd71b6ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb255828640 1 -- 172.21.15.112:0/3585685691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 msgr2=0x55cd71b6a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb255828640 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b6a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb255828640 1 -- 172.21.15.112:0/3585685691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd717b3070 msgr2=0x55cd71b6ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb255828640 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd717b3070 0x55cd71b6ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb255828640 1 -- 172.21.15.112:0/3585685691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd71a88c90 con 0x55cd7178a1b0 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb255828640 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd7178a1b0 0x55cd71b6a0d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb248009a60 tx=0x7fb248003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12611206fc7fbeb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.813+0000 7fb238ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:46.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.814+0000 7fb238ff9640 1 -- 172.21.15.112:0/3585685691 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248059060 con 0x55cd7178a1b0 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.814+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3585685691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb248036030 con 0x55cd7178a1b0 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.814+0000 7fb238ff9640 1 -- 172.21.15.112:0/3585685691 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fb24802f660 con 0x55cd7178a1b0 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.814+0000 7fb238ff9640 1 -- 172.21.15.112:0/3585685691 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb24802f860 con 0x55cd7178a1b0 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.814+0000 7fb25d7ff740 10 client.16197 fetch_fsmap learned FSMap version 28 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.814+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3585685691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd71b78c90 con 0x55cd7178a1b0 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.815+0000 7fb238ff9640 1 -- 172.21.15.112:0/3585685691 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb248036030 con 0x55cd7178a1b0 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.815+0000 7fb238ff9640 1 -- 172.21.15.112:0/3585685691 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fb248043070 con 0x55cd7178a1b0 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.815+0000 7fb25d7ff740 10 client.16197 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.815+0000 7fb25d7ff740 10 client.16197 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.815+0000 7fb25d7ff740 20 client.16197 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.815+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55cd717719f0 0x55cd71b7bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.815+0000 7fb25d7ff740 4 client.16197 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.815+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3585685691 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd71ad2df0 con 0x55cd717719f0 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.815+0000 7fb254826640 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55cd717719f0 0x55cd71b7bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.815+0000 7fb238ff9640 10 client.16197 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.816+0000 7fb254826640 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55cd717719f0 0x55cd71b7bff0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bad0c68a55f5105a server_cookie=f5cb27f4022f36ca in_seq=0 out_seq=0 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.816+0000 7fb238ff9640 1 -- 172.21.15.112:0/3585685691 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cd71ad2df0 con 0x55cd717719f0 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.816+0000 7fb238ff9640 10 client.16197 handle_command_reply: tid=0 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.816+0000 7fb25d7ff740 1 client.16197 shutdown 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.816+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3585685691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd7178a1b0 msgr2=0x55cd71b6a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.816+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd7178a1b0 0x55cd71b6a0d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb248009a60 tx=0x7fb248003cc0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.816+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3585685691 shutdown_connections 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.816+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55cd717719f0 0x55cd71b7bff0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.816+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd717b3070 0x55cd71b6ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.816+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b6a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.816+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/3585685691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd7178a1b0 0x55cd71b6a0d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.817+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3585685691 >> 172.21.15.112:0/3585685691 conn(0x55cd717a7460 msgr2=0x55cd71b56bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.817+0000 7fb238ff9640 0 client.16197 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.817+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3585685691 shutdown_connections 2024-09-21T03:32:46.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.817+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3585685691 wait complete. 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.817+0000 7fb25d7ff740 20 client.16197 trim_cache size 0 max 16384 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb25d7ff740 1 Processor -- start 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb25d7ff740 1 -- start start 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd71792390 0x55cd71b5bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd717b3070 0x55cd71b672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501afbf0 con 0x55cd71792390 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501ad4c0 con 0x55cd7178a1b0 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501ab310 con 0x55cd717b3070 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb254826640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd71792390 0x55cd71b5bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb254826640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd71792390 0x55cd71b5bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54008/0 (socket says 172.21.15.112:54008) 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb254826640 1 -- 172.21.15.112:0/3230020542 learned_addr learned my addr 172.21.15.112:0/3230020542 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb255828640 1 --2- 172.21.15.112:0/3230020542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb255828640 1 --2- 172.21.15.112:0/3230020542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd717b3070 0x55cd71b672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb254826640 1 -- 172.21.15.112:0/3230020542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd717b3070 msgr2=0x55cd71b672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb254826640 1 --2- 172.21.15.112:0/3230020542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd717b3070 0x55cd71b672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb254826640 1 -- 172.21.15.112:0/3230020542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 msgr2=0x55cd71b585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb254826640 1 --2- 172.21.15.112:0/3230020542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb254826640 1 -- 172.21.15.112:0/3230020542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd71b5cbc0 con 0x55cd71792390 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb255828640 1 --2- 172.21.15.112:0/3230020542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.831+0000 7fb254826640 1 --2- 172.21.15.112:0/3230020542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd71792390 0x55cd71b5bde0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009d60 tx=0x7fb24c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26f24d753fd5ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.832+0000 7fb23affd640 1 -- 172.21.15.112:0/3230020542 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c004250 con 0x55cd71792390 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.832+0000 7fb23affd640 1 -- 172.21.15.112:0/3230020542 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb24c0043f0 con 0x55cd71792390 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.832+0000 7fb23affd640 1 -- 172.21.15.112:0/3230020542 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c01f8c0 con 0x55cd71792390 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.833+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3230020542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd71792390 msgr2=0x55cd71b5bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.833+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/3230020542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd71792390 0x55cd71b5bde0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009d60 tx=0x7fb24c005f90 comp rx=0 tx=0).stop 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.833+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3230020542 shutdown_connections 2024-09-21T03:32:46.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.833+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/3230020542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd717b3070 0x55cd71b672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.833+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/3230020542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd71792390 0x55cd71b5bde0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.833+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/3230020542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.833+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3230020542 >> 172.21.15.112:0/3230020542 conn(0x55cd717a7460 msgr2=0x55cd71b5b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.833+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3230020542 shutdown_connections 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.833+0000 7fb25d7ff740 1 -- 172.21.15.112:0/3230020542 wait complete. 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.833+0000 7fb25d7ff740 1 -- start start 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.833+0000 7fb25d7ff740 1 Processor -- start 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b7cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b7d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb25d7ff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 0x55cd71b7da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501ab980 con 0x55cd717b3070 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501acdf0 con 0x55cd7178a1b0 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb25d7ff740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501ac050 con 0x55cd71792390 2024-09-21T03:32:46.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb25d7ff740 10 client.0 _ll_register_callbacks cb 0x7fb2440ec040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb255828640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 0x55cd71b7da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb255828640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b7cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb254826640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b7d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb255828640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b7cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:45498/0 (socket says 172.21.15.112:45498) 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb255828640 1 -- 172.21.15.112:0/97170636 learned_addr learned my addr 172.21.15.112:0/97170636 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb255828640 1 -- 172.21.15.112:0/97170636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 msgr2=0x55cd71b7d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb255828640 1 --2- 172.21.15.112:0/97170636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b7d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb255828640 1 -- 172.21.15.112:0/97170636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 msgr2=0x55cd71b7da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb255828640 1 --2- 172.21.15.112:0/97170636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 0x55cd71b7da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb255828640 1 -- 172.21.15.112:0/97170636 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd71be5770 con 0x55cd7178a1b0 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb255828640 1 --2- 172.21.15.112:0/97170636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b7cfa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb24802ade0 tx=0x7fb24802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1231c8cabc5ef98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.834+0000 7fb238ff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb238ff9640 1 -- 172.21.15.112:0/97170636 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248059070 con 0x55cd7178a1b0 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb238ff9640 1 -- 172.21.15.112:0/97170636 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fb248031e00 con 0x55cd7178a1b0 2024-09-21T03:32:46.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb25d7ff740 1 -- 172.21.15.112:0/97170636 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb24c0043f0 con 0x55cd7178a1b0 2024-09-21T03:32:46.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb238ff9640 1 -- 172.21.15.112:0/97170636 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb248062080 con 0x55cd7178a1b0 2024-09-21T03:32:46.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb25d7ff740 10 client.25766 fetch_fsmap learned FSMap version 28 2024-09-21T03:32:46.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb25d7ff740 1 -- 172.21.15.112:0/97170636 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd71be6960 con 0x55cd7178a1b0 2024-09-21T03:32:46.857 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:32:46.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb238ff9640 1 -- 172.21.15.112:0/97170636 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb24c0043f0 con 0x55cd7178a1b0 2024-09-21T03:32:46.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb238ff9640 1 -- 172.21.15.112:0/97170636 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fb248043040 con 0x55cd7178a1b0 2024-09-21T03:32:46.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb25d7ff740 10 client.25766 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb25d7ff740 10 client.25766 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb25d7ff740 20 client.25766 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/97170636 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55cd71746da0 0x55cd71be8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb25d7ff740 4 client.25766 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb25d7ff740 1 -- 172.21.15.112:0/97170636 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd71ace8a0 con 0x55cd71746da0 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.835+0000 7fb254826640 1 --2- 172.21.15.112:0/97170636 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55cd71746da0 0x55cd71be8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb238ff9640 10 client.25766 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb254826640 1 --2- 172.21.15.112:0/97170636 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55cd71746da0 0x55cd71be8f20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea4c00a89e7c3c server_cookie=752416d3541e940a in_seq=0 out_seq=0 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb238ff9640 1 -- 172.21.15.112:0/97170636 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd71ace8a0 con 0x55cd71746da0 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb238ff9640 10 client.25766 handle_command_reply: tid=0 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb25d7ff740 1 client.25766 shutdown 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb25d7ff740 1 -- 172.21.15.112:0/97170636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 msgr2=0x55cd71b7cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/97170636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b7cfa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb24802ade0 tx=0x7fb24802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb25d7ff740 1 -- 172.21.15.112:0/97170636 shutdown_connections 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/97170636 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55cd71746da0 0x55cd71be8f20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/97170636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cd717b3070 0x55cd71b7da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb238ff9640 0 client.25766 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/97170636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd71792390 0x55cd71b7d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb25d7ff740 1 --2- 172.21.15.112:0/97170636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd7178a1b0 0x55cd71b7cfa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.836+0000 7fb25d7ff740 1 -- 172.21.15.112:0/97170636 >> 172.21.15.112:0/97170636 conn(0x55cd717a7460 msgr2=0x55cd71b5be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25d7ff740 1 -- 172.21.15.112:0/97170636 shutdown_connections 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25d7ff740 1 -- 172.21.15.112:0/97170636 wait complete. 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25d7ff740 20 client.25766 trim_cache size 0 max 16384 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25728b640 1 -- 172.21.15.112:0/973737607 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb23c0b6680 msgr2=0x7fb23c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25728b640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb23c0b6680 0x7fb23c0b8b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb2440e5280 tx=0x7fb2440eb4c0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25728b640 1 -- 172.21.15.112:0/973737607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb250102a60 msgr2=0x7fb25019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25728b640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb250102a60 0x7fb25019a460 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb24400e9f0 tx=0x7fb24400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25728b640 1 -- 172.21.15.112:0/973737607 shutdown_connections 2024-09-21T03:32:46.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25728b640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb23c0b6680 0x7fb23c0b8b90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25728b640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb250108cf0 0x7fb25019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25728b640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2501083a0 0x7fb25019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25728b640 1 --2- 172.21.15.112:0/973737607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb250102a60 0x7fb25019a460 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:46.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25728b640 1 -- 172.21.15.112:0/973737607 >> 172.21.15.112:0/973737607 conn(0x7fb250079730 msgr2=0x7fb2501019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:46.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.837+0000 7fb25728b640 1 -- 172.21.15.112:0/973737607 shutdown_connections 2024-09-21T03:32:46.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:46.838+0000 7fb25728b640 1 -- 172.21.15.112:0/973737607 wait complete. 2024-09-21T03:32:46.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:46 smithi203 ceph-mon[34723]: pgmap v431: 97 pgs: 97 active+clean; 203 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 297 op/s 2024-09-21T03:32:46.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:46 smithi187 ceph-mon[34603]: pgmap v431: 97 pgs: 97 active+clean; 203 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 297 op/s 2024-09-21T03:32:47.219 INFO:tasks.workunit.client.0.smithi112.stdout: 1 348299 11.40 MB/sec execute 47 sec latency 13.955 ms 2024-09-21T03:32:47.619 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:32:47.909 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:48.231 INFO:tasks.workunit.client.0.smithi112.stdout: 1 351233 11.52 MB/sec execute 48 sec latency 19.730 ms 2024-09-21T03:32:48.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.430+0000 7f6409bb4640 1 -- 172.21.15.112:0/247931750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6404108480 msgr2=0x7f640410ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.430+0000 7f6409bb4640 1 --2- 172.21.15.112:0/247931750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6404108480 0x7f640410ecb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00adb0 tx=0x7f63fc02b7a0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.430+0000 7f6409bb4640 1 -- 172.21.15.112:0/247931750 shutdown_connections 2024-09-21T03:32:48.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.430+0000 7f6409bb4640 1 --2- 172.21.15.112:0/247931750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6404108480 0x7f640410ecb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.430+0000 7f6409bb4640 1 --2- 172.21.15.112:0/247931750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6404107b30 0x7f6404107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.430+0000 7f6409bb4640 1 --2- 172.21.15.112:0/247931750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64041021f0 0x7f6404102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.430+0000 7f6409bb4640 1 -- 172.21.15.112:0/247931750 >> 172.21.15.112:0/247931750 conn(0x7f6404077c60 msgr2=0x7f64040780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.430+0000 7f6409bb4640 1 -- 172.21.15.112:0/247931750 shutdown_connections 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.430+0000 7f6409bb4640 1 -- 172.21.15.112:0/247931750 wait complete. 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.430+0000 7f6409bb4640 1 Processor -- start 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6409bb4640 1 -- start start 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6409bb4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64041021f0 0x7f640419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6409bb4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6404107b30 0x7f640419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6409bb4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6404108480 0x7f640419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6409bb4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64041118c0 con 0x7f6404107b30 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6409bb4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6404111740 con 0x7f6404108480 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6409bb4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6404111a40 con 0x7f64041021f0 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6402ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6404107b30 0x7f640419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6402ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6404107b30 0x7f640419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54028/0 (socket says 172.21.15.112:54028) 2024-09-21T03:32:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6402ffd640 1 -- 172.21.15.112:0/2855976710 learned_addr learned my addr 172.21.15.112:0/2855976710 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f64037fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64041021f0 0x7f640419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6403fff640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6404108480 0x7f640419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6402ffd640 1 -- 172.21.15.112:0/2855976710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64041021f0 msgr2=0x7f640419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6402ffd640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64041021f0 0x7f640419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6402ffd640 1 -- 172.21.15.112:0/2855976710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6404108480 msgr2=0x7f640419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6402ffd640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6404108480 0x7f640419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.431+0000 7f6402ffd640 1 -- 172.21.15.112:0/2855976710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64041a0400 con 0x7f6404107b30 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.432+0000 7f6402ffd640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6404107b30 0x7f640419aa30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f63f800e9c0 tx=0x7f63f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.432+0000 7f6400ff9640 1 -- 172.21.15.112:0/2855976710 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f800ccc0 con 0x7f6404107b30 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.432+0000 7f6400ff9640 1 -- 172.21.15.112:0/2855976710 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f80054d0 con 0x7f6404107b30 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.432+0000 7f6409bb4640 1 -- 172.21.15.112:0/2855976710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64041a06f0 con 0x7f6404107b30 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.432+0000 7f6400ff9640 1 -- 172.21.15.112:0/2855976710 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8010430 con 0x7f6404107b30 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.432+0000 7f6409bb4640 1 -- 172.21.15.112:0/2855976710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64041a6e50 con 0x7f6404107b30 2024-09-21T03:32:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.434+0000 7f6410128740 1 Processor -- start 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.434+0000 7f6410128740 1 -- start start 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6400ff9640 1 -- 172.21.15.112:0/2855976710 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f63f80105d0 con 0x7f6404107b30 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6410128740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d448782c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6410128740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d44878800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6410128740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444b0390 0x559d44878d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6410128740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d447eeea0 con 0x559d444d1070 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6410128740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559d447de2d0 con 0x559d444a81b0 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6410128740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559d447e20e0 con 0x559d444b0390 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6403fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d448782c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6403fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d448782c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54036/0 (socket says 172.21.15.112:54036) 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6403fff640 1 -- 172.21.15.112:0/2478424338 learned_addr learned my addr 172.21.15.112:0/2478424338 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f64037fe640 1 --2- 172.21.15.112:0/2478424338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d44878800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6402ffd640 1 --2- 172.21.15.112:0/2478424338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444b0390 0x559d44878d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6403fff640 1 -- 172.21.15.112:0/2478424338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444b0390 msgr2=0x559d44878d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6403fff640 1 --2- 172.21.15.112:0/2478424338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444b0390 0x559d44878d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6403fff640 1 -- 172.21.15.112:0/2478424338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 msgr2=0x559d44878800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6403fff640 1 --2- 172.21.15.112:0/2478424338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d44878800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6403fff640 1 -- 172.21.15.112:0/2478424338 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d4453c650 con 0x559d444d1070 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6403fff640 1 --2- 172.21.15.112:0/2478424338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d448782c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00adb0 tx=0x7f63fc00a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb983f39826076eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.435+0000 7f6400ff9640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63ec0b6680 0x7f63ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.436+0000 7f63e9ffb640 1 -- 172.21.15.112:0/2478424338 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc032b10 con 0x559d444d1070 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.436+0000 7f64037fe640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63ec0b6680 0x7f63ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.436+0000 7f63e9ffb640 1 -- 172.21.15.112:0/2478424338 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63fc003e20 con 0x559d444d1070 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.436+0000 7f63e9ffb640 1 -- 172.21.15.112:0/2478424338 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc004260 con 0x559d444d1070 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.436+0000 7f6400ff9640 1 -- 172.21.15.112:0/2855976710 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f63f80e7a40 con 0x7f6404107b30 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f6410128740 1 -- 172.21.15.112:0/2478424338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 msgr2=0x559d448782c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f64037fe640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63ec0b6680 0x7f63ec0b8b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009da0 tx=0x7f63f4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f6410128740 1 --2- 172.21.15.112:0/2478424338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d448782c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00adb0 tx=0x7f63fc00a4b0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f6410128740 1 -- 172.21.15.112:0/2478424338 shutdown_connections 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f6410128740 1 --2- 172.21.15.112:0/2478424338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444b0390 0x559d44878d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f6410128740 1 --2- 172.21.15.112:0/2478424338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d44878800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f6410128740 1 --2- 172.21.15.112:0/2478424338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d448782c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f6410128740 1 -- 172.21.15.112:0/2478424338 >> 172.21.15.112:0/2478424338 conn(0x559d444c5460 msgr2=0x559d448761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f6410128740 1 -- 172.21.15.112:0/2478424338 shutdown_connections 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f6410128740 1 -- 172.21.15.112:0/2478424338 wait complete. 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f6410128740 1 -- start start 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.437+0000 7f6410128740 1 Processor -- start 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6410128740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444a81b0 0x559d44887d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6410128740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d44888260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6410128740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444d1070 0x559d448940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6410128740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d4463f700 con 0x559d444b0390 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6410128740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559d44779160 con 0x559d444d1070 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6410128740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559d4488ad70 con 0x559d444a81b0 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6410128740 10 client.0 _ll_register_callbacks cb 0x559d447929e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6402ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d44888260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6402ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d44888260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54044/0 (socket says 172.21.15.112:54044) 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6402ffd640 1 -- 172.21.15.112:0/2493796911 learned_addr learned my addr 172.21.15.112:0/2493796911 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f64037fe640 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444a81b0 0x559d44887d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6403fff640 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444d1070 0x559d448940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6403fff640 1 -- 172.21.15.112:0/2493796911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444a81b0 msgr2=0x559d44887d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6403fff640 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444a81b0 0x559d44887d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6403fff640 1 -- 172.21.15.112:0/2493796911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 msgr2=0x559d44888260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6403fff640 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d44888260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6403fff640 1 -- 172.21.15.112:0/2493796911 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d447a6c90 con 0x559d444d1070 2024-09-21T03:32:48.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6402ffd640 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d44888260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f64037fe640 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444a81b0 0x559d44887d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f6403fff640 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444d1070 0x559d448940a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63fc006010 tx=0x7f63fc00a670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7596fb5a407ea68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.438+0000 7f63e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f63e2ffd640 1 -- 172.21.15.112:0/2493796911 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc055060 con 0x559d444d1070 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f63e2ffd640 1 -- 172.21.15.112:0/2493796911 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f63fc003b80 con 0x559d444d1070 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f6410128740 1 -- 172.21.15.112:0/2493796911 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63fc003e20 con 0x559d444d1070 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f63e2ffd640 1 -- 172.21.15.112:0/2493796911 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63fc032b10 con 0x559d444d1070 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f6410128740 10 client.25778 fetch_fsmap learned FSMap version 28 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f6410128740 1 -- 172.21.15.112:0/2493796911 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d44896a60 con 0x559d444d1070 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f63e2ffd640 1 -- 172.21.15.112:0/2493796911 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63fc003e20 con 0x559d444d1070 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f63e2ffd640 1 -- 172.21.15.112:0/2493796911 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f63fc031610 con 0x559d444d1070 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f6410128740 10 client.25778 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:32:48.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f6410128740 10 client.25778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f6410128740 20 client.25778 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f6410128740 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559d4448f9f0 0x559d44899dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f6410128740 4 client.25778 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.439+0000 7f6410128740 1 -- 172.21.15.112:0/2493796911 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d447f0df0 con 0x559d4448f9f0 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.440+0000 7f64037fe640 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559d4448f9f0 0x559d44899dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.440+0000 7f63e2ffd640 10 client.25778 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.440+0000 7f64037fe640 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559d4448f9f0 0x559d44899dc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afae600d83916dcc server_cookie=e1fa9f76120f340b in_seq=0 out_seq=0 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f63e2ffd640 1 -- 172.21.15.112:0/2493796911 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d447f0df0 con 0x559d4448f9f0 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f63e2ffd640 10 client.25778 handle_command_reply: tid=0 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f6410128740 1 client.25778 shutdown 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f6410128740 1 -- 172.21.15.112:0/2493796911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444d1070 msgr2=0x559d448940a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f6410128740 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444d1070 0x559d448940a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63fc006010 tx=0x7f63fc00a670 comp rx=0 tx=0).stop 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f6410128740 1 -- 172.21.15.112:0/2493796911 shutdown_connections 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f6410128740 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559d4448f9f0 0x559d44899dc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f6410128740 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444d1070 0x559d448940a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f6410128740 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d44888260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f63e2ffd640 0 client.25778 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:48.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f6410128740 1 --2- 172.21.15.112:0/2493796911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444a81b0 0x559d44887d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f6410128740 1 -- 172.21.15.112:0/2493796911 >> 172.21.15.112:0/2493796911 conn(0x559d444c5460 msgr2=0x559d4487b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f6410128740 1 -- 172.21.15.112:0/2493796911 shutdown_connections 2024-09-21T03:32:48.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.441+0000 7f6410128740 1 -- 172.21.15.112:0/2493796911 wait complete. 2024-09-21T03:32:48.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.442+0000 7f6410128740 20 client.25778 trim_cache size 0 max 16384 2024-09-21T03:32:48.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6410128740 1 Processor -- start 2024-09-21T03:32:48.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6410128740 1 -- start start 2024-09-21T03:32:48.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6410128740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d448881e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6410128740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444b0390 0x559d44875760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6410128740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d44875ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6410128740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64041afc30 con 0x559d444d1070 2024-09-21T03:32:48.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6410128740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f64041ad500 con 0x559d444a81b0 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6410128740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f64041ab350 con 0x559d444b0390 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6403fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d44875ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f64037fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d448881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6403fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d44875ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54054/0 (socket says 172.21.15.112:54054) 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6403fff640 1 -- 172.21.15.112:0/1597443967 learned_addr learned my addr 172.21.15.112:0/1597443967 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.456+0000 7f6402ffd640 1 --2- 172.21.15.112:0/1597443967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444b0390 0x559d44875760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.457+0000 7f64037fe640 1 -- 172.21.15.112:0/1597443967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444b0390 msgr2=0x559d44875760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.457+0000 7f64037fe640 1 --2- 172.21.15.112:0/1597443967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444b0390 0x559d44875760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.457+0000 7f64037fe640 1 -- 172.21.15.112:0/1597443967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 msgr2=0x559d44875ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.457+0000 7f64037fe640 1 --2- 172.21.15.112:0/1597443967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d44875ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.457+0000 7f64037fe640 1 -- 172.21.15.112:0/1597443967 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d44888720 con 0x559d444a81b0 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.457+0000 7f6403fff640 1 --2- 172.21.15.112:0/1597443967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d44875ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.457+0000 7f64037fe640 1 --2- 172.21.15.112:0/1597443967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d448881e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x559d447de400 tx=0x7f63f401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eea8223506219b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.457+0000 7f63e97fa640 1 -- 172.21.15.112:0/1597443967 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4003e10 con 0x559d444a81b0 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.457+0000 7f63e97fa640 1 -- 172.21.15.112:0/1597443967 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f40054d0 con 0x559d444a81b0 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.457+0000 7f63e97fa640 1 -- 172.21.15.112:0/1597443967 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4004a00 con 0x559d444a81b0 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.458+0000 7f6410128740 1 -- 172.21.15.112:0/1597443967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 msgr2=0x559d448881e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.458+0000 7f6410128740 1 --2- 172.21.15.112:0/1597443967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d448881e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x559d447de400 tx=0x7f63f401b000 comp rx=0 tx=0).stop 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.458+0000 7f6410128740 1 -- 172.21.15.112:0/1597443967 shutdown_connections 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.458+0000 7f6410128740 1 --2- 172.21.15.112:0/1597443967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444d1070 0x559d44875ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.458+0000 7f6410128740 1 --2- 172.21.15.112:0/1597443967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444b0390 0x559d44875760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.458+0000 7f6410128740 1 --2- 172.21.15.112:0/1597443967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d448881e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.458+0000 7f6410128740 1 -- 172.21.15.112:0/1597443967 >> 172.21.15.112:0/1597443967 conn(0x559d444c5460 msgr2=0x559d44887ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.458+0000 7f6410128740 1 -- 172.21.15.112:0/1597443967 shutdown_connections 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.458+0000 7f6410128740 1 -- 172.21.15.112:0/1597443967 wait complete. 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.458+0000 7f6410128740 1 -- start start 2024-09-21T03:32:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6410128740 1 Processor -- start 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6410128740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d4489d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6410128740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d4489a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6410128740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444d1070 0x559d4489af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6410128740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f64041ab9c0 con 0x559d444b0390 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6410128740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f64041ace30 con 0x559d444a81b0 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6410128740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f64041ac090 con 0x559d444d1070 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6410128740 10 client.0 _ll_register_callbacks cb 0x7f63f40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6402ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d4489a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f64037fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d4489d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6402ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d4489a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54068/0 (socket says 172.21.15.112:54068) 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6402ffd640 1 -- 172.21.15.112:0/3951187589 learned_addr learned my addr 172.21.15.112:0/3951187589 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f64037fe640 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d4489d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:45540/0 (socket says 172.21.15.112:45540) 2024-09-21T03:32:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6403fff640 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444d1070 0x559d4489af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6402ffd640 1 -- 172.21.15.112:0/3951187589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444d1070 msgr2=0x559d4489af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6402ffd640 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444d1070 0x559d4489af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6402ffd640 1 -- 172.21.15.112:0/3951187589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 msgr2=0x559d4489d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6402ffd640 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d4489d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f6402ffd640 1 -- 172.21.15.112:0/3951187589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d4489b620 con 0x559d444b0390 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.459+0000 7f64037fe640 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d4489d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f6402ffd640 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d4489a9f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f63f80eb330 tx=0x7f63f80b71c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872e884ae333db3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f63e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f63e2ffd640 1 -- 172.21.15.112:0/3951187589 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f80ad6c0 con 0x559d444b0390 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f6410128740 1 -- 172.21.15.112:0/3951187589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f40054d0 con 0x559d444b0390 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f63e2ffd640 1 -- 172.21.15.112:0/3951187589 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f63f80a9240 con 0x559d444b0390 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f63e2ffd640 1 -- 172.21.15.112:0/3951187589 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63f80a9060 con 0x559d444b0390 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f6410128740 10 client.16221 fetch_fsmap learned FSMap version 28 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f6410128740 1 -- 172.21.15.112:0/3951187589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d44904480 con 0x559d444b0390 2024-09-21T03:32:48.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f63e2ffd640 1 -- 172.21.15.112:0/3951187589 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f40054d0 con 0x559d444b0390 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f63e2ffd640 1 -- 172.21.15.112:0/3951187589 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f63f80a9440 con 0x559d444b0390 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f6410128740 10 client.16221 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f6410128740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f6410128740 20 client.16221 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f6410128740 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559d44464da0 0x559d44906b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f6410128740 4 client.16221 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.460+0000 7f6410128740 1 -- 172.21.15.112:0/3951187589 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d447ec8a0 con 0x559d44464da0 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.461+0000 7f64037fe640 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559d44464da0 0x559d44906b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.461+0000 7f63e2ffd640 10 client.16221 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.461+0000 7f64037fe640 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559d44464da0 0x559d44906b60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5794a3dc60d293ac server_cookie=eca0e6be3c99b1c7 in_seq=0 out_seq=0 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.461+0000 7f63e2ffd640 1 -- 172.21.15.112:0/3951187589 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d447ec8a0 con 0x559d44464da0 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.461+0000 7f63e2ffd640 10 client.16221 handle_command_reply: tid=0 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.461+0000 7f6410128740 1 client.16221 shutdown 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6410128740 1 -- 172.21.15.112:0/3951187589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 msgr2=0x559d4489a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6410128740 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d4489a9f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f63f80eb330 tx=0x7f63f80b71c0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6410128740 1 -- 172.21.15.112:0/3951187589 shutdown_connections 2024-09-21T03:32:48.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6410128740 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559d44464da0 0x559d44906b60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6410128740 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559d444d1070 0x559d4489af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6410128740 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d444b0390 0x559d4489a9f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6410128740 1 --2- 172.21.15.112:0/3951187589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d444a81b0 0x559d4489d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6410128740 1 -- 172.21.15.112:0/3951187589 >> 172.21.15.112:0/3951187589 conn(0x559d444c5460 msgr2=0x559d4487b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f63e2ffd640 0 client.16221 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:48.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6410128740 1 -- 172.21.15.112:0/3951187589 shutdown_connections 2024-09-21T03:32:48.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6410128740 1 -- 172.21.15.112:0/3951187589 wait complete. 2024-09-21T03:32:48.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6410128740 20 client.16221 trim_cache size 0 max 16384 2024-09-21T03:32:48.473 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:32:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6409bb4640 1 -- 172.21.15.112:0/2855976710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63ec0b6680 msgr2=0x7f63ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6409bb4640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63ec0b6680 0x7f63ec0b8b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009da0 tx=0x7f63f4005f90 comp rx=0 tx=0).stop 2024-09-21T03:32:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6409bb4640 1 -- 172.21.15.112:0/2855976710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6404107b30 msgr2=0x7f640419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.462+0000 7f6409bb4640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6404107b30 0x7f640419aa30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f63f800e9c0 tx=0x7f63f800ee80 comp rx=0 tx=0).stop 2024-09-21T03:32:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.463+0000 7f6409bb4640 1 -- 172.21.15.112:0/2855976710 shutdown_connections 2024-09-21T03:32:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.463+0000 7f6409bb4640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f63ec0b6680 0x7f63ec0b8b90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.463+0000 7f6409bb4640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6404108480 0x7f640419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.463+0000 7f6409bb4640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6404107b30 0x7f640419aa30 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.463+0000 7f6409bb4640 1 --2- 172.21.15.112:0/2855976710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64041021f0 0x7f640419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:48.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.463+0000 7f6409bb4640 1 -- 172.21.15.112:0/2855976710 >> 172.21.15.112:0/2855976710 conn(0x7f6404077c60 msgr2=0x7f6404105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:48.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.463+0000 7f6409bb4640 1 -- 172.21.15.112:0/2855976710 shutdown_connections 2024-09-21T03:32:48.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:48.463+0000 7f6409bb4640 1 -- 172.21.15.112:0/2855976710 wait complete. 2024-09-21T03:32:48.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:48 smithi112 ceph-mon[31442]: pgmap v432: 97 pgs: 97 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 265 op/s 2024-09-21T03:32:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:48 smithi203 ceph-mon[34723]: pgmap v432: 97 pgs: 97 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 265 op/s 2024-09-21T03:32:48.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:48 smithi187 ceph-mon[34603]: pgmap v432: 97 pgs: 97 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 265 op/s 2024-09-21T03:32:49.231 INFO:tasks.workunit.client.0.smithi112.stdout: 1 352584 11.42 MB/sec execute 49 sec latency 59.028 ms 2024-09-21T03:32:50.231 INFO:tasks.workunit.client.0.smithi112.stdout: 1 355000 11.46 MB/sec execute 50 sec latency 51.615 ms 2024-09-21T03:32:50.748 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:32:50.748 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:32:50.771 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:50 smithi112 ceph-mon[31442]: pgmap v433: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-09-21T03:32:50.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:50 smithi203 ceph-mon[34723]: pgmap v433: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-09-21T03:32:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:50 smithi187 ceph-mon[34603]: pgmap v433: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-09-21T03:32:51.011 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:51.231 INFO:tasks.workunit.client.0.smithi112.stdout: 1 356737 11.35 MB/sec execute 51 sec latency 11.916 ms 2024-09-21T03:32:51.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.510+0000 7fec4bee1640 1 -- 172.21.15.112:0/348498447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec44106280 msgr2=0x7fec44106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.510+0000 7fec4bee1640 1 --2- 172.21.15.112:0/348498447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec44106280 0x7fec44106690 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fec34009900 tx=0x7fec3402a410 comp rx=0 tx=0).stop 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.510+0000 7fec4bee1640 1 -- 172.21.15.112:0/348498447 shutdown_connections 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.510+0000 7fec4bee1640 1 --2- 172.21.15.112:0/348498447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec44106bd0 0x7fec44110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.510+0000 7fec4bee1640 1 --2- 172.21.15.112:0/348498447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec44106280 0x7fec44106690 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.510+0000 7fec4bee1640 1 --2- 172.21.15.112:0/348498447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44102030 0x7fec44102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.510+0000 7fec4bee1640 1 -- 172.21.15.112:0/348498447 >> 172.21.15.112:0/348498447 conn(0x7fec44077c60 msgr2=0x7fec440780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.510+0000 7fec4bee1640 1 -- 172.21.15.112:0/348498447 shutdown_connections 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.510+0000 7fec4bee1640 1 -- 172.21.15.112:0/348498447 wait complete. 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.510+0000 7fec4bee1640 1 Processor -- start 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec4bee1640 1 -- start start 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec4bee1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec44102030 0x7fec4419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec4bee1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44106280 0x7fec4419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec4bee1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec44106bd0 0x7fec4419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec4bee1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec44113aa0 con 0x7fec44102030 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec4bee1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec44113920 con 0x7fec44106bd0 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec4bee1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec44113c20 con 0x7fec44106280 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec49c7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec44102030 0x7fec4419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec49c7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec44102030 0x7fec4419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54096/0 (socket says 172.21.15.112:54096) 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec49c7d640 1 -- 172.21.15.112:0/2619958558 learned_addr learned my addr 172.21.15.112:0/2619958558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec4947c640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44106280 0x7fec4419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec4a47e640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec44106bd0 0x7fec4419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec49c7d640 1 -- 172.21.15.112:0/2619958558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44106280 msgr2=0x7fec4419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec49c7d640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44106280 0x7fec4419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec49c7d640 1 -- 172.21.15.112:0/2619958558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec44106bd0 msgr2=0x7fec4419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec49c7d640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec44106bd0 0x7fec4419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec49c7d640 1 -- 172.21.15.112:0/2619958558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec441a0400 con 0x7fec44102030 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.511+0000 7fec4947c640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44106280 0x7fec4419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.512+0000 7fec49c7d640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec44102030 0x7fec4419a490 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e9c0 tx=0x7fec4000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.512+0000 7fec3affd640 1 -- 172.21.15.112:0/2619958558 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4000ccc0 con 0x7fec44102030 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.512+0000 7fec3affd640 1 -- 172.21.15.112:0/2619958558 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec400035f0 con 0x7fec44102030 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.512+0000 7fec4bee1640 1 -- 172.21.15.112:0/2619958558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec441a06f0 con 0x7fec44102030 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.512+0000 7fec3affd640 1 -- 172.21.15.112:0/2619958558 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4000ce60 con 0x7fec44102030 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.512+0000 7fec4bee1640 1 -- 172.21.15.112:0/2619958558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec441a6e50 con 0x7fec44102030 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.515+0000 7fec3affd640 1 -- 172.21.15.112:0/2619958558 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fec40004000 con 0x7fec44102030 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.515+0000 7fec52455740 1 Processor -- start 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.515+0000 7fec52455740 1 -- start start 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.515+0000 7fec52455740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25bec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.515+0000 7fec52455740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bec970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.515+0000 7fec52455740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25beceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.515+0000 7fec52455740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560f25b62e60 con 0x560f25824670 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.515+0000 7fec52455740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560f25b52290 con 0x560f2581c490 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.515+0000 7fec52455740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560f25b560a0 con 0x560f25845350 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec49c7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec4a47e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25bec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec4a47e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25bec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60630/0 (socket says 172.21.15.112:60630) 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec49c7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bec970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:45574/0 (socket says 172.21.15.112:45574) 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec4a47e640 1 -- 172.21.15.112:0/33022758 learned_addr learned my addr 172.21.15.112:0/33022758 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec4947c640 1 --2- 172.21.15.112:0/33022758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25beceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec3affd640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec300b6680 0x7fec300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec49c7d640 1 -- 172.21.15.112:0/33022758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 msgr2=0x560f25bec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec49c7d640 1 --2- 172.21.15.112:0/33022758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25bec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec49c7d640 1 -- 172.21.15.112:0/33022758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 msgr2=0x560f25beceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:51.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec49c7d640 1 --2- 172.21.15.112:0/33022758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25beceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec49c7d640 1 -- 172.21.15.112:0/33022758 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f258b0930 con 0x560f2581c490 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec4a47e640 1 --2- 172.21.15.112:0/33022758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25bec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec4a47e640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec300b6680 0x7fec300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec3affd640 1 -- 172.21.15.112:0/2619958558 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fec400e7c00 con 0x7fec44102030 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec49c7d640 1 --2- 172.21.15.112:0/33022758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bec970 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fec400e5350 tx=0x7fec40022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa10560b24c49477 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec2ffff640 1 -- 172.21.15.112:0/33022758 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec400ad820 con 0x560f2581c490 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec4a47e640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec300b6680 0x7fec300b8b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fec3c003dd0 tx=0x7fec3c004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.516+0000 7fec2ffff640 1 -- 172.21.15.112:0/33022758 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec40087070 con 0x560f2581c490 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.517+0000 7fec2ffff640 1 -- 172.21.15.112:0/33022758 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec400d1110 con 0x560f2581c490 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.517+0000 7fec52455740 1 -- 172.21.15.112:0/33022758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 msgr2=0x560f25bec970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.517+0000 7fec52455740 1 --2- 172.21.15.112:0/33022758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bec970 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fec400e5350 tx=0x7fec40022000 comp rx=0 tx=0).stop 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.517+0000 7fec52455740 1 -- 172.21.15.112:0/33022758 shutdown_connections 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.518+0000 7fec52455740 1 --2- 172.21.15.112:0/33022758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25beceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.518+0000 7fec52455740 1 --2- 172.21.15.112:0/33022758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bec970 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.518+0000 7fec52455740 1 --2- 172.21.15.112:0/33022758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25bec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.518+0000 7fec52455740 1 -- 172.21.15.112:0/33022758 >> 172.21.15.112:0/33022758 conn(0x560f25839740 msgr2=0x560f25bea320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.518+0000 7fec52455740 1 -- 172.21.15.112:0/33022758 shutdown_connections 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.518+0000 7fec52455740 1 -- 172.21.15.112:0/33022758 wait complete. 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.518+0000 7fec52455740 1 -- start start 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.518+0000 7fec52455740 1 Processor -- start 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec52455740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bf7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec52455740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25bf84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec52455740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25c03f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec52455740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560f259b3890 con 0x560f25824670 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec52455740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560f25aecd40 con 0x560f2581c490 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec52455740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560f25bfabb0 con 0x560f25845350 2024-09-21T03:32:51.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec52455740 10 client.0 _ll_register_callbacks cb 0x560f25b50c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec49c7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bf7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec49c7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bf7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:45580/0 (socket says 172.21.15.112:45580) 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec49c7d640 1 -- 172.21.15.112:0/2453082153 learned_addr learned my addr 172.21.15.112:0/2453082153 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec4947c640 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25bf84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec4a47e640 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25c03f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec49c7d640 1 -- 172.21.15.112:0/2453082153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 msgr2=0x560f25c03f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec49c7d640 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25c03f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec49c7d640 1 -- 172.21.15.112:0/2453082153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 msgr2=0x560f25bf84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec49c7d640 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25bf84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec49c7d640 1 -- 172.21.15.112:0/2453082153 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f25c08880 con 0x560f2581c490 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.519+0000 7fec4947c640 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25bf84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec49c7d640 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bf7fa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fec400e5320 tx=0x7fec400ad620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc336c6b575bbb86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:32:51.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2453082153 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec400230c0 con 0x560f2581c490 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2453082153 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fec400042f0 con 0x560f2581c490 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec52455740 1 -- 172.21.15.112:0/2453082153 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec40087070 con 0x560f2581c490 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2453082153 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec4008c190 con 0x560f2581c490 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec52455740 10 client.25796 fetch_fsmap learned FSMap version 28 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec52455740 1 -- 172.21.15.112:0/2453082153 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f25c0b180 con 0x560f2581c490 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2453082153 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec40087070 con 0x560f2581c490 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2453082153 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fec400ee030 con 0x560f2581c490 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec52455740 10 client.25796 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec52455740 10 client.25796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec52455740 20 client.25796 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec52455740 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560f25803cd0 0x560f25c0e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec52455740 4 client.25796 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec52455740 1 -- 172.21.15.112:0/2453082153 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f25b64db0 con 0x560f25803cd0 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.520+0000 7fec4947c640 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560f25803cd0 0x560f25c0e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.521+0000 7fec2d7fa640 10 client.25796 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.521+0000 7fec4947c640 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560f25803cd0 0x560f25c0e4e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7656a2308a0e05b server_cookie=89b6771badb3d940 in_seq=0 out_seq=0 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2453082153 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f25b64db0 con 0x560f25803cd0 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec2d7fa640 10 client.25796 handle_command_reply: tid=0 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 1 client.25796 shutdown 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 1 -- 172.21.15.112:0/2453082153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 msgr2=0x560f25bf7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bf7fa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fec400e5320 tx=0x7fec400ad620 comp rx=0 tx=0).stop 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 1 -- 172.21.15.112:0/2453082153 shutdown_connections 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560f25803cd0 0x560f25c0e4e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25c03f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25bf84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 1 --2- 172.21.15.112:0/2453082153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25bf7fa0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec2d7fa640 0 client.25796 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:51.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 1 -- 172.21.15.112:0/2453082153 >> 172.21.15.112:0/2453082153 conn(0x560f25839740 msgr2=0x560f25bede20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 1 -- 172.21.15.112:0/2453082153 shutdown_connections 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 1 -- 172.21.15.112:0/2453082153 wait complete. 2024-09-21T03:32:51.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.522+0000 7fec52455740 20 client.25796 trim_cache size 0 max 16384 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec52455740 1 Processor -- start 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec52455740 1 -- start start 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec52455740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f2581c490 0x560f25c03ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec52455740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f25824670 0x560f25bea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec52455740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25845350 0x560f25bf6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec52455740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec441afc30 con 0x560f25845350 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec52455740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec441ad500 con 0x560f25824670 2024-09-21T03:32:51.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec52455740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec441ab370 con 0x560f2581c490 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec4a47e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25845350 0x560f25bf6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec49c7d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f2581c490 0x560f25c03ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec4a47e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25845350 0x560f25bf6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54122/0 (socket says 172.21.15.112:54122) 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec4a47e640 1 -- 172.21.15.112:0/69573263 learned_addr learned my addr 172.21.15.112:0/69573263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.541+0000 7fec4947c640 1 --2- 172.21.15.112:0/69573263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f25824670 0x560f25bea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.542+0000 7fec4a47e640 1 -- 172.21.15.112:0/69573263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f2581c490 msgr2=0x560f25c03ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.542+0000 7fec4a47e640 1 --2- 172.21.15.112:0/69573263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f2581c490 0x560f25c03ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.542+0000 7fec4a47e640 1 -- 172.21.15.112:0/69573263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f25824670 msgr2=0x560f25bea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.542+0000 7fec4a47e640 1 --2- 172.21.15.112:0/69573263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f25824670 0x560f25bea170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.542+0000 7fec4a47e640 1 -- 172.21.15.112:0/69573263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f25beb0b0 con 0x560f25845350 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.542+0000 7fec49c7d640 1 --2- 172.21.15.112:0/69573263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f2581c490 0x560f25c03ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.542+0000 7fec4a47e640 1 --2- 172.21.15.112:0/69573263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25845350 0x560f25bf6620 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009e40 tx=0x7fec3c013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d762142f7540c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.542+0000 7fec2f7fe640 1 -- 172.21.15.112:0/69573263 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c01c470 con 0x560f25845350 2024-09-21T03:32:51.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.542+0000 7fec2f7fe640 1 -- 172.21.15.112:0/69573263 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec3c01cac0 con 0x560f25845350 2024-09-21T03:32:51.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.542+0000 7fec2f7fe640 1 -- 172.21.15.112:0/69573263 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c02aa90 con 0x560f25845350 2024-09-21T03:32:51.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.543+0000 7fec52455740 1 -- 172.21.15.112:0/69573263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25845350 msgr2=0x560f25bf6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.543+0000 7fec52455740 1 --2- 172.21.15.112:0/69573263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25845350 0x560f25bf6620 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009e40 tx=0x7fec3c013030 comp rx=0 tx=0).stop 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.543+0000 7fec52455740 1 -- 172.21.15.112:0/69573263 shutdown_connections 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.543+0000 7fec52455740 1 --2- 172.21.15.112:0/69573263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25845350 0x560f25bf6620 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.543+0000 7fec52455740 1 --2- 172.21.15.112:0/69573263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f25824670 0x560f25bea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.543+0000 7fec52455740 1 --2- 172.21.15.112:0/69573263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f2581c490 0x560f25c03ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.543+0000 7fec52455740 1 -- 172.21.15.112:0/69573263 >> 172.21.15.112:0/69573263 conn(0x560f25839740 msgr2=0x560f25bf2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.543+0000 7fec52455740 1 -- 172.21.15.112:0/69573263 shutdown_connections 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.543+0000 7fec52455740 1 -- 172.21.15.112:0/69573263 wait complete. 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.543+0000 7fec52455740 1 -- start start 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec52455740 1 Processor -- start 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec52455740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25c11f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec52455740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25c0f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec52455740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25c0fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec52455740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec441ab9c0 con 0x560f25824670 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec52455740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec441ace30 con 0x560f2581c490 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec52455740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec441ac090 con 0x560f25845350 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec52455740 10 client.0 _ll_register_callbacks cb 0x7fec40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec4947c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25c0f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec4947c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25c0f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54138/0 (socket says 172.21.15.112:54138) 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec49c7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25c11f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec4947c640 1 -- 172.21.15.112:0/2205998668 learned_addr learned my addr 172.21.15.112:0/2205998668 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.544+0000 7fec4a47e640 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25c0fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec4947c640 1 -- 172.21.15.112:0/2205998668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 msgr2=0x560f25c0fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec4947c640 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25c0fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec4947c640 1 -- 172.21.15.112:0/2205998668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 msgr2=0x560f25c11f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:51.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec4947c640 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25c11f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec4947c640 1 -- 172.21.15.112:0/2205998668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f25c77c60 con 0x560f25824670 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec49c7d640 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25c11f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec4947c640 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25c0f420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fec34000c00 tx=0x7fec34004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbffc5b2ba289a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2205998668 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34035040 con 0x560f25824670 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2205998668 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fec34005040 con 0x560f25824670 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec52455740 1 -- 172.21.15.112:0/2205998668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3c01cac0 con 0x560f25824670 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2205998668 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec34005240 con 0x560f25824670 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec52455740 10 client.16239 fetch_fsmap learned FSMap version 28 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec52455740 1 -- 172.21.15.112:0/2205998668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f25c78e50 con 0x560f25824670 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.545+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2205998668 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec3c01cac0 con 0x560f25824670 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.546+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2205998668 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fec34031720 con 0x560f25824670 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.546+0000 7fec52455740 10 client.16239 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.546+0000 7fec52455740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:51.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.546+0000 7fec52455740 20 client.16239 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.546+0000 7fec52455740 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560f257d9080 0x560f25c7b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.546+0000 7fec52455740 4 client.16239 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.546+0000 7fec52455740 1 -- 172.21.15.112:0/2205998668 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560f25b60860 con 0x560f257d9080 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.546+0000 7fec49c7d640 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560f257d9080 0x560f25c7b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.546+0000 7fec2d7fa640 10 client.16239 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:51.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.546+0000 7fec49c7d640 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560f257d9080 0x560f25c7b610 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26de0ad2dbb98383 server_cookie=42eb4371bb85de8c in_seq=0 out_seq=0 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec2d7fa640 1 -- 172.21.15.112:0/2205998668 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560f25b60860 con 0x560f257d9080 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec2d7fa640 10 client.16239 handle_command_reply: tid=0 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 1 client.16239 shutdown 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 1 -- 172.21.15.112:0/2205998668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 msgr2=0x560f25c0f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25c0f420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fec34000c00 tx=0x7fec34004130 comp rx=0 tx=0).stop 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 1 -- 172.21.15.112:0/2205998668 shutdown_connections 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560f257d9080 0x560f25c7b610 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f25845350 0x560f25c0fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560f25824670 0x560f25c0f420 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 1 --2- 172.21.15.112:0/2205998668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f2581c490 0x560f25c11f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec2d7fa640 0 client.16239 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 1 -- 172.21.15.112:0/2205998668 >> 172.21.15.112:0/2205998668 conn(0x560f25839740 msgr2=0x560f25bf1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 1 -- 172.21.15.112:0/2205998668 shutdown_connections 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 1 -- 172.21.15.112:0/2205998668 wait complete. 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.549+0000 7fec52455740 20 client.16239 trim_cache size 0 max 16384 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:32:51.554 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "cd748fd9-1e14-47f0-aa33-872010daf567", 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 -- 172.21.15.112:0/2619958558 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec300b6680 msgr2=0x7fec300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec300b6680 0x7fec300b8b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fec3c003dd0 tx=0x7fec3c004a40 comp rx=0 tx=0).stop 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 -- 172.21.15.112:0/2619958558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec44102030 msgr2=0x7fec4419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec44102030 0x7fec4419a490 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e9c0 tx=0x7fec4000edc0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 -- 172.21.15.112:0/2619958558 shutdown_connections 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec300b6680 0x7fec300b8b90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec44106bd0 0x7fec4419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44106280 0x7fec4419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 --2- 172.21.15.112:0/2619958558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec44102030 0x7fec4419a490 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 -- 172.21.15.112:0/2619958558 >> 172.21.15.112:0/2619958558 conn(0x7fec44077c60 msgr2=0x7fec441074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 -- 172.21.15.112:0/2619958558 shutdown_connections 2024-09-21T03:32:51.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:51.550+0000 7fec4bee1640 1 -- 172.21.15.112:0/2619958558 wait complete. 2024-09-21T03:32:52.231 INFO:tasks.workunit.client.0.smithi112.stdout: 1 358688 11.40 MB/sec execute 52 sec latency 39.774 ms 2024-09-21T03:32:52.271 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:32:52.334 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:52 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:32:52.334 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:52 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:52.334 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:52 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:32:52.335 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:52 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:32:52.335 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:52 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:52.335 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:52 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:52.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:52 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:32:52.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:52 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:52.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:52 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:32:52.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:52 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:32:52.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:52 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:52.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:52 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:52.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:52 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:32:52.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:52 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:52.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:52 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:32:52.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:52 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:32:52.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:52 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:52.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:52 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:52.574 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:32:53.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.073+0000 7f0c87190640 1 -- 172.21.15.112:0/813813744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c80108cf0 msgr2=0x7f0c8010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:53.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.073+0000 7f0c87190640 1 --2- 172.21.15.112:0/813813744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c80108cf0 0x7f0c8010f520 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009ca0 tx=0x7f0c7802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.073+0000 7f0c87190640 1 -- 172.21.15.112:0/813813744 shutdown_connections 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.073+0000 7f0c87190640 1 --2- 172.21.15.112:0/813813744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c80108cf0 0x7f0c8010f520 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.073+0000 7f0c87190640 1 --2- 172.21.15.112:0/813813744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c801083a0 0x7f0c801087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.073+0000 7f0c87190640 1 --2- 172.21.15.112:0/813813744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c80102a60 0x7f0c80102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.073+0000 7f0c87190640 1 -- 172.21.15.112:0/813813744 >> 172.21.15.112:0/813813744 conn(0x7f0c80079720 msgr2=0x7f0c80079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.073+0000 7f0c87190640 1 -- 172.21.15.112:0/813813744 shutdown_connections 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.074+0000 7f0c87190640 1 -- 172.21.15.112:0/813813744 wait complete. 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.074+0000 7f0c87190640 1 Processor -- start 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.074+0000 7f0c87190640 1 -- start start 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c87190640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c80102a60 0x7f0c8019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c87190640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c801083a0 0x7f0c8019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c87190640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c80108cf0 0x7f0c8019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c87190640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80113a80 con 0x7f0c80102a60 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c87190640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80113900 con 0x7f0c801083a0 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c87190640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80113c00 con 0x7f0c80108cf0 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c84f2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c80102a60 0x7f0c8019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c84f2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c80102a60 0x7f0c8019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54154/0 (socket says 172.21.15.112:54154) 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c84f2c640 1 -- 172.21.15.112:0/3095422425 learned_addr learned my addr 172.21.15.112:0/3095422425 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c8572d640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c80108cf0 0x7f0c8019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c801083a0 0x7f0c8019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c84f2c640 1 -- 172.21.15.112:0/3095422425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c80108cf0 msgr2=0x7f0c8019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c84f2c640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c80108cf0 0x7f0c8019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c84f2c640 1 -- 172.21.15.112:0/3095422425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c801083a0 msgr2=0x7f0c8019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c84f2c640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c801083a0 0x7f0c8019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c84f2c640 1 -- 172.21.15.112:0/3095422425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c801a0320 con 0x7f0c80102a60 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c801083a0 0x7f0c8019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:32:53.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.075+0000 7f0c84f2c640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c80102a60 0x7f0c8019a3b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e9f0 tx=0x7f0c7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.076+0000 7f0c7dffb640 1 -- 172.21.15.112:0/3095422425 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7400cd80 con 0x7f0c80102a60 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.076+0000 7f0c87190640 1 -- 172.21.15.112:0/3095422425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c801a0610 con 0x7f0c80102a60 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.076+0000 7f0c7dffb640 1 -- 172.21.15.112:0/3095422425 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c740108e0 con 0x7f0c80102a60 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.076+0000 7f0c7dffb640 1 -- 172.21.15.112:0/3095422425 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74018980 con 0x7f0c80102a60 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.076+0000 7f0c87190640 1 -- 172.21.15.112:0/3095422425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c801a6d70 con 0x7f0c80102a60 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.078+0000 7f0c8d704740 1 Processor -- start 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c8d704740 1 -- start start 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c7dffb640 1 -- 172.21.15.112:0/3095422425 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0c74018b20 con 0x7f0c80102a60 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac7e070 0x56416b025270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac551b0 0x56416b0257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac5d390 0x56416b025cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c8d704740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56416af9be50 con 0x56416ac7e070 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c8d704740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56416af8b280 con 0x56416ac5d390 2024-09-21T03:32:53.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c8d704740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56416af8f090 con 0x56416ac551b0 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c8572d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac7e070 0x56416b025270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c7ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac5d390 0x56416b025cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c8572d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac7e070 0x56416b025270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54164/0 (socket says 172.21.15.112:54164) 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c8572d640 1 -- 172.21.15.112:0/3872419329 learned_addr learned my addr 172.21.15.112:0/3872419329 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c7ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac5d390 0x56416b025cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:45634/0 (socket says 172.21.15.112:45634) 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.079+0000 7f0c84f2c640 1 --2- 172.21.15.112:0/3872419329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac551b0 0x56416b0257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c8572d640 1 -- 172.21.15.112:0/3872419329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac551b0 msgr2=0x56416b0257b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c8572d640 1 --2- 172.21.15.112:0/3872419329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac551b0 0x56416b0257b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c8572d640 1 -- 172.21.15.112:0/3872419329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac5d390 msgr2=0x56416b025cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c8572d640 1 --2- 172.21.15.112:0/3872419329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac5d390 0x56416b025cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c8572d640 1 -- 172.21.15.112:0/3872419329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56416ace9650 con 0x56416ac7e070 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/3872419329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac5d390 0x56416b025cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c7dffb640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c700b6680 0x7f0c700b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c8572d640 1 --2- 172.21.15.112:0/3872419329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac7e070 0x56416b025270 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0c7802f410 tx=0x7f0c7802f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c922f93489a5457b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:53.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c84f2c640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c700b6680 0x7f0c700b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c7dffb640 1 -- 172.21.15.112:0/3095422425 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f0c740e7b30 con 0x7f0c80102a60 2024-09-21T03:32:53.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c6affd640 1 -- 172.21.15.112:0/3872419329 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78031cf0 con 0x56416ac7e070 2024-09-21T03:32:53.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c6affd640 1 -- 172.21.15.112:0/3872419329 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c78036030 con 0x56416ac7e070 2024-09-21T03:32:53.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.080+0000 7f0c6affd640 1 -- 172.21.15.112:0/3872419329 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c780042a0 con 0x56416ac7e070 2024-09-21T03:32:53.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.081+0000 7f0c8d704740 1 -- 172.21.15.112:0/3872419329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac7e070 msgr2=0x56416b025270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:53.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.081+0000 7f0c84f2c640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c700b6680 0x7f0c700b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x56416b023ef0 tx=0x7f0c740eb5e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:53.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.081+0000 7f0c8d704740 1 --2- 172.21.15.112:0/3872419329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac7e070 0x56416b025270 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0c7802f410 tx=0x7f0c7802f440 comp rx=0 tx=0).stop 2024-09-21T03:32:53.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.081+0000 7f0c8d704740 1 -- 172.21.15.112:0/3872419329 shutdown_connections 2024-09-21T03:32:53.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.081+0000 7f0c8d704740 1 --2- 172.21.15.112:0/3872419329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac5d390 0x56416b025cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.081+0000 7f0c8d704740 1 --2- 172.21.15.112:0/3872419329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac551b0 0x56416b0257b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.081+0000 7f0c8d704740 1 --2- 172.21.15.112:0/3872419329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac7e070 0x56416b025270 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.081+0000 7f0c8d704740 1 -- 172.21.15.112:0/3872419329 >> 172.21.15.112:0/3872419329 conn(0x56416ac72460 msgr2=0x56416b023160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:53.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.081+0000 7f0c8d704740 1 -- 172.21.15.112:0/3872419329 shutdown_connections 2024-09-21T03:32:53.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.081+0000 7f0c8d704740 1 -- 172.21.15.112:0/3872419329 wait complete. 2024-09-21T03:32:53.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.082+0000 7f0c8d704740 1 -- start start 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.082+0000 7f0c8d704740 1 Processor -- start 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.082+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac551b0 0x56416b030990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.082+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac5d390 0x56416b030ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.082+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac7e070 0x56416b03cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.082+0000 7f0c8d704740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56416adec700 con 0x56416ac551b0 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.082+0000 7f0c8d704740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56416af26110 con 0x56416ac7e070 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.082+0000 7f0c8d704740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56416b033870 con 0x56416ac5d390 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.082+0000 7f0c8d704740 10 client.0 _ll_register_callbacks cb 0x56416af3f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c8572d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac551b0 0x56416b030990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c8572d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac551b0 0x56416b030990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54176/0 (socket says 172.21.15.112:54176) 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c8572d640 1 -- 172.21.15.112:0/1446618477 learned_addr learned my addr 172.21.15.112:0/1446618477 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c7ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac5d390 0x56416b030ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c8572d640 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac7e070 0x56416b03cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c8572d640 1 -- 172.21.15.112:0/1446618477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac5d390 msgr2=0x56416b030ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:53.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c8572d640 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac5d390 0x56416b030ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c8572d640 1 -- 172.21.15.112:0/1446618477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac7e070 msgr2=0x56416b03cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c8572d640 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac7e070 0x56416b03cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c8572d640 1 -- 172.21.15.112:0/1446618477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56416af53c40 con 0x56416ac551b0 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac5d390 0x56416b030ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c8572d640 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac551b0 0x56416b030990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009620 tx=0x7f0c78003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f117dcd4e7ba97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c5ffff640 1 -- 172.21.15.112:0/1446618477 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78058060 con 0x56416ac551b0 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c5ffff640 1 -- 172.21.15.112:0/1446618477 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f0c78031860 con 0x56416ac551b0 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.083+0000 7f0c8d704740 1 -- 172.21.15.112:0/1446618477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c78036030 con 0x56416ac551b0 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c5ffff640 1 -- 172.21.15.112:0/1446618477 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c7802f660 con 0x56416ac551b0 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c8d704740 10 client.16251 fetch_fsmap learned FSMap version 28 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c8d704740 1 -- 172.21.15.112:0/1446618477 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56416b03f5b0 con 0x56416ac551b0 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c5ffff640 1 -- 172.21.15.112:0/1446618477 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c78036030 con 0x56416ac551b0 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c5ffff640 1 -- 172.21.15.112:0/1446618477 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f0c78043070 con 0x56416ac551b0 2024-09-21T03:32:53.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c8d704740 10 client.16251 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c8d704740 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c8d704740 20 client.16251 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c8d704740 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56416ac3c9f0 0x56416b042910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c8d704740 4 client.16251 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c8d704740 1 -- 172.21.15.112:0/1446618477 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56416af9dda0 con 0x56416ac3c9f0 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.084+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56416ac3c9f0 0x56416b042910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.085+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56416ac3c9f0 0x56416b042910 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d68d45764f7f0ad server_cookie=c4dec03064bf784d in_seq=0 out_seq=0 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.085+0000 7f0c5ffff640 10 client.16251 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.085+0000 7f0c5ffff640 1 -- 172.21.15.112:0/1446618477 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56416af9dda0 con 0x56416ac3c9f0 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.085+0000 7f0c5ffff640 10 client.16251 handle_command_reply: tid=0 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.085+0000 7f0c8d704740 1 client.16251 shutdown 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c8d704740 1 -- 172.21.15.112:0/1446618477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac551b0 msgr2=0x56416b030990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c8d704740 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac551b0 0x56416b030990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009620 tx=0x7f0c78003d70 comp rx=0 tx=0).stop 2024-09-21T03:32:53.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c8d704740 1 -- 172.21.15.112:0/1446618477 shutdown_connections 2024-09-21T03:32:53.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c8d704740 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56416ac3c9f0 0x56416b042910 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c8d704740 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac7e070 0x56416b03cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c8d704740 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac5d390 0x56416b030ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c8d704740 1 --2- 172.21.15.112:0/1446618477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac551b0 0x56416b030990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c8d704740 1 -- 172.21.15.112:0/1446618477 >> 172.21.15.112:0/1446618477 conn(0x56416ac72460 msgr2=0x56416b028420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:53.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c5ffff640 0 client.16251 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:53.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c8d704740 1 -- 172.21.15.112:0/1446618477 shutdown_connections 2024-09-21T03:32:53.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c8d704740 1 -- 172.21.15.112:0/1446618477 wait complete. 2024-09-21T03:32:53.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.086+0000 7f0c8d704740 20 client.16251 trim_cache size 0 max 16384 2024-09-21T03:32:53.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.104+0000 7f0c8d704740 1 Processor -- start 2024-09-21T03:32:53.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.104+0000 7f0c8d704740 1 -- start start 2024-09-21T03:32:53.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac551b0 0x56416b028a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b03ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac7e070 0x56416b02db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c8d704740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801afb10 con 0x56416ac5d390 2024-09-21T03:32:53.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c8d704740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801ad3e0 con 0x56416ac551b0 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c8d704740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801ab230 con 0x56416ac7e070 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c7ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b03ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c7ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b03ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54190/0 (socket says 172.21.15.112:54190) 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c7ffff640 1 -- 172.21.15.112:0/1923420832 learned_addr learned my addr 172.21.15.112:0/1923420832 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c8572d640 1 --2- 172.21.15.112:0/1923420832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac551b0 0x56416b028a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c8572d640 1 --2- 172.21.15.112:0/1923420832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac7e070 0x56416b02db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c7ffff640 1 -- 172.21.15.112:0/1923420832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac7e070 msgr2=0x56416b02db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/1923420832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac7e070 0x56416b02db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c7ffff640 1 -- 172.21.15.112:0/1923420832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac551b0 msgr2=0x56416b028a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/1923420832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac551b0 0x56416b028a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c7ffff640 1 -- 172.21.15.112:0/1923420832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56416b031a30 con 0x56416ac5d390 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/1923420832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b03ca90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0137c0 tx=0x7f0c6c014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4c30014a9a27ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:53.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.105+0000 7f0c6a7fc640 1 -- 172.21.15.112:0/1923420832 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c0035f0 con 0x56416ac5d390 2024-09-21T03:32:53.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.106+0000 7f0c6a7fc640 1 -- 172.21.15.112:0/1923420832 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c6c003c40 con 0x56416ac5d390 2024-09-21T03:32:53.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.106+0000 7f0c6a7fc640 1 -- 172.21.15.112:0/1923420832 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c009d30 con 0x56416ac5d390 2024-09-21T03:32:53.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.106+0000 7f0c8d704740 1 -- 172.21.15.112:0/1923420832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 msgr2=0x56416b03ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:53.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.106+0000 7f0c8d704740 1 --2- 172.21.15.112:0/1923420832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b03ca90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0137c0 tx=0x7f0c6c014f90 comp rx=0 tx=0).stop 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.106+0000 7f0c8d704740 1 -- 172.21.15.112:0/1923420832 shutdown_connections 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.106+0000 7f0c8d704740 1 --2- 172.21.15.112:0/1923420832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac7e070 0x56416b02db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.106+0000 7f0c8d704740 1 --2- 172.21.15.112:0/1923420832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b03ca90 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.106+0000 7f0c8d704740 1 --2- 172.21.15.112:0/1923420832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac551b0 0x56416b028a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.107+0000 7f0c8d704740 1 -- 172.21.15.112:0/1923420832 >> 172.21.15.112:0/1923420832 conn(0x56416ac72460 msgr2=0x56416b028370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.107+0000 7f0c8d704740 1 -- 172.21.15.112:0/1923420832 shutdown_connections 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.107+0000 7f0c8d704740 1 -- 172.21.15.112:0/1923420832 wait complete. 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.107+0000 7f0c8d704740 1 -- start start 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.107+0000 7f0c8d704740 1 Processor -- start 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.107+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac551b0 0x56416b034320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b034900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c8d704740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac7e070 0x56416b0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c8d704740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801ab8a0 con 0x56416ac5d390 2024-09-21T03:32:53.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c8d704740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801acd10 con 0x56416ac551b0 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c8d704740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801abf70 con 0x56416ac7e070 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c8d704740 10 client.0 _ll_register_callbacks cb 0x7f0c6c013810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c7ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b034900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c7ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b034900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54200/0 (socket says 172.21.15.112:54200) 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c7ffff640 1 -- 172.21.15.112:0/457806827 learned_addr learned my addr 172.21.15.112:0/457806827 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c8572d640 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac7e070 0x56416b0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c8572d640 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac551b0 0x56416b034320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c7ffff640 1 -- 172.21.15.112:0/457806827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac7e070 msgr2=0x56416b0a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac7e070 0x56416b0a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c7ffff640 1 -- 172.21.15.112:0/457806827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac551b0 msgr2=0x56416b034320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac551b0 0x56416b034320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c7ffff640 1 -- 172.21.15.112:0/457806827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56416b0a5710 con 0x56416ac5d390 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c7ffff640 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b034900 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0135c0 tx=0x7f0c6c013670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864f600671176ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:32:53.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c5ffff640 1 -- 172.21.15.112:0/457806827 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c009ab0 con 0x56416ac5d390 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c5ffff640 1 -- 172.21.15.112:0/457806827 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f0c6c009c50 con 0x56416ac5d390 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.108+0000 7f0c8d704740 1 -- 172.21.15.112:0/457806827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c6c003c40 con 0x56416ac5d390 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c5ffff640 1 -- 172.21.15.112:0/457806827 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c6c009e50 con 0x56416ac5d390 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c8d704740 10 client.16257 fetch_fsmap learned FSMap version 28 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c8d704740 1 -- 172.21.15.112:0/457806827 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56416b0aca40 con 0x56416ac5d390 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c5ffff640 1 -- 172.21.15.112:0/457806827 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c6c003c40 con 0x56416ac5d390 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c5ffff640 1 -- 172.21.15.112:0/457806827 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f0c6c030030 con 0x56416ac5d390 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c8d704740 10 client.16257 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c8d704740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c8d704740 20 client.16257 populate_metadata read hostname 'smithi112' 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c8d704740 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56416ac11da0 0x56416b0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:32:53.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c8d704740 4 client.16257 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:32:53.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c8d704740 1 -- 172.21.15.112:0/457806827 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56416af99850 con 0x56416ac11da0 2024-09-21T03:32:53.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.109+0000 7f0c8572d640 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56416ac11da0 0x56416b0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:32:53.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.110+0000 7f0c5ffff640 10 client.16257 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:53.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.110+0000 7f0c8572d640 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56416ac11da0 0x56416b0af1d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c08967f1f94b8b server_cookie=a4991960ec406d6b in_seq=0 out_seq=0 2024-09-21T03:32:53.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.110+0000 7f0c5ffff640 1 -- 172.21.15.112:0/457806827 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x56416af99850 con 0x56416ac11da0 2024-09-21T03:32:53.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.110+0000 7f0c5ffff640 10 client.16257 handle_command_reply: tid=0 2024-09-21T03:32:53.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 1 client.16257 shutdown 2024-09-21T03:32:53.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 1 -- 172.21.15.112:0/457806827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 msgr2=0x56416b034900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:53.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b034900 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0135c0 tx=0x7f0c6c013670 comp rx=0 tx=0).stop 2024-09-21T03:32:53.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 1 -- 172.21.15.112:0/457806827 shutdown_connections 2024-09-21T03:32:53.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56416ac11da0 0x56416b0af1d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56416ac7e070 0x56416b0a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c5ffff640 0 client.16257 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:32:53.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56416ac5d390 0x56416b034900 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 1 --2- 172.21.15.112:0/457806827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56416ac551b0 0x56416b034320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 1 -- 172.21.15.112:0/457806827 >> 172.21.15.112:0/457806827 conn(0x56416ac72460 msgr2=0x56416b026280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:53.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 1 -- 172.21.15.112:0/457806827 shutdown_connections 2024-09-21T03:32:53.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 1 -- 172.21.15.112:0/457806827 wait complete. 2024-09-21T03:32:53.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.111+0000 7f0c8d704740 20 client.16257 trim_cache size 0 max 16384 2024-09-21T03:32:53.117 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:32:53.117 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:32:53.117 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:32:53.117 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:32:53.117 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:53.117 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "cd748fd9-1e14-47f0-aa33-872010daf567", 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:32:53.118 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:53.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:53.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:32:53.122 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:32:53.122 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:32:53.122 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:32:53.122 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:32:53.122 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:32:53.122 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:32:53.122 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:32:53.122 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:32:53.122 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:32:53.122 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 -- 172.21.15.112:0/3095422425 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c700b6680 msgr2=0x7f0c700b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c700b6680 0x7f0c700b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x56416b023ef0 tx=0x7f0c740eb5e0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 -- 172.21.15.112:0/3095422425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c80102a60 msgr2=0x7f0c8019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c80102a60 0x7f0c8019a3b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e9f0 tx=0x7f0c7400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 -- 172.21.15.112:0/3095422425 shutdown_connections 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c700b6680 0x7f0c700b8b90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c80108cf0 0x7f0c8019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c801083a0 0x7f0c8019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 --2- 172.21.15.112:0/3095422425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c80102a60 0x7f0c8019a3b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 -- 172.21.15.112:0/3095422425 >> 172.21.15.112:0/3095422425 conn(0x7f0c80079720 msgr2=0x7f0c80105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 -- 172.21.15.112:0/3095422425 shutdown_connections 2024-09-21T03:32:53.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:32:53.112+0000 7f0c87190640 1 -- 172.21.15.112:0/3095422425 wait complete. 2024-09-21T03:32:53.231 INFO:tasks.workunit.client.0.smithi112.stdout: 1 361189 11.42 MB/sec execute 53 sec latency 34.312 ms 2024-09-21T03:32:53.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:53 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:32:53.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:53 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:53.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:53 smithi112 ceph-mon[31442]: pgmap v434: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 341 op/s 2024-09-21T03:32:53.410 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:53 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:32:53.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:53 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:53.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:53 smithi203 ceph-mon[34723]: pgmap v434: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 341 op/s 2024-09-21T03:32:53.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:53 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:32:53.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:53 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:32:53.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:53 smithi187 ceph-mon[34603]: pgmap v434: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 341 op/s 2024-09-21T03:32:53.948 INFO:tasks.cephfs.filesystem:scrub status for tag:cd748fd9-1e14-47f0-aa33-872010daf567 - {'path': '/', 'tag': 'cd748fd9-1e14-47f0-aa33-872010daf567', 'options': 'recursive,force'} 2024-09-21T03:32:54.231 INFO:tasks.workunit.client.0.smithi112.stdout: 1 362735 11.34 MB/sec execute 54 sec latency 15.272 ms 2024-09-21T03:32:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:54 smithi112 ceph-mon[31442]: pgmap v435: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 271 op/s 2024-09-21T03:32:54.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:54 smithi203 ceph-mon[34723]: pgmap v435: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 271 op/s 2024-09-21T03:32:54.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:54 smithi187 ceph-mon[34603]: pgmap v435: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 271 op/s 2024-09-21T03:32:55.231 INFO:tasks.workunit.client.0.smithi112.stdout: 1 365540 11.46 MB/sec execute 55 sec latency 45.403 ms 2024-09-21T03:32:56.231 INFO:tasks.workunit.client.0.smithi112.stdout: 1 366958 11.36 MB/sec execute 56 sec latency 18.801 ms 2024-09-21T03:32:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:56 smithi112 ceph-mon[31442]: pgmap v436: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 325 op/s 2024-09-21T03:32:56.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:56 smithi203 ceph-mon[34723]: pgmap v436: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 325 op/s 2024-09-21T03:32:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:56 smithi187 ceph-mon[34603]: pgmap v436: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 325 op/s 2024-09-21T03:32:57.231 INFO:tasks.workunit.client.0.smithi112.stdout: 1 369285 11.40 MB/sec execute 57 sec latency 10.232 ms 2024-09-21T03:32:58.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 371455 11.34 MB/sec execute 58 sec latency 13.327 ms 2024-09-21T03:32:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:58 smithi112 ceph-mon[31442]: pgmap v437: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-09-21T03:32:58.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:58 smithi203 ceph-mon[34723]: pgmap v437: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-09-21T03:32:58.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:58 smithi187 ceph-mon[34603]: pgmap v437: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-09-21T03:32:59.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 373171 11.35 MB/sec execute 59 sec latency 19.120 ms 2024-09-21T03:32:59.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:59 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:32:59.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:59 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:32:59.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:32:59 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:32:59.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:59 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:32:59.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:59 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:32:59.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:32:59 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:32:59.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:59 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:32:59.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:59 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:32:59.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:32:59 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:33:00.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 375665 11.38 MB/sec execute 60 sec latency 44.282 ms 2024-09-21T03:33:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:00 smithi112 ceph-mon[31442]: pgmap v438: 97 pgs: 97 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-09-21T03:33:00.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:00 smithi203 ceph-mon[34723]: pgmap v438: 97 pgs: 97 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-09-21T03:33:00.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:00 smithi187 ceph-mon[34603]: pgmap v438: 97 pgs: 97 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 315 op/s 2024-09-21T03:33:01.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 376594 11.28 MB/sec execute 61 sec latency 25.715 ms 2024-09-21T03:33:02.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 379352 11.33 MB/sec execute 62 sec latency 15.493 ms 2024-09-21T03:33:02.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:02 smithi112 ceph-mon[31442]: pgmap v439: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-09-21T03:33:02.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:02 smithi203 ceph-mon[34723]: pgmap v439: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-09-21T03:33:02.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:02 smithi187 ceph-mon[34603]: pgmap v439: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-09-21T03:33:03.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 380781 11.26 MB/sec execute 63 sec latency 17.357 ms 2024-09-21T03:33:04.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 383368 11.35 MB/sec execute 64 sec latency 12.813 ms 2024-09-21T03:33:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:04 smithi112 ceph-mon[31442]: pgmap v440: 97 pgs: 97 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 268 op/s 2024-09-21T03:33:04.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:04 smithi203 ceph-mon[34723]: pgmap v440: 97 pgs: 97 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 268 op/s 2024-09-21T03:33:04.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:04 smithi187 ceph-mon[34603]: pgmap v440: 97 pgs: 97 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 268 op/s 2024-09-21T03:33:05.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 384808 11.26 MB/sec execute 65 sec latency 101.599 ms 2024-09-21T03:33:06.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 387155 11.30 MB/sec execute 66 sec latency 19.714 ms 2024-09-21T03:33:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:06 smithi112 ceph-mon[31442]: pgmap v441: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 335 op/s 2024-09-21T03:33:06.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:06 smithi203 ceph-mon[34723]: pgmap v441: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 335 op/s 2024-09-21T03:33:06.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:06 smithi187 ceph-mon[34603]: pgmap v441: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 335 op/s 2024-09-21T03:33:07.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 389844 11.33 MB/sec execute 67 sec latency 14.045 ms 2024-09-21T03:33:08.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 391442 11.28 MB/sec execute 68 sec latency 14.265 ms 2024-09-21T03:33:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:08 smithi112 ceph-mon[31442]: pgmap v442: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-09-21T03:33:08.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:08 smithi203 ceph-mon[34723]: pgmap v442: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-09-21T03:33:08.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:08 smithi187 ceph-mon[34603]: pgmap v442: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 284 op/s 2024-09-21T03:33:09.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 394008 11.36 MB/sec execute 69 sec latency 16.437 ms 2024-09-21T03:33:10.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 395435 11.27 MB/sec execute 70 sec latency 43.729 ms 2024-09-21T03:33:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:10 smithi112 ceph-mon[31442]: pgmap v443: 97 pgs: 97 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 359 op/s 2024-09-21T03:33:10.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:10 smithi203 ceph-mon[34723]: pgmap v443: 97 pgs: 97 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 359 op/s 2024-09-21T03:33:10.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:10 smithi187 ceph-mon[34603]: pgmap v443: 97 pgs: 97 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 359 op/s 2024-09-21T03:33:11.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 397747 11.31 MB/sec execute 71 sec latency 25.664 ms 2024-09-21T03:33:11.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:11 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:11 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:11.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:11 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:33:12.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 399905 11.26 MB/sec execute 72 sec latency 17.192 ms 2024-09-21T03:33:12.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:12 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:12.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:12 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:12.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:12 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:33:12.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:12 smithi112 ceph-mon[31442]: pgmap v444: 97 pgs: 97 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-09-21T03:33:12.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:12 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:12.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:12 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:12.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:12 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:33:12.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:12 smithi203 ceph-mon[34723]: pgmap v444: 97 pgs: 97 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-09-21T03:33:12.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:12 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:33:12.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:12 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:33:12.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:12 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:33:12.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:12 smithi187 ceph-mon[34603]: pgmap v444: 97 pgs: 97 active+clean; 238 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-09-21T03:33:13.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 401639 11.27 MB/sec execute 73 sec latency 13.038 ms 2024-09-21T03:33:14.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 404480 11.35 MB/sec execute 74 sec latency 13.292 ms 2024-09-21T03:33:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:14 smithi112 ceph-mon[31442]: pgmap v445: 97 pgs: 97 active+clean; 239 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 286 op/s 2024-09-21T03:33:14.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:14 smithi203 ceph-mon[34723]: pgmap v445: 97 pgs: 97 active+clean; 239 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 286 op/s 2024-09-21T03:33:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:14 smithi187 ceph-mon[34603]: pgmap v445: 97 pgs: 97 active+clean; 239 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 286 op/s 2024-09-21T03:33:15.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 405658 11.24 MB/sec execute 75 sec latency 70.253 ms 2024-09-21T03:33:16.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 408253 11.32 MB/sec execute 76 sec latency 29.737 ms 2024-09-21T03:33:16.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:16 smithi112 ceph-mon[31442]: pgmap v446: 97 pgs: 97 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 365 op/s 2024-09-21T03:33:16.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:16 smithi203 ceph-mon[34723]: pgmap v446: 97 pgs: 97 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 365 op/s 2024-09-21T03:33:16.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:16 smithi187 ceph-mon[34603]: pgmap v446: 97 pgs: 97 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 365 op/s 2024-09-21T03:33:17.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 410212 11.25 MB/sec execute 77 sec latency 12.608 ms 2024-09-21T03:33:18.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 412111 11.28 MB/sec execute 78 sec latency 13.285 ms 2024-09-21T03:33:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:18 smithi112 ceph-mon[31442]: pgmap v447: 97 pgs: 97 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-09-21T03:33:18.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:18 smithi203 ceph-mon[34723]: pgmap v447: 97 pgs: 97 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-09-21T03:33:18.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:18 smithi187 ceph-mon[34603]: pgmap v447: 97 pgs: 97 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-09-21T03:33:19.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 415045 11.35 MB/sec execute 79 sec latency 13.963 ms 2024-09-21T03:33:20.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 416320 11.26 MB/sec execute 80 sec latency 44.863 ms 2024-09-21T03:33:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:20 smithi112 ceph-mon[31442]: pgmap v448: 97 pgs: 97 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 282 op/s 2024-09-21T03:33:20.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:20 smithi203 ceph-mon[34723]: pgmap v448: 97 pgs: 97 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 282 op/s 2024-09-21T03:33:20.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:20 smithi187 ceph-mon[34603]: pgmap v448: 97 pgs: 97 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 282 op/s 2024-09-21T03:33:21.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 418947 11.33 MB/sec execute 81 sec latency 19.086 ms 2024-09-21T03:33:22.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 421297 11.33 MB/sec execute 82 sec latency 12.213 ms 2024-09-21T03:33:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:22 smithi112 ceph-mon[31442]: pgmap v449: 97 pgs: 97 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 292 op/s 2024-09-21T03:33:22.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:22 smithi203 ceph-mon[34723]: pgmap v449: 97 pgs: 97 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 292 op/s 2024-09-21T03:33:22.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:22 smithi187 ceph-mon[34603]: pgmap v449: 97 pgs: 97 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 292 op/s 2024-09-21T03:33:23.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 422922 11.29 MB/sec execute 83 sec latency 15.863 ms 2024-09-21T03:33:23.950 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:33:24.212 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:24.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 425782 11.36 MB/sec execute 84 sec latency 14.147 ms 2024-09-21T03:33:24.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:24 smithi112 ceph-mon[31442]: pgmap v450: 97 pgs: 97 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-09-21T03:33:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.694+0000 7f5dce923640 1 -- 172.21.15.112:0/3779657629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc8102a60 msgr2=0x7f5dc8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.694+0000 7f5dce923640 1 --2- 172.21.15.112:0/3779657629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc8102a60 0x7f5dc8102e70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009900 tx=0x7f5db802a410 comp rx=0 tx=0).stop 2024-09-21T03:33:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.694+0000 7f5dce923640 1 -- 172.21.15.112:0/3779657629 shutdown_connections 2024-09-21T03:33:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.694+0000 7f5dce923640 1 --2- 172.21.15.112:0/3779657629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc8108cf0 0x7f5dc810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.694+0000 7f5dce923640 1 --2- 172.21.15.112:0/3779657629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc81083a0 0x7f5dc81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.694+0000 7f5dce923640 1 --2- 172.21.15.112:0/3779657629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc8102a60 0x7f5dc8102e70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.694+0000 7f5dce923640 1 -- 172.21.15.112:0/3779657629 >> 172.21.15.112:0/3779657629 conn(0x7f5dc8079730 msgr2=0x7f5dc8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.694+0000 7f5dce923640 1 -- 172.21.15.112:0/3779657629 shutdown_connections 2024-09-21T03:33:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.694+0000 7f5dce923640 1 -- 172.21.15.112:0/3779657629 wait complete. 2024-09-21T03:33:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.694+0000 7f5dce923640 1 Processor -- start 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dce923640 1 -- start start 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dce923640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc8102a60 0x7f5dc8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dce923640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc81083a0 0x7f5dc81998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dce923640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc8108cf0 0x7f5dc819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dce923640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc8112960 con 0x7f5dc81083a0 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dce923640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc81127e0 con 0x7f5dc8102a60 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dce923640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc8112ae0 con 0x7f5dc8108cf0 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dc77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc81083a0 0x7f5dc81998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dc77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc81083a0 0x7f5dc81998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54652/0 (socket says 172.21.15.112:54652) 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dc77fe640 1 -- 172.21.15.112:0/3130548426 learned_addr learned my addr 172.21.15.112:0/3130548426 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc8102a60 0x7f5dc8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dccec0640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc8108cf0 0x7f5dc819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dc77fe640 1 -- 172.21.15.112:0/3130548426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc8108cf0 msgr2=0x7f5dc819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dc77fe640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc8108cf0 0x7f5dc819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dc77fe640 1 -- 172.21.15.112:0/3130548426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc8102a60 msgr2=0x7f5dc8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.695+0000 7f5dc77fe640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc8102a60 0x7f5dc8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.696+0000 7f5dc77fe640 1 -- 172.21.15.112:0/3130548426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc819f280 con 0x7f5dc81083a0 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.696+0000 7f5dc77fe640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc81083a0 0x7f5dc81998b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc005fd0 tx=0x7f5dbc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.696+0000 7f5dc57fa640 1 -- 172.21.15.112:0/3130548426 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc017070 con 0x7f5dc81083a0 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.696+0000 7f5dce923640 1 -- 172.21.15.112:0/3130548426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc819f570 con 0x7f5dc81083a0 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.696+0000 7f5dc57fa640 1 -- 172.21.15.112:0/3130548426 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dbc009c30 con 0x7f5dc81083a0 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.696+0000 7f5dc57fa640 1 -- 172.21.15.112:0/3130548426 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc003d30 con 0x7f5dc81083a0 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.696+0000 7f5dce923640 1 -- 172.21.15.112:0/3130548426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc81a5cd0 con 0x7f5dc81083a0 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dd4e97740 1 Processor -- start 2024-09-21T03:33:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dc57fa640 1 -- 172.21.15.112:0/3130548426 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f5dbc003ed0 con 0x7f5dc81083a0 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dd4e97740 1 -- start start 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d389070 0x55893d732d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3601b0 0x55893d730270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 0x55893d730920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55893d6a6e50 con 0x55893d368390 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55893d696280 con 0x55893d3601b0 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55893d69a090 con 0x55893d389070 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dc77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 0x55893d730920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dccec0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d389070 0x55893d732d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dc77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 0x55893d730920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54656/0 (socket says 172.21.15.112:54656) 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dccec0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d389070 0x55893d732d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60426/0 (socket says 172.21.15.112:60426) 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dc77fe640 1 -- 172.21.15.112:0/3751963827 learned_addr learned my addr 172.21.15.112:0/3751963827 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.699+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/3751963827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3601b0 0x55893d730270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dccec0640 1 -- 172.21.15.112:0/3751963827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3601b0 msgr2=0x55893d730270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dccec0640 1 --2- 172.21.15.112:0/3751963827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3601b0 0x55893d730270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dccec0640 1 -- 172.21.15.112:0/3751963827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 msgr2=0x55893d730920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dccec0640 1 --2- 172.21.15.112:0/3751963827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 0x55893d730920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dccec0640 1 -- 172.21.15.112:0/3751963827 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893d3f4650 con 0x55893d389070 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dc57fa640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5db00b6680 0x7f5db00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5db00b6680 0x7f5db00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dccec0640 1 --2- 172.21.15.112:0/3751963827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d389070 0x55893d732d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009f00 tx=0x7f5dc0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2024abcd0e843487 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dc57fa640 1 -- 172.21.15.112:0/3130548426 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f5dbc0e7320 con 0x7f5dc81083a0 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dae7fc640 1 -- 172.21.15.112:0/3751963827 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0004d20 con 0x55893d389070 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dae7fc640 1 -- 172.21.15.112:0/3751963827 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc0004050 con 0x55893d389070 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.700+0000 7f5dae7fc640 1 -- 172.21.15.112:0/3751963827 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0005280 con 0x55893d389070 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.701+0000 7f5dd4e97740 1 -- 172.21.15.112:0/3751963827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d389070 msgr2=0x55893d732d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.701+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5db00b6680 0x7f5db00b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5db8031440 tx=0x7f5db8005890 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.701+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/3751963827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d389070 0x55893d732d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009f00 tx=0x7f5dc0011810 comp rx=0 tx=0).stop 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.701+0000 7f5dd4e97740 1 -- 172.21.15.112:0/3751963827 shutdown_connections 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.701+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/3751963827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 0x55893d730920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.701+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/3751963827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3601b0 0x55893d730270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.701+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/3751963827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d389070 0x55893d732d40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.701+0000 7f5dd4e97740 1 -- 172.21.15.112:0/3751963827 >> 172.21.15.112:0/3751963827 conn(0x55893d37d460 msgr2=0x55893d72e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.701+0000 7f5dd4e97740 1 -- 172.21.15.112:0/3751963827 shutdown_connections 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.701+0000 7f5dd4e97740 1 -- 172.21.15.112:0/3751963827 wait complete. 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dd4e97740 1 -- start start 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dd4e97740 1 Processor -- start 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3601b0 0x55893d73fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d368390 0x55893d740350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d389070 0x55893d740890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55893d4f7700 con 0x55893d389070 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55893d631110 con 0x55893d3601b0 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55893d742dd0 con 0x55893d368390 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dd4e97740 10 client.0 _ll_register_callbacks cb 0x55893d64a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dccec0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d389070 0x55893d740890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dccec0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d389070 0x55893d740890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54658/0 (socket says 172.21.15.112:54658) 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.702+0000 7f5dc77fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d368390 0x55893d740350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5dccec0640 1 -- 172.21.15.112:0/65208506 learned_addr learned my addr 172.21.15.112:0/65208506 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3601b0 0x55893d73fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5dccec0640 1 -- 172.21.15.112:0/65208506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d368390 msgr2=0x55893d740350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5dccec0640 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d368390 0x55893d740350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5dccec0640 1 -- 172.21.15.112:0/65208506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3601b0 msgr2=0x55893d73fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5dccec0640 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3601b0 0x55893d73fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5dccec0640 1 -- 172.21.15.112:0/65208506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55893d65ec40 con 0x55893d389070 2024-09-21T03:33:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5dccec0640 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d389070 0x55893d740890 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0004970 tx=0x7f5dc0004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e784dbe8e8f38bf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5da77fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5da77fe640 1 -- 172.21.15.112:0/65208506 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0028020 con 0x55893d389070 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5dd4e97740 1 -- 172.21.15.112:0/65208506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc0004050 con 0x55893d389070 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5da77fe640 1 -- 172.21.15.112:0/65208506 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f5dc0004c50 con 0x55893d389070 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5da77fe640 1 -- 172.21.15.112:0/65208506 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5dc0033050 con 0x55893d389070 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5dd4e97740 10 client.16269 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.703+0000 7f5dd4e97740 1 -- 172.21.15.112:0/65208506 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55893d74ea40 con 0x55893d389070 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.704+0000 7f5da77fe640 1 -- 172.21.15.112:0/65208506 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc0004050 con 0x55893d389070 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.704+0000 7f5da77fe640 1 -- 172.21.15.112:0/65208506 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f5dc0023450 con 0x55893d389070 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.704+0000 7f5dd4e97740 10 client.16269 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.704+0000 7f5dd4e97740 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.704+0000 7f5dd4e97740 20 client.16269 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.704+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55893d3479f0 0x55893d751f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.704+0000 7f5dd4e97740 4 client.16269 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:24.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.704+0000 7f5dd4e97740 1 -- 172.21.15.112:0/65208506 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55893d6a8da0 con 0x55893d3479f0 2024-09-21T03:33:24.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.704+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55893d3479f0 0x55893d751f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.705+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55893d3479f0 0x55893d751f10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b174d8bf96b36912 server_cookie=ecd38095aff370c6 in_seq=0 out_seq=0 2024-09-21T03:33:24.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.705+0000 7f5da77fe640 10 client.16269 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.705+0000 7f5da77fe640 1 -- 172.21.15.112:0/65208506 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55893d6a8da0 con 0x55893d3479f0 2024-09-21T03:33:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.705+0000 7f5da77fe640 10 client.16269 handle_command_reply: tid=0 2024-09-21T03:33:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 1 client.16269 shutdown 2024-09-21T03:33:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 1 -- 172.21.15.112:0/65208506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d389070 msgr2=0x55893d740890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d389070 0x55893d740890 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0004970 tx=0x7f5dc0004490 comp rx=0 tx=0).stop 2024-09-21T03:33:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 1 -- 172.21.15.112:0/65208506 shutdown_connections 2024-09-21T03:33:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55893d3479f0 0x55893d751f10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d389070 0x55893d740890 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5da77fe640 0 client.16269 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d368390 0x55893d740350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/65208506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3601b0 0x55893d73fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 1 -- 172.21.15.112:0/65208506 >> 172.21.15.112:0/65208506 conn(0x55893d37d460 msgr2=0x55893d72c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 1 -- 172.21.15.112:0/65208506 shutdown_connections 2024-09-21T03:33:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 1 -- 172.21.15.112:0/65208506 wait complete. 2024-09-21T03:33:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.706+0000 7f5dd4e97740 20 client.16269 trim_cache size 0 max 16384 2024-09-21T03:33:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.724+0000 7f5dd4e97740 1 Processor -- start 2024-09-21T03:33:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.724+0000 7f5dd4e97740 1 -- start start 2024-09-21T03:33:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3479f0 0x55893d7319b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d3601b0 0x55893d730060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 0x55893d7305a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc81afcf0 con 0x55893d368390 2024-09-21T03:33:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc81ad5c0 con 0x55893d3479f0 2024-09-21T03:33:24.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc81ab410 con 0x55893d3601b0 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dccec0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 0x55893d7305a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dccec0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 0x55893d7305a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54672/0 (socket says 172.21.15.112:54672) 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dccec0640 1 -- 172.21.15.112:0/3023571397 learned_addr learned my addr 172.21.15.112:0/3023571397 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dc7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3479f0 0x55893d7319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dc77fe640 1 --2- 172.21.15.112:0/3023571397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d3601b0 0x55893d730060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dccec0640 1 -- 172.21.15.112:0/3023571397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d3601b0 msgr2=0x55893d730060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dccec0640 1 --2- 172.21.15.112:0/3023571397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d3601b0 0x55893d730060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dccec0640 1 -- 172.21.15.112:0/3023571397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3479f0 msgr2=0x55893d7319b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dccec0640 1 --2- 172.21.15.112:0/3023571397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3479f0 0x55893d7319b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dccec0640 1 -- 172.21.15.112:0/3023571397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893d728410 con 0x55893d368390 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/3023571397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3479f0 0x55893d7319b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dccec0640 1 --2- 172.21.15.112:0/3023571397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 0x55893d7305a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00238f0 tx=0x7f5dc0011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960e3cb72931d902 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dadffb640 1 -- 172.21.15.112:0/3023571397 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0028070 con 0x55893d368390 2024-09-21T03:33:24.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.725+0000 7f5dadffb640 1 -- 172.21.15.112:0/3023571397 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc0023ae0 con 0x55893d368390 2024-09-21T03:33:24.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.726+0000 7f5dadffb640 1 -- 172.21.15.112:0/3023571397 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0023450 con 0x55893d368390 2024-09-21T03:33:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.726+0000 7f5dd4e97740 1 -- 172.21.15.112:0/3023571397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 msgr2=0x55893d7305a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.726+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/3023571397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 0x55893d7305a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00238f0 tx=0x7f5dc0011d10 comp rx=0 tx=0).stop 2024-09-21T03:33:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.726+0000 7f5dd4e97740 1 -- 172.21.15.112:0/3023571397 shutdown_connections 2024-09-21T03:33:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.726+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/3023571397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d368390 0x55893d7305a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.726+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/3023571397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d3601b0 0x55893d730060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.726+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/3023571397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d3479f0 0x55893d7319b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.726+0000 7f5dd4e97740 1 -- 172.21.15.112:0/3023571397 >> 172.21.15.112:0/3023571397 conn(0x55893d37d460 msgr2=0x55893d7312a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.726+0000 7f5dd4e97740 1 -- 172.21.15.112:0/3023571397 shutdown_connections 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.727+0000 7f5dd4e97740 1 -- 172.21.15.112:0/3023571397 wait complete. 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.727+0000 7f5dd4e97740 1 -- start start 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.727+0000 7f5dd4e97740 1 Processor -- start 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.727+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d3479f0 0x55893d748790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.727+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d3601b0 0x55893d757130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.727+0000 7f5dd4e97740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d368390 0x55893d757670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.727+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc81aba80 con 0x55893d3479f0 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.727+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc81acef0 con 0x55893d368390 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.727+0000 7f5dd4e97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc81ac150 con 0x55893d3601b0 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.727+0000 7f5dd4e97740 10 client.0 _ll_register_callbacks cb 0x7f5dbc0ead40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d3479f0 0x55893d748790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d3479f0 0x55893d748790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54682/0 (socket says 172.21.15.112:54682) 2024-09-21T03:33:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dc7fff640 1 -- 172.21.15.112:0/1221429642 learned_addr learned my addr 172.21.15.112:0/1221429642 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dc77fe640 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d3601b0 0x55893d757130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dccec0640 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d368390 0x55893d757670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dc7fff640 1 -- 172.21.15.112:0/1221429642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d3601b0 msgr2=0x55893d757130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d3601b0 0x55893d757130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dc7fff640 1 -- 172.21.15.112:0/1221429642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d368390 msgr2=0x55893d757670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d368390 0x55893d757670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dc7fff640 1 -- 172.21.15.112:0/1221429642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55893d757cb0 con 0x55893d3479f0 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dc7fff640 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d3479f0 0x55893d748790 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5db802a930 tx=0x7f5db803e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6fd6af67313449 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5da77fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5da77fe640 1 -- 172.21.15.112:0/1221429642 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8059060 con 0x55893d3479f0 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5dd4e97740 1 -- 172.21.15.112:0/1221429642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc0023ae0 con 0x55893d3479f0 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.728+0000 7f5da77fe640 1 -- 172.21.15.112:0/1221429642 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f5db8005040 con 0x55893d3479f0 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5da77fe640 1 -- 172.21.15.112:0/1221429642 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5db8035040 con 0x55893d3479f0 2024-09-21T03:33:24.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5dd4e97740 10 client.16275 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5dd4e97740 1 -- 172.21.15.112:0/1221429642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55893d7bc6c0 con 0x55893d3479f0 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5da77fe640 1 -- 172.21.15.112:0/1221429642 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc0023ae0 con 0x55893d3479f0 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5da77fe640 1 -- 172.21.15.112:0/1221429642 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f5db80035f0 con 0x55893d3479f0 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5dd4e97740 10 client.16275 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5dd4e97740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5dd4e97740 20 client.16275 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55893d389070 0x55893d7beeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5dd4e97740 4 client.16275 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5dd4e97740 1 -- 172.21.15.112:0/1221429642 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55893d6a4850 con 0x55893d389070 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.729+0000 7f5dc77fe640 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55893d389070 0x55893d7beeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5da77fe640 10 client.16275 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dc77fe640 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55893d389070 0x55893d7beeb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63320dc0c9832923 server_cookie=db6f9e018c21e282 in_seq=0 out_seq=0 2024-09-21T03:33:24.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5da77fe640 1 -- 172.21.15.112:0/1221429642 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x55893d6a4850 con 0x55893d389070 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5da77fe640 10 client.16275 handle_command_reply: tid=0 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dd4e97740 1 client.16275 shutdown 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dd4e97740 1 -- 172.21.15.112:0/1221429642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d3479f0 msgr2=0x55893d748790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d3479f0 0x55893d748790 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5db802a930 tx=0x7f5db803e070 comp rx=0 tx=0).stop 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dd4e97740 1 -- 172.21.15.112:0/1221429642 shutdown_connections 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55893d389070 0x55893d7beeb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5da77fe640 0 client.16275 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55893d368390 0x55893d757670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55893d3601b0 0x55893d757130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dd4e97740 1 --2- 172.21.15.112:0/1221429642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55893d3479f0 0x55893d748790 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dd4e97740 1 -- 172.21.15.112:0/1221429642 >> 172.21.15.112:0/1221429642 conn(0x55893d37d460 msgr2=0x55893d7411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dd4e97740 1 -- 172.21.15.112:0/1221429642 shutdown_connections 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.730+0000 7f5dd4e97740 1 -- 172.21.15.112:0/1221429642 wait complete. 2024-09-21T03:33:24.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dd4e97740 20 client.16275 trim_cache size 0 max 16384 2024-09-21T03:33:24.737 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:33:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:33:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:33:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:33:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:33:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:33:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:33:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:24.742 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:24.742 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:33:24.742 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:33:24.742 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:33:24.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 -- 172.21.15.112:0/3130548426 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5db00b6680 msgr2=0x7f5db00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5db00b6680 0x7f5db00b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5db8031440 tx=0x7f5db8005890 comp rx=0 tx=0).stop 2024-09-21T03:33:24.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 -- 172.21.15.112:0/3130548426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc81083a0 msgr2=0x7f5dc81998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc81083a0 0x7f5dc81998b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc005fd0 tx=0x7f5dbc00dde0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 -- 172.21.15.112:0/3130548426 shutdown_connections 2024-09-21T03:33:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5db00b6680 0x7f5db00b8b90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc8108cf0 0x7f5dc819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dc81083a0 0x7f5dc81998b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 --2- 172.21.15.112:0/3130548426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc8102a60 0x7f5dc8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 -- 172.21.15.112:0/3130548426 >> 172.21.15.112:0/3130548426 conn(0x7f5dc8079730 msgr2=0x7f5dc8101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 -- 172.21.15.112:0/3130548426 shutdown_connections 2024-09-21T03:33:24.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:24.731+0000 7f5dce923640 1 -- 172.21.15.112:0/3130548426 wait complete. 2024-09-21T03:33:24.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:24 smithi203 ceph-mon[34723]: pgmap v450: 97 pgs: 97 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-09-21T03:33:24.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:24 smithi187 ceph-mon[34603]: pgmap v450: 97 pgs: 97 active+clean; 255 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-09-21T03:33:25.232 INFO:tasks.workunit.client.0.smithi112.stdout: 1 426794 11.27 MB/sec execute 85 sec latency 26.583 ms 2024-09-21T03:33:25.601 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:33:25.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:33:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:26.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 429460 11.33 MB/sec execute 86 sec latency 21.845 ms 2024-09-21T03:33:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.506+0000 7f646eb8c640 1 -- 172.21.15.112:0/882320229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6468106bd0 msgr2=0x7f6468110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.506+0000 7f646eb8c640 1 --2- 172.21.15.112:0/882320229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6468106bd0 0x7f6468110660 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6460009ca0 tx=0x7f646002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.506+0000 7f646eb8c640 1 -- 172.21.15.112:0/882320229 shutdown_connections 2024-09-21T03:33:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.506+0000 7f646eb8c640 1 --2- 172.21.15.112:0/882320229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6468106bd0 0x7f6468110660 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.506+0000 7f646eb8c640 1 --2- 172.21.15.112:0/882320229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6468106280 0x7f6468106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.506+0000 7f646eb8c640 1 --2- 172.21.15.112:0/882320229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468102030 0x7f6468102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.506+0000 7f646eb8c640 1 -- 172.21.15.112:0/882320229 >> 172.21.15.112:0/882320229 conn(0x7f6468077c60 msgr2=0x7f64680780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:26.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.506+0000 7f646eb8c640 1 -- 172.21.15.112:0/882320229 shutdown_connections 2024-09-21T03:33:26.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.506+0000 7f646eb8c640 1 -- 172.21.15.112:0/882320229 wait complete. 2024-09-21T03:33:26.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f646eb8c640 1 Processor -- start 2024-09-21T03:33:26.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f646eb8c640 1 -- start start 2024-09-21T03:33:26.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f646eb8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6468102030 0x7f646819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f646eb8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6468106280 0x7f646819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f646eb8c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468106bd0 0x7f646819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f646eb8c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6468113aa0 con 0x7f6468106280 2024-09-21T03:33:26.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f646eb8c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6468113920 con 0x7f6468106bd0 2024-09-21T03:33:26.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f646eb8c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6468113c20 con 0x7f6468102030 2024-09-21T03:33:26.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f6467fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6468106280 0x7f646819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f646c928640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6468102030 0x7f646819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f6467fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6468106280 0x7f646819a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37088/0 (socket says 172.21.15.112:37088) 2024-09-21T03:33:26.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f6467fff640 1 -- 172.21.15.112:0/1338963023 learned_addr learned my addr 172.21.15.112:0/1338963023 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:26.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f646d129640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468106bd0 0x7f646819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f6467fff640 1 -- 172.21.15.112:0/1338963023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6468102030 msgr2=0x7f646819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f6467fff640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6468102030 0x7f646819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f6467fff640 1 -- 172.21.15.112:0/1338963023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468106bd0 msgr2=0x7f646819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f6467fff640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468106bd0 0x7f646819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.507+0000 7f6467fff640 1 -- 172.21.15.112:0/1338963023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64681a0360 con 0x7f6468106280 2024-09-21T03:33:26.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.508+0000 7f6467fff640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6468106280 0x7f646819a990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f645400e9f0 tx=0x7f645400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:26.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.510+0000 7f6465ffb640 1 -- 172.21.15.112:0/1338963023 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645400ccc0 con 0x7f6468106280 2024-09-21T03:33:26.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.510+0000 7f6465ffb640 1 -- 172.21.15.112:0/1338963023 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64540108e0 con 0x7f6468106280 2024-09-21T03:33:26.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.511+0000 7f646eb8c640 1 -- 172.21.15.112:0/1338963023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64681a0650 con 0x7f6468106280 2024-09-21T03:33:26.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.511+0000 7f6465ffb640 1 -- 172.21.15.112:0/1338963023 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454018860 con 0x7f6468106280 2024-09-21T03:33:26.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.511+0000 7f646eb8c640 1 -- 172.21.15.112:0/1338963023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64681a6db0 con 0x7f6468106280 2024-09-21T03:33:26.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.512+0000 7f646eb8c640 1 -- 172.21.15.112:0/1338963023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6468102440 con 0x7f6468106280 2024-09-21T03:33:26.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.514+0000 7f6465ffb640 1 -- 172.21.15.112:0/1338963023 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f6454010430 con 0x7f6468106280 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.514+0000 7f6465ffb640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f64580b6700 0x7f64580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.514+0000 7f646c928640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f64580b6700 0x7f64580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.514+0000 7f6465ffb640 1 -- 172.21.15.112:0/1338963023 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f64540e7db0 con 0x7f6468106280 2024-09-21T03:33:26.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.515+0000 7f646c928640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f64580b6700 0x7f64580b8c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f646806ae30 tx=0x7f645c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:26.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.518+0000 7f6465ffb640 1 -- 172.21.15.112:0/1338963023 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f64540b2740 con 0x7f6468106280 2024-09-21T03:33:26.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.775+0000 7f646eb8c640 1 -- 172.21.15.112:0/1338963023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f646819b520 con 0x7f6468106280 2024-09-21T03:33:26.777 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:26 smithi112 ceph-mon[31442]: pgmap v451: 97 pgs: 97 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 357 op/s 2024-09-21T03:33:26.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.776+0000 7f6465ffb640 1 -- 172.21.15.112:0/1338963023 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11929 (secure 0 0 0) 0x7f646819b520 con 0x7f6468106280 2024-09-21T03:33:26.778 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:33:26.778 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"btime":"2024-09-21T03:32:44:502736+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:32:44.054356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:33:26.778 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 28 2024-09-21T03:33:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.778+0000 7f646eb8c640 1 -- 172.21.15.112:0/1338963023 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f64580b6700 msgr2=0x7f64580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.778+0000 7f646eb8c640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f64580b6700 0x7f64580b8c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f646806ae30 tx=0x7f645c005f90 comp rx=0 tx=0).stop 2024-09-21T03:33:26.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.778+0000 7f646eb8c640 1 -- 172.21.15.112:0/1338963023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6468106280 msgr2=0x7f646819a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.778+0000 7f646eb8c640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6468106280 0x7f646819a990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f645400e9f0 tx=0x7f645400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.778+0000 7f646eb8c640 1 -- 172.21.15.112:0/1338963023 shutdown_connections 2024-09-21T03:33:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.778+0000 7f646eb8c640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f64580b6700 0x7f64580b8c10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.778+0000 7f646eb8c640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468106bd0 0x7f646819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.778+0000 7f646eb8c640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6468106280 0x7f646819a990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.778+0000 7f646eb8c640 1 --2- 172.21.15.112:0/1338963023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6468102030 0x7f646819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.778+0000 7f646eb8c640 1 -- 172.21.15.112:0/1338963023 >> 172.21.15.112:0/1338963023 conn(0x7f6468077c60 msgr2=0x7f6468107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.779+0000 7f646eb8c640 1 -- 172.21.15.112:0/1338963023 shutdown_connections 2024-09-21T03:33:26.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:26.779+0000 7f646eb8c640 1 -- 172.21.15.112:0/1338963023 wait complete. 2024-09-21T03:33:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:26 smithi203 ceph-mon[34723]: pgmap v451: 97 pgs: 97 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 357 op/s 2024-09-21T03:33:26.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:26 smithi187 ceph-mon[34603]: pgmap v451: 97 pgs: 97 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 357 op/s 2024-09-21T03:33:27.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 431201 11.27 MB/sec execute 87 sec latency 25.759 ms 2024-09-21T03:33:27.626 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:33:27.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:27 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1338963023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:27.895 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:27.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:27 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1338963023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:27.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:27 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1338963023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:33:28.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 433222 11.30 MB/sec execute 88 sec latency 38.437 ms 2024-09-21T03:33:28.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.380+0000 7f3d23923640 1 -- 172.21.15.112:0/3175276294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d1c102060 msgr2=0x7f3d1c102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.380+0000 7f3d23923640 1 --2- 172.21.15.112:0/3175276294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d1c102060 0x7f3d1c102470 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009ca0 tx=0x7f3d1802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.380+0000 7f3d23923640 1 -- 172.21.15.112:0/3175276294 shutdown_connections 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.381+0000 7f3d23923640 1 --2- 172.21.15.112:0/3175276294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c106c00 0x7f3d1c110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.381+0000 7f3d23923640 1 --2- 172.21.15.112:0/3175276294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d1c1062b0 0x7f3d1c1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.381+0000 7f3d23923640 1 --2- 172.21.15.112:0/3175276294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d1c102060 0x7f3d1c102470 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.381+0000 7f3d23923640 1 -- 172.21.15.112:0/3175276294 >> 172.21.15.112:0/3175276294 conn(0x7f3d1c077cc0 msgr2=0x7f3d1c078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.381+0000 7f3d23923640 1 -- 172.21.15.112:0/3175276294 shutdown_connections 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.381+0000 7f3d23923640 1 -- 172.21.15.112:0/3175276294 wait complete. 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.381+0000 7f3d23923640 1 Processor -- start 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.381+0000 7f3d23923640 1 -- start start 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d23923640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c102060 0x7f3d1c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d23923640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d1c1062b0 0x7f3d1c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d23923640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d1c106c00 0x7f3d1c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d23923640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c113af0 con 0x7f3d1c106c00 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d23923640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c113970 con 0x7f3d1c1062b0 2024-09-21T03:33:28.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d23923640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c113c70 con 0x7f3d1c102060 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d216bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c102060 0x7f3d1c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d21ec0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d1c106c00 0x7f3d1c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d216bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c102060 0x7f3d1c19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46988/0 (socket says 172.21.15.112:46988) 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d216bf640 1 -- 172.21.15.112:0/1449359063 learned_addr learned my addr 172.21.15.112:0/1449359063 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d20ebe640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d1c1062b0 0x7f3d1c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d21ec0640 1 -- 172.21.15.112:0/1449359063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c102060 msgr2=0x7f3d1c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d21ec0640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c102060 0x7f3d1c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d21ec0640 1 -- 172.21.15.112:0/1449359063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d1c1062b0 msgr2=0x7f3d1c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d21ec0640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d1c1062b0 0x7f3d1c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.382+0000 7f3d21ec0640 1 -- 172.21.15.112:0/1449359063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c1a0510 con 0x7f3d1c106c00 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.383+0000 7f3d21ec0640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d1c106c00 0x7f3d1c19fde0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400e970 tx=0x7f3d1400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.383+0000 7f3d127fc640 1 -- 172.21.15.112:0/1449359063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1400ccc0 con 0x7f3d1c106c00 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.383+0000 7f3d23923640 1 -- 172.21.15.112:0/1449359063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1c1a0800 con 0x7f3d1c106c00 2024-09-21T03:33:28.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.383+0000 7f3d127fc640 1 -- 172.21.15.112:0/1449359063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d14005020 con 0x7f3d1c106c00 2024-09-21T03:33:28.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.383+0000 7f3d127fc640 1 -- 172.21.15.112:0/1449359063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14004a00 con 0x7f3d1c106c00 2024-09-21T03:33:28.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.383+0000 7f3d23923640 1 -- 172.21.15.112:0/1449359063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1c1a6f60 con 0x7f3d1c106c00 2024-09-21T03:33:28.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.386+0000 7f3d29e97740 1 Processor -- start 2024-09-21T03:33:28.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.386+0000 7f3d127fc640 1 -- 172.21.15.112:0/1449359063 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3d14004ba0 con 0x7f3d1c106c00 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.386+0000 7f3d29e97740 1 -- start start 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d5fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 0x55de2d5fc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 0x55de2d5fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d29e97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de2d572ea0 con 0x55de2d234390 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d29e97740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55de2d5622d0 con 0x55de2d255070 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d29e97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55de2d5660e0 con 0x55de2d22c1b0 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d216bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 0x55de2d5fc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d20ebe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d5fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d216bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 0x55de2d5fc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:47002/0 (socket says 172.21.15.112:47002) 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d216bf640 1 -- 172.21.15.112:0/3546381372 learned_addr learned my addr 172.21.15.112:0/3546381372 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:28.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d20ebe640 1 --2- 172.21.15.112:0/3546381372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 0x55de2d5fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d216bf640 1 -- 172.21.15.112:0/3546381372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 msgr2=0x55de2d5fc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d216bf640 1 --2- 172.21.15.112:0/3546381372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d5fc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d216bf640 1 -- 172.21.15.112:0/3546381372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 msgr2=0x55de2d5fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d127fc640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d080b6680 0x7f3d080b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d216bf640 1 --2- 172.21.15.112:0/3546381372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 0x55de2d5fcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d216bf640 1 -- 172.21.15.112:0/3546381372 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2d2c0650 con 0x55de2d22c1b0 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.387+0000 7f3d21ec0640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d080b6680 0x7f3d080b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.388+0000 7f3d216bf640 1 --2- 172.21.15.112:0/3546381372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 0x55de2d5fc800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009450 tx=0x7f3d18009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58fbe745202acc77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.388+0000 7f3d127fc640 1 -- 172.21.15.112:0/1449359063 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3d140e7590 con 0x7f3d1c106c00 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.388+0000 7f3d077fe640 1 -- 172.21.15.112:0/3546381372 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18031ce0 con 0x55de2d22c1b0 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.388+0000 7f3d077fe640 1 -- 172.21.15.112:0/3546381372 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d18036030 con 0x55de2d22c1b0 2024-09-21T03:33:28.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.388+0000 7f3d077fe640 1 -- 172.21.15.112:0/3546381372 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18004040 con 0x55de2d22c1b0 2024-09-21T03:33:28.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.389+0000 7f3d29e97740 1 -- 172.21.15.112:0/3546381372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 msgr2=0x55de2d5fc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.389+0000 7f3d21ec0640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d080b6680 0x7f3d080b8b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d140ad3d0 tx=0x7f3d140a92e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.389+0000 7f3d29e97740 1 --2- 172.21.15.112:0/3546381372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 0x55de2d5fc800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009450 tx=0x7f3d18009480 comp rx=0 tx=0).stop 2024-09-21T03:33:28.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.389+0000 7f3d29e97740 1 -- 172.21.15.112:0/3546381372 shutdown_connections 2024-09-21T03:33:28.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.389+0000 7f3d29e97740 1 --2- 172.21.15.112:0/3546381372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 0x55de2d5fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.389+0000 7f3d29e97740 1 --2- 172.21.15.112:0/3546381372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 0x55de2d5fc800 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.389+0000 7f3d29e97740 1 --2- 172.21.15.112:0/3546381372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d5fc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.389+0000 7f3d29e97740 1 -- 172.21.15.112:0/3546381372 >> 172.21.15.112:0/3546381372 conn(0x55de2d249460 msgr2=0x55de2d5fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.389+0000 7f3d29e97740 1 -- 172.21.15.112:0/3546381372 shutdown_connections 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.389+0000 7f3d29e97740 1 -- 172.21.15.112:0/3546381372 wait complete. 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.389+0000 7f3d29e97740 1 -- start start 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d29e97740 1 Processor -- start 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d22c1b0 0x55de2d607910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 0x55de2d607e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d255070 0x55de2d613c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d29e97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de2d3c3700 con 0x55de2d234390 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d29e97740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55de2d4fd160 con 0x55de2d22c1b0 2024-09-21T03:33:28.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d29e97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55de2d60a8c0 con 0x55de2d255070 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d29e97740 10 client.0 _ll_register_callbacks cb 0x55de2d5169e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d216bf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 0x55de2d607e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d216bf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 0x55de2d607e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37142/0 (socket says 172.21.15.112:37142) 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d20ebe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d255070 0x55de2d613c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d216bf640 1 -- 172.21.15.112:0/272384333 learned_addr learned my addr 172.21.15.112:0/272384333 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.390+0000 7f3d20ebe640 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d22c1b0 0x55de2d607910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d216bf640 1 -- 172.21.15.112:0/272384333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d255070 msgr2=0x55de2d613c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d216bf640 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d255070 0x55de2d613c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d216bf640 1 -- 172.21.15.112:0/272384333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d22c1b0 msgr2=0x55de2d607910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d216bf640 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d22c1b0 0x55de2d607910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d216bf640 1 -- 172.21.15.112:0/272384333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de2d52ac90 con 0x55de2d234390 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d216bf640 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 0x55de2d607e50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3d18004af0 tx=0x7f3d18003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ef57301ad1792 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:33:28.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d04ff9640 1 -- 172.21.15.112:0/272384333 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1804a040 con 0x55de2d234390 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d04ff9640 1 -- 172.21.15.112:0/272384333 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3d18031650 con 0x55de2d234390 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d29e97740 1 -- 172.21.15.112:0/272384333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d18036030 con 0x55de2d234390 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d04ff9640 1 -- 172.21.15.112:0/272384333 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d18031850 con 0x55de2d234390 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d29e97740 10 client.16293 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d29e97740 1 -- 172.21.15.112:0/272384333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de2d6165d0 con 0x55de2d234390 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.391+0000 7f3d04ff9640 1 -- 172.21.15.112:0/272384333 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d18036030 con 0x55de2d234390 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.392+0000 7f3d04ff9640 1 -- 172.21.15.112:0/272384333 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f3d1804d030 con 0x55de2d234390 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.392+0000 7f3d29e97740 10 client.16293 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.392+0000 7f3d29e97740 10 client.16293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.392+0000 7f3d29e97740 20 client.16293 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.392+0000 7f3d29e97740 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55de2d2139f0 0x55de2d619930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.392+0000 7f3d29e97740 4 client.16293 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.392+0000 7f3d29e97740 1 -- 172.21.15.112:0/272384333 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de2d574df0 con 0x55de2d2139f0 2024-09-21T03:33:28.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.392+0000 7f3d20ebe640 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55de2d2139f0 0x55de2d619930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.392+0000 7f3d04ff9640 10 client.16293 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:33:28.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.392+0000 7f3d20ebe640 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55de2d2139f0 0x55de2d619930 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d5c3cfeafe07e7b server_cookie=d481fbdbf177cc10 in_seq=0 out_seq=0 2024-09-21T03:33:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d04ff9640 1 -- 172.21.15.112:0/272384333 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de2d574df0 con 0x55de2d2139f0 2024-09-21T03:33:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d04ff9640 10 client.16293 handle_command_reply: tid=0 2024-09-21T03:33:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d29e97740 1 client.16293 shutdown 2024-09-21T03:33:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d29e97740 1 -- 172.21.15.112:0/272384333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 msgr2=0x55de2d607e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d29e97740 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 0x55de2d607e50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3d18004af0 tx=0x7f3d18003bd0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d29e97740 1 -- 172.21.15.112:0/272384333 shutdown_connections 2024-09-21T03:33:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d29e97740 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55de2d2139f0 0x55de2d619930 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d29e97740 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d255070 0x55de2d613c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d29e97740 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d234390 0x55de2d607e50 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d29e97740 1 --2- 172.21.15.112:0/272384333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d22c1b0 0x55de2d607910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d29e97740 1 -- 172.21.15.112:0/272384333 >> 172.21.15.112:0/272384333 conn(0x55de2d249460 msgr2=0x55de2d5ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d04ff9640 0 client.16293 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:33:28.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d29e97740 1 -- 172.21.15.112:0/272384333 shutdown_connections 2024-09-21T03:33:28.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.393+0000 7f3d29e97740 1 -- 172.21.15.112:0/272384333 wait complete. 2024-09-21T03:33:28.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.394+0000 7f3d29e97740 20 client.16293 trim_cache size 0 max 16384 2024-09-21T03:33:28.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d29e97740 1 Processor -- start 2024-09-21T03:33:28.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d29e97740 1 -- start start 2024-09-21T03:33:28.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 0x55de2d613b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d5f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d1e8da0 0x55de2d5f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d29e97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1b4090 con 0x55de2d1e8da0 2024-09-21T03:33:28.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d29e97740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1b1960 con 0x55de2d255070 2024-09-21T03:33:28.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d29e97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1af7b0 con 0x55de2d22c1b0 2024-09-21T03:33:28.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d20ebe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d1e8da0 0x55de2d5f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d216bf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d5f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d20ebe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d1e8da0 0x55de2d5f9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37150/0 (socket says 172.21.15.112:37150) 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d216bf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d5f9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40538/0 (socket says 172.21.15.112:40538) 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d20ebe640 1 -- 172.21.15.112:0/2402754413 learned_addr learned my addr 172.21.15.112:0/2402754413 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.409+0000 7f3d20ebe640 1 --2- 172.21.15.112:0/2402754413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 0x55de2d613b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.410+0000 7f3d216bf640 1 -- 172.21.15.112:0/2402754413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 msgr2=0x55de2d613b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.410+0000 7f3d216bf640 1 --2- 172.21.15.112:0/2402754413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 0x55de2d613b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.410+0000 7f3d216bf640 1 -- 172.21.15.112:0/2402754413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d1e8da0 msgr2=0x55de2d5f9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.410+0000 7f3d216bf640 1 --2- 172.21.15.112:0/2402754413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d1e8da0 0x55de2d5f9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.410+0000 7f3d216bf640 1 -- 172.21.15.112:0/2402754413 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2d5feb00 con 0x55de2d255070 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.410+0000 7f3d216bf640 1 --2- 172.21.15.112:0/2402754413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d5f9760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3d1804d000 tx=0x7f3d18032fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79e4400308fca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.410+0000 7f3d06ffd640 1 -- 172.21.15.112:0/2402754413 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18031650 con 0x55de2d255070 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.410+0000 7f3d06ffd640 1 -- 172.21.15.112:0/2402754413 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d18036070 con 0x55de2d255070 2024-09-21T03:33:28.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.410+0000 7f3d06ffd640 1 -- 172.21.15.112:0/2402754413 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1802fa60 con 0x55de2d255070 2024-09-21T03:33:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.411+0000 7f3d29e97740 1 -- 172.21.15.112:0/2402754413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 msgr2=0x55de2d5f9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.411+0000 7f3d29e97740 1 --2- 172.21.15.112:0/2402754413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d5f9760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3d1804d000 tx=0x7f3d18032fe0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.411+0000 7f3d29e97740 1 -- 172.21.15.112:0/2402754413 shutdown_connections 2024-09-21T03:33:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.411+0000 7f3d29e97740 1 --2- 172.21.15.112:0/2402754413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d1e8da0 0x55de2d5f9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.411+0000 7f3d29e97740 1 --2- 172.21.15.112:0/2402754413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d5f9760 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.411+0000 7f3d29e97740 1 --2- 172.21.15.112:0/2402754413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d22c1b0 0x55de2d613b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.411+0000 7f3d29e97740 1 -- 172.21.15.112:0/2402754413 >> 172.21.15.112:0/2402754413 conn(0x55de2d249460 msgr2=0x55de2d606190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.411+0000 7f3d29e97740 1 -- 172.21.15.112:0/2402754413 shutdown_connections 2024-09-21T03:33:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.411+0000 7f3d29e97740 1 -- 172.21.15.112:0/2402754413 wait complete. 2024-09-21T03:33:28.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.411+0000 7f3d29e97740 1 -- start start 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.411+0000 7f3d29e97740 1 Processor -- start 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d1e8da0 0x55de2d60be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d22c1b0 0x55de2d60c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d29e97740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d683200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d29e97740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1afe20 con 0x55de2d22c1b0 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d29e97740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1b1290 con 0x55de2d255070 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d29e97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1b04f0 con 0x55de2d1e8da0 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d29e97740 10 client.0 _ll_register_callbacks cb 0x7f3d0c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d216bf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d22c1b0 0x55de2d60c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d20ebe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d1e8da0 0x55de2d60be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d216bf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d22c1b0 0x55de2d60c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37166/0 (socket says 172.21.15.112:37166) 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d216bf640 1 -- 172.21.15.112:0/1839901719 learned_addr learned my addr 172.21.15.112:0/1839901719 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:28.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d20ebe640 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d683200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d216bf640 1 -- 172.21.15.112:0/1839901719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d1e8da0 msgr2=0x55de2d60be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d216bf640 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d1e8da0 0x55de2d60be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d216bf640 1 -- 172.21.15.112:0/1839901719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 msgr2=0x55de2d683200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d216bf640 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d683200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.412+0000 7f3d216bf640 1 -- 172.21.15.112:0/1839901719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de2d60cb30 con 0x55de2d22c1b0 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d216bf640 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d22c1b0 0x55de2d60c340 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3d1804e040 tx=0x7f3d180317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d336efd2aa9d737d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d04ff9640 1 -- 172.21.15.112:0/1839901719 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18030c30 con 0x55de2d22c1b0 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d04ff9640 1 -- 172.21.15.112:0/1839901719 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3d18030dd0 con 0x55de2d22c1b0 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d29e97740 1 -- 172.21.15.112:0/1839901719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d18036070 con 0x55de2d22c1b0 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d04ff9640 1 -- 172.21.15.112:0/1839901719 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d18009360 con 0x55de2d22c1b0 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d29e97740 10 client.16299 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d29e97740 1 -- 172.21.15.112:0/1839901719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de2d60cb30 con 0x55de2d22c1b0 2024-09-21T03:33:28.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d04ff9640 1 -- 172.21.15.112:0/1839901719 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d18036070 con 0x55de2d22c1b0 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d04ff9640 1 -- 172.21.15.112:0/1839901719 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f3d18043070 con 0x55de2d22c1b0 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d29e97740 10 client.16299 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d29e97740 10 client.16299 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d29e97740 20 client.16299 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.413+0000 7f3d29e97740 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55de2d684a60 0x55de2d686e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.414+0000 7f3d29e97740 4 client.16299 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.414+0000 7f3d29e97740 1 -- 172.21.15.112:0/1839901719 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de2d5708a0 con 0x55de2d684a60 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.414+0000 7f3d20ebe640 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55de2d684a60 0x55de2d686e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.414+0000 7f3d04ff9640 10 client.16299 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.414+0000 7f3d20ebe640 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55de2d684a60 0x55de2d686e80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63289137ec0b0e7f server_cookie=c05857784e4f0679 in_seq=0 out_seq=0 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.414+0000 7f3d04ff9640 1 -- 172.21.15.112:0/1839901719 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de2d5708a0 con 0x55de2d684a60 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.414+0000 7f3d04ff9640 10 client.16299 handle_command_reply: tid=0 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.414+0000 7f3d29e97740 1 client.16299 shutdown 2024-09-21T03:33:28.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d29e97740 1 -- 172.21.15.112:0/1839901719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d22c1b0 msgr2=0x55de2d60c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d29e97740 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d22c1b0 0x55de2d60c340 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3d1804e040 tx=0x7f3d180317c0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d29e97740 1 -- 172.21.15.112:0/1839901719 shutdown_connections 2024-09-21T03:33:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d29e97740 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55de2d684a60 0x55de2d686e80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d29e97740 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de2d255070 0x55de2d683200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d29e97740 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de2d22c1b0 0x55de2d60c340 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d29e97740 1 --2- 172.21.15.112:0/1839901719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55de2d1e8da0 0x55de2d60be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d04ff9640 0 client.16299 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:33:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d29e97740 1 -- 172.21.15.112:0/1839901719 >> 172.21.15.112:0/1839901719 conn(0x55de2d249460 msgr2=0x55de2d609730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d29e97740 1 -- 172.21.15.112:0/1839901719 shutdown_connections 2024-09-21T03:33:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d29e97740 1 -- 172.21.15.112:0/1839901719 wait complete. 2024-09-21T03:33:28.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d29e97740 20 client.16299 trim_cache size 0 max 16384 2024-09-21T03:33:28.422 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:33:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d23923640 1 -- 172.21.15.112:0/1449359063 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d080b6680 msgr2=0x7f3d080b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d23923640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d080b6680 0x7f3d080b8b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d140ad3d0 tx=0x7f3d140a92e0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d23923640 1 -- 172.21.15.112:0/1449359063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d1c106c00 msgr2=0x7f3d1c19fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.415+0000 7f3d23923640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d1c106c00 0x7f3d1c19fde0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400e970 tx=0x7f3d1400ee30 comp rx=0 tx=0).stop 2024-09-21T03:33:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.416+0000 7f3d23923640 1 -- 172.21.15.112:0/1449359063 shutdown_connections 2024-09-21T03:33:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.416+0000 7f3d23923640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3d080b6680 0x7f3d080b8b90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.416+0000 7f3d23923640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d1c106c00 0x7f3d1c19fde0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.416+0000 7f3d23923640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d1c1062b0 0x7f3d1c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.416+0000 7f3d23923640 1 --2- 172.21.15.112:0/1449359063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c102060 0x7f3d1c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.416+0000 7f3d23923640 1 -- 172.21.15.112:0/1449359063 >> 172.21.15.112:0/1449359063 conn(0x7f3d1c077cc0 msgr2=0x7f3d1c1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.416+0000 7f3d23923640 1 -- 172.21.15.112:0/1449359063 shutdown_connections 2024-09-21T03:33:28.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:28.416+0000 7f3d23923640 1 -- 172.21.15.112:0/1449359063 wait complete. 2024-09-21T03:33:28.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:28 smithi112 ceph-mon[31442]: pgmap v452: 97 pgs: 97 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 248 op/s 2024-09-21T03:33:28.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:28 smithi203 ceph-mon[34723]: pgmap v452: 97 pgs: 97 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 248 op/s 2024-09-21T03:33:28.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:28 smithi187 ceph-mon[34603]: pgmap v452: 97 pgs: 97 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 248 op/s 2024-09-21T03:33:29.144 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:33:29.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 435642 11.30 MB/sec execute 89 sec latency 14.451 ms 2024-09-21T03:33:29.407 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.934+0000 7efd9008d640 1 -- 172.21.15.112:0/3504860158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd881084b0 msgr2=0x7efd8810ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.934+0000 7efd9008d640 1 --2- 172.21.15.112:0/3504860158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd881084b0 0x7efd8810ece0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efd80009ca0 tx=0x7efd8002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.934+0000 7efd9008d640 1 -- 172.21.15.112:0/3504860158 shutdown_connections 2024-09-21T03:33:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.934+0000 7efd9008d640 1 --2- 172.21.15.112:0/3504860158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd881084b0 0x7efd8810ece0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.934+0000 7efd9008d640 1 --2- 172.21.15.112:0/3504860158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd88107b60 0x7efd88107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.934+0000 7efd9008d640 1 --2- 172.21.15.112:0/3504860158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd88102270 0x7efd88102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.934+0000 7efd9008d640 1 -- 172.21.15.112:0/3504860158 >> 172.21.15.112:0/3504860158 conn(0x7efd88077cc0 msgr2=0x7efd88078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.934+0000 7efd9008d640 1 -- 172.21.15.112:0/3504860158 shutdown_connections 2024-09-21T03:33:29.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.934+0000 7efd9008d640 1 -- 172.21.15.112:0/3504860158 wait complete. 2024-09-21T03:33:29.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.934+0000 7efd9008d640 1 Processor -- start 2024-09-21T03:33:29.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd9008d640 1 -- start start 2024-09-21T03:33:29.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd9008d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd88102270 0x7efd88199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd9008d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd88107b60 0x7efd881998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd9008d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd881084b0 0x7efd8819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd9008d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd881129f0 con 0x7efd881084b0 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd9008d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd88112870 con 0x7efd88107b60 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd9008d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efd88112b70 con 0x7efd88102270 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd8e62a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd881084b0 0x7efd8819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd8de29640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd88102270 0x7efd88199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd8e62a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd881084b0 0x7efd8819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37176/0 (socket says 172.21.15.112:37176) 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd8e62a640 1 -- 172.21.15.112:0/352763415 learned_addr learned my addr 172.21.15.112:0/352763415 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd8d628640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd88107b60 0x7efd881998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd8e62a640 1 -- 172.21.15.112:0/352763415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd88102270 msgr2=0x7efd88199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd8e62a640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd88102270 0x7efd88199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd8e62a640 1 -- 172.21.15.112:0/352763415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd88107b60 msgr2=0x7efd881998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd8e62a640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd88107b60 0x7efd881998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.935+0000 7efd8e62a640 1 -- 172.21.15.112:0/352763415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8819f2b0 con 0x7efd881084b0 2024-09-21T03:33:29.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.936+0000 7efd8e62a640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd881084b0 0x7efd8819eb80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efd80005cc0 tx=0x7efd80031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.936+0000 7efd7effd640 1 -- 172.21.15.112:0/352763415 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8004a070 con 0x7efd881084b0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.936+0000 7efd9008d640 1 -- 172.21.15.112:0/352763415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8819f540 con 0x7efd881084b0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.936+0000 7efd9008d640 1 -- 172.21.15.112:0/352763415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8810f6a0 con 0x7efd881084b0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.936+0000 7efd7effd640 1 -- 172.21.15.112:0/352763415 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd80003e20 con 0x7efd881084b0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.936+0000 7efd7effd640 1 -- 172.21.15.112:0/352763415 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8002f8c0 con 0x7efd881084b0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd7effd640 1 -- 172.21.15.112:0/352763415 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7efd8004d070 con 0x7efd881084b0 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd96601740 1 Processor -- start 2024-09-21T03:33:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd96601740 1 -- start start 2024-09-21T03:33:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd96601740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3bf070 0x56112e7666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd96601740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e766be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd96601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 0x56112e767120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd96601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56112e6dcea0 con 0x56112e39e390 2024-09-21T03:33:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd96601740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56112e6cc2d0 con 0x56112e3bf070 2024-09-21T03:33:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd96601740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56112e6d00e0 con 0x56112e3961b0 2024-09-21T03:33:29.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd8de29640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e766be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd8d628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 0x56112e767120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd8de29640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e766be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:47050/0 (socket says 172.21.15.112:47050) 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.939+0000 7efd8de29640 1 -- 172.21.15.112:0/3215696433 learned_addr learned my addr 172.21.15.112:0/3215696433 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd8d628640 1 --2- 172.21.15.112:0/3215696433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3bf070 0x56112e7666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd7effd640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd740b6680 0x7efd740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd8de29640 1 -- 172.21.15.112:0/3215696433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3bf070 msgr2=0x56112e7666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd8e62a640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd740b6680 0x7efd740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd8de29640 1 --2- 172.21.15.112:0/3215696433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3bf070 0x56112e7666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd8de29640 1 -- 172.21.15.112:0/3215696433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 msgr2=0x56112e767120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd8de29640 1 --2- 172.21.15.112:0/3215696433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 0x56112e767120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd8de29640 1 -- 172.21.15.112:0/3215696433 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112e42a650 con 0x56112e3961b0 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd8d628640 1 --2- 172.21.15.112:0/3215696433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3bf070 0x56112e7666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:29.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd7effd640 1 -- 172.21.15.112:0/352763415 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7efd80109ac0 con 0x7efd881084b0 2024-09-21T03:33:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd8de29640 1 --2- 172.21.15.112:0/3215696433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e766be0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efd84015cf0 tx=0x7efd84009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=938e87d216b61b9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd8e62a640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd740b6680 0x7efd740b8b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efd800cf9b0 tx=0x7efd800f3040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd73fff640 1 -- 172.21.15.112:0/3215696433 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84004230 con 0x56112e3961b0 2024-09-21T03:33:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd73fff640 1 -- 172.21.15.112:0/3215696433 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd84009e10 con 0x56112e3961b0 2024-09-21T03:33:29.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.940+0000 7efd73fff640 1 -- 172.21.15.112:0/3215696433 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84015070 con 0x56112e3961b0 2024-09-21T03:33:29.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.941+0000 7efd96601740 1 -- 172.21.15.112:0/3215696433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 msgr2=0x56112e766be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.941+0000 7efd96601740 1 --2- 172.21.15.112:0/3215696433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e766be0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efd84015cf0 tx=0x7efd84009c20 comp rx=0 tx=0).stop 2024-09-21T03:33:29.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.941+0000 7efd96601740 1 -- 172.21.15.112:0/3215696433 shutdown_connections 2024-09-21T03:33:29.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.941+0000 7efd96601740 1 --2- 172.21.15.112:0/3215696433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 0x56112e767120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.941+0000 7efd96601740 1 --2- 172.21.15.112:0/3215696433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e766be0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.941+0000 7efd96601740 1 --2- 172.21.15.112:0/3215696433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3bf070 0x56112e7666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.941+0000 7efd96601740 1 -- 172.21.15.112:0/3215696433 >> 172.21.15.112:0/3215696433 conn(0x56112e3b3460 msgr2=0x56112e764590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.942+0000 7efd96601740 1 -- 172.21.15.112:0/3215696433 shutdown_connections 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.942+0000 7efd96601740 1 -- 172.21.15.112:0/3215696433 wait complete. 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.942+0000 7efd96601740 1 -- start start 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.942+0000 7efd96601740 1 Processor -- start 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd96601740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3961b0 0x56112e771d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd96601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 0x56112e772250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd96601740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3bf070 0x56112e77e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd96601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56112e52d700 con 0x56112e39e390 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd96601740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56112e667160 con 0x56112e3961b0 2024-09-21T03:33:29.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd96601740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56112e774e80 con 0x56112e3bf070 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd96601740 10 client.0 _ll_register_callbacks cb 0x56112e6809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8de29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 0x56112e772250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8de29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 0x56112e772250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37194/0 (socket says 172.21.15.112:37194) 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8de29640 1 -- 172.21.15.112:0/9243311 learned_addr learned my addr 172.21.15.112:0/9243311 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8d628640 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3bf070 0x56112e77e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8d628640 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3961b0 0x56112e771d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8de29640 1 -- 172.21.15.112:0/9243311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3bf070 msgr2=0x56112e77e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8de29640 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3bf070 0x56112e77e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8de29640 1 -- 172.21.15.112:0/9243311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3961b0 msgr2=0x56112e771d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8de29640 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3961b0 0x56112e771d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8de29640 1 -- 172.21.15.112:0/9243311 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56112e694c90 con 0x56112e39e390 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8d628640 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3961b0 0x56112e771d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:29.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd8de29640 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 0x56112e772250 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efd84005bc0 tx=0x7efd84009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505602643c1735e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.943+0000 7efd717fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd717fa640 1 -- 172.21.15.112:0/9243311 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84005260 con 0x56112e39e390 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd717fa640 1 -- 172.21.15.112:0/9243311 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7efd84005400 con 0x56112e39e390 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd96601740 1 -- 172.21.15.112:0/9243311 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd84009e10 con 0x56112e39e390 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd717fa640 1 -- 172.21.15.112:0/9243311 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd84005600 con 0x56112e39e390 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd96601740 10 client.16311 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd96601740 1 -- 172.21.15.112:0/9243311 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56112e780b50 con 0x56112e39e390 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd717fa640 1 -- 172.21.15.112:0/9243311 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd84009e10 con 0x56112e39e390 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd717fa640 1 -- 172.21.15.112:0/9243311 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7efd8401e8c0 con 0x56112e39e390 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd96601740 10 client.16311 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd96601740 10 client.16311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd96601740 20 client.16311 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd96601740 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x56112e37d9f0 0x56112e783eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd96601740 4 client.16311 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:29.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.944+0000 7efd96601740 1 -- 172.21.15.112:0/9243311 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56112e6dedf0 con 0x56112e37d9f0 2024-09-21T03:33:29.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.945+0000 7efd8d628640 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x56112e37d9f0 0x56112e783eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.945+0000 7efd717fa640 10 client.16311 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:33:29.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.945+0000 7efd8d628640 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x56112e37d9f0 0x56112e783eb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16b3b72b86ae8e03 server_cookie=3e053e8d9c8781f7 in_seq=0 out_seq=0 2024-09-21T03:33:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd717fa640 1 -- 172.21.15.112:0/9243311 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56112e6dedf0 con 0x56112e37d9f0 2024-09-21T03:33:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd717fa640 10 client.16311 handle_command_reply: tid=0 2024-09-21T03:33:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 1 client.16311 shutdown 2024-09-21T03:33:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 1 -- 172.21.15.112:0/9243311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 msgr2=0x56112e772250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 0x56112e772250 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efd84005bc0 tx=0x7efd84009680 comp rx=0 tx=0).stop 2024-09-21T03:33:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 1 -- 172.21.15.112:0/9243311 shutdown_connections 2024-09-21T03:33:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x56112e37d9f0 0x56112e783eb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3bf070 0x56112e77e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e39e390 0x56112e772250 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 1 --2- 172.21.15.112:0/9243311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e3961b0 0x56112e771d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd717fa640 0 client.16311 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:33:29.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 1 -- 172.21.15.112:0/9243311 >> 172.21.15.112:0/9243311 conn(0x56112e3b3460 msgr2=0x56112e769880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 1 -- 172.21.15.112:0/9243311 shutdown_connections 2024-09-21T03:33:29.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 1 -- 172.21.15.112:0/9243311 wait complete. 2024-09-21T03:33:29.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.946+0000 7efd96601740 20 client.16311 trim_cache size 0 max 16384 2024-09-21T03:33:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd96601740 1 Processor -- start 2024-09-21T03:33:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd96601740 1 -- start start 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd96601740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e768090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd96601740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e39e390 0x56112e769eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd96601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e76ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd96601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd881afcc0 con 0x56112e3bf070 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd96601740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd881ad590 con 0x56112e39e390 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd96601740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efd881ab3e0 con 0x56112e3961b0 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd8d628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e76ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd8d628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e76ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37208/0 (socket says 172.21.15.112:37208) 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd8d628640 1 -- 172.21.15.112:0/730224821 learned_addr learned my addr 172.21.15.112:0/730224821 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd8d628640 1 --2- 172.21.15.112:0/730224821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e768090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.961+0000 7efd8de29640 1 --2- 172.21.15.112:0/730224821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e39e390 0x56112e769eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.962+0000 7efd8d628640 1 -- 172.21.15.112:0/730224821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 msgr2=0x56112e768090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.962+0000 7efd8d628640 1 --2- 172.21.15.112:0/730224821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e768090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.962+0000 7efd8d628640 1 -- 172.21.15.112:0/730224821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e39e390 msgr2=0x56112e769eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.962+0000 7efd8d628640 1 --2- 172.21.15.112:0/730224821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e39e390 0x56112e769eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.962+0000 7efd8d628640 1 -- 172.21.15.112:0/730224821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112e772810 con 0x56112e3bf070 2024-09-21T03:33:29.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.962+0000 7efd8d628640 1 --2- 172.21.15.112:0/730224821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e76ef00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7efd78000c00 tx=0x7efd780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2302999cf35c23f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.962+0000 7efd737fe640 1 -- 172.21.15.112:0/730224821 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7801a040 con 0x56112e3bf070 2024-09-21T03:33:29.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.962+0000 7efd737fe640 1 -- 172.21.15.112:0/730224821 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd780054e0 con 0x56112e3bf070 2024-09-21T03:33:29.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.962+0000 7efd737fe640 1 -- 172.21.15.112:0/730224821 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78004ab0 con 0x56112e3bf070 2024-09-21T03:33:29.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.963+0000 7efd96601740 1 -- 172.21.15.112:0/730224821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 msgr2=0x56112e76ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.963+0000 7efd96601740 1 --2- 172.21.15.112:0/730224821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e76ef00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7efd78000c00 tx=0x7efd780046c0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.963+0000 7efd96601740 1 -- 172.21.15.112:0/730224821 shutdown_connections 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.963+0000 7efd96601740 1 --2- 172.21.15.112:0/730224821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e76ef00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.963+0000 7efd96601740 1 --2- 172.21.15.112:0/730224821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e39e390 0x56112e769eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.963+0000 7efd96601740 1 --2- 172.21.15.112:0/730224821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e768090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.963+0000 7efd96601740 1 -- 172.21.15.112:0/730224821 >> 172.21.15.112:0/730224821 conn(0x56112e3b3460 msgr2=0x56112e7697a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.963+0000 7efd96601740 1 -- 172.21.15.112:0/730224821 shutdown_connections 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.963+0000 7efd96601740 1 -- 172.21.15.112:0/730224821 wait complete. 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.963+0000 7efd96601740 1 -- start start 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.963+0000 7efd96601740 1 Processor -- start 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd96601740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e7e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd96601740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e39e390 0x56112e7e5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd96601740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e7779d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd96601740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd881aba50 con 0x56112e3bf070 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd96601740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd881acec0 con 0x56112e39e390 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd96601740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efd881ac120 con 0x56112e3961b0 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd96601740 10 client.0 _ll_register_callbacks cb 0x7efd78009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd8d628640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e7e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd8d628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e7779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd8d628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e7779d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37210/0 (socket says 172.21.15.112:37210) 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd8d628640 1 -- 172.21.15.112:0/1889140298 learned_addr learned my addr 172.21.15.112:0/1889140298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd8de29640 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e39e390 0x56112e7e5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd8d628640 1 -- 172.21.15.112:0/1889140298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 msgr2=0x56112e7e5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd8d628640 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e7e5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd8d628640 1 -- 172.21.15.112:0/1889140298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e39e390 msgr2=0x56112e7e5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:29.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd8d628640 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e39e390 0x56112e7e5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.964+0000 7efd8d628640 1 -- 172.21.15.112:0/1889140298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56112e768ee0 con 0x56112e3bf070 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.965+0000 7efd8de29640 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e39e390 0x56112e7e5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.965+0000 7efd8d628640 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e7779d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7efd78014010 tx=0x7efd78005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b36e67db5d2e10e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.965+0000 7efd717fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.965+0000 7efd717fa640 1 -- 172.21.15.112:0/1889140298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78012230 con 0x56112e3bf070 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.965+0000 7efd717fa640 1 -- 172.21.15.112:0/1889140298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7efd780114f0 con 0x56112e3bf070 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.965+0000 7efd96601740 1 -- 172.21.15.112:0/1889140298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd780054e0 con 0x56112e3bf070 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.965+0000 7efd717fa640 1 -- 172.21.15.112:0/1889140298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd780116f0 con 0x56112e3bf070 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.965+0000 7efd96601740 10 client.16317 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.965+0000 7efd96601740 1 -- 172.21.15.112:0/1889140298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56112e7ee140 con 0x56112e3bf070 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.966+0000 7efd717fa640 1 -- 172.21.15.112:0/1889140298 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd780054e0 con 0x56112e3bf070 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.966+0000 7efd717fa640 1 -- 172.21.15.112:0/1889140298 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7efd78021690 con 0x56112e3bf070 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.966+0000 7efd96601740 10 client.16317 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:29.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.966+0000 7efd96601740 10 client.16317 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:33:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.966+0000 7efd96601740 20 client.16317 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.966+0000 7efd96601740 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x56112e352da0 0x56112e7f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.966+0000 7efd96601740 4 client.16317 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:33:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.966+0000 7efd96601740 1 -- 172.21.15.112:0/1889140298 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56112e6da8a0 con 0x56112e352da0 2024-09-21T03:33:29.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.966+0000 7efd8de29640 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x56112e352da0 0x56112e7f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:29.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.966+0000 7efd717fa640 10 client.16317 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:33:29.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd8de29640 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x56112e352da0 0x56112e7f07e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db3380efb7e186b6 server_cookie=5f90095f546ffcba in_seq=0 out_seq=0 2024-09-21T03:33:29.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd717fa640 1 -- 172.21.15.112:0/1889140298 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56112e6da8a0 con 0x56112e352da0 2024-09-21T03:33:29.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd717fa640 10 client.16317 handle_command_reply: tid=0 2024-09-21T03:33:29.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 1 client.16317 shutdown 2024-09-21T03:33:29.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 1 -- 172.21.15.112:0/1889140298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 msgr2=0x56112e7779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e7779d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7efd78014010 tx=0x7efd78005f90 comp rx=0 tx=0).stop 2024-09-21T03:33:29.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 1 -- 172.21.15.112:0/1889140298 shutdown_connections 2024-09-21T03:33:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x56112e352da0 0x56112e7f07e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd717fa640 0 client.16317 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:33:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56112e3bf070 0x56112e7779d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56112e39e390 0x56112e7e5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 1 --2- 172.21.15.112:0/1889140298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56112e3961b0 0x56112e7e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 1 -- 172.21.15.112:0/1889140298 >> 172.21.15.112:0/1889140298 conn(0x56112e3b3460 msgr2=0x56112e767e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 1 -- 172.21.15.112:0/1889140298 shutdown_connections 2024-09-21T03:33:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 1 -- 172.21.15.112:0/1889140298 wait complete. 2024-09-21T03:33:29.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.967+0000 7efd96601740 20 client.16317 trim_cache size 0 max 16384 2024-09-21T03:33:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:33:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 -- 172.21.15.112:0/352763415 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd740b6680 msgr2=0x7efd740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd740b6680 0x7efd740b8b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efd800cf9b0 tx=0x7efd800f3040 comp rx=0 tx=0).stop 2024-09-21T03:33:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 -- 172.21.15.112:0/352763415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd881084b0 msgr2=0x7efd8819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd881084b0 0x7efd8819eb80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efd80005cc0 tx=0x7efd80031680 comp rx=0 tx=0).stop 2024-09-21T03:33:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 -- 172.21.15.112:0/352763415 shutdown_connections 2024-09-21T03:33:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd740b6680 0x7efd740b8b90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd881084b0 0x7efd8819eb80 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd88107b60 0x7efd881998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 --2- 172.21.15.112:0/352763415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd88102270 0x7efd88199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:29.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 -- 172.21.15.112:0/352763415 >> 172.21.15.112:0/352763415 conn(0x7efd88077cc0 msgr2=0x7efd88103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:29.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 -- 172.21.15.112:0/352763415 shutdown_connections 2024-09-21T03:33:29.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:29.968+0000 7efd9008d640 1 -- 172.21.15.112:0/352763415 wait complete. 2024-09-21T03:33:30.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 437132 11.26 MB/sec execute 90 sec latency 32.114 ms 2024-09-21T03:33:30.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:30 smithi112 ceph-mon[31442]: pgmap v453: 97 pgs: 97 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 306 op/s 2024-09-21T03:33:30.857 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:33:30.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:30 smithi203 ceph-mon[34723]: pgmap v453: 97 pgs: 97 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 306 op/s 2024-09-21T03:33:30.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:30 smithi187 ceph-mon[34603]: pgmap v453: 97 pgs: 97 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 306 op/s 2024-09-21T03:33:31.118 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:31.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 439854 11.33 MB/sec execute 91 sec latency 43.560 ms 2024-09-21T03:33:31.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.657+0000 7f5621993640 1 -- 172.21.15.112:0/3050063779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c102a60 msgr2=0x7f561c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.657+0000 7f5621993640 1 --2- 172.21.15.112:0/3050063779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c102a60 0x7f561c102e70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f560c009860 tx=0x7f560c02a340 comp rx=0 tx=0).stop 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.657+0000 7f5621993640 1 -- 172.21.15.112:0/3050063779 shutdown_connections 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.657+0000 7f5621993640 1 --2- 172.21.15.112:0/3050063779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f561c108cf0 0x7f561c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.657+0000 7f5621993640 1 --2- 172.21.15.112:0/3050063779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f561c1083a0 0x7f561c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.657+0000 7f5621993640 1 --2- 172.21.15.112:0/3050063779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c102a60 0x7f561c102e70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.657+0000 7f5621993640 1 -- 172.21.15.112:0/3050063779 >> 172.21.15.112:0/3050063779 conn(0x7f561c079730 msgr2=0x7f561c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.657+0000 7f5621993640 1 -- 172.21.15.112:0/3050063779 shutdown_connections 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.657+0000 7f5621993640 1 -- 172.21.15.112:0/3050063779 wait complete. 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5621993640 1 Processor -- start 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5621993640 1 -- start start 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5621993640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c102a60 0x7f561c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5621993640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f561c1083a0 0x7f561c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5621993640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f561c108cf0 0x7f561c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5621993640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f561c112960 con 0x7f561c1083a0 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5621993640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1127e0 con 0x7f561c102a60 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5621993640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f561c112ae0 con 0x7f561c108cf0 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5613fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f561c1083a0 0x7f561c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5613fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f561c1083a0 0x7f561c1998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37240/0 (socket says 172.21.15.112:37240) 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5613fff640 1 -- 172.21.15.112:0/1864490649 learned_addr learned my addr 172.21.15.112:0/1864490649 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f561b7fe640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f561c108cf0 0x7f561c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f561affd640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c102a60 0x7f561c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5613fff640 1 -- 172.21.15.112:0/1864490649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f561c108cf0 msgr2=0x7f561c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5613fff640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f561c108cf0 0x7f561c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5613fff640 1 -- 172.21.15.112:0/1864490649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c102a60 msgr2=0x7f561c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5613fff640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c102a60 0x7f561c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5613fff640 1 -- 172.21.15.112:0/1864490649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c19f2a0 con 0x7f561c1083a0 2024-09-21T03:33:31.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f561affd640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c102a60 0x7f561c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:31.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.658+0000 7f5613fff640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f561c1083a0 0x7f561c1998d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f560400e9a0 tx=0x7f560400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.659+0000 7f5620991640 1 -- 172.21.15.112:0/1864490649 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560400cd80 con 0x7f561c1083a0 2024-09-21T03:33:31.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.659+0000 7f5621993640 1 -- 172.21.15.112:0/1864490649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561c19f590 con 0x7f561c1083a0 2024-09-21T03:33:31.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.659+0000 7f5620991640 1 -- 172.21.15.112:0/1864490649 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56040035f0 con 0x7f561c1083a0 2024-09-21T03:33:31.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.659+0000 7f5620991640 1 -- 172.21.15.112:0/1864490649 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604010690 con 0x7f561c1083a0 2024-09-21T03:33:31.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.659+0000 7f5621993640 1 -- 172.21.15.112:0/1864490649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f561c1a5cf0 con 0x7f561c1083a0 2024-09-21T03:33:31.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.661+0000 7f5627f07740 1 Processor -- start 2024-09-21T03:33:31.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f5627f07740 1 -- start start 2024-09-21T03:33:31.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f5620991640 1 -- 172.21.15.112:0/1864490649 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f5604003fb0 con 0x7f561c1083a0 2024-09-21T03:33:31.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f5627f07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e424070 0x562c9e7cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f5627f07740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f5627f07740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f5627f07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562c9e741ea0 con 0x562c9e424070 2024-09-21T03:33:31.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f5627f07740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562c9e7312d0 con 0x562c9e403390 2024-09-21T03:33:31.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f5627f07740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562c9e7350e0 con 0x562c9e3fb1b0 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e424070 0x562c9e7cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f561affd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f561affd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:47104/0 (socket says 172.21.15.112:47104) 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f561affd640 1 -- 172.21.15.112:0/1606243620 learned_addr learned my addr 172.21.15.112:0/1606243620 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f5613fff640 1 --2- 172.21.15.112:0/1606243620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f561affd640 1 -- 172.21.15.112:0/1606243620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 msgr2=0x562c9e7cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f561affd640 1 --2- 172.21.15.112:0/1606243620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f561affd640 1 -- 172.21.15.112:0/1606243620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e424070 msgr2=0x562c9e7cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f561affd640 1 --2- 172.21.15.112:0/1606243620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e424070 0x562c9e7cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.662+0000 7f561affd640 1 -- 172.21.15.112:0/1606243620 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9e48f650 con 0x562c9e3fb1b0 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.663+0000 7f5620991640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f56080b6680 0x7f56080b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.663+0000 7f561affd640 1 --2- 172.21.15.112:0/1606243620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7cb800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f560c009860 tx=0x7f560c030b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7e1c0d80cf4066d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.663+0000 7f561b7fe640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f56080b6680 0x7f56080b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.663+0000 7f5620991640 1 -- 172.21.15.112:0/1864490649 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f56040e7a20 con 0x7f561c1083a0 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.663+0000 7f5611ffb640 1 -- 172.21.15.112:0/1606243620 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c037040 con 0x562c9e3fb1b0 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.663+0000 7f5611ffb640 1 -- 172.21.15.112:0/1606243620 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f560c030450 con 0x562c9e3fb1b0 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.663+0000 7f5611ffb640 1 -- 172.21.15.112:0/1606243620 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c005580 con 0x562c9e3fb1b0 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.664+0000 7f5627f07740 1 -- 172.21.15.112:0/1606243620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 msgr2=0x562c9e7cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.664+0000 7f5627f07740 1 --2- 172.21.15.112:0/1606243620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7cb800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f560c009860 tx=0x7f560c030b50 comp rx=0 tx=0).stop 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.664+0000 7f561b7fe640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f56080b6680 0x7f56080b8b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5614005fd0 tx=0x7f5614009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.664+0000 7f5627f07740 1 -- 172.21.15.112:0/1606243620 shutdown_connections 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.664+0000 7f5627f07740 1 --2- 172.21.15.112:0/1606243620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.664+0000 7f5627f07740 1 --2- 172.21.15.112:0/1606243620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7cb800 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.664+0000 7f5627f07740 1 --2- 172.21.15.112:0/1606243620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e424070 0x562c9e7cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.664+0000 7f5627f07740 1 -- 172.21.15.112:0/1606243620 >> 172.21.15.112:0/1606243620 conn(0x562c9e418460 msgr2=0x562c9e7c91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.664+0000 7f5627f07740 1 -- 172.21.15.112:0/1606243620 shutdown_connections 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.664+0000 7f5627f07740 1 -- 172.21.15.112:0/1606243620 wait complete. 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.665+0000 7f5627f07740 1 -- start start 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.665+0000 7f5627f07740 1 Processor -- start 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.665+0000 7f5627f07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3fb1b0 0x562c9e7d6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.665+0000 7f5627f07740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7d7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.665+0000 7f5627f07740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e424070 0x562c9e7e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.665+0000 7f5627f07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562c9e592700 con 0x562c9e3fb1b0 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.665+0000 7f5627f07740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562c9e6cc160 con 0x562c9e403390 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.665+0000 7f5627f07740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562c9e7dbb00 con 0x562c9e424070 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.665+0000 7f5627f07740 10 client.0 _ll_register_callbacks cb 0x562c9e6e59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.665+0000 7f561affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3fb1b0 0x562c9e7d6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.665+0000 7f561affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3fb1b0 0x562c9e7d6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37248/0 (socket says 172.21.15.112:37248) 2024-09-21T03:33:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f561affd640 1 -- 172.21.15.112:0/522734002 learned_addr learned my addr 172.21.15.112:0/522734002 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f561b7fe640 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e424070 0x562c9e7e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f5613fff640 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7d7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f561affd640 1 -- 172.21.15.112:0/522734002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e424070 msgr2=0x562c9e7e4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f561affd640 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e424070 0x562c9e7e4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f561affd640 1 -- 172.21.15.112:0/522734002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 msgr2=0x562c9e7d7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f561affd640 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7d7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f561affd640 1 -- 172.21.15.112:0/522734002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c9e6f9c90 con 0x562c9e3fb1b0 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f561affd640 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3fb1b0 0x562c9e7d6b90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f560c035040 tx=0x7f560c030a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98921923dd1b1873 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f55faffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f55faffd640 1 -- 172.21.15.112:0/522734002 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c037030 con 0x562c9e3fb1b0 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f55faffd640 1 -- 172.21.15.112:0/522734002 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f560c004440 con 0x562c9e3fb1b0 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f5627f07740 1 -- 172.21.15.112:0/522734002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f560c030450 con 0x562c9e3fb1b0 2024-09-21T03:33:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.666+0000 7f55faffd640 1 -- 172.21.15.112:0/522734002 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f560c005180 con 0x562c9e3fb1b0 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.667+0000 7f5627f07740 10 client.16329 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.667+0000 7f5627f07740 1 -- 172.21.15.112:0/522734002 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c9e7e7ad0 con 0x562c9e3fb1b0 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.667+0000 7f55faffd640 1 -- 172.21.15.112:0/522734002 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f560c030450 con 0x562c9e3fb1b0 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.667+0000 7f55faffd640 1 -- 172.21.15.112:0/522734002 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f560c0035f0 con 0x562c9e3fb1b0 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.667+0000 7f5627f07740 10 client.16329 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.667+0000 7f5627f07740 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.667+0000 7f5627f07740 20 client.16329 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.667+0000 7f5627f07740 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562c9e3e29f0 0x562c9e7eae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.667+0000 7f5627f07740 4 client.16329 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.667+0000 7f5627f07740 1 -- 172.21.15.112:0/522734002 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c9e743df0 con 0x562c9e3e29f0 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.667+0000 7f5613fff640 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562c9e3e29f0 0x562c9e7eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.668+0000 7f5613fff640 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562c9e3e29f0 0x562c9e7eae30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac6076eb30d6735 server_cookie=66cbec1110c0a19a in_seq=0 out_seq=0 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.668+0000 7f55faffd640 10 client.16329 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f55faffd640 1 -- 172.21.15.112:0/522734002 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562c9e743df0 con 0x562c9e3e29f0 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f55faffd640 10 client.16329 handle_command_reply: tid=0 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 1 client.16329 shutdown 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 1 -- 172.21.15.112:0/522734002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3fb1b0 msgr2=0x562c9e7d6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3fb1b0 0x562c9e7d6b90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f560c035040 tx=0x7f560c030a00 comp rx=0 tx=0).stop 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 1 -- 172.21.15.112:0/522734002 shutdown_connections 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562c9e3e29f0 0x562c9e7eae30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e424070 0x562c9e7e4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7d7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 1 --2- 172.21.15.112:0/522734002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3fb1b0 0x562c9e7d6b90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f55faffd640 0 client.16329 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 1 -- 172.21.15.112:0/522734002 >> 172.21.15.112:0/522734002 conn(0x562c9e418460 msgr2=0x562c9e7ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 1 -- 172.21.15.112:0/522734002 shutdown_connections 2024-09-21T03:33:31.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 1 -- 172.21.15.112:0/522734002 wait complete. 2024-09-21T03:33:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.669+0000 7f5627f07740 20 client.16329 trim_cache size 0 max 16384 2024-09-21T03:33:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f5627f07740 1 Processor -- start 2024-09-21T03:33:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f5627f07740 1 -- start start 2024-09-21T03:33:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f5627f07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3e29f0 0x562c9e7ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f5627f07740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7d87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f5627f07740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7e4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f5627f07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1afc80 con 0x562c9e3e29f0 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f5627f07740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1ad550 con 0x562c9e403390 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f5627f07740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1ab3c0 con 0x562c9e3fb1b0 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f561affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3e29f0 0x562c9e7ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f561affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3e29f0 0x562c9e7ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37252/0 (socket says 172.21.15.112:37252) 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f561affd640 1 -- 172.21.15.112:0/2583554362 learned_addr learned my addr 172.21.15.112:0/2583554362 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f5613fff640 1 --2- 172.21.15.112:0/2583554362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7d87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f561b7fe640 1 --2- 172.21.15.112:0/2583554362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7e4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f561affd640 1 -- 172.21.15.112:0/2583554362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 msgr2=0x562c9e7d87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f561affd640 1 --2- 172.21.15.112:0/2583554362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7d87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f561affd640 1 -- 172.21.15.112:0/2583554362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 msgr2=0x562c9e7e4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f561affd640 1 --2- 172.21.15.112:0/2583554362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7e4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f561affd640 1 -- 172.21.15.112:0/2583554362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9e7d75d0 con 0x562c9e3e29f0 2024-09-21T03:33:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.683+0000 7f561affd640 1 --2- 172.21.15.112:0/2583554362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3e29f0 0x562c9e7ced80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f560c005eb0 tx=0x7f560c030910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e46afc56c68b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.684+0000 7f56117fa640 1 -- 172.21.15.112:0/2583554362 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c037070 con 0x562c9e3e29f0 2024-09-21T03:33:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.684+0000 7f56117fa640 1 -- 172.21.15.112:0/2583554362 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f560c030450 con 0x562c9e3e29f0 2024-09-21T03:33:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.684+0000 7f56117fa640 1 -- 172.21.15.112:0/2583554362 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c0046c0 con 0x562c9e3e29f0 2024-09-21T03:33:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.685+0000 7f5627f07740 1 -- 172.21.15.112:0/2583554362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3e29f0 msgr2=0x562c9e7ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.685+0000 7f5627f07740 1 --2- 172.21.15.112:0/2583554362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3e29f0 0x562c9e7ced80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f560c005eb0 tx=0x7f560c030910 comp rx=0 tx=0).stop 2024-09-21T03:33:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.685+0000 7f5627f07740 1 -- 172.21.15.112:0/2583554362 shutdown_connections 2024-09-21T03:33:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.685+0000 7f5627f07740 1 --2- 172.21.15.112:0/2583554362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e403390 0x562c9e7e4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.685+0000 7f5627f07740 1 --2- 172.21.15.112:0/2583554362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7d87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.685+0000 7f5627f07740 1 --2- 172.21.15.112:0/2583554362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e3e29f0 0x562c9e7ced80 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.685+0000 7f5627f07740 1 -- 172.21.15.112:0/2583554362 >> 172.21.15.112:0/2583554362 conn(0x562c9e418460 msgr2=0x562c9e7d80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.685+0000 7f5627f07740 1 -- 172.21.15.112:0/2583554362 shutdown_connections 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.685+0000 7f5627f07740 1 -- 172.21.15.112:0/2583554362 wait complete. 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.685+0000 7f5627f07740 1 -- start start 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.685+0000 7f5627f07740 1 Processor -- start 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5627f07740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e3e29f0 0x562c9e7ee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5627f07740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5627f07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e403390 0x562c9e7ec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5627f07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1aba10 con 0x562c9e403390 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5627f07740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1ace80 con 0x562c9e3e29f0 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5627f07740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1ac0e0 con 0x562c9e3fb1b0 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5627f07740 10 client.0 _ll_register_callbacks cb 0x7f5614009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f561affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e3e29f0 0x562c9e7ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5613fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e403390 0x562c9e7ec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f561affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e3e29f0 0x562c9e7ee640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40660/0 (socket says 172.21.15.112:40660) 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f561affd640 1 -- 172.21.15.112:0/3808162136 learned_addr learned my addr 172.21.15.112:0/3808162136 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5613fff640 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7ebb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:47122/0 (socket says 172.21.15.112:47122) 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5613fff640 1 -- 172.21.15.112:0/3808162136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e3e29f0 msgr2=0x562c9e7ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5613fff640 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e3e29f0 0x562c9e7ee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5613fff640 1 -- 172.21.15.112:0/3808162136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e403390 msgr2=0x562c9e7ec050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5613fff640 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e403390 0x562c9e7ec050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5613fff640 1 -- 172.21.15.112:0/3808162136 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c9e7ec7e0 con 0x562c9e3fb1b0 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f561affd640 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e3e29f0 0x562c9e7ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5613fff640 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7ebb10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f56040e51d0 tx=0x7f5604003e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6798a54d0eb7e1c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:31.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f55faffd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f55faffd640 1 -- 172.21.15.112:0/3808162136 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56040a9770 con 0x562c9e3fb1b0 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f5627f07740 1 -- 172.21.15.112:0/3808162136 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f560c030450 con 0x562c9e3fb1b0 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.686+0000 7f55faffd640 1 -- 172.21.15.112:0/3808162136 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f56040e5450 con 0x562c9e3fb1b0 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f55faffd640 1 -- 172.21.15.112:0/3808162136 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56040b21b0 con 0x562c9e3fb1b0 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f5627f07740 10 client.25903 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f5627f07740 1 -- 172.21.15.112:0/3808162136 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c9e7d9860 con 0x562c9e3fb1b0 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f55faffd640 1 -- 172.21.15.112:0/3808162136 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f560c030450 con 0x562c9e3fb1b0 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f55faffd640 1 -- 172.21.15.112:0/3808162136 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f56040d3020 con 0x562c9e3fb1b0 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f5627f07740 10 client.25903 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f5627f07740 10 client.25903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f5627f07740 20 client.25903 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f5627f07740 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562c9e424070 0x562c9e7d9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f5627f07740 4 client.25903 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:33:31.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f5627f07740 1 -- 172.21.15.112:0/3808162136 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c9e7d2740 con 0x562c9e424070 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.687+0000 7f561affd640 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562c9e424070 0x562c9e7d9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f561affd640 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562c9e424070 0x562c9e7d9fd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13c906881673d142 server_cookie=451529f04c953cd5 in_seq=0 out_seq=0 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f55faffd640 10 client.25903 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f55faffd640 1 -- 172.21.15.112:0/3808162136 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c9e7d2740 con 0x562c9e424070 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f55faffd640 10 client.25903 handle_command_reply: tid=0 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 1 client.25903 shutdown 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 1 -- 172.21.15.112:0/3808162136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 msgr2=0x562c9e7ebb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7ebb10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f56040e51d0 tx=0x7f5604003e70 comp rx=0 tx=0).stop 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 1 -- 172.21.15.112:0/3808162136 shutdown_connections 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562c9e424070 0x562c9e7d9fd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c9e403390 0x562c9e7ec050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c9e3fb1b0 0x562c9e7ebb10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f55faffd640 0 client.25903 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:31.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 1 --2- 172.21.15.112:0/3808162136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c9e3e29f0 0x562c9e7ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 1 -- 172.21.15.112:0/3808162136 >> 172.21.15.112:0/3808162136 conn(0x562c9e418460 msgr2=0x562c9e7cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 1 -- 172.21.15.112:0/3808162136 shutdown_connections 2024-09-21T03:33:31.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 1 -- 172.21.15.112:0/3808162136 wait complete. 2024-09-21T03:33:31.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.688+0000 7f5627f07740 20 client.25903 trim_cache size 0 max 16384 2024-09-21T03:33:31.697 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 -- 172.21.15.112:0/1864490649 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f56080b6680 msgr2=0x7f56080b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f56080b6680 0x7f56080b8b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5614005fd0 tx=0x7f5614009500 comp rx=0 tx=0).stop 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 -- 172.21.15.112:0/1864490649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f561c1083a0 msgr2=0x7f561c1998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f561c1083a0 0x7f561c1998d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f560400e9a0 tx=0x7f560400ee60 comp rx=0 tx=0).stop 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 -- 172.21.15.112:0/1864490649 shutdown_connections 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f56080b6680 0x7f56080b8b90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f561c108cf0 0x7f561c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f561c1083a0 0x7f561c1998d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 --2- 172.21.15.112:0/1864490649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c102a60 0x7f561c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 -- 172.21.15.112:0/1864490649 >> 172.21.15.112:0/1864490649 conn(0x7f561c079730 msgr2=0x7f561c101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 -- 172.21.15.112:0/1864490649 shutdown_connections 2024-09-21T03:33:31.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:31.689+0000 7f5621993640 1 -- 172.21.15.112:0/1864490649 wait complete. 2024-09-21T03:33:32.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 441343 11.28 MB/sec execute 92 sec latency 27.182 ms 2024-09-21T03:33:32.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:32 smithi112 ceph-mon[31442]: pgmap v454: 97 pgs: 97 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 328 op/s 2024-09-21T03:33:32.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:32 smithi203 ceph-mon[34723]: pgmap v454: 97 pgs: 97 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 328 op/s 2024-09-21T03:33:32.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:32 smithi187 ceph-mon[34603]: pgmap v454: 97 pgs: 97 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 328 op/s 2024-09-21T03:33:33.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 443813 11.31 MB/sec execute 93 sec latency 10.968 ms 2024-09-21T03:33:33.533 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:33:33.534 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:33:33.802 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:34.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 445883 11.27 MB/sec execute 94 sec latency 40.303 ms 2024-09-21T03:33:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.310+0000 7f7af96cc640 1 -- 172.21.15.112:0/516713080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7af4103bb0 msgr2=0x7f7af4103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.310+0000 7f7af96cc640 1 --2- 172.21.15.112:0/516713080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7af4103bb0 0x7f7af4103fc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009900 tx=0x7f7ae402a410 comp rx=0 tx=0).stop 2024-09-21T03:33:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.310+0000 7f7af96cc640 1 -- 172.21.15.112:0/516713080 shutdown_connections 2024-09-21T03:33:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.310+0000 7f7af96cc640 1 --2- 172.21.15.112:0/516713080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af4109e40 0x7f7af4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.310+0000 7f7af96cc640 1 --2- 172.21.15.112:0/516713080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7af41094f0 0x7f7af4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.310+0000 7f7af96cc640 1 --2- 172.21.15.112:0/516713080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7af4103bb0 0x7f7af4103fc0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.310+0000 7f7af96cc640 1 -- 172.21.15.112:0/516713080 >> 172.21.15.112:0/516713080 conn(0x7f7af40ff600 msgr2=0x7f7af4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.311+0000 7f7af96cc640 1 -- 172.21.15.112:0/516713080 shutdown_connections 2024-09-21T03:33:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.311+0000 7f7af96cc640 1 -- 172.21.15.112:0/516713080 wait complete. 2024-09-21T03:33:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.311+0000 7f7af96cc640 1 Processor -- start 2024-09-21T03:33:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.311+0000 7f7af96cc640 1 -- start start 2024-09-21T03:33:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af96cc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af4103bb0 0x7f7af4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af96cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7af41094f0 0x7f7af4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af96cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7af4109e40 0x7f7af4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af96cc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4113b00 con 0x7f7af4109e40 2024-09-21T03:33:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af96cc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4113980 con 0x7f7af4103bb0 2024-09-21T03:33:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af96cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4113c80 con 0x7f7af41094f0 2024-09-21T03:33:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af4103bb0 0x7f7af4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7af4109e40 0x7f7af4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af4103bb0 0x7f7af4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40670/0 (socket says 172.21.15.112:40670) 2024-09-21T03:33:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af2ffd640 1 -- 172.21.15.112:0/2209700573 learned_addr learned my addr 172.21.15.112:0/2209700573 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7af41094f0 0x7f7af4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af37fe640 1 -- 172.21.15.112:0/2209700573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7af41094f0 msgr2=0x7f7af4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af37fe640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7af41094f0 0x7f7af4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af37fe640 1 -- 172.21.15.112:0/2209700573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af4103bb0 msgr2=0x7f7af4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af37fe640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af4103bb0 0x7f7af4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af37fe640 1 -- 172.21.15.112:0/2209700573 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af41a8790 con 0x7f7af4109e40 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.312+0000 7f7af37fe640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7af4109e40 0x7f7af4076190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009fd0 tx=0x7f7aec00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.313+0000 7f7af0ff9640 1 -- 172.21.15.112:0/2209700573 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec00ce20 con 0x7f7af4109e40 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.313+0000 7f7af96cc640 1 -- 172.21.15.112:0/2209700573 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af41a8a80 con 0x7f7af4109e40 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.313+0000 7f7af0ff9640 1 -- 172.21.15.112:0/2209700573 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aec005020 con 0x7f7af4109e40 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.313+0000 7f7af0ff9640 1 -- 172.21.15.112:0/2209700573 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec010690 con 0x7f7af4109e40 2024-09-21T03:33:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.313+0000 7f7af96cc640 1 -- 172.21.15.112:0/2209700573 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af41a8fc0 con 0x7f7af4109e40 2024-09-21T03:33:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.316+0000 7f7affc40740 1 Processor -- start 2024-09-21T03:33:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.316+0000 7f7af0ff9640 1 -- 172.21.15.112:0/2209700573 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7aec010830 con 0x7f7af4109e40 2024-09-21T03:33:34.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.316+0000 7f7affc40740 1 -- start start 2024-09-21T03:33:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7affc40740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedac6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7affc40740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6f6490 0x560dedac6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7affc40740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6fe670 0x560dedac6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7affc40740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560deda3ce60 con 0x560ded71f350 2024-09-21T03:33:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7affc40740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560deda2c290 con 0x560ded6fe670 2024-09-21T03:33:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7affc40740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560deda300a0 con 0x560ded6f6490 2024-09-21T03:33:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7aea7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedac6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7aea7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedac6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37284/0 (socket says 172.21.15.112:37284) 2024-09-21T03:33:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7aea7fc640 1 -- 172.21.15.112:0/449879787 learned_addr learned my addr 172.21.15.112:0/449879787 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7af2ffd640 1 --2- 172.21.15.112:0/449879787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6f6490 0x560dedac6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/449879787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6fe670 0x560dedac6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7af0ff9640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7ae00b6680 0x7f7ae00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7af2ffd640 1 -- 172.21.15.112:0/449879787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6fe670 msgr2=0x560dedac6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7af2ffd640 1 --2- 172.21.15.112:0/449879787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6fe670 0x560dedac6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7af2ffd640 1 -- 172.21.15.112:0/449879787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 msgr2=0x560dedac6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7af2ffd640 1 --2- 172.21.15.112:0/449879787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedac6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7af2ffd640 1 -- 172.21.15.112:0/449879787 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ded78a930 con 0x560ded6f6490 2024-09-21T03:33:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.317+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/449879787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedac6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.318+0000 7f7af0ff9640 1 -- 172.21.15.112:0/2209700573 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f7aec0e7d70 con 0x7f7af4109e40 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.318+0000 7f7af37fe640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7ae00b6680 0x7f7ae00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.318+0000 7f7af2ffd640 1 --2- 172.21.15.112:0/449879787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6f6490 0x560dedac6970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4004770 tx=0x7f7ae4031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef86738709ff41da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.318+0000 7f7ae97fa640 1 -- 172.21.15.112:0/449879787 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4040070 con 0x560ded6f6490 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.318+0000 7f7ae97fa640 1 -- 172.21.15.112:0/449879787 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ae4004030 con 0x560ded6f6490 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.318+0000 7f7ae97fa640 1 -- 172.21.15.112:0/449879787 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4035040 con 0x560ded6f6490 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.319+0000 7f7affc40740 1 -- 172.21.15.112:0/449879787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6f6490 msgr2=0x560dedac6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.319+0000 7f7affc40740 1 --2- 172.21.15.112:0/449879787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6f6490 0x560dedac6970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4004770 tx=0x7f7ae4031b10 comp rx=0 tx=0).stop 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.319+0000 7f7af37fe640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7ae00b6680 0x7f7ae00b8b90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0e5580 tx=0x7f7aec005870 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.319+0000 7f7affc40740 1 -- 172.21.15.112:0/449879787 shutdown_connections 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.319+0000 7f7affc40740 1 --2- 172.21.15.112:0/449879787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6fe670 0x560dedac6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.319+0000 7f7affc40740 1 --2- 172.21.15.112:0/449879787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6f6490 0x560dedac6970 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.319+0000 7f7affc40740 1 --2- 172.21.15.112:0/449879787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedac6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.319+0000 7f7affc40740 1 -- 172.21.15.112:0/449879787 >> 172.21.15.112:0/449879787 conn(0x560ded713740 msgr2=0x560dedac4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:34.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.319+0000 7f7affc40740 1 -- 172.21.15.112:0/449879787 shutdown_connections 2024-09-21T03:33:34.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.319+0000 7f7affc40740 1 -- 172.21.15.112:0/449879787 wait complete. 2024-09-21T03:33:34.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.319+0000 7f7affc40740 1 -- start start 2024-09-21T03:33:34.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7affc40740 1 Processor -- start 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7affc40740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedad1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7affc40740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 0x560dedad2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7affc40740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedaddea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7affc40740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560ded88d890 con 0x560ded71f350 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7affc40740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560ded9c6d40 con 0x560ded6f6490 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7affc40740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560dedad4bb0 con 0x560ded6fe670 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7affc40740 10 client.0 _ll_register_callbacks cb 0x560deda2ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7aea7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedad1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 0x560dedad2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 0x560dedad2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:47158/0 (socket says 172.21.15.112:47158) 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7aea7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedad1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40692/0 (socket says 172.21.15.112:40692) 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7af2ffd640 1 -- 172.21.15.112:0/3970943579 learned_addr learned my addr 172.21.15.112:0/3970943579 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedaddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7af2ffd640 1 -- 172.21.15.112:0/3970943579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 msgr2=0x560dedad1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7af2ffd640 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedad1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7af2ffd640 1 -- 172.21.15.112:0/3970943579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 msgr2=0x560dedaddea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7af2ffd640 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedaddea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7af2ffd640 1 -- 172.21.15.112:0/3970943579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dedade460 con 0x560ded6fe670 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.320+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedad1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.321+0000 7f7af2ffd640 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 0x560dedad2510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40044c0 tx=0x7f7ae40044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e72d27b86e64a419 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.321+0000 7f7ada7fc640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.321+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3970943579 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4040030 con 0x560ded6fe670 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.321+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3970943579 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f7ae4005120 con 0x560ded6fe670 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.321+0000 7f7affc40740 1 -- 172.21.15.112:0/3970943579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae4004030 con 0x560ded6fe670 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.321+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3970943579 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ae4056050 con 0x560ded6fe670 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.321+0000 7f7affc40740 10 client.25915 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.321+0000 7f7affc40740 1 -- 172.21.15.112:0/3970943579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560dedae0d70 con 0x560ded6fe670 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.321+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3970943579 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ae4004030 con 0x560ded6fe670 2024-09-21T03:33:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.321+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3970943579 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f7ae4004d40 con 0x560ded6fe670 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.322+0000 7f7affc40740 10 client.25915 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.322+0000 7f7affc40740 10 client.25915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.322+0000 7f7affc40740 20 client.25915 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.322+0000 7f7affc40740 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560ded6ddcd0 0x560dedae40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.322+0000 7f7affc40740 4 client.25915 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.322+0000 7f7affc40740 1 -- 172.21.15.112:0/3970943579 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560deda3edb0 con 0x560ded6ddcd0 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.322+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560ded6ddcd0 0x560dedae40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.322+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560ded6ddcd0 0x560dedae40d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522865016b5e51ce server_cookie=77502720013271e1 in_seq=0 out_seq=0 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.322+0000 7f7ada7fc640 10 client.25915 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3970943579 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560deda3edb0 con 0x560ded6ddcd0 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7ada7fc640 10 client.25915 handle_command_reply: tid=0 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 1 client.25915 shutdown 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 1 -- 172.21.15.112:0/3970943579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 msgr2=0x560dedad2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 0x560dedad2510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40044c0 tx=0x7f7ae40044f0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 1 -- 172.21.15.112:0/3970943579 shutdown_connections 2024-09-21T03:33:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560ded6ddcd0 0x560dedae40d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedaddea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 0x560dedad2510 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7ada7fc640 0 client.25915 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 1 --2- 172.21.15.112:0/3970943579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedad1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 1 -- 172.21.15.112:0/3970943579 >> 172.21.15.112:0/3970943579 conn(0x560ded713740 msgr2=0x560dedac9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 1 -- 172.21.15.112:0/3970943579 shutdown_connections 2024-09-21T03:33:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 1 -- 172.21.15.112:0/3970943579 wait complete. 2024-09-21T03:33:34.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.323+0000 7f7affc40740 20 client.25915 trim_cache size 0 max 16384 2024-09-21T03:33:34.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.341+0000 7f7affc40740 1 Processor -- start 2024-09-21T03:33:34.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.341+0000 7f7affc40740 1 -- start start 2024-09-21T03:33:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.341+0000 7f7affc40740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedadde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.341+0000 7f7affc40740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 0x560dedacc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.341+0000 7f7affc40740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedacefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.341+0000 7f7affc40740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41b3ff0 con 0x560ded71f350 2024-09-21T03:33:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.341+0000 7f7affc40740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41b18c0 con 0x560ded6f6490 2024-09-21T03:33:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.341+0000 7f7affc40740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41af710 con 0x560ded6fe670 2024-09-21T03:33:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7aea7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedadde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7aea7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedadde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40698/0 (socket says 172.21.15.112:40698) 2024-09-21T03:33:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7aea7fc640 1 -- 172.21.15.112:0/4117548083 learned_addr learned my addr 172.21.15.112:0/4117548083 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:34.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7af2ffd640 1 --2- 172.21.15.112:0/4117548083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 0x560dedacc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/4117548083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedacefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7aea7fc640 1 -- 172.21.15.112:0/4117548083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 msgr2=0x560dedacc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/4117548083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 0x560dedacc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7aea7fc640 1 -- 172.21.15.112:0/4117548083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 msgr2=0x560dedacefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/4117548083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedacefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7aea7fc640 1 -- 172.21.15.112:0/4117548083 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dedac9290 con 0x560ded6f6490 2024-09-21T03:33:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/4117548083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedadde10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x560ded9c6d70 tx=0x7f7adc005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f23fcf47ad874bca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7ae8ff9640 1 -- 172.21.15.112:0/4117548083 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc028040 con 0x560ded6f6490 2024-09-21T03:33:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7ae8ff9640 1 -- 172.21.15.112:0/4117548083 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7adc003e70 con 0x560ded6f6490 2024-09-21T03:33:34.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.342+0000 7f7ae8ff9640 1 -- 172.21.15.112:0/4117548083 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc010d50 con 0x560ded6f6490 2024-09-21T03:33:34.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.343+0000 7f7affc40740 1 -- 172.21.15.112:0/4117548083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 msgr2=0x560dedadde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.343+0000 7f7affc40740 1 --2- 172.21.15.112:0/4117548083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedadde10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x560ded9c6d70 tx=0x7f7adc005850 comp rx=0 tx=0).stop 2024-09-21T03:33:34.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.343+0000 7f7affc40740 1 -- 172.21.15.112:0/4117548083 shutdown_connections 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.343+0000 7f7affc40740 1 --2- 172.21.15.112:0/4117548083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded71f350 0x560dedacefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.343+0000 7f7affc40740 1 --2- 172.21.15.112:0/4117548083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded6fe670 0x560dedacc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.343+0000 7f7affc40740 1 --2- 172.21.15.112:0/4117548083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6f6490 0x560dedadde10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.343+0000 7f7affc40740 1 -- 172.21.15.112:0/4117548083 >> 172.21.15.112:0/4117548083 conn(0x560ded713740 msgr2=0x560dedad0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.343+0000 7f7affc40740 1 -- 172.21.15.112:0/4117548083 shutdown_connections 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.343+0000 7f7affc40740 1 -- 172.21.15.112:0/4117548083 wait complete. 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7affc40740 1 -- start start 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7affc40740 1 Processor -- start 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7affc40740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded6f6490 0x560dedae5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7affc40740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6fe670 0x560dedae5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7affc40740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded71f350 0x560dedb4d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7affc40740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41afd80 con 0x560ded6f6490 2024-09-21T03:33:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7affc40740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41b11f0 con 0x560ded6fe670 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7affc40740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41b0450 con 0x560ded71f350 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7aea7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded6f6490 0x560dedae5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6fe670 0x560dedae5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7affc40740 10 client.0 _ll_register_callbacks cb 0x7f7adc0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6fe670 0x560dedae5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40704/0 (socket says 172.21.15.112:40704) 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7af2ffd640 1 -- 172.21.15.112:0/3381592219 learned_addr learned my addr 172.21.15.112:0/3381592219 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.344+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded71f350 0x560dedb4d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7aea7fc640 1 -- 172.21.15.112:0/3381592219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded71f350 msgr2=0x560dedb4d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded71f350 0x560dedb4d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7aea7fc640 1 -- 172.21.15.112:0/3381592219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6fe670 msgr2=0x560dedae5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6fe670 0x560dedae5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7aea7fc640 1 -- 172.21.15.112:0/3381592219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dedae6230 con 0x560ded6f6490 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7aea7fc640 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded6f6490 0x560dedae5420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009f70 tx=0x7f7adc003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637dfed43bd3d394 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:34.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7ada7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3381592219 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc028030 con 0x560ded6f6490 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7affc40740 1 -- 172.21.15.112:0/3381592219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7adc003e70 con 0x560ded6f6490 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3381592219 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f7adc013870 con 0x560ded6f6490 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3381592219 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7adc013a70 con 0x560ded6f6490 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7affc40740 10 client.16353 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7affc40740 1 -- 172.21.15.112:0/3381592219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560dedb4ee30 con 0x560ded6f6490 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.345+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3381592219 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7adc003e70 con 0x560ded6f6490 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.346+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3381592219 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f7adc02d4e0 con 0x560ded6f6490 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.346+0000 7f7affc40740 10 client.16353 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.346+0000 7f7affc40740 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.346+0000 7f7affc40740 20 client.16353 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.346+0000 7f7affc40740 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560ded6b3080 0x560dedb51510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:34.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.346+0000 7f7affc40740 4 client.16353 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:33:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.346+0000 7f7affc40740 1 -- 172.21.15.112:0/3381592219 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560deda3a860 con 0x560ded6b3080 2024-09-21T03:33:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.346+0000 7f7af2ffd640 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560ded6b3080 0x560dedb51510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.346+0000 7f7af2ffd640 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560ded6b3080 0x560dedb51510 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b364815cc942c116 server_cookie=c10d2d488bfe86f8 in_seq=0 out_seq=0 2024-09-21T03:33:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.346+0000 7f7ada7fc640 10 client.16353 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.348+0000 7f7ada7fc640 1 -- 172.21.15.112:0/3381592219 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560deda3a860 con 0x560ded6b3080 2024-09-21T03:33:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.348+0000 7f7ada7fc640 10 client.16353 handle_command_reply: tid=0 2024-09-21T03:33:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.348+0000 7f7affc40740 1 client.16353 shutdown 2024-09-21T03:33:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.348+0000 7f7affc40740 1 -- 172.21.15.112:0/3381592219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded6f6490 msgr2=0x560dedae5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.348+0000 7f7affc40740 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded6f6490 0x560dedae5420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009f70 tx=0x7f7adc003b20 comp rx=0 tx=0).stop 2024-09-21T03:33:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7affc40740 1 -- 172.21.15.112:0/3381592219 shutdown_connections 2024-09-21T03:33:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7affc40740 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560ded6b3080 0x560dedb51510 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7affc40740 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560ded71f350 0x560dedb4d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7affc40740 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ded6fe670 0x560dedae5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7ada7fc640 0 client.16353 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7affc40740 1 --2- 172.21.15.112:0/3381592219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560ded6f6490 0x560dedae5420 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7affc40740 1 -- 172.21.15.112:0/3381592219 >> 172.21.15.112:0/3381592219 conn(0x560ded713740 msgr2=0x560dedac7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7affc40740 1 -- 172.21.15.112:0/3381592219 shutdown_connections 2024-09-21T03:33:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7affc40740 1 -- 172.21.15.112:0/3381592219 wait complete. 2024-09-21T03:33:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7affc40740 20 client.16353 trim_cache size 0 max 16384 2024-09-21T03:33:34.354 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:33:34.354 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:33:34.354 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "b1f22425-3f0c-43b0-aebe-8cd8df2d8753", 2024-09-21T03:33:34.355 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:33:34.355 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:33:34.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7af96cc640 1 -- 172.21.15.112:0/2209700573 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7ae00b6680 msgr2=0x7f7ae00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7af96cc640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7ae00b6680 0x7f7ae00b8b90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0e5580 tx=0x7f7aec005870 comp rx=0 tx=0).stop 2024-09-21T03:33:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7af96cc640 1 -- 172.21.15.112:0/2209700573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7af4109e40 msgr2=0x7f7af4076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7af96cc640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7af4109e40 0x7f7af4076190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009fd0 tx=0x7f7aec00ef10 comp rx=0 tx=0).stop 2024-09-21T03:33:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7af96cc640 1 -- 172.21.15.112:0/2209700573 shutdown_connections 2024-09-21T03:33:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7af96cc640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7ae00b6680 0x7f7ae00b8b90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7af96cc640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7af4109e40 0x7f7af4076190 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7af96cc640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7af41094f0 0x7f7af4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7af96cc640 1 --2- 172.21.15.112:0/2209700573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af4103bb0 0x7f7af4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.349+0000 7f7af96cc640 1 -- 172.21.15.112:0/2209700573 >> 172.21.15.112:0/2209700573 conn(0x7f7af40ff600 msgr2=0x7f7af4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.350+0000 7f7af96cc640 1 -- 172.21.15.112:0/2209700573 shutdown_connections 2024-09-21T03:33:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:34.350+0000 7f7af96cc640 1 -- 172.21.15.112:0/2209700573 wait complete. 2024-09-21T03:33:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:34 smithi112 ceph-mon[31442]: pgmap v455: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 249 op/s 2024-09-21T03:33:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:34 smithi203 ceph-mon[34723]: pgmap v455: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 249 op/s 2024-09-21T03:33:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:34 smithi187 ceph-mon[34603]: pgmap v455: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 249 op/s 2024-09-21T03:33:35.173 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:33:35.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 447611 11.27 MB/sec execute 95 sec latency 33.597 ms 2024-09-21T03:33:35.463 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:33:35.775 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:35 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:33:35.776 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:35 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:35.776 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:35 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:33:35.776 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:35 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:33:35.776 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:35 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:33:35.776 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:35 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:35.776 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:35 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:35.776 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:35 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:35.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:35 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:33:35.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:35 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:35.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:35 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:33:35.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:35 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:33:35.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:35 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:33:35.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:35 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:35.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:35 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:35.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:35 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:35.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:35 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:33:35.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:35 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:35.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:35 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:33:35.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:35 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:33:35.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:35 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:33:35.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:35 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:35.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:35 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:35.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:35 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:33:36.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.997+0000 7f5c65953640 1 -- 172.21.15.112:0/3119009079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60107b30 msgr2=0x7f5c60107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.997+0000 7f5c65953640 1 --2- 172.21.15.112:0/3119009079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60107b30 0x7f5c60107f40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009900 tx=0x7f5c4802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.997+0000 7f5c65953640 1 -- 172.21.15.112:0/3119009079 shutdown_connections 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.997+0000 7f5c65953640 1 --2- 172.21.15.112:0/3119009079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c60108480 0x7f5c6010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.997+0000 7f5c65953640 1 --2- 172.21.15.112:0/3119009079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60107b30 0x7f5c60107f40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.997+0000 7f5c65953640 1 --2- 172.21.15.112:0/3119009079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c601021f0 0x7f5c60102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.997+0000 7f5c65953640 1 -- 172.21.15.112:0/3119009079 >> 172.21.15.112:0/3119009079 conn(0x7f5c60077c60 msgr2=0x7f5c600780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.997+0000 7f5c65953640 1 -- 172.21.15.112:0/3119009079 shutdown_connections 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.997+0000 7f5c65953640 1 -- 172.21.15.112:0/3119009079 wait complete. 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c65953640 1 Processor -- start 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c65953640 1 -- start start 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c65953640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c601021f0 0x7f5c6019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c65953640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60107b30 0x7f5c6019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c65953640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c60108480 0x7f5c6019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c65953640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c601129c0 con 0x7f5c60107b30 2024-09-21T03:33:36.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c65953640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5c60112840 con 0x7f5c60108480 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c65953640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c60112b40 con 0x7f5c601021f0 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60107b30 0x7f5c6019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60107b30 0x7f5c6019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41066/0 (socket says 172.21.15.112:41066) 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c567fc640 1 -- 172.21.15.112:0/3764396249 learned_addr learned my addr 172.21.15.112:0/3764396249 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c5effd640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c601021f0 0x7f5c6019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c5f7fe640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c60108480 0x7f5c6019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c567fc640 1 -- 172.21.15.112:0/3764396249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c601021f0 msgr2=0x7f5c6019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c567fc640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c601021f0 0x7f5c6019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c567fc640 1 -- 172.21.15.112:0/3764396249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c60108480 msgr2=0x7f5c6019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c567fc640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c60108480 0x7f5c6019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c567fc640 1 -- 172.21.15.112:0/3764396249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c601a0430 con 0x7f5c60107b30 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.998+0000 7f5c5f7fe640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c60108480 0x7f5c6019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.999+0000 7f5c567fc640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60107b30 0x7f5c6019aa60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006010 tx=0x7f5c48004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:36.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.999+0000 7f5c64951640 1 -- 172.21.15.112:0/3764396249 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4803f030 con 0x7f5c60107b30 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.999+0000 7f5c65953640 1 -- 172.21.15.112:0/3764396249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c601a06c0 con 0x7f5c60107b30 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.999+0000 7f5c64951640 1 -- 172.21.15.112:0/3764396249 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c48003aa0 con 0x7f5c60107b30 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.999+0000 7f5c64951640 1 -- 172.21.15.112:0/3764396249 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48035070 con 0x7f5c60107b30 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:35.999+0000 7f5c65953640 1 -- 172.21.15.112:0/3764396249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c601a6e80 con 0x7f5c60107b30 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.001+0000 7f5c6bec7740 1 Processor -- start 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.001+0000 7f5c6bec7740 1 -- start start 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113261070 0x561113608270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611132381b0 0x5611136087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561113240390 0x561113608cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56111357ee50 con 0x5611132381b0 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56111356e280 con 0x561113261070 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561113572090 con 0x561113240390 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c5effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611132381b0 0x5611136087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c5effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611132381b0 0x5611136087b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41072/0 (socket says 172.21.15.112:41072) 2024-09-21T03:33:36.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c5effd640 1 -- 172.21.15.112:0/1841147902 learned_addr learned my addr 172.21.15.112:0/1841147902 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c64951640 1 -- 172.21.15.112:0/3764396249 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f5c48042030 con 0x7f5c60107b30 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c5f7fe640 1 --2- 172.21.15.112:0/1841147902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113261070 0x561113608270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c567fc640 1 --2- 172.21.15.112:0/1841147902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561113240390 0x561113608cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c5effd640 1 -- 172.21.15.112:0/1841147902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561113240390 msgr2=0x561113608cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c5effd640 1 --2- 172.21.15.112:0/1841147902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561113240390 0x561113608cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c5effd640 1 -- 172.21.15.112:0/1841147902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113261070 msgr2=0x561113608270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c5effd640 1 --2- 172.21.15.112:0/1841147902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113261070 0x561113608270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.002+0000 7f5c5effd640 1 -- 172.21.15.112:0/1841147902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611132cc650 con 0x5611132381b0 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.003+0000 7f5c567fc640 1 --2- 172.21.15.112:0/1841147902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561113240390 0x561113608cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.003+0000 7f5c5f7fe640 1 --2- 172.21.15.112:0/1841147902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113261070 0x561113608270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.003+0000 7f5c5effd640 1 --2- 172.21.15.112:0/1841147902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611132381b0 0x5611136087b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000bd10 tx=0x7f5c500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797ab3f4678fe707 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.003+0000 7f5c55ffb640 1 -- 172.21.15.112:0/1841147902 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50004030 con 0x5611132381b0 2024-09-21T03:33:36.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.003+0000 7f5c55ffb640 1 -- 172.21.15.112:0/1841147902 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c500035f0 con 0x5611132381b0 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.003+0000 7f5c55ffb640 1 -- 172.21.15.112:0/1841147902 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c500116a0 con 0x5611132381b0 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.003+0000 7f5c64951640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5c4c0b6680 0x7f5c4c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.003+0000 7f5c5f7fe640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5c4c0b6680 0x7f5c4c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.003+0000 7f5c64951640 1 -- 172.21.15.112:0/3764396249 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f5c48107b90 con 0x7f5c60107b30 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.004+0000 7f5c6bec7740 1 -- 172.21.15.112:0/1841147902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611132381b0 msgr2=0x5611136087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.004+0000 7f5c5f7fe640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5c4c0b6680 0x7f5c4c0b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5c58005fd0 tx=0x7f5c58009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.004+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/1841147902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611132381b0 0x5611136087b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000bd10 tx=0x7f5c500107a0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.004+0000 7f5c6bec7740 1 -- 172.21.15.112:0/1841147902 shutdown_connections 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.004+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/1841147902 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561113240390 0x561113608cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.004+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/1841147902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5611132381b0 0x5611136087b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.004+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/1841147902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113261070 0x561113608270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.004+0000 7f5c6bec7740 1 -- 172.21.15.112:0/1841147902 >> 172.21.15.112:0/1841147902 conn(0x561113255460 msgr2=0x561113606160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.004+0000 7f5c6bec7740 1 -- 172.21.15.112:0/1841147902 shutdown_connections 2024-09-21T03:33:36.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.004+0000 7f5c6bec7740 1 -- 172.21.15.112:0/1841147902 wait complete. 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c6bec7740 1 -- start start 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c6bec7740 1 Processor -- start 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x561113617cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113618230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561113261070 0x5611136187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5611133cf700 con 0x561113261070 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561113509110 con 0x561113240390 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56111361adb0 con 0x5611132381b0 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c6bec7740 10 client.0 _ll_register_callbacks cb 0x561113522990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c5f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561113261070 0x5611136187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c5effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x561113617cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c5f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561113261070 0x5611136187b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41086/0 (socket says 172.21.15.112:41086) 2024-09-21T03:33:36.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c5f7fe640 1 -- 172.21.15.112:0/929008940 learned_addr learned my addr 172.21.15.112:0/929008940 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.005+0000 7f5c567fc640 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113618230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c5effd640 1 -- 172.21.15.112:0/929008940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 msgr2=0x561113618230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c5effd640 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113618230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c5effd640 1 -- 172.21.15.112:0/929008940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561113261070 msgr2=0x5611136187b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c5effd640 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561113261070 0x5611136187b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c5effd640 1 -- 172.21.15.112:0/929008940 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561113536c40 con 0x5611132381b0 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c5f7fe640 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561113261070 0x5611136187b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c5effd640 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x561113617cf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5c50003f10 tx=0x7f5c50003e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a07d0df9624f26c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c3effd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c3effd640 1 -- 172.21.15.112:0/929008940 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50029060 con 0x5611132381b0 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c6bec7740 1 -- 172.21.15.112:0/929008940 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c500035f0 con 0x5611132381b0 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c3effd640 1 -- 172.21.15.112:0/929008940 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f5c50003af0 con 0x5611132381b0 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c3effd640 1 -- 172.21.15.112:0/929008940 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c50003f70 con 0x5611132381b0 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c6bec7740 10 client.25930 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:36.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.006+0000 7f5c6bec7740 1 -- 172.21.15.112:0/929008940 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561113626a00 con 0x5611132381b0 2024-09-21T03:33:36.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.007+0000 7f5c3effd640 1 -- 172.21.15.112:0/929008940 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c500035f0 con 0x5611132381b0 2024-09-21T03:33:36.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.007+0000 7f5c3effd640 1 -- 172.21.15.112:0/929008940 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f5c500047c0 con 0x5611132381b0 2024-09-21T03:33:36.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.007+0000 7f5c6bec7740 10 client.25930 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:36.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.007+0000 7f5c6bec7740 10 client.25930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:33:36.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.007+0000 7f5c6bec7740 20 client.25930 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:36.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.007+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56111321f9f0 0x561113629dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.007+0000 7f5c6bec7740 4 client.25930 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:33:36.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.007+0000 7f5c6bec7740 1 -- 172.21.15.112:0/929008940 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561113580da0 con 0x56111321f9f0 2024-09-21T03:33:36.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.007+0000 7f5c567fc640 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56111321f9f0 0x561113629dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.008+0000 7f5c567fc640 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56111321f9f0 0x561113629dc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3b74e4ca0107cbc server_cookie=2e0e03467bbafa2 in_seq=0 out_seq=0 2024-09-21T03:33:36.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.008+0000 7f5c3effd640 10 client.25930 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:36.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.008+0000 7f5c3effd640 1 -- 172.21.15.112:0/929008940 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561113580da0 con 0x56111321f9f0 2024-09-21T03:33:36.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.008+0000 7f5c3effd640 10 client.25930 handle_command_reply: tid=0 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.008+0000 7f5c6bec7740 1 client.25930 shutdown 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c6bec7740 1 -- 172.21.15.112:0/929008940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 msgr2=0x561113617cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x561113617cf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5c50003f10 tx=0x7f5c50003e20 comp rx=0 tx=0).stop 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c6bec7740 1 -- 172.21.15.112:0/929008940 shutdown_connections 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56111321f9f0 0x561113629dc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561113261070 0x5611136187b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c3effd640 0 client.25930 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113618230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/929008940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x561113617cf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c6bec7740 1 -- 172.21.15.112:0/929008940 >> 172.21.15.112:0/929008940 conn(0x561113255460 msgr2=0x561113609c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c6bec7740 1 -- 172.21.15.112:0/929008940 shutdown_connections 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c6bec7740 1 -- 172.21.15.112:0/929008940 wait complete. 2024-09-21T03:33:36.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.009+0000 7f5c6bec7740 20 client.25930 trim_cache size 0 max 16384 2024-09-21T03:33:36.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.028+0000 7f5c6bec7740 1 Processor -- start 2024-09-21T03:33:36.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.028+0000 7f5c6bec7740 1 -- start start 2024-09-21T03:33:36.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136134a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x5611136099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113614ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c601afd20 con 0x56111321f9f0 2024-09-21T03:33:36.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5c601ad5f0 con 0x561113240390 2024-09-21T03:33:36.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c601ab440 con 0x5611132381b0 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c5effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c5effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136134a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41092/0 (socket says 172.21.15.112:41092) 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c5effd640 1 -- 172.21.15.112:0/3375157977 learned_addr learned my addr 172.21.15.112:0/3375157977 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c567fc640 1 --2- 172.21.15.112:0/3375157977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x5611136099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c5f7fe640 1 --2- 172.21.15.112:0/3375157977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113614ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c5effd640 1 -- 172.21.15.112:0/3375157977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 msgr2=0x5611136099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c5effd640 1 --2- 172.21.15.112:0/3375157977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x5611136099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c5effd640 1 -- 172.21.15.112:0/3375157977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 msgr2=0x561113614ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c5effd640 1 --2- 172.21.15.112:0/3375157977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113614ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c5effd640 1 -- 172.21.15.112:0/3375157977 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611136090b0 con 0x56111321f9f0 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.029+0000 7f5c5effd640 1 --2- 172.21.15.112:0/3375157977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136134a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5c50001eb0 tx=0x7f5c5000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ae52df7acea4b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.030+0000 7f5c557fa640 1 -- 172.21.15.112:0/3375157977 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50029070 con 0x56111321f9f0 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.030+0000 7f5c557fa640 1 -- 172.21.15.112:0/3375157977 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c500035f0 con 0x56111321f9f0 2024-09-21T03:33:36.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.030+0000 7f5c557fa640 1 -- 172.21.15.112:0/3375157977 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50005060 con 0x56111321f9f0 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.030+0000 7f5c6bec7740 1 -- 172.21.15.112:0/3375157977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 msgr2=0x5611136134a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.030+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/3375157977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136134a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5c50001eb0 tx=0x7f5c5000b390 comp rx=0 tx=0).stop 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.030+0000 7f5c6bec7740 1 -- 172.21.15.112:0/3375157977 shutdown_connections 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.031+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/3375157977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113614ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.031+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/3375157977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x5611136099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.031+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/3375157977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136134a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.031+0000 7f5c6bec7740 1 -- 172.21.15.112:0/3375157977 >> 172.21.15.112:0/3375157977 conn(0x561113255460 msgr2=0x5611136092a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.031+0000 7f5c6bec7740 1 -- 172.21.15.112:0/3375157977 shutdown_connections 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.031+0000 7f5c6bec7740 1 -- 172.21.15.112:0/3375157977 wait complete. 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.031+0000 7f5c6bec7740 1 -- start start 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.031+0000 7f5c6bec7740 1 Processor -- start 2024-09-21T03:33:36.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136884d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x561113688a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c6bec7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113693550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c601abab0 con 0x56111321f9f0 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5c601acf20 con 0x561113240390 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c6bec7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c601ac180 con 0x5611132381b0 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c6bec7740 10 client.0 _ll_register_callbacks cb 0x7f5c580099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c5effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136884d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x561113688a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c5effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136884d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41096/0 (socket says 172.21.15.112:41096) 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c5effd640 1 -- 172.21.15.112:0/2440807618 learned_addr learned my addr 172.21.15.112:0/2440807618 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x561113688a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:34068/0 (socket says 172.21.15.112:34068) 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c5f7fe640 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113693550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c5effd640 1 -- 172.21.15.112:0/2440807618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 msgr2=0x561113688a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c5effd640 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x561113688a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c5effd640 1 -- 172.21.15.112:0/2440807618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 msgr2=0x561113693550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:33:36.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c5effd640 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113693550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c5effd640 1 -- 172.21.15.112:0/2440807618 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561113693a90 con 0x56111321f9f0 2024-09-21T03:33:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c5f7fe640 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113693550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:33:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.032+0000 7f5c567fc640 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x561113688a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:33:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.033+0000 7f5c5effd640 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136884d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000b4e0 tx=0x7f5c5002eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb530fba474a023 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:33:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.033+0000 7f5c3effd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:33:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.033+0000 7f5c3effd640 1 -- 172.21.15.112:0/2440807618 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50029040 con 0x56111321f9f0 2024-09-21T03:33:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.033+0000 7f5c3effd640 1 -- 172.21.15.112:0/2440807618 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f5c50004e00 con 0x56111321f9f0 2024-09-21T03:33:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.033+0000 7f5c6bec7740 1 -- 172.21.15.112:0/2440807618 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c500035f0 con 0x56111321f9f0 2024-09-21T03:33:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.033+0000 7f5c3effd640 1 -- 172.21.15.112:0/2440807618 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c50047050 con 0x56111321f9f0 2024-09-21T03:33:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.033+0000 7f5c6bec7740 10 client.16371 fetch_fsmap learned FSMap version 28 2024-09-21T03:33:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.033+0000 7f5c6bec7740 1 -- 172.21.15.112:0/2440807618 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561113694b90 con 0x56111321f9f0 2024-09-21T03:33:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.033+0000 7f5c3effd640 1 -- 172.21.15.112:0/2440807618 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c500035f0 con 0x56111321f9f0 2024-09-21T03:33:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.034+0000 7f5c3effd640 1 -- 172.21.15.112:0/2440807618 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f5c500044c0 con 0x56111321f9f0 2024-09-21T03:33:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.034+0000 7f5c6bec7740 10 client.16371 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T03:33:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.034+0000 7f5c6bec7740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:33:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.034+0000 7f5c6bec7740 20 client.16371 populate_metadata read hostname 'smithi112' 2024-09-21T03:33:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.034+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561113261070 0x561113697230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:33:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.034+0000 7f5c6bec7740 4 client.16371 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:33:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.034+0000 7f5c6bec7740 1 -- 172.21.15.112:0/2440807618 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56111357c850 con 0x561113261070 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.034+0000 7f5c567fc640 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561113261070 0x561113697230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.034+0000 7f5c567fc640 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561113261070 0x561113697230 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8a5e73785519e6 server_cookie=bef59c05e1d8fccd in_seq=0 out_seq=0 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.034+0000 7f5c3effd640 10 client.16371 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c3effd640 1 -- 172.21.15.112:0/2440807618 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x56111357c850 con 0x561113261070 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c3effd640 10 client.16371 handle_command_reply: tid=0 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 1 client.16371 shutdown 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 1 -- 172.21.15.112:0/2440807618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 msgr2=0x5611136884d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136884d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000b4e0 tx=0x7f5c5002eda0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 1 -- 172.21.15.112:0/2440807618 shutdown_connections 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561113261070 0x561113697230 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561113240390 0x561113693550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611132381b0 0x561113688a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 1 --2- 172.21.15.112:0/2440807618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56111321f9f0 0x5611136884d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c3effd640 0 client.16371 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:33:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 1 -- 172.21.15.112:0/2440807618 >> 172.21.15.112:0/2440807618 conn(0x561113255460 msgr2=0x561113619100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:36.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 1 -- 172.21.15.112:0/2440807618 shutdown_connections 2024-09-21T03:33:36.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 1 -- 172.21.15.112:0/2440807618 wait complete. 2024-09-21T03:33:36.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.036+0000 7f5c6bec7740 20 client.16371 trim_cache size 0 max 16384 2024-09-21T03:33:36.042 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:33:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:33:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:33:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:33:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "b1f22425-3f0c-43b0-aebe-8cd8df2d8753", 2024-09-21T03:33:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:33:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:33:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:33:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:33:36.048 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:33:36.048 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:33:36.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:33:36.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:33:36.048 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:33:36.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 -- 172.21.15.112:0/3764396249 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5c4c0b6680 msgr2=0x7f5c4c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5c4c0b6680 0x7f5c4c0b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5c58005fd0 tx=0x7f5c58009500 comp rx=0 tx=0).stop 2024-09-21T03:33:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 -- 172.21.15.112:0/3764396249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60107b30 msgr2=0x7f5c6019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:33:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60107b30 0x7f5c6019aa60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006010 tx=0x7f5c48004060 comp rx=0 tx=0).stop 2024-09-21T03:33:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 -- 172.21.15.112:0/3764396249 shutdown_connections 2024-09-21T03:33:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f5c4c0b6680 0x7f5c4c0b8b90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c60108480 0x7f5c6019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c60107b30 0x7f5c6019aa60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 --2- 172.21.15.112:0/3764396249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c601021f0 0x7f5c6019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:33:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 -- 172.21.15.112:0/3764396249 >> 172.21.15.112:0/3764396249 conn(0x7f5c60077c60 msgr2=0x7f5c6010fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:33:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 -- 172.21.15.112:0/3764396249 shutdown_connections 2024-09-21T03:33:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:33:36.037+0000 7f5c65953640 1 -- 172.21.15.112:0/3764396249 wait complete. 2024-09-21T03:33:36.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 450321 11.30 MB/sec execute 96 sec latency 46.577 ms 2024-09-21T03:33:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:36 smithi112 ceph-mon[31442]: pgmap v456: 97 pgs: 97 active+clean; 271 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 346 op/s 2024-09-21T03:33:36.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:36 smithi203 ceph-mon[34723]: pgmap v456: 97 pgs: 97 active+clean; 271 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 346 op/s 2024-09-21T03:33:36.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:36 smithi187 ceph-mon[34603]: pgmap v456: 97 pgs: 97 active+clean; 271 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 346 op/s 2024-09-21T03:33:36.981 INFO:tasks.cephfs.filesystem:scrub status for tag:b1f22425-3f0c-43b0-aebe-8cd8df2d8753 - {'path': '/', 'tag': 'b1f22425-3f0c-43b0-aebe-8cd8df2d8753', 'options': 'recursive,force'} 2024-09-21T03:33:37.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 451905 11.29 MB/sec execute 97 sec latency 13.140 ms 2024-09-21T03:33:38.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 454431 11.32 MB/sec execute 98 sec latency 9.944 ms 2024-09-21T03:33:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:38 smithi112 ceph-mon[31442]: pgmap v457: 97 pgs: 97 active+clean; 271 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 246 op/s 2024-09-21T03:33:38.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:38 smithi203 ceph-mon[34723]: pgmap v457: 97 pgs: 97 active+clean; 271 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 246 op/s 2024-09-21T03:33:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:38 smithi187 ceph-mon[34603]: pgmap v457: 97 pgs: 97 active+clean; 271 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 246 op/s 2024-09-21T03:33:39.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 456629 11.28 MB/sec execute 99 sec latency 16.293 ms 2024-09-21T03:33:40.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 458344 11.28 MB/sec execute 100 sec latency 18.211 ms 2024-09-21T03:33:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:40 smithi112 ceph-mon[31442]: pgmap v458: 97 pgs: 97 active+clean; 276 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 305 op/s 2024-09-21T03:33:40.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:40 smithi203 ceph-mon[34723]: pgmap v458: 97 pgs: 97 active+clean; 276 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 305 op/s 2024-09-21T03:33:40.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:40 smithi187 ceph-mon[34603]: pgmap v458: 97 pgs: 97 active+clean; 276 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 305 op/s 2024-09-21T03:33:41.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 459772 11.43 MB/sec execute 101 sec latency 73.909 ms 2024-09-21T03:33:42.233 INFO:tasks.workunit.client.0.smithi112.stdout: 1 462272 11.45 MB/sec execute 102 sec latency 11.442 ms 2024-09-21T03:33:43.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 462499 11.34 MB/sec execute 103 sec latency 759.967 ms 2024-09-21T03:33:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:43 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:33:43.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:43 smithi203 ceph-mon[34723]: pgmap v459: 97 pgs: 97 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 288 op/s 2024-09-21T03:33:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:43 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:33:43.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:43 smithi187 ceph-mon[34603]: pgmap v459: 97 pgs: 97 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 288 op/s 2024-09-21T03:33:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:43 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:33:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:43 smithi112 ceph-mon[31442]: pgmap v459: 97 pgs: 97 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 288 op/s 2024-09-21T03:33:44.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 462499 11.23 MB/sec execute 104 sec latency 1760.045 ms 2024-09-21T03:33:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:44 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:33:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:44 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:33:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:44 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:33:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:44 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:33:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:44 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:33:44.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:44 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:33:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:44 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:33:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:44 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:33:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:44 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:33:45.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 462499 11.13 MB/sec execute 105 sec latency 2760.173 ms 2024-09-21T03:33:45.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:45 smithi203 ceph-mon[34723]: pgmap v460: 97 pgs: 97 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-21T03:33:45.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:45 smithi187 ceph-mon[34603]: pgmap v460: 97 pgs: 97 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-21T03:33:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:45 smithi112 ceph-mon[31442]: pgmap v460: 97 pgs: 97 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-21T03:33:46.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 462499 11.02 MB/sec execute 106 sec latency 3760.274 ms 2024-09-21T03:33:46.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:46 smithi112 ceph-mon[31442]: pgmap v461: 97 pgs: 97 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-09-21T03:33:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:46 smithi203 ceph-mon[34723]: pgmap v461: 97 pgs: 97 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-09-21T03:33:46.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:46 smithi187 ceph-mon[34603]: pgmap v461: 97 pgs: 97 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 296 op/s 2024-09-21T03:33:47.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 462499 10.92 MB/sec execute 107 sec latency 4760.376 ms 2024-09-21T03:33:48.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 464923 10.93 MB/sec execute 108 sec latency 4838.188 ms 2024-09-21T03:33:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:48 smithi112 ceph-mon[31442]: pgmap v462: 97 pgs: 97 active+clean; 281 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 195 op/s 2024-09-21T03:33:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:48 smithi203 ceph-mon[34723]: pgmap v462: 97 pgs: 97 active+clean; 281 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 195 op/s 2024-09-21T03:33:48.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:48 smithi187 ceph-mon[34603]: pgmap v462: 97 pgs: 97 active+clean; 281 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 195 op/s 2024-09-21T03:33:49.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 466538 10.91 MB/sec execute 109 sec latency 13.970 ms 2024-09-21T03:33:50.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 469274 10.97 MB/sec execute 110 sec latency 13.450 ms 2024-09-21T03:33:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:50 smithi112 ceph-mon[31442]: pgmap v463: 97 pgs: 97 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 227 op/s 2024-09-21T03:33:50.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:50 smithi203 ceph-mon[34723]: pgmap v463: 97 pgs: 97 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 227 op/s 2024-09-21T03:33:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:50 smithi187 ceph-mon[34603]: pgmap v463: 97 pgs: 97 active+clean; 283 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 227 op/s 2024-09-21T03:33:51.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 470855 10.92 MB/sec execute 111 sec latency 72.474 ms 2024-09-21T03:33:52.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 473021 10.94 MB/sec execute 112 sec latency 17.831 ms 2024-09-21T03:33:52.789 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:52 smithi112 ceph-mon[31442]: pgmap v464: 97 pgs: 97 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.9 MiB/s wr, 206 op/s 2024-09-21T03:33:52.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:52 smithi203 ceph-mon[34723]: pgmap v464: 97 pgs: 97 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.9 MiB/s wr, 206 op/s 2024-09-21T03:33:52.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:52 smithi187 ceph-mon[34603]: pgmap v464: 97 pgs: 97 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.9 MiB/s wr, 206 op/s 2024-09-21T03:33:53.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 475560 10.96 MB/sec execute 113 sec latency 31.818 ms 2024-09-21T03:33:54.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 477473 10.96 MB/sec execute 114 sec latency 17.465 ms 2024-09-21T03:33:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:54 smithi112 ceph-mon[31442]: pgmap v465: 97 pgs: 97 active+clean; 285 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 2.5 MiB/s wr, 168 op/s 2024-09-21T03:33:54.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:54 smithi203 ceph-mon[34723]: pgmap v465: 97 pgs: 97 active+clean; 285 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 2.5 MiB/s wr, 168 op/s 2024-09-21T03:33:54.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:54 smithi187 ceph-mon[34603]: pgmap v465: 97 pgs: 97 active+clean; 285 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 2.5 MiB/s wr, 168 op/s 2024-09-21T03:33:55.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 479991 10.99 MB/sec execute 115 sec latency 15.154 ms 2024-09-21T03:33:56.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 481852 10.95 MB/sec execute 116 sec latency 44.925 ms 2024-09-21T03:33:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:56 smithi112 ceph-mon[31442]: pgmap v466: 97 pgs: 97 active+clean; 292 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 273 op/s 2024-09-21T03:33:56.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:56 smithi203 ceph-mon[34723]: pgmap v466: 97 pgs: 97 active+clean; 292 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 273 op/s 2024-09-21T03:33:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:56 smithi187 ceph-mon[34603]: pgmap v466: 97 pgs: 97 active+clean; 292 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 273 op/s 2024-09-21T03:33:57.234 INFO:tasks.workunit.client.0.smithi112.stdout: 1 483811 10.97 MB/sec execute 117 sec latency 44.934 ms 2024-09-21T03:33:58.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 486588 11.00 MB/sec execute 118 sec latency 30.172 ms 2024-09-21T03:33:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:33:58 smithi112 ceph-mon[31442]: pgmap v467: 97 pgs: 97 active+clean; 293 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-09-21T03:33:58.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:33:58 smithi203 ceph-mon[34723]: pgmap v467: 97 pgs: 97 active+clean; 293 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-09-21T03:33:58.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:33:58 smithi187 ceph-mon[34603]: pgmap v467: 97 pgs: 97 active+clean; 293 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-09-21T03:33:59.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 488186 10.99 MB/sec execute 119 sec latency 18.587 ms 2024-09-21T03:34:00.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 490634 11.01 MB/sec execute 120 sec latency 12.102 ms 2024-09-21T03:34:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:00 smithi112 ceph-mon[31442]: pgmap v468: 97 pgs: 97 active+clean; 295 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-09-21T03:34:00.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:00 smithi203 ceph-mon[34723]: pgmap v468: 97 pgs: 97 active+clean; 295 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-09-21T03:34:00.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:00 smithi187 ceph-mon[34603]: pgmap v468: 97 pgs: 97 active+clean; 295 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-09-21T03:34:01.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 491693 10.96 MB/sec execute 121 sec latency 58.636 ms 2024-09-21T03:34:02.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 494096 10.99 MB/sec execute 122 sec latency 20.041 ms 2024-09-21T03:34:02.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:02 smithi112 ceph-mon[31442]: pgmap v469: 97 pgs: 97 active+clean; 298 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-09-21T03:34:02.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:02 smithi203 ceph-mon[34723]: pgmap v469: 97 pgs: 97 active+clean; 298 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-09-21T03:34:02.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:02 smithi187 ceph-mon[34603]: pgmap v469: 97 pgs: 97 active+clean; 298 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-09-21T03:34:03.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 495769 10.94 MB/sec execute 123 sec latency 17.293 ms 2024-09-21T03:34:04.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 497993 10.96 MB/sec execute 124 sec latency 12.663 ms 2024-09-21T03:34:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:04 smithi112 ceph-mon[31442]: pgmap v470: 97 pgs: 97 active+clean; 300 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-09-21T03:34:04.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:04 smithi203 ceph-mon[34723]: pgmap v470: 97 pgs: 97 active+clean; 300 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-09-21T03:34:04.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:04 smithi187 ceph-mon[34603]: pgmap v470: 97 pgs: 97 active+clean; 300 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-09-21T03:34:05.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 500424 10.98 MB/sec execute 125 sec latency 19.143 ms 2024-09-21T03:34:06.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 501921 10.95 MB/sec execute 126 sec latency 74.638 ms 2024-09-21T03:34:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:06 smithi112 ceph-mon[31442]: pgmap v471: 97 pgs: 97 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 380 op/s 2024-09-21T03:34:06.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:06 smithi203 ceph-mon[34723]: pgmap v471: 97 pgs: 97 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 380 op/s 2024-09-21T03:34:06.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:06 smithi187 ceph-mon[34603]: pgmap v471: 97 pgs: 97 active+clean; 308 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 380 op/s 2024-09-21T03:34:06.983 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:34:07.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 504606 11.00 MB/sec execute 127 sec latency 18.877 ms 2024-09-21T03:34:07.255 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:07.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.770+0000 7f91def15640 1 -- 172.21.15.112:0/1421969381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d81083d0 msgr2=0x7f91d81087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.770+0000 7f91def15640 1 --2- 172.21.15.112:0/1421969381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d81083d0 0x7f91d81087e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009ca0 tx=0x7f91c802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.770+0000 7f91def15640 1 -- 172.21.15.112:0/1421969381 shutdown_connections 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.770+0000 7f91def15640 1 --2- 172.21.15.112:0/1421969381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d8108d20 0x7f91d810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.770+0000 7f91def15640 1 --2- 172.21.15.112:0/1421969381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d81083d0 0x7f91d81087e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.770+0000 7f91def15640 1 --2- 172.21.15.112:0/1421969381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d8102ae0 0x7f91d8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.770+0000 7f91def15640 1 -- 172.21.15.112:0/1421969381 >> 172.21.15.112:0/1421969381 conn(0x7f91d8079760 msgr2=0x7f91d8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.770+0000 7f91def15640 1 -- 172.21.15.112:0/1421969381 shutdown_connections 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.770+0000 7f91def15640 1 -- 172.21.15.112:0/1421969381 wait complete. 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.771+0000 7f91def15640 1 Processor -- start 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.771+0000 7f91def15640 1 -- start start 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.771+0000 7f91def15640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d8102ae0 0x7f91d819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.771+0000 7f91def15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d81083d0 0x7f91d819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.771+0000 7f91def15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d8108d20 0x7f91d819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.771+0000 7f91def15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91d8113ab0 con 0x7f91d81083d0 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.771+0000 7f91def15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f91d8113930 con 0x7f91d8108d20 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.771+0000 7f91def15640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f91d8113c30 con 0x7f91d8102ae0 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d81083d0 0x7f91d819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d81083d0 0x7f91d819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60678/0 (socket says 172.21.15.112:60678) 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d7fff640 1 -- 172.21.15.112:0/1510760009 learned_addr learned my addr 172.21.15.112:0/1510760009 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91dccb1640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d8102ae0 0x7f91d819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d8108d20 0x7f91d819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d7fff640 1 -- 172.21.15.112:0/1510760009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d8102ae0 msgr2=0x7f91d819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d7fff640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d8102ae0 0x7f91d819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d7fff640 1 -- 172.21.15.112:0/1510760009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d8108d20 msgr2=0x7f91d819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d7fff640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d8108d20 0x7f91d819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d7fff640 1 -- 172.21.15.112:0/1510760009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d81a03d0 con 0x7f91d81083d0 2024-09-21T03:34:07.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91dccb1640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d8102ae0 0x7f91d819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d7fff640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d81083d0 0x7f91d819aa00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f91c802ade0 tx=0x7f91c8031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d5ffb640 1 -- 172.21.15.112:0/1510760009 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8003b70 con 0x7f91d81083d0 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d5ffb640 1 -- 172.21.15.112:0/1510760009 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c8003d10 con 0x7f91d81083d0 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91d5ffb640 1 -- 172.21.15.112:0/1510760009 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c802fa50 con 0x7f91d81083d0 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.772+0000 7f91def15640 1 -- 172.21.15.112:0/1510760009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d81a0660 con 0x7f91d81083d0 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.773+0000 7f91def15640 1 -- 172.21.15.112:0/1510760009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d81a6e20 con 0x7f91d81083d0 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.775+0000 7f91e5489740 1 Processor -- start 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.775+0000 7f91d5ffb640 1 -- 172.21.15.112:0/1510760009 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f91c8004580 con 0x7f91d81083d0 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.775+0000 7f91e5489740 1 -- start start 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91e5489740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee82a070 0x5608eebd1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91e5489740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebd1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91e5489740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebd20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91e5489740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5608eeb47e50 con 0x5608ee8011b0 2024-09-21T03:34:07.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91e5489740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5608eeb37280 con 0x5608ee82a070 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91e5489740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5608eeb3b090 con 0x5608ee809390 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dccb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebd1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dccb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebd1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60680/0 (socket says 172.21.15.112:60680) 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dccb1640 1 -- 172.21.15.112:0/2016266036 learned_addr learned my addr 172.21.15.112:0/2016266036 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91d7fff640 1 --2- 172.21.15.112:0/2016266036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebd20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/2016266036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee82a070 0x5608eebd1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dccb1640 1 -- 172.21.15.112:0/2016266036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 msgr2=0x5608eebd20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91d5ffb640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91c40b6700 0x7f91c40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dccb1640 1 --2- 172.21.15.112:0/2016266036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebd20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dccb1640 1 -- 172.21.15.112:0/2016266036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee82a070 msgr2=0x5608eebd1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dccb1640 1 --2- 172.21.15.112:0/2016266036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee82a070 0x5608eebd1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dccb1640 1 -- 172.21.15.112:0/2016266036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608ee895650 con 0x5608ee8011b0 2024-09-21T03:34:07.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91d7fff640 1 --2- 172.21.15.112:0/2016266036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebd20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/2016266036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee82a070 0x5608eebd1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91c40b6700 0x7f91c40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91dccb1640 1 --2- 172.21.15.112:0/2016266036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebd1b90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f91d0010580 tx=0x7f91d0010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4263774216c889c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.776+0000 7f91d5ffb640 1 -- 172.21.15.112:0/1510760009 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f91c8108900 con 0x7f91d81083d0 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.777+0000 7f91ceffd640 1 -- 172.21.15.112:0/2016266036 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0011760 con 0x5608ee8011b0 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.777+0000 7f91ceffd640 1 -- 172.21.15.112:0/2016266036 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d0011db0 con 0x5608ee8011b0 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.777+0000 7f91ceffd640 1 -- 172.21.15.112:0/2016266036 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d000e430 con 0x5608ee8011b0 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.777+0000 7f91e5489740 1 -- 172.21.15.112:0/2016266036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 msgr2=0x5608eebd1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.777+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91c40b6700 0x7f91c40b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009c20 tx=0x7f91c00046c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.777+0000 7f91e5489740 1 --2- 172.21.15.112:0/2016266036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebd1b90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f91d0010580 tx=0x7f91d0010a40 comp rx=0 tx=0).stop 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.778+0000 7f91e5489740 1 -- 172.21.15.112:0/2016266036 shutdown_connections 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.778+0000 7f91e5489740 1 --2- 172.21.15.112:0/2016266036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebd20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.778+0000 7f91e5489740 1 --2- 172.21.15.112:0/2016266036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebd1b90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.778+0000 7f91e5489740 1 --2- 172.21.15.112:0/2016266036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee82a070 0x5608eebd1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.778+0000 7f91e5489740 1 -- 172.21.15.112:0/2016266036 >> 172.21.15.112:0/2016266036 conn(0x5608ee81e460 msgr2=0x5608eebcf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.778+0000 7f91e5489740 1 -- 172.21.15.112:0/2016266036 shutdown_connections 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.778+0000 7f91e5489740 1 -- 172.21.15.112:0/2016266036 wait complete. 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.778+0000 7f91e5489740 1 -- start start 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.778+0000 7f91e5489740 1 Processor -- start 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91e5489740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebdccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91e5489740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebdd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91e5489740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee82a070 0x5608eebdd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91e5489740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5608ee998700 con 0x5608ee8011b0 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91e5489740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5608eead2110 con 0x5608ee82a070 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91e5489740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5608eebdfe30 con 0x5608ee809390 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91e5489740 10 client.0 _ll_register_callbacks cb 0x5608eeaeb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91dccb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebdccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91dccb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebdccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60694/0 (socket says 172.21.15.112:60694) 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91dccb1640 1 -- 172.21.15.112:0/920417144 learned_addr learned my addr 172.21.15.112:0/920417144 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91d7fff640 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebdd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee82a070 0x5608eebdd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91dccb1640 1 -- 172.21.15.112:0/920417144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 msgr2=0x5608eebdd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91dccb1640 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebdd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91dccb1640 1 -- 172.21.15.112:0/920417144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee82a070 msgr2=0x5608eebdd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91dccb1640 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee82a070 0x5608eebdd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91dccb1640 1 -- 172.21.15.112:0/920417144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eeaffc40 con 0x5608ee8011b0 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91dccb1640 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebdccf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f91d001ca20 tx=0x7f91d001dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b666556d1763480 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91b7fff640 1 -- 172.21.15.112:0/920417144 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0011560 con 0x5608ee8011b0 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91b7fff640 1 -- 172.21.15.112:0/920417144 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f91d0011700 con 0x5608ee8011b0 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.779+0000 7f91e5489740 1 -- 172.21.15.112:0/920417144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91d0011db0 con 0x5608ee8011b0 2024-09-21T03:34:07.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91b7fff640 1 -- 172.21.15.112:0/920417144 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f91d0011900 con 0x5608ee8011b0 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91e5489740 10 client.16383 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91e5489740 1 -- 172.21.15.112:0/920417144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5608eebebb00 con 0x5608ee8011b0 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91b7fff640 1 -- 172.21.15.112:0/920417144 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d0011db0 con 0x5608ee8011b0 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91b7fff640 1 -- 172.21.15.112:0/920417144 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f91d0027b20 con 0x5608ee8011b0 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91e5489740 10 client.16383 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91e5489740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91e5489740 20 client.16383 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91e5489740 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5608ee7e89f0 0x5608eebeeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91e5489740 4 client.16383 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91e5489740 1 -- 172.21.15.112:0/920417144 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608eeb49da0 con 0x5608ee7e89f0 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.780+0000 7f91d7fff640 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5608ee7e89f0 0x5608eebeeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.781+0000 7f91d7fff640 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5608ee7e89f0 0x5608eebeeec0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc5c73e0620b116 server_cookie=7e78368192a6118d in_seq=0 out_seq=0 2024-09-21T03:34:07.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.781+0000 7f91b7fff640 10 client.16383 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.781+0000 7f91b7fff640 1 -- 172.21.15.112:0/920417144 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608eeb49da0 con 0x5608ee7e89f0 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.781+0000 7f91b7fff640 10 client.16383 handle_command_reply: tid=0 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.781+0000 7f91e5489740 1 client.16383 shutdown 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.781+0000 7f91e5489740 1 -- 172.21.15.112:0/920417144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 msgr2=0x5608eebdccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.782+0000 7f91e5489740 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebdccf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f91d001ca20 tx=0x7f91d001dee0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.782+0000 7f91e5489740 1 -- 172.21.15.112:0/920417144 shutdown_connections 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.782+0000 7f91e5489740 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5608ee7e89f0 0x5608eebeeec0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.782+0000 7f91b7fff640 0 client.16383 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.782+0000 7f91e5489740 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee82a070 0x5608eebdd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.782+0000 7f91e5489740 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebdd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.782+0000 7f91e5489740 1 --2- 172.21.15.112:0/920417144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee8011b0 0x5608eebdccf0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.782+0000 7f91e5489740 1 -- 172.21.15.112:0/920417144 >> 172.21.15.112:0/920417144 conn(0x5608ee81e460 msgr2=0x5608eebd47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.782+0000 7f91e5489740 1 -- 172.21.15.112:0/920417144 shutdown_connections 2024-09-21T03:34:07.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.782+0000 7f91e5489740 1 -- 172.21.15.112:0/920417144 wait complete. 2024-09-21T03:34:07.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.782+0000 7f91e5489740 20 client.16383 trim_cache size 0 max 16384 2024-09-21T03:34:07.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91e5489740 1 Processor -- start 2024-09-21T03:34:07.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91e5489740 1 -- start start 2024-09-21T03:34:07.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91e5489740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee8011b0 0x5608eebdc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91e5489740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebde210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91e5489740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee82a070 0x5608eebceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91e5489740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81b1ed0 con 0x5608ee82a070 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91e5489740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81af770 con 0x5608ee8011b0 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91e5489740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81ad5c0 con 0x5608ee809390 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91dd4b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee82a070 0x5608eebceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91dd4b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee82a070 0x5608eebceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60696/0 (socket says 172.21.15.112:60696) 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91dd4b2640 1 -- 172.21.15.112:0/814189069 learned_addr learned my addr 172.21.15.112:0/814189069 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91d7fff640 1 --2- 172.21.15.112:0/814189069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebde210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91dccb1640 1 --2- 172.21.15.112:0/814189069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee8011b0 0x5608eebdc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91dd4b2640 1 -- 172.21.15.112:0/814189069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 msgr2=0x5608eebde210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/814189069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebde210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91dd4b2640 1 -- 172.21.15.112:0/814189069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee8011b0 msgr2=0x5608eebdc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/814189069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee8011b0 0x5608eebdc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.801+0000 7f91dd4b2640 1 -- 172.21.15.112:0/814189069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608eebd3940 con 0x5608ee82a070 2024-09-21T03:34:07.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.802+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/814189069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee82a070 0x5608eebceb70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f91c001f4a0 tx=0x7f91c0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f007790b354318e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:07.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.802+0000 7f91ce7fc640 1 -- 172.21.15.112:0/814189069 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0004020 con 0x5608ee82a070 2024-09-21T03:34:07.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.802+0000 7f91ce7fc640 1 -- 172.21.15.112:0/814189069 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c00041c0 con 0x5608ee82a070 2024-09-21T03:34:07.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.802+0000 7f91ce7fc640 1 -- 172.21.15.112:0/814189069 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0029760 con 0x5608ee82a070 2024-09-21T03:34:07.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.803+0000 7f91e5489740 1 -- 172.21.15.112:0/814189069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee82a070 msgr2=0x5608eebceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.803+0000 7f91e5489740 1 --2- 172.21.15.112:0/814189069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee82a070 0x5608eebceb70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f91c001f4a0 tx=0x7f91c0016000 comp rx=0 tx=0).stop 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.803+0000 7f91e5489740 1 -- 172.21.15.112:0/814189069 shutdown_connections 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.803+0000 7f91e5489740 1 --2- 172.21.15.112:0/814189069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee82a070 0x5608eebceb70 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.803+0000 7f91e5489740 1 --2- 172.21.15.112:0/814189069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee809390 0x5608eebde210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.803+0000 7f91e5489740 1 --2- 172.21.15.112:0/814189069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee8011b0 0x5608eebdc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.803+0000 7f91e5489740 1 -- 172.21.15.112:0/814189069 >> 172.21.15.112:0/814189069 conn(0x5608ee81e460 msgr2=0x5608eebd3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.803+0000 7f91e5489740 1 -- 172.21.15.112:0/814189069 shutdown_connections 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.803+0000 7f91e5489740 1 -- 172.21.15.112:0/814189069 wait complete. 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.803+0000 7f91e5489740 1 -- start start 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.803+0000 7f91e5489740 1 Processor -- start 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91e5489740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee8011b0 0x5608eebe59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91e5489740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee809390 0x5608eebe0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91e5489740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee82a070 0x5608eebe1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91e5489740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81adc30 con 0x5608ee809390 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91e5489740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81af0a0 con 0x5608ee8011b0 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91e5489740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f91d81ae300 con 0x5608ee82a070 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91e5489740 10 client.0 _ll_register_callbacks cb 0x7f91c810e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee809390 0x5608eebe0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91dccb1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee8011b0 0x5608eebe59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91dd4b2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee82a070 0x5608eebe1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91dccb1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee8011b0 0x5608eebe59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:39840/0 (socket says 172.21.15.112:39840) 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91dccb1640 1 -- 172.21.15.112:0/2739732166 learned_addr learned my addr 172.21.15.112:0/2739732166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91d7fff640 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee809390 0x5608eebe0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60710/0 (socket says 172.21.15.112:60710) 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91dd4b2640 1 -- 172.21.15.112:0/2739732166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee8011b0 msgr2=0x5608eebe59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee8011b0 0x5608eebe59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91dd4b2640 1 -- 172.21.15.112:0/2739732166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee809390 msgr2=0x5608eebe0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:07.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee809390 0x5608eebe0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91dd4b2640 1 -- 172.21.15.112:0/2739732166 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eebe1c80 con 0x5608ee82a070 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91dd4b2640 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee82a070 0x5608eebe1510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009a50 tx=0x7f91c00056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d72316e524e24e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.804+0000 7f91b7fff640 1 -- 172.21.15.112:0/2739732166 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c00035f0 con 0x5608ee82a070 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91b7fff640 1 -- 172.21.15.112:0/2739732166 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f91c0003790 con 0x5608ee82a070 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91e5489740 1 -- 172.21.15.112:0/2739732166 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c00041c0 con 0x5608ee82a070 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91b7fff640 1 -- 172.21.15.112:0/2739732166 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f91c0003990 con 0x5608ee82a070 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91e5489740 10 client.25954 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91e5489740 1 -- 172.21.15.112:0/2739732166 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5608eec596d0 con 0x5608ee82a070 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91b7fff640 1 -- 172.21.15.112:0/2739732166 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c00041c0 con 0x5608ee82a070 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91b7fff640 1 -- 172.21.15.112:0/2739732166 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f91c0030020 con 0x5608ee82a070 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91e5489740 10 client.25954 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91e5489740 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:07.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91e5489740 20 client.25954 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:07.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91e5489740 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5608ee7bdda0 0x5608eec5bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:07.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91e5489740 4 client.25954 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:34:07.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91e5489740 1 -- 172.21.15.112:0/2739732166 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608eeb45850 con 0x5608ee7bdda0 2024-09-21T03:34:07.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.805+0000 7f91dccb1640 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5608ee7bdda0 0x5608eec5bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:07.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.806+0000 7f91b7fff640 10 client.25954 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:07.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.806+0000 7f91dccb1640 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5608ee7bdda0 0x5608eec5bdc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff94f84b1c3226e server_cookie=ccc1e30e673f7036 in_seq=0 out_seq=0 2024-09-21T03:34:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91b7fff640 1 -- 172.21.15.112:0/2739732166 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x5608eeb45850 con 0x5608ee7bdda0 2024-09-21T03:34:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91b7fff640 10 client.25954 handle_command_reply: tid=0 2024-09-21T03:34:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 1 client.25954 shutdown 2024-09-21T03:34:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 1 -- 172.21.15.112:0/2739732166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee82a070 msgr2=0x5608eebe1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee82a070 0x5608eebe1510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009a50 tx=0x7f91c00056d0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 1 -- 172.21.15.112:0/2739732166 shutdown_connections 2024-09-21T03:34:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5608ee7bdda0 0x5608eec5bdc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608ee82a070 0x5608eebe1510 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608ee809390 0x5608eebe0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 1 --2- 172.21.15.112:0/2739732166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5608ee8011b0 0x5608eebe59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91b7fff640 0 client.25954 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 1 -- 172.21.15.112:0/2739732166 >> 172.21.15.112:0/2739732166 conn(0x5608ee81e460 msgr2=0x5608eebde1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 1 -- 172.21.15.112:0/2739732166 shutdown_connections 2024-09-21T03:34:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 1 -- 172.21.15.112:0/2739732166 wait complete. 2024-09-21T03:34:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.808+0000 7f91e5489740 20 client.25954 trim_cache size 0 max 16384 2024-09-21T03:34:07.814 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:34:07.814 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:34:07.814 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:07.815 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:34:07.816 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:07.817 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:07.818 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:07.819 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:07.820 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:07.820 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:07.820 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:34:07.820 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:34:07.820 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:34:07.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 -- 172.21.15.112:0/1510760009 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91c40b6700 msgr2=0x7f91c40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91c40b6700 0x7f91c40b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009c20 tx=0x7f91c00046c0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 -- 172.21.15.112:0/1510760009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d81083d0 msgr2=0x7f91d819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d81083d0 0x7f91d819aa00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f91c802ade0 tx=0x7f91c8031770 comp rx=0 tx=0).stop 2024-09-21T03:34:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 -- 172.21.15.112:0/1510760009 shutdown_connections 2024-09-21T03:34:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91c40b6700 0x7f91c40b8c10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d8108d20 0x7f91d819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91d81083d0 0x7f91d819aa00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 --2- 172.21.15.112:0/1510760009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d8102ae0 0x7f91d819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 -- 172.21.15.112:0/1510760009 >> 172.21.15.112:0/1510760009 conn(0x7f91d8079760 msgr2=0x7f91d8105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 -- 172.21.15.112:0/1510760009 shutdown_connections 2024-09-21T03:34:07.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:07.809+0000 7f91def15640 1 -- 172.21.15.112:0/1510760009 wait complete. 2024-09-21T03:34:08.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 506010 10.96 MB/sec execute 128 sec latency 44.795 ms 2024-09-21T03:34:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:08 smithi112 ceph-mon[31442]: pgmap v472: 97 pgs: 97 active+clean; 307 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-09-21T03:34:08.879 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:34:08.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:34:08.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:08 smithi203 ceph-mon[34723]: pgmap v472: 97 pgs: 97 active+clean; 307 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-09-21T03:34:08.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:08 smithi187 ceph-mon[34603]: pgmap v472: 97 pgs: 97 active+clean; 307 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-09-21T03:34:09.148 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:09.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 508402 10.98 MB/sec execute 129 sec latency 17.048 ms 2024-09-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.697+0000 7fb7d948a640 1 -- 172.21.15.112:0/1578919160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4103c30 msgr2=0x7fb7d4104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.697+0000 7fb7d948a640 1 --2- 172.21.15.112:0/1578919160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4103c30 0x7fb7d4104040 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009900 tx=0x7fb7c402a410 comp rx=0 tx=0).stop 2024-09-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.697+0000 7fb7d948a640 1 -- 172.21.15.112:0/1578919160 shutdown_connections 2024-09-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.697+0000 7fb7d948a640 1 --2- 172.21.15.112:0/1578919160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7d4109ec0 0x7fb7d41106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.697+0000 7fb7d948a640 1 --2- 172.21.15.112:0/1578919160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d4109570 0x7fb7d4109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.697+0000 7fb7d948a640 1 --2- 172.21.15.112:0/1578919160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4103c30 0x7fb7d4104040 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.697+0000 7fb7d948a640 1 -- 172.21.15.112:0/1578919160 >> 172.21.15.112:0/1578919160 conn(0x7fb7d407b8d0 msgr2=0x7fb7d407bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.697+0000 7fb7d948a640 1 -- 172.21.15.112:0/1578919160 shutdown_connections 2024-09-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.697+0000 7fb7d948a640 1 -- 172.21.15.112:0/1578919160 wait complete. 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.697+0000 7fb7d948a640 1 Processor -- start 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d948a640 1 -- start start 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d948a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4103c30 0x7fb7d419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d948a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d4109570 0x7fb7d419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d948a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7d4109ec0 0x7fb7d419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d948a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4113ae0 con 0x7fb7d4103c30 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d948a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4113960 con 0x7fb7d4109ec0 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d948a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4113c60 con 0x7fb7d4109570 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4103c30 0x7fb7d419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4103c30 0x7fb7d419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60716/0 (socket says 172.21.15.112:60716) 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d2ffd640 1 -- 172.21.15.112:0/2315157644 learned_addr learned my addr 172.21.15.112:0/2315157644 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d27fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d4109570 0x7fb7d419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.698+0000 7fb7d37fe640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7d4109ec0 0x7fb7d419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7d2ffd640 1 -- 172.21.15.112:0/2315157644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d4109570 msgr2=0x7fb7d419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7d2ffd640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d4109570 0x7fb7d419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7d2ffd640 1 -- 172.21.15.112:0/2315157644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7d4109ec0 msgr2=0x7fb7d419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7d2ffd640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7d4109ec0 0x7fb7d419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7d2ffd640 1 -- 172.21.15.112:0/2315157644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d41a04e0 con 0x7fb7d4103c30 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7d27fc640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d4109570 0x7fb7d419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7d2ffd640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4103c30 0x7fb7d419a570 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4005940 tx=0x7fb7c40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7bbfff640 1 -- 172.21.15.112:0/2315157644 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c403f030 con 0x7fb7d4103c30 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7bbfff640 1 -- 172.21.15.112:0/2315157644 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c40044c0 con 0x7fb7d4103c30 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7bbfff640 1 -- 172.21.15.112:0/2315157644 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4035070 con 0x7fb7d4103c30 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7d948a640 1 -- 172.21.15.112:0/2315157644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d41a0770 con 0x7fb7d4103c30 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.699+0000 7fb7d948a640 1 -- 172.21.15.112:0/2315157644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d41a6f30 con 0x7fb7d4103c30 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.700+0000 7fb7d948a640 1 -- 172.21.15.112:0/2315157644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d4104040 con 0x7fb7d4103c30 2024-09-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.702+0000 7fb7bbfff640 1 -- 172.21.15.112:0/2315157644 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb7c4042030 con 0x7fb7d4103c30 2024-09-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.703+0000 7fb7bbfff640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7bc0b6680 0x7fb7bc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.703+0000 7fb7d27fc640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7bc0b6680 0x7fb7bc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.703+0000 7fb7bbfff640 1 -- 172.21.15.112:0/2315157644 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fb7c4107c60 con 0x7fb7d4103c30 2024-09-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.704+0000 7fb7d27fc640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7bc0b6680 0x7fb7bc0b8b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4106600 tx=0x7fb7c8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.707+0000 7fb7bbfff640 1 -- 172.21.15.112:0/2315157644 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb7c40ce210 con 0x7fb7d4103c30 2024-09-21T03:34:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.942+0000 7fb7d948a640 1 -- 172.21.15.112:0/2315157644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7d410bcc0 con 0x7fb7d4103c30 2024-09-21T03:34:09.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.942+0000 7fb7bbfff640 1 -- 172.21.15.112:0/2315157644 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11930 (secure 0 0 0) 0x7fb7c40d25f0 con 0x7fb7d4103c30 2024-09-21T03:34:09.945 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:34:09.946 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"btime":"2024-09-21T03:33:42:353851+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:33:42.303719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:34:09.946 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-09-21T03:34:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 -- 172.21.15.112:0/2315157644 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7bc0b6680 msgr2=0x7fb7bc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7bc0b6680 0x7fb7bc0b8b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4106600 tx=0x7fb7c8005f90 comp rx=0 tx=0).stop 2024-09-21T03:34:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 -- 172.21.15.112:0/2315157644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4103c30 msgr2=0x7fb7d419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4103c30 0x7fb7d419a570 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4005940 tx=0x7fb7c40040c0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 -- 172.21.15.112:0/2315157644 shutdown_connections 2024-09-21T03:34:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb7bc0b6680 0x7fb7bc0b8b90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7d4109ec0 0x7fb7d419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d4109570 0x7fb7d419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 --2- 172.21.15.112:0/2315157644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7d4103c30 0x7fb7d419a570 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 -- 172.21.15.112:0/2315157644 >> 172.21.15.112:0/2315157644 conn(0x7fb7d407b8d0 msgr2=0x7fb7d41076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 -- 172.21.15.112:0/2315157644 shutdown_connections 2024-09-21T03:34:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:09.945+0000 7fb7d948a640 1 -- 172.21.15.112:0/2315157644 wait complete. 2024-09-21T03:34:10.235 INFO:tasks.workunit.client.0.smithi112.stdout: 1 510370 10.95 MB/sec execute 130 sec latency 21.064 ms 2024-09-21T03:34:10.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:09 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2315157644' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:34:10.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:09 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2315157644' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:34:10.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:09 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2315157644' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:34:10.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:34:10.970 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:11.236 INFO:tasks.workunit.client.0.smithi112.stdout: 1 512152 10.96 MB/sec execute 131 sec latency 29.458 ms 2024-09-21T03:34:11.283 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:11 smithi112 ceph-mon[31442]: pgmap v473: 97 pgs: 97 active+clean; 313 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 336 op/s 2024-09-21T03:34:11.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:11 smithi203 ceph-mon[34723]: pgmap v473: 97 pgs: 97 active+clean; 313 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 336 op/s 2024-09-21T03:34:11.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:11 smithi187 ceph-mon[34603]: pgmap v473: 97 pgs: 97 active+clean; 313 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 336 op/s 2024-09-21T03:34:11.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.496+0000 7f854279e640 1 -- 172.21.15.112:0/2430186005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853c105320 msgr2=0x7f853c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.496+0000 7f854279e640 1 --2- 172.21.15.112:0/2430186005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853c105320 0x7f853c10d870 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8524009900 tx=0x7f852402a410 comp rx=0 tx=0).stop 2024-09-21T03:34:11.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.496+0000 7f854279e640 1 -- 172.21.15.112:0/2430186005 shutdown_connections 2024-09-21T03:34:11.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.496+0000 7f854279e640 1 --2- 172.21.15.112:0/2430186005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f853c10de30 0x7f853c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.496+0000 7f854279e640 1 --2- 172.21.15.112:0/2430186005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853c105320 0x7f853c10d870 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.496+0000 7f854279e640 1 --2- 172.21.15.112:0/2430186005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853c1049d0 0x7f853c104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.496+0000 7f854279e640 1 -- 172.21.15.112:0/2430186005 >> 172.21.15.112:0/2430186005 conn(0x7f853c079740 msgr2=0x7f853c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:11.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.496+0000 7f854279e640 1 -- 172.21.15.112:0/2430186005 shutdown_connections 2024-09-21T03:34:11.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.496+0000 7f854279e640 1 -- 172.21.15.112:0/2430186005 wait complete. 2024-09-21T03:34:11.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.497+0000 7f854279e640 1 Processor -- start 2024-09-21T03:34:11.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.497+0000 7f854279e640 1 -- start start 2024-09-21T03:34:11.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f854279e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853c1049d0 0x7f853c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f854279e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f853c105320 0x7f853c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f854279e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853c10de30 0x7f853c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f854279e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f853c113a90 con 0x7f853c1049d0 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f854279e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f853c113910 con 0x7f853c105320 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f854279e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f853c113c10 con 0x7f853c10de30 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f853bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853c1049d0 0x7f853c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f853bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853c1049d0 0x7f853c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60728/0 (socket says 172.21.15.112:60728) 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f853bfff640 1 -- 172.21.15.112:0/1467299021 learned_addr learned my addr 172.21.15.112:0/1467299021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f853b7fe640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f853c105320 0x7f853c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f8540d3b640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853c10de30 0x7f853c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f853bfff640 1 -- 172.21.15.112:0/1467299021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853c10de30 msgr2=0x7f853c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f853bfff640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853c10de30 0x7f853c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f853bfff640 1 -- 172.21.15.112:0/1467299021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f853c105320 msgr2=0x7f853c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f853bfff640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f853c105320 0x7f853c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f853bfff640 1 -- 172.21.15.112:0/1467299021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c1a0450 con 0x7f853c1049d0 2024-09-21T03:34:11.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f853b7fe640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f853c105320 0x7f853c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:11.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.498+0000 7f8540d3b640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853c10de30 0x7f853c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:11.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.499+0000 7f853bfff640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853c1049d0 0x7f853c19a4e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f852c00e9c0 tx=0x7f852c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:11.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.499+0000 7f85397fa640 1 -- 172.21.15.112:0/1467299021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c00cde0 con 0x7f853c1049d0 2024-09-21T03:34:11.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.499+0000 7f85397fa640 1 -- 172.21.15.112:0/1467299021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852c0108e0 con 0x7f853c1049d0 2024-09-21T03:34:11.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.499+0000 7f85397fa640 1 -- 172.21.15.112:0/1467299021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c018960 con 0x7f853c1049d0 2024-09-21T03:34:11.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.499+0000 7f854279e640 1 -- 172.21.15.112:0/1467299021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c1a0740 con 0x7f853c1049d0 2024-09-21T03:34:11.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.499+0000 7f854279e640 1 -- 172.21.15.112:0/1467299021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c1a6ea0 con 0x7f853c1049d0 2024-09-21T03:34:11.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f85397fa640 1 -- 172.21.15.112:0/1467299021 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f852c01f020 con 0x7f853c1049d0 2024-09-21T03:34:11.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f8548d12740 1 Processor -- start 2024-09-21T03:34:11.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f8548d12740 1 -- start start 2024-09-21T03:34:11.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f8548d12740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27c3070 0x556ca2b6a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f8548d12740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 0x556ca2b6a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f8548d12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b6ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f8548d12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556ca2ae0ea0 con 0x556ca27a2390 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f8548d12740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556ca2ad02d0 con 0x556ca27c3070 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f8548d12740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556ca2ad40e0 con 0x556ca279a1b0 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f853bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 0x556ca2b6a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f8540d3b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27c3070 0x556ca2b6a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f853b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b6ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f8540d3b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27c3070 0x556ca2b6a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:39890/0 (socket says 172.21.15.112:39890) 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.502+0000 7f8540d3b640 1 -- 172.21.15.112:0/541338419 learned_addr learned my addr 172.21.15.112:0/541338419 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f8540d3b640 1 -- 172.21.15.112:0/541338419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 msgr2=0x556ca2b6a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f8540d3b640 1 --2- 172.21.15.112:0/541338419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 0x556ca2b6a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f8540d3b640 1 -- 172.21.15.112:0/541338419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 msgr2=0x556ca2b6ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:11.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f8540d3b640 1 --2- 172.21.15.112:0/541338419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b6ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f85397fa640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85280b6680 0x7f85280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f8540d3b640 1 -- 172.21.15.112:0/541338419 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca282e650 con 0x556ca27c3070 2024-09-21T03:34:11.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f853bfff640 1 --2- 172.21.15.112:0/541338419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 0x556ca2b6a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:11.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f85397fa640 1 -- 172.21.15.112:0/1467299021 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f852c0e7970 con 0x7f853c1049d0 2024-09-21T03:34:11.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f8540d3b640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85280b6680 0x7f85280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f8540d3b640 1 --2- 172.21.15.112:0/541338419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27c3070 0x556ca2b6a2c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f85340117e0 tx=0x7f8534011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86251c93c45c95a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:11.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f85327fc640 1 -- 172.21.15.112:0/541338419 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853400fb00 con 0x556ca27c3070 2024-09-21T03:34:11.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f85327fc640 1 -- 172.21.15.112:0/541338419 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8534010e00 con 0x556ca27c3070 2024-09-21T03:34:11.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.503+0000 7f85327fc640 1 -- 172.21.15.112:0/541338419 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853401ca80 con 0x556ca27c3070 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.504+0000 7f8548d12740 1 -- 172.21.15.112:0/541338419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27c3070 msgr2=0x556ca2b6a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.504+0000 7f8540d3b640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85280b6680 0x7f85280b8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8534011490 tx=0x7f8534021000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.504+0000 7f8548d12740 1 --2- 172.21.15.112:0/541338419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27c3070 0x556ca2b6a2c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f85340117e0 tx=0x7f8534011ca0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.504+0000 7f8548d12740 1 -- 172.21.15.112:0/541338419 shutdown_connections 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.504+0000 7f8548d12740 1 --2- 172.21.15.112:0/541338419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b6ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.504+0000 7f8548d12740 1 --2- 172.21.15.112:0/541338419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 0x556ca2b6a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.504+0000 7f8548d12740 1 --2- 172.21.15.112:0/541338419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27c3070 0x556ca2b6a2c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.504+0000 7f8548d12740 1 -- 172.21.15.112:0/541338419 >> 172.21.15.112:0/541338419 conn(0x556ca27b7460 msgr2=0x556ca2b681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.504+0000 7f8548d12740 1 -- 172.21.15.112:0/541338419 shutdown_connections 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.504+0000 7f8548d12740 1 -- 172.21.15.112:0/541338419 wait complete. 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.505+0000 7f8548d12740 1 -- start start 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.505+0000 7f8548d12740 1 Processor -- start 2024-09-21T03:34:11.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.505+0000 7f8548d12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca279a1b0 0x556ca2b759c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.505+0000 7f8548d12740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27a2390 0x556ca2b75f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.505+0000 7f8548d12740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca27c3070 0x556ca2b861a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.505+0000 7f8548d12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556ca2931700 con 0x556ca279a1b0 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.505+0000 7f8548d12740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556ca2a6b160 con 0x556ca27a2390 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.505+0000 7f8548d12740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556ca2b7ceb0 con 0x556ca27c3070 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f8548d12740 10 client.0 _ll_register_callbacks cb 0x556ca2a849e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f853bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca279a1b0 0x556ca2b759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f853bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca279a1b0 0x556ca2b759c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60742/0 (socket says 172.21.15.112:60742) 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f853bfff640 1 -- 172.21.15.112:0/4185427765 learned_addr learned my addr 172.21.15.112:0/4185427765 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f8540d3b640 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca27c3070 0x556ca2b861a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f853b7fe640 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27a2390 0x556ca2b75f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f853bfff640 1 -- 172.21.15.112:0/4185427765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca27c3070 msgr2=0x556ca2b861a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f853bfff640 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca27c3070 0x556ca2b861a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f853bfff640 1 -- 172.21.15.112:0/4185427765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27a2390 msgr2=0x556ca2b75f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f853bfff640 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27a2390 0x556ca2b75f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f853bfff640 1 -- 172.21.15.112:0/4185427765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca2a98c90 con 0x556ca279a1b0 2024-09-21T03:34:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f8540d3b640 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca27c3070 0x556ca2b861a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f853bfff640 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca279a1b0 0x556ca2b759c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f852c00cfd0 tx=0x7f852c0a9e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01cd720aa6f9953 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f851b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f851b7fe640 1 -- 172.21.15.112:0/4185427765 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0ad110 con 0x556ca279a1b0 2024-09-21T03:34:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f851b7fe640 1 -- 172.21.15.112:0/4185427765 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f852c010630 con 0x556ca279a1b0 2024-09-21T03:34:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.506+0000 7f8548d12740 1 -- 172.21.15.112:0/4185427765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8534010e00 con 0x556ca279a1b0 2024-09-21T03:34:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f851b7fe640 1 -- 172.21.15.112:0/4185427765 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f852c0e5550 con 0x556ca279a1b0 2024-09-21T03:34:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f8548d12740 10 client.16407 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:11.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f8548d12740 1 -- 172.21.15.112:0/4185427765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556ca2b88c80 con 0x556ca279a1b0 2024-09-21T03:34:11.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f851b7fe640 1 -- 172.21.15.112:0/4185427765 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8534010e00 con 0x556ca279a1b0 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f851b7fe640 1 -- 172.21.15.112:0/4185427765 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f852c0aeaf0 con 0x556ca279a1b0 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f8548d12740 10 client.16407 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f8548d12740 10 client.16407 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f8548d12740 20 client.16407 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f8548d12740 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556ca27819f0 0x556ca2b8c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f8548d12740 4 client.16407 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f8548d12740 1 -- 172.21.15.112:0/4185427765 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ca2ae2df0 con 0x556ca27819f0 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.507+0000 7f853b7fe640 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556ca27819f0 0x556ca2b8c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.508+0000 7f851b7fe640 10 client.16407 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.508+0000 7f853b7fe640 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556ca27819f0 0x556ca2b8c070 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f99fee8eff54c6b server_cookie=a15b6898eaf4f848 in_seq=0 out_seq=0 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.509+0000 7f851b7fe640 1 -- 172.21.15.112:0/4185427765 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ca2ae2df0 con 0x556ca27819f0 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.509+0000 7f851b7fe640 10 client.16407 handle_command_reply: tid=0 2024-09-21T03:34:11.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.509+0000 7f8548d12740 1 client.16407 shutdown 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f8548d12740 1 -- 172.21.15.112:0/4185427765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca279a1b0 msgr2=0x556ca2b759c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f8548d12740 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca279a1b0 0x556ca2b759c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f852c00cfd0 tx=0x7f852c0a9e10 comp rx=0 tx=0).stop 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f8548d12740 1 -- 172.21.15.112:0/4185427765 shutdown_connections 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f8548d12740 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556ca27819f0 0x556ca2b8c070 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f851b7fe640 0 client.16407 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f8548d12740 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca27c3070 0x556ca2b861a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f8548d12740 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27a2390 0x556ca2b75f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f8548d12740 1 --2- 172.21.15.112:0/4185427765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca279a1b0 0x556ca2b759c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f8548d12740 1 -- 172.21.15.112:0/4185427765 >> 172.21.15.112:0/4185427765 conn(0x556ca27b7460 msgr2=0x556ca2b6d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f8548d12740 1 -- 172.21.15.112:0/4185427765 shutdown_connections 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f8548d12740 1 -- 172.21.15.112:0/4185427765 wait complete. 2024-09-21T03:34:11.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.510+0000 7f8548d12740 20 client.16407 trim_cache size 0 max 16384 2024-09-21T03:34:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.524+0000 7f8548d12740 1 Processor -- start 2024-09-21T03:34:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.524+0000 7f8548d12740 1 -- start start 2024-09-21T03:34:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.524+0000 7f8548d12740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca27b7460 0x556ca2b778a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.524+0000 7f8548d12740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca279a1b0 0x556ca2b6b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.524+0000 7f8548d12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b860a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.524+0000 7f8548d12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1afdb0 con 0x556ca27a2390 2024-09-21T03:34:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.524+0000 7f8548d12740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1ad650 con 0x556ca279a1b0 2024-09-21T03:34:11.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.524+0000 7f8548d12740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1ab4a0 con 0x556ca27b7460 2024-09-21T03:34:11.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.524+0000 7f853b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca279a1b0 0x556ca2b6b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.524+0000 7f8540d3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b860a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f8540d3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b860a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60750/0 (socket says 172.21.15.112:60750) 2024-09-21T03:34:11.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f8540d3b640 1 -- 172.21.15.112:0/3356050900 learned_addr learned my addr 172.21.15.112:0/3356050900 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:11.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f853bfff640 1 --2- 172.21.15.112:0/3356050900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca27b7460 0x556ca2b778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f853b7fe640 1 -- 172.21.15.112:0/3356050900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca27b7460 msgr2=0x556ca2b778a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:11.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f853b7fe640 1 --2- 172.21.15.112:0/3356050900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca27b7460 0x556ca2b778a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f853b7fe640 1 -- 172.21.15.112:0/3356050900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 msgr2=0x556ca2b860a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f853b7fe640 1 --2- 172.21.15.112:0/3356050900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b860a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f853b7fe640 1 -- 172.21.15.112:0/3356050900 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca2b73ec0 con 0x556ca279a1b0 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f853b7fe640 1 --2- 172.21.15.112:0/3356050900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca279a1b0 0x556ca2b6b2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f852402a930 tx=0x7f8524000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc65e699da7e8d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f8531ffb640 1 -- 172.21.15.112:0/3356050900 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524035040 con 0x556ca279a1b0 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f8531ffb640 1 -- 172.21.15.112:0/3356050900 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8524054070 con 0x556ca279a1b0 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.525+0000 7f8531ffb640 1 -- 172.21.15.112:0/3356050900 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85240053a0 con 0x556ca279a1b0 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.526+0000 7f8548d12740 1 -- 172.21.15.112:0/3356050900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca279a1b0 msgr2=0x556ca2b6b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.526+0000 7f8548d12740 1 --2- 172.21.15.112:0/3356050900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca279a1b0 0x556ca2b6b2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f852402a930 tx=0x7f8524000bc0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.526+0000 7f8548d12740 1 -- 172.21.15.112:0/3356050900 shutdown_connections 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.526+0000 7f8548d12740 1 --2- 172.21.15.112:0/3356050900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b860a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.526+0000 7f8548d12740 1 --2- 172.21.15.112:0/3356050900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca279a1b0 0x556ca2b6b2f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.526+0000 7f8548d12740 1 --2- 172.21.15.112:0/3356050900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca27b7460 0x556ca2b778a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.526+0000 7f8548d12740 1 -- 172.21.15.112:0/3356050900 >> 172.21.15.112:0/3356050900 conn(0x556ca2b76080 msgr2=0x556ca2b6f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:11.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.526+0000 7f8548d12740 1 -- 172.21.15.112:0/3356050900 shutdown_connections 2024-09-21T03:34:11.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.526+0000 7f8548d12740 1 -- 172.21.15.112:0/3356050900 wait complete. 2024-09-21T03:34:11.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f8548d12740 1 -- start start 2024-09-21T03:34:11.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f8548d12740 1 Processor -- start 2024-09-21T03:34:11.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f8548d12740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 0x556ca2b82100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f8548d12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b82640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f8548d12740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27b7460 0x556ca2b79ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f8548d12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1abb10 con 0x556ca27a2390 2024-09-21T03:34:11.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f8548d12740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1acf80 con 0x556ca27b7460 2024-09-21T03:34:11.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f8548d12740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1ac1e0 con 0x556ca279a1b0 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f8548d12740 10 client.0 _ll_register_callbacks cb 0x7f852c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f853b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b82640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f853b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b82640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60754/0 (socket says 172.21.15.112:60754) 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f853b7fe640 1 -- 172.21.15.112:0/3382308111 learned_addr learned my addr 172.21.15.112:0/3382308111 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.527+0000 7f853bfff640 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 0x556ca2b82100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f8540d3b640 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27b7460 0x556ca2b79ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f853bfff640 1 -- 172.21.15.112:0/3382308111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27b7460 msgr2=0x556ca2b79ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f853bfff640 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27b7460 0x556ca2b79ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f853bfff640 1 -- 172.21.15.112:0/3382308111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 msgr2=0x556ca2b82640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f853bfff640 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b82640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f853bfff640 1 -- 172.21.15.112:0/3382308111 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca2b82e30 con 0x556ca279a1b0 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f853b7fe640 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b82640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f853bfff640 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 0x556ca2b82100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f852c0058c0 tx=0x7f852c0a9f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8339d8e508b83358 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:11.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f851b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f851b7fe640 1 -- 172.21.15.112:0/3382308111 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0acf30 con 0x556ca279a1b0 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f851b7fe640 1 -- 172.21.15.112:0/3382308111 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f852c010640 con 0x556ca279a1b0 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f8548d12740 1 -- 172.21.15.112:0/3382308111 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8524054070 con 0x556ca279a1b0 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f851b7fe640 1 -- 172.21.15.112:0/3382308111 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f852c0ad110 con 0x556ca279a1b0 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f8548d12740 10 client.25978 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.528+0000 7f8548d12740 1 -- 172.21.15.112:0/3382308111 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556ca2bf2f80 con 0x556ca279a1b0 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.529+0000 7f851b7fe640 1 -- 172.21.15.112:0/3382308111 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8524054070 con 0x556ca279a1b0 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.529+0000 7f851b7fe640 1 -- 172.21.15.112:0/3382308111 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f852c0ac490 con 0x556ca279a1b0 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.529+0000 7f8548d12740 10 client.25978 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.529+0000 7f8548d12740 10 client.25978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.529+0000 7f8548d12740 20 client.25978 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.529+0000 7f8548d12740 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556ca27c3070 0x556ca2bf5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.529+0000 7f8548d12740 4 client.25978 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:34:11.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.529+0000 7f8548d12740 1 -- 172.21.15.112:0/3382308111 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ca2ade8a0 con 0x556ca27c3070 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.529+0000 7f853b7fe640 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556ca27c3070 0x556ca2bf5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.529+0000 7f851b7fe640 10 client.25978 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.529+0000 7f853b7fe640 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556ca27c3070 0x556ca2bf5530 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5264710936e1e45c server_cookie=e2906babe4db9c9b in_seq=0 out_seq=0 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f851b7fe640 1 -- 172.21.15.112:0/3382308111 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ca2ade8a0 con 0x556ca27c3070 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f851b7fe640 10 client.25978 handle_command_reply: tid=0 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 1 client.25978 shutdown 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 1 -- 172.21.15.112:0/3382308111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 msgr2=0x556ca2b82100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 0x556ca2b82100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f852c0058c0 tx=0x7f852c0a9f40 comp rx=0 tx=0).stop 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 1 -- 172.21.15.112:0/3382308111 shutdown_connections 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556ca27c3070 0x556ca2bf5530 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556ca27b7460 0x556ca2b79ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556ca27a2390 0x556ca2b82640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f851b7fe640 0 client.25978 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:34:11.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 1 --2- 172.21.15.112:0/3382308111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca279a1b0 0x556ca2b82100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 1 -- 172.21.15.112:0/3382308111 >> 172.21.15.112:0/3382308111 conn(0x556ca2b76080 msgr2=0x556ca2b77580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:11.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 1 -- 172.21.15.112:0/3382308111 shutdown_connections 2024-09-21T03:34:11.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 1 -- 172.21.15.112:0/3382308111 wait complete. 2024-09-21T03:34:11.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.530+0000 7f8548d12740 20 client.25978 trim_cache size 0 max 16384 2024-09-21T03:34:11.538 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:34:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 -- 172.21.15.112:0/1467299021 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85280b6680 msgr2=0x7f85280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85280b6680 0x7f85280b8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8534011490 tx=0x7f8534021000 comp rx=0 tx=0).stop 2024-09-21T03:34:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 -- 172.21.15.112:0/1467299021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853c1049d0 msgr2=0x7f853c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853c1049d0 0x7f853c19a4e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f852c00e9c0 tx=0x7f852c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:34:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 -- 172.21.15.112:0/1467299021 shutdown_connections 2024-09-21T03:34:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f85280b6680 0x7f85280b8b90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f853c10de30 0x7f853c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f853c105320 0x7f853c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 --2- 172.21.15.112:0/1467299021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f853c1049d0 0x7f853c19a4e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 -- 172.21.15.112:0/1467299021 >> 172.21.15.112:0/1467299021 conn(0x7f853c079740 msgr2=0x7f853c1007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 -- 172.21.15.112:0/1467299021 shutdown_connections 2024-09-21T03:34:11.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:11.531+0000 7f854279e640 1 -- 172.21.15.112:0/1467299021 wait complete. 2024-09-21T03:34:12.236 INFO:tasks.workunit.client.0.smithi112.stdout: 1 514010 10.93 MB/sec execute 132 sec latency 33.604 ms 2024-09-21T03:34:12.243 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:12 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:12.273 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:34:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:12 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:12.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:12 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:34:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.075+0000 7f8eed219640 1 -- 172.21.15.112:0/1171566627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102a60 msgr2=0x7f8ee8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.075+0000 7f8eed219640 1 --2- 172.21.15.112:0/1171566627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102a60 0x7f8ee8102e70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009ca0 tx=0x7f8ed802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.075+0000 7f8eed219640 1 -- 172.21.15.112:0/1171566627 shutdown_connections 2024-09-21T03:34:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.075+0000 7f8eed219640 1 --2- 172.21.15.112:0/1171566627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee8108cf0 0x7f8ee810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.075+0000 7f8eed219640 1 --2- 172.21.15.112:0/1171566627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee81083a0 0x7f8ee81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.075+0000 7f8eed219640 1 --2- 172.21.15.112:0/1171566627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102a60 0x7f8ee8102e70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.075+0000 7f8eed219640 1 -- 172.21.15.112:0/1171566627 >> 172.21.15.112:0/1171566627 conn(0x7f8ee8079730 msgr2=0x7f8ee8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.075+0000 7f8eed219640 1 -- 172.21.15.112:0/1171566627 shutdown_connections 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.075+0000 7f8eed219640 1 -- 172.21.15.112:0/1171566627 wait complete. 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.076+0000 7f8eed219640 1 Processor -- start 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8eed219640 1 -- start start 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8eed219640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102a60 0x7f8ee819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8eed219640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81083a0 0x7f8ee819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8eed219640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8108cf0 0x7f8ee819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8eed219640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8112960 con 0x7f8ee8102a60 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8eed219640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81127e0 con 0x7f8ee8108cf0 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8eed219640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8112ae0 con 0x7f8ee81083a0 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8ee6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102a60 0x7f8ee819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8ede59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81083a0 0x7f8ee819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8ee6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102a60 0x7f8ee819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60774/0 (socket says 172.21.15.112:60774) 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8ee6d9d640 1 -- 172.21.15.112:0/1389952602 learned_addr learned my addr 172.21.15.112:0/1389952602 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8ede59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81083a0 0x7f8ee819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:37630/0 (socket says 172.21.15.112:37630) 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.077+0000 7f8ee759e640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8108cf0 0x7f8ee819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8ee6d9d640 1 -- 172.21.15.112:0/1389952602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81083a0 msgr2=0x7f8ee819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8ee6d9d640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81083a0 0x7f8ee819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8ee6d9d640 1 -- 172.21.15.112:0/1389952602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8108cf0 msgr2=0x7f8ee819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8ee6d9d640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8108cf0 0x7f8ee819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8ee6d9d640 1 -- 172.21.15.112:0/1389952602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee81a03b0 con 0x7f8ee8102a60 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8ee759e640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8108cf0 0x7f8ee819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8ede59c640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81083a0 0x7f8ee819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8ee6d9d640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102a60 0x7f8ee819a440 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8006010 tx=0x7f8ed80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8ee4d99640 1 -- 172.21.15.112:0/1389952602 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8003bf0 con 0x7f8ee8102a60 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8eed219640 1 -- 172.21.15.112:0/1389952602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee81a0640 con 0x7f8ee8102a60 2024-09-21T03:34:13.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8ee4d99640 1 -- 172.21.15.112:0/1389952602 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed8003d90 con 0x7f8ee8102a60 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8ee4d99640 1 -- 172.21.15.112:0/1389952602 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed802fc20 con 0x7f8ee8102a60 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.078+0000 7f8eed219640 1 -- 172.21.15.112:0/1389952602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee81a6e00 con 0x7f8ee8102a60 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.081+0000 7f8ef378d740 1 Processor -- start 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.081+0000 7f8ef378d740 1 -- start start 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.081+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c00543c070 0x55c0057e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.081+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0054131b0 0x55c0057e3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.081+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00541b390 0x55c0057e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.081+0000 7f8ef378d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c005759ea0 con 0x55c00543c070 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.081+0000 7f8ef378d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c0057492d0 con 0x55c0054131b0 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.081+0000 7f8ef378d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c00574d0e0 con 0x55c00541b390 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.081+0000 7f8ee4d99640 1 -- 172.21.15.112:0/1389952602 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8ed80046a0 con 0x7f8ee8102a60 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ee759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c00543c070 0x55c0057e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ee759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c00543c070 0x55c0057e36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60786/0 (socket says 172.21.15.112:60786) 2024-09-21T03:34:13.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ee759e640 1 -- 172.21.15.112:0/716980606 learned_addr learned my addr 172.21.15.112:0/716980606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ede59c640 1 --2- 172.21.15.112:0/716980606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00541b390 0x55c0057e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ee6d9d640 1 --2- 172.21.15.112:0/716980606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0054131b0 0x55c0057e3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ee759e640 1 -- 172.21.15.112:0/716980606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00541b390 msgr2=0x55c0057e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ee759e640 1 --2- 172.21.15.112:0/716980606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00541b390 0x55c0057e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ee759e640 1 -- 172.21.15.112:0/716980606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0054131b0 msgr2=0x55c0057e3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ee759e640 1 --2- 172.21.15.112:0/716980606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0054131b0 0x55c0057e3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ee759e640 1 -- 172.21.15.112:0/716980606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0054a7650 con 0x55c00543c070 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ede59c640 1 --2- 172.21.15.112:0/716980606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00541b390 0x55c0057e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ee6d9d640 1 --2- 172.21.15.112:0/716980606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0054131b0 0x55c0057e3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8ee759e640 1 --2- 172.21.15.112:0/716980606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c00543c070 0x55c0057e36a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0010400 tx=0x7f8ee00108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf91f78d91e20abe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8edd59a640 1 -- 172.21.15.112:0/716980606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0004230 con 0x55c00543c070 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8edd59a640 1 -- 172.21.15.112:0/716980606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee00043d0 con 0x55c00543c070 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.082+0000 7f8edd59a640 1 -- 172.21.15.112:0/716980606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00039f0 con 0x55c00543c070 2024-09-21T03:34:13.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ee4d99640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed40b6680 0x7f8ed40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ee6d9d640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed40b6680 0x7f8ed40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ee4d99640 1 -- 172.21.15.112:0/1389952602 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f8ed8108960 con 0x7f8ee8102a60 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ef378d740 1 -- 172.21.15.112:0/716980606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c00543c070 msgr2=0x55c0057e36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ee6d9d640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed40b6680 0x7f8ed40b8b90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8ed81061c0 tx=0x7f8ed80095c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ef378d740 1 --2- 172.21.15.112:0/716980606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c00543c070 0x55c0057e36a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0010400 tx=0x7f8ee00108c0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ef378d740 1 -- 172.21.15.112:0/716980606 shutdown_connections 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ef378d740 1 --2- 172.21.15.112:0/716980606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00541b390 0x55c0057e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ef378d740 1 --2- 172.21.15.112:0/716980606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0054131b0 0x55c0057e3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ef378d740 1 --2- 172.21.15.112:0/716980606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c00543c070 0x55c0057e36a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ef378d740 1 -- 172.21.15.112:0/716980606 >> 172.21.15.112:0/716980606 conn(0x55c005430460 msgr2=0x55c0057e1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.083+0000 7f8ef378d740 1 -- 172.21.15.112:0/716980606 shutdown_connections 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ef378d740 1 -- 172.21.15.112:0/716980606 wait complete. 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ef378d740 1 -- start start 2024-09-21T03:34:13.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ef378d740 1 Processor -- start 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00541b390 0x55c0057ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00543c070 0x55c0057fb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ef378d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c0055aa700 con 0x55c0054131b0 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ef378d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c0056e4160 con 0x55c00541b390 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ef378d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c0057f1e80 con 0x55c00543c070 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ef378d740 10 client.0 _ll_register_callbacks cb 0x55c0056fd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ee759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ee759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057eeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60790/0 (socket says 172.21.15.112:60790) 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.084+0000 7f8ee759e640 1 -- 172.21.15.112:0/2203733897 learned_addr learned my addr 172.21.15.112:0/2203733897 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ee759e640 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00543c070 0x55c0057fb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ede59c640 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00541b390 0x55c0057ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ee759e640 1 -- 172.21.15.112:0/2203733897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00543c070 msgr2=0x55c0057fb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:13.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ee759e640 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00543c070 0x55c0057fb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ee759e640 1 -- 172.21.15.112:0/2203733897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00541b390 msgr2=0x55c0057ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ee759e640 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00541b390 0x55c0057ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ee759e640 1 -- 172.21.15.112:0/2203733897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c005711c90 con 0x55c0054131b0 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ee759e640 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057eeda0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0003650 tx=0x7f8ee0011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bad60ea58b7b9fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ece7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2203733897 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0025030 con 0x55c0054131b0 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2203733897 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f8ee0005450 con 0x55c0054131b0 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ef378d740 1 -- 172.21.15.112:0/2203733897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee00043d0 con 0x55c0054131b0 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2203733897 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ee0005650 con 0x55c0054131b0 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ef378d740 10 client.16425 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.085+0000 7f8ef378d740 1 -- 172.21.15.112:0/2203733897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0057fdbf0 con 0x55c0054131b0 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.086+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2203733897 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee00043d0 con 0x55c0054131b0 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.086+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2203733897 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f8ee00356f0 con 0x55c0054131b0 2024-09-21T03:34:13.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.086+0000 7f8ef378d740 10 client.16425 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.086+0000 7f8ef378d740 10 client.16425 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.086+0000 7f8ef378d740 20 client.16425 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.086+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c0053fa9f0 0x55c005800fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.086+0000 7f8ef378d740 4 client.16425 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.086+0000 7f8ef378d740 1 -- 172.21.15.112:0/2203733897 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c00575bdf0 con 0x55c0053fa9f0 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.086+0000 7f8ede59c640 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c0053fa9f0 0x55c005800fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.087+0000 7f8ede59c640 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c0053fa9f0 0x55c005800fb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f0994c97fe8d24b server_cookie=544d1da554779f69 in_seq=0 out_seq=0 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.087+0000 7f8ece7fc640 10 client.16425 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.087+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2203733897 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c00575bdf0 con 0x55c0053fa9f0 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.087+0000 7f8ece7fc640 10 client.16425 handle_command_reply: tid=0 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.087+0000 7f8ef378d740 1 client.16425 shutdown 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.087+0000 7f8ef378d740 1 -- 172.21.15.112:0/2203733897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 msgr2=0x55c0057eeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:13.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.087+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057eeda0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0003650 tx=0x7f8ee0011f10 comp rx=0 tx=0).stop 2024-09-21T03:34:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.088+0000 7f8ef378d740 1 -- 172.21.15.112:0/2203733897 shutdown_connections 2024-09-21T03:34:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.088+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c0053fa9f0 0x55c005800fb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.088+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00543c070 0x55c0057fb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.088+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00541b390 0x55c0057ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.088+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2203733897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057eeda0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.088+0000 7f8ef378d740 1 -- 172.21.15.112:0/2203733897 >> 172.21.15.112:0/2203733897 conn(0x55c005430460 msgr2=0x55c0057e6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.088+0000 7f8ece7fc640 0 client.16425 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:34:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.088+0000 7f8ef378d740 1 -- 172.21.15.112:0/2203733897 shutdown_connections 2024-09-21T03:34:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.088+0000 7f8ef378d740 1 -- 172.21.15.112:0/2203733897 wait complete. 2024-09-21T03:34:13.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.088+0000 7f8ef378d740 20 client.16425 trim_cache size 0 max 16384 2024-09-21T03:34:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.101+0000 7f8ef378d740 1 Processor -- start 2024-09-21T03:34:13.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.101+0000 7f8ef378d740 1 -- start start 2024-09-21T03:34:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00541b390 0x55c0057ebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00543c070 0x55c0057ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ef378d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81b1ed0 con 0x55c0054131b0 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ef378d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81af770 con 0x55c00541b390 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ef378d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81ad5c0 con 0x55c00543c070 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ee759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ee759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057e4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60798/0 (socket says 172.21.15.112:60798) 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ee759e640 1 -- 172.21.15.112:0/17148095 learned_addr learned my addr 172.21.15.112:0/17148095 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ede59c640 1 --2- 172.21.15.112:0/17148095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00541b390 0x55c0057ebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ee759e640 1 --2- 172.21.15.112:0/17148095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00543c070 0x55c0057ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ee759e640 1 -- 172.21.15.112:0/17148095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00543c070 msgr2=0x55c0057ec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ee759e640 1 --2- 172.21.15.112:0/17148095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00543c070 0x55c0057ec4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ee759e640 1 -- 172.21.15.112:0/17148095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00541b390 msgr2=0x55c0057ebf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ee759e640 1 --2- 172.21.15.112:0/17148095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00541b390 0x55c0057ebf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ee759e640 1 -- 172.21.15.112:0/17148095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0057e5f30 con 0x55c0054131b0 2024-09-21T03:34:13.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ede59c640 1 --2- 172.21.15.112:0/17148095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00541b390 0x55c0057ebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:13.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.102+0000 7f8ee759e640 1 --2- 172.21.15.112:0/17148095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057e4de0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0006010 tx=0x7f8ee0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1f3c85c70124cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:13.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.103+0000 7f8edcd99640 1 -- 172.21.15.112:0/17148095 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0025070 con 0x55c0054131b0 2024-09-21T03:34:13.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.103+0000 7f8edcd99640 1 -- 172.21.15.112:0/17148095 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee0023070 con 0x55c0054131b0 2024-09-21T03:34:13.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.103+0000 7f8edcd99640 1 -- 172.21.15.112:0/17148095 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0010dc0 con 0x55c0054131b0 2024-09-21T03:34:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.104+0000 7f8ef378d740 1 -- 172.21.15.112:0/17148095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 msgr2=0x55c0057e4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.104+0000 7f8ef378d740 1 --2- 172.21.15.112:0/17148095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057e4de0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0006010 tx=0x7f8ee0001e50 comp rx=0 tx=0).stop 2024-09-21T03:34:13.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.104+0000 7f8ef378d740 1 -- 172.21.15.112:0/17148095 shutdown_connections 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.104+0000 7f8ef378d740 1 --2- 172.21.15.112:0/17148095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00543c070 0x55c0057ec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.104+0000 7f8ef378d740 1 --2- 172.21.15.112:0/17148095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00541b390 0x55c0057ebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.104+0000 7f8ef378d740 1 --2- 172.21.15.112:0/17148095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c0057e4de0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.104+0000 7f8ef378d740 1 -- 172.21.15.112:0/17148095 >> 172.21.15.112:0/17148095 conn(0x55c005430460 msgr2=0x55c0057e46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.104+0000 7f8ef378d740 1 -- 172.21.15.112:0/17148095 shutdown_connections 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.104+0000 7f8ef378d740 1 -- 172.21.15.112:0/17148095 wait complete. 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.104+0000 7f8ef378d740 1 -- start start 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.104+0000 7f8ef378d740 1 Processor -- start 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c005803c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00541b390 0x55c005804270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ef378d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00543c070 0x55c0058639a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ef378d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81adc30 con 0x55c0054131b0 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ef378d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81af0a0 con 0x55c00543c070 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ef378d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee81ae300 con 0x55c00541b390 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ef378d740 10 client.0 _ll_register_callbacks cb 0x7f8ed000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ee759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c005803c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ee759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c005803c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60810/0 (socket says 172.21.15.112:60810) 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ee759e640 1 -- 172.21.15.112:0/2821523166 learned_addr learned my addr 172.21.15.112:0/2821523166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ede59c640 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00541b390 0x55c005804270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ee759e640 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00543c070 0x55c0058639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ee759e640 1 -- 172.21.15.112:0/2821523166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00541b390 msgr2=0x55c005804270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ee759e640 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00541b390 0x55c005804270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ee759e640 1 -- 172.21.15.112:0/2821523166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00543c070 msgr2=0x55c0058639a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ee759e640 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00543c070 0x55c0058639a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ee759e640 1 -- 172.21.15.112:0/2821523166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c005864010 con 0x55c0054131b0 2024-09-21T03:34:13.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ee759e640 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c005803c90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0011740 tx=0x7f8ee000b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caf4e823fda3ac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ece7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2821523166 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0025050 con 0x55c0054131b0 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.105+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2821523166 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f8ee0035420 con 0x55c0054131b0 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ef378d740 1 -- 172.21.15.112:0/2821523166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee0023070 con 0x55c0054131b0 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2821523166 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ee0012e90 con 0x55c0054131b0 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ef378d740 10 client.16431 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ef378d740 1 -- 172.21.15.112:0/2821523166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c00586b400 con 0x55c0054131b0 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2821523166 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee0023070 con 0x55c0054131b0 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2821523166 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f8ee002e510 con 0x55c0054131b0 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ef378d740 10 client.16431 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ef378d740 10 client.16431 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ef378d740 20 client.16431 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:13.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c0053cfda0 0x55c00586db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ef378d740 4 client.16431 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ef378d740 1 -- 172.21.15.112:0/2821523166 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0057578a0 con 0x55c0053cfda0 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.106+0000 7f8ede59c640 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c0053cfda0 0x55c00586db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ece7fc640 10 client.16431 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ede59c640 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c0053cfda0 0x55c00586db50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=953c4c947ea0850c server_cookie=87b9fb4be75cdebc in_seq=0 out_seq=0 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ece7fc640 1 -- 172.21.15.112:0/2821523166 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0057578a0 con 0x55c0053cfda0 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ece7fc640 10 client.16431 handle_command_reply: tid=0 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 1 client.16431 shutdown 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 1 -- 172.21.15.112:0/2821523166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 msgr2=0x55c005803c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c005803c90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0011740 tx=0x7f8ee000b470 comp rx=0 tx=0).stop 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 1 -- 172.21.15.112:0/2821523166 shutdown_connections 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c0053cfda0 0x55c00586db50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c00543c070 0x55c0058639a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c00541b390 0x55c005804270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 1 --2- 172.21.15.112:0/2821523166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0054131b0 0x55c005803c90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 1 -- 172.21.15.112:0/2821523166 >> 172.21.15.112:0/2821523166 conn(0x55c005430460 msgr2=0x55c0057f0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ece7fc640 0 client.16431 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:34:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 1 -- 172.21.15.112:0/2821523166 shutdown_connections 2024-09-21T03:34:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 1 -- 172.21.15.112:0/2821523166 wait complete. 2024-09-21T03:34:13.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.107+0000 7f8ef378d740 20 client.16431 trim_cache size 0 max 16384 2024-09-21T03:34:13.114 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:34:13.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 -- 172.21.15.112:0/1389952602 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed40b6680 msgr2=0x7f8ed40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:13.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed40b6680 0x7f8ed40b8b90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8ed81061c0 tx=0x7f8ed80095c0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 -- 172.21.15.112:0/1389952602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102a60 msgr2=0x7f8ee819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:13.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102a60 0x7f8ee819a440 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8006010 tx=0x7f8ed80319f0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 -- 172.21.15.112:0/1389952602 shutdown_connections 2024-09-21T03:34:13.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8ed40b6680 0x7f8ed40b8b90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ee8108cf0 0x7f8ee819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ee81083a0 0x7f8ee819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 --2- 172.21.15.112:0/1389952602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee8102a60 0x7f8ee819a440 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:13.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 -- 172.21.15.112:0/1389952602 >> 172.21.15.112:0/1389952602 conn(0x7f8ee8079730 msgr2=0x7f8ee8101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:13.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 -- 172.21.15.112:0/1389952602 shutdown_connections 2024-09-21T03:34:13.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:13.108+0000 7f8eed219640 1 -- 172.21.15.112:0/1389952602 wait complete. 2024-09-21T03:34:13.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 515744 10.94 MB/sec execute 133 sec latency 25.987 ms 2024-09-21T03:34:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:34:13.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:13 smithi112 ceph-mon[31442]: pgmap v474: 97 pgs: 97 active+clean; 317 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 322 op/s 2024-09-21T03:34:13.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:13.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:13.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:34:13.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:13 smithi203 ceph-mon[34723]: pgmap v474: 97 pgs: 97 active+clean; 317 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 322 op/s 2024-09-21T03:34:13.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:34:13.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:34:13.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:34:13.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:13 smithi187 ceph-mon[34603]: pgmap v474: 97 pgs: 97 active+clean; 317 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 322 op/s 2024-09-21T03:34:13.801 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:34:14.065 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:14.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 518324 10.96 MB/sec execute 134 sec latency 19.132 ms 2024-09-21T03:34:14.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.567+0000 7f9f17472640 1 -- 172.21.15.112:0/1526424637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f10102a60 msgr2=0x7f9f10102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.567+0000 7f9f17472640 1 --2- 172.21.15.112:0/1526424637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f10102a60 0x7f9f10102e70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009900 tx=0x7f9f0c02a410 comp rx=0 tx=0).stop 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.567+0000 7f9f17472640 1 -- 172.21.15.112:0/1526424637 shutdown_connections 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.567+0000 7f9f17472640 1 --2- 172.21.15.112:0/1526424637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f10108cf0 0x7f9f1010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.567+0000 7f9f17472640 1 --2- 172.21.15.112:0/1526424637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f101083a0 0x7f9f101087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.567+0000 7f9f17472640 1 --2- 172.21.15.112:0/1526424637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f10102a60 0x7f9f10102e70 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.567+0000 7f9f17472640 1 -- 172.21.15.112:0/1526424637 >> 172.21.15.112:0/1526424637 conn(0x7f9f10079730 msgr2=0x7f9f10079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.567+0000 7f9f17472640 1 -- 172.21.15.112:0/1526424637 shutdown_connections 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.567+0000 7f9f17472640 1 -- 172.21.15.112:0/1526424637 wait complete. 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f17472640 1 Processor -- start 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f17472640 1 -- start start 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f17472640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f10102a60 0x7f9f1019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f17472640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f101083a0 0x7f9f1019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f17472640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f10108cf0 0x7f9f101a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f17472640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10113a80 con 0x7f9f101083a0 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f17472640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10113900 con 0x7f9f10102a60 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f17472640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10113c00 con 0x7f9f10108cf0 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f14a0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f101083a0 0x7f9f1019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f14a0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f101083a0 0x7f9f1019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60830/0 (socket says 172.21.15.112:60830) 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f14a0d640 1 -- 172.21.15.112:0/2294257989 learned_addr learned my addr 172.21.15.112:0/2294257989 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.568+0000 7f9f1520e640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f10102a60 0x7f9f1019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f10108cf0 0x7f9f101a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f1520e640 1 -- 172.21.15.112:0/2294257989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f10108cf0 msgr2=0x7f9f101a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f1520e640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f10108cf0 0x7f9f101a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f1520e640 1 -- 172.21.15.112:0/2294257989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f101083a0 msgr2=0x7f9f1019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f1520e640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f101083a0 0x7f9f1019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f1520e640 1 -- 172.21.15.112:0/2294257989 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f101a25e0 con 0x7f9f10102a60 2024-09-21T03:34:14.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f14a0d640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f101083a0 0x7f9f1019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:14.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f1520e640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f10102a60 0x7f9f1019c670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c02a930 tx=0x7f9f0c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:14.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f067fc640 1 -- 172.21.15.112:0/2294257989 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c03f030 con 0x7f9f10102a60 2024-09-21T03:34:14.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f067fc640 1 -- 172.21.15.112:0/2294257989 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f0c004490 con 0x7f9f10102a60 2024-09-21T03:34:14.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f17472640 1 -- 172.21.15.112:0/2294257989 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f101a2870 con 0x7f9f10102a60 2024-09-21T03:34:14.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f067fc640 1 -- 172.21.15.112:0/2294257989 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c035070 con 0x7f9f10102a60 2024-09-21T03:34:14.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.569+0000 7f9f17472640 1 -- 172.21.15.112:0/2294257989 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f101a9030 con 0x7f9f10102a60 2024-09-21T03:34:14.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f1d9e6740 1 Processor -- start 2024-09-21T03:34:14.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f1d9e6740 1 -- start start 2024-09-21T03:34:14.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f067fc640 1 -- 172.21.15.112:0/2294257989 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9f0c042030 con 0x7f9f10102a60 2024-09-21T03:34:14.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 0x55945878d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x55945878d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 0x55945878dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559458703ea0 con 0x5594583c5390 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5594586f32d0 con 0x5594583bd1b0 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5594586f70e0 con 0x5594583e6070 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f15a0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 0x55945878d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f1520e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x55945878d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f15a0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 0x55945878d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:37676/0 (socket says 172.21.15.112:37676) 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f15a0f640 1 -- 172.21.15.112:0/803648631 learned_addr learned my addr 172.21.15.112:0/803648631 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.572+0000 7f9f14a0d640 1 --2- 172.21.15.112:0/803648631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 0x55945878dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9f15a0f640 1 -- 172.21.15.112:0/803648631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 msgr2=0x55945878d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/803648631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x55945878d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9f15a0f640 1 -- 172.21.15.112:0/803648631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 msgr2=0x55945878dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/803648631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 0x55945878dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9f15a0f640 1 -- 172.21.15.112:0/803648631 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559458451650 con 0x5594583e6070 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9f067fc640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9efc0b6700 0x7f9efc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/803648631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 0x55945878d2c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009fd0 tx=0x7f9f080118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5d7398c52aca1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9f1520e640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9efc0b6700 0x7f9efc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9efb7fe640 1 -- 172.21.15.112:0/803648631 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08004d20 con 0x5594583e6070 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9f067fc640 1 -- 172.21.15.112:0/2294257989 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f9f0c107880 con 0x7f9f10102a60 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9efb7fe640 1 -- 172.21.15.112:0/803648631 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f08004050 con 0x5594583e6070 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.573+0000 7f9efb7fe640 1 -- 172.21.15.112:0/803648631 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08005260 con 0x5594583e6070 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.574+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/803648631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 msgr2=0x55945878d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.574+0000 7f9f1520e640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9efc0b6700 0x7f9efc0b8c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c1050e0 tx=0x7f9f0c10d550 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.574+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/803648631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 0x55945878d2c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009fd0 tx=0x7f9f080118d0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.574+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/803648631 shutdown_connections 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.574+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/803648631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 0x55945878dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.574+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/803648631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x55945878d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.574+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/803648631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 0x55945878d2c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.574+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/803648631 >> 172.21.15.112:0/803648631 conn(0x5594583da460 msgr2=0x55945878b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.574+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/803648631 shutdown_connections 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.574+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/803648631 wait complete. 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.575+0000 7f9f1d9e6740 1 -- start start 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.575+0000 7f9f1d9e6740 1 Processor -- start 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.575+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x5594587988f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.575+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 0x559458798e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.575+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 0x5594587993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.575+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559458554700 con 0x5594583c5390 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.575+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55945868e160 con 0x5594583bd1b0 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.575+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55945879baa0 con 0x5594583e6070 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.575+0000 7f9f1d9e6740 10 client.0 _ll_register_callbacks cb 0x5594586a79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:14.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f14a0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 0x559458798e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f15a0f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x5594587988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f14a0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 0x559458798e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60842/0 (socket says 172.21.15.112:60842) 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f14a0d640 1 -- 172.21.15.112:0/2678216472 learned_addr learned my addr 172.21.15.112:0/2678216472 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x5594587988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40022/0 (socket says 172.21.15.112:40022) 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 0x5594587993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f14a0d640 1 -- 172.21.15.112:0/2678216472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 msgr2=0x5594587993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f14a0d640 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 0x5594587993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f14a0d640 1 -- 172.21.15.112:0/2678216472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 msgr2=0x5594587988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f14a0d640 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x5594587988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f14a0d640 1 -- 172.21.15.112:0/2678216472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594586bbc90 con 0x5594583c5390 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f14a0d640 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 0x559458798e30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9f00012d20 tx=0x7f9f0000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a5fdd1aade38df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9ef8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/2678216472 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f000120d0 con 0x5594583c5390 2024-09-21T03:34:14.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/2678216472 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f9f000047a0 con 0x5594583c5390 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/2678216472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f08004050 con 0x5594583c5390 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/2678216472 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f00012270 con 0x5594583c5390 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f1d9e6740 10 client.16443 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.576+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/2678216472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5594587a77d0 con 0x5594583c5390 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.577+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/2678216472 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f08004050 con 0x5594583c5390 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.577+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/2678216472 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f9f00010660 con 0x5594583c5390 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.577+0000 7f9f1d9e6740 10 client.16443 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.577+0000 7f9f1d9e6740 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.577+0000 7f9f1d9e6740 20 client.16443 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.577+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5594583a49f0 0x5594587aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.577+0000 7f9f1d9e6740 4 client.16443 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.577+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/2678216472 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559458705df0 con 0x5594583a49f0 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.577+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5594583a49f0 0x5594587aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.577+0000 7f9ef8ff9640 10 client.16443 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:14.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5594583a49f0 0x5594587aacd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bed2a420f2fb407 server_cookie=52925186be050152 in_seq=0 out_seq=0 2024-09-21T03:34:14.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/2678216472 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559458705df0 con 0x5594583a49f0 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9ef8ff9640 10 client.16443 handle_command_reply: tid=0 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9f1d9e6740 1 client.16443 shutdown 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/2678216472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 msgr2=0x559458798e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 0x559458798e30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9f00012d20 tx=0x7f9f0000f790 comp rx=0 tx=0).stop 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/2678216472 shutdown_connections 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5594583a49f0 0x5594587aacd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583e6070 0x5594587993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583c5390 0x559458798e30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/2678216472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x5594587988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.578+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/2678216472 >> 172.21.15.112:0/2678216472 conn(0x5594583da460 msgr2=0x559458790450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.579+0000 7f9ef8ff9640 0 client.16443 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:14.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.579+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/2678216472 shutdown_connections 2024-09-21T03:34:14.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.579+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/2678216472 wait complete. 2024-09-21T03:34:14.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.579+0000 7f9f1d9e6740 20 client.16443 trim_cache size 0 max 16384 2024-09-21T03:34:14.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.592+0000 7f9f1d9e6740 1 Processor -- start 2024-09-21T03:34:14.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.592+0000 7f9f1d9e6740 1 -- start start 2024-09-21T03:34:14.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583bd1b0 0x55945879a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 0x55945878e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583e6070 0x5594587a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101b40d0 con 0x5594583bd1b0 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101b1970 con 0x5594583e6070 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101af7c0 con 0x5594583c5390 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f15a0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583bd1b0 0x55945879a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f15a0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583bd1b0 0x55945879a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60848/0 (socket says 172.21.15.112:60848) 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f15a0f640 1 -- 172.21.15.112:0/4079199647 learned_addr learned my addr 172.21.15.112:0/4079199647 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/4079199647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583e6070 0x5594587a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f14a0d640 1 --2- 172.21.15.112:0/4079199647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 0x55945878e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f15a0f640 1 -- 172.21.15.112:0/4079199647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 msgr2=0x55945878e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/4079199647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 0x55945878e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f15a0f640 1 -- 172.21.15.112:0/4079199647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583e6070 msgr2=0x5594587a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/4079199647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583e6070 0x5594587a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f15a0f640 1 -- 172.21.15.112:0/4079199647 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945878b6f0 con 0x5594583bd1b0 2024-09-21T03:34:14.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.593+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/4079199647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583bd1b0 0x55945879a7d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9f08011910 tx=0x7f9f08005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352a87eaa1d9031e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:14.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.594+0000 7f9efaffd640 1 -- 172.21.15.112:0/4079199647 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08003fd0 con 0x5594583bd1b0 2024-09-21T03:34:14.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.594+0000 7f9efaffd640 1 -- 172.21.15.112:0/4079199647 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f08004620 con 0x5594583bd1b0 2024-09-21T03:34:14.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.594+0000 7f9efaffd640 1 -- 172.21.15.112:0/4079199647 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0800fd40 con 0x5594583bd1b0 2024-09-21T03:34:14.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.594+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/4079199647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583bd1b0 msgr2=0x55945879a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:14.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.594+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/4079199647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583bd1b0 0x55945879a7d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9f08011910 tx=0x7f9f08005bd0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.594+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/4079199647 shutdown_connections 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.594+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/4079199647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583e6070 0x5594587a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.594+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/4079199647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 0x55945878e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.594+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/4079199647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583bd1b0 0x55945879a7d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.594+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/4079199647 >> 172.21.15.112:0/4079199647 conn(0x5594583da460 msgr2=0x5594587a3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.595+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/4079199647 shutdown_connections 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.595+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/4079199647 wait complete. 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.595+0000 7f9f1d9e6740 1 -- start start 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.595+0000 7f9f1d9e6740 1 Processor -- start 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.595+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x5594587a1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.595+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 0x55945879ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.595+0000 7f9f1d9e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583e6070 0x55945879cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.595+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101afe30 con 0x5594583e6070 2024-09-21T03:34:14.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.595+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101b12a0 con 0x5594583bd1b0 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.595+0000 7f9f1d9e6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101b0500 con 0x5594583c5390 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f1d9e6740 10 client.0 _ll_register_callbacks cb 0x7f9f00009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f15a0f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x5594587a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f14a0d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 0x55945879ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f15a0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583e6070 0x55945879cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f15a0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583e6070 0x55945879cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60858/0 (socket says 172.21.15.112:60858) 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f14a0d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 0x55945879ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:37710/0 (socket says 172.21.15.112:37710) 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f15a0f640 1 -- 172.21.15.112:0/505133193 learned_addr learned my addr 172.21.15.112:0/505133193 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f14a0d640 1 -- 172.21.15.112:0/505133193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 msgr2=0x5594587a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f14a0d640 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x5594587a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f14a0d640 1 -- 172.21.15.112:0/505133193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583e6070 msgr2=0x55945879cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f14a0d640 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583e6070 0x55945879cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f14a0d640 1 -- 172.21.15.112:0/505133193 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945879d750 con 0x5594583c5390 2024-09-21T03:34:14.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583e6070 0x55945879cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f14a0d640 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 0x55945879ca20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f00010a30 tx=0x7f9f0000fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44b607ab028c0582 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9ef8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/505133193 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00005250 con 0x5594583c5390 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/505133193 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f9f000053f0 con 0x5594583c5390 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.596+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/505133193 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f08004620 con 0x5594583c5390 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/505133193 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f000055f0 con 0x5594583c5390 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9f1d9e6740 10 client.26014 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/505133193 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559458815690 con 0x5594583c5390 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/505133193 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f08004620 con 0x5594583c5390 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/505133193 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f9f00025470 con 0x5594583c5390 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9f1d9e6740 10 client.26014 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9f1d9e6740 10 client.26014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:14.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9f1d9e6740 20 client.26014 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559458379da0 0x559458817e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9f1d9e6740 4 client.26014 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/505133193 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594587018a0 con 0x559458379da0 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.597+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559458379da0 0x559458817e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9ef8ff9640 10 client.26014 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f15a0f640 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559458379da0 0x559458817e30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ebde4d0cefce1aa server_cookie=b42262f42a963c7e in_seq=0 out_seq=0 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9ef8ff9640 1 -- 172.21.15.112:0/505133193 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594587018a0 con 0x559458379da0 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9ef8ff9640 10 client.26014 handle_command_reply: tid=0 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 1 client.26014 shutdown 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/505133193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 msgr2=0x55945879ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 0x55945879ca20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f00010a30 tx=0x7f9f0000fc90 comp rx=0 tx=0).stop 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/505133193 shutdown_connections 2024-09-21T03:34:14.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x559458379da0 0x559458817e30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5594583e6070 0x55945879cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5594583c5390 0x55945879ca20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9ef8ff9640 0 client.26014 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 1 --2- 172.21.15.112:0/505133193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5594583bd1b0 0x5594587a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/505133193 >> 172.21.15.112:0/505133193 conn(0x5594583da460 msgr2=0x5594587a4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/505133193 shutdown_connections 2024-09-21T03:34:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 1 -- 172.21.15.112:0/505133193 wait complete. 2024-09-21T03:34:14.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.598+0000 7f9f1d9e6740 20 client.26014 trim_cache size 0 max 16384 2024-09-21T03:34:14.605 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:34:14.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 -- 172.21.15.112:0/2294257989 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9efc0b6700 msgr2=0x7f9efc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:14.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9efc0b6700 0x7f9efc0b8c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c1050e0 tx=0x7f9f0c10d550 comp rx=0 tx=0).stop 2024-09-21T03:34:14.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 -- 172.21.15.112:0/2294257989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f10102a60 msgr2=0x7f9f1019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:14.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f10102a60 0x7f9f1019c670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c02a930 tx=0x7f9f0c031a20 comp rx=0 tx=0).stop 2024-09-21T03:34:14.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 -- 172.21.15.112:0/2294257989 shutdown_connections 2024-09-21T03:34:14.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f9efc0b6700 0x7f9efc0b8c10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f10108cf0 0x7f9f101a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f101083a0 0x7f9f1019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 --2- 172.21.15.112:0/2294257989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f10102a60 0x7f9f1019c670 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 -- 172.21.15.112:0/2294257989 >> 172.21.15.112:0/2294257989 conn(0x7f9f10079730 msgr2=0x7f9f1010d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 -- 172.21.15.112:0/2294257989 shutdown_connections 2024-09-21T03:34:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:14.599+0000 7f9f17472640 1 -- 172.21.15.112:0/2294257989 wait complete. 2024-09-21T03:34:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:14 smithi112 ceph-mon[31442]: pgmap v475: 97 pgs: 97 active+clean; 323 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 300 op/s 2024-09-21T03:34:14.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:14 smithi203 ceph-mon[34723]: pgmap v475: 97 pgs: 97 active+clean; 323 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 300 op/s 2024-09-21T03:34:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:14 smithi187 ceph-mon[34603]: pgmap v475: 97 pgs: 97 active+clean; 323 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 300 op/s 2024-09-21T03:34:15.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 519836 10.94 MB/sec execute 135 sec latency 25.283 ms 2024-09-21T03:34:16.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 522443 10.98 MB/sec execute 136 sec latency 15.269 ms 2024-09-21T03:34:16.273 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:34:16.273 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:34:16.538 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:16.659 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:16 smithi112 ceph-mon[31442]: pgmap v476: 97 pgs: 97 active+clean; 340 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 320 op/s 2024-09-21T03:34:16.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:16 smithi203 ceph-mon[34723]: pgmap v476: 97 pgs: 97 active+clean; 340 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 320 op/s 2024-09-21T03:34:16.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:16 smithi187 ceph-mon[34603]: pgmap v476: 97 pgs: 97 active+clean; 340 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 320 op/s 2024-09-21T03:34:17.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.051+0000 7f0eb1455640 1 -- 172.21.15.112:0/3419168260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eac1083a0 msgr2=0x7f0eac1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.051+0000 7f0eb1455640 1 --2- 172.21.15.112:0/3419168260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eac1083a0 0x7f0eac1087b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009900 tx=0x7f0e9402a410 comp rx=0 tx=0).stop 2024-09-21T03:34:17.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.051+0000 7f0eb1455640 1 -- 172.21.15.112:0/3419168260 shutdown_connections 2024-09-21T03:34:17.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.051+0000 7f0eb1455640 1 --2- 172.21.15.112:0/3419168260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac108cf0 0x7f0eac10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.051+0000 7f0eb1455640 1 --2- 172.21.15.112:0/3419168260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eac1083a0 0x7f0eac1087b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.051+0000 7f0eb1455640 1 --2- 172.21.15.112:0/3419168260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eac102a60 0x7f0eac102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.051+0000 7f0eb1455640 1 -- 172.21.15.112:0/3419168260 >> 172.21.15.112:0/3419168260 conn(0x7f0eac079730 msgr2=0x7f0eac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:17.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.051+0000 7f0eb1455640 1 -- 172.21.15.112:0/3419168260 shutdown_connections 2024-09-21T03:34:17.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.051+0000 7f0eb1455640 1 -- 172.21.15.112:0/3419168260 wait complete. 2024-09-21T03:34:17.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eb1455640 1 Processor -- start 2024-09-21T03:34:17.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eb1455640 1 -- start start 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eb1455640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eac102a60 0x7f0eac19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eb1455640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eac1083a0 0x7f0eac19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eb1455640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac108cf0 0x7f0eac19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eb1455640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac113a80 con 0x7f0eac102a60 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eb1455640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac113900 con 0x7f0eac1083a0 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eb1455640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac113c00 con 0x7f0eac108cf0 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eaaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eac102a60 0x7f0eac19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0ea3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eac1083a0 0x7f0eac19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eaaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eac102a60 0x7f0eac19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39898/0 (socket says 172.21.15.112:39898) 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eaaffd640 1 -- 172.21.15.112:0/1295572036 learned_addr learned my addr 172.21.15.112:0/1295572036 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0ea3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eac1083a0 0x7f0eac19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:54626/0 (socket says 172.21.15.112:54626) 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.052+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac108cf0 0x7f0eac19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0eaaffd640 1 -- 172.21.15.112:0/1295572036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac108cf0 msgr2=0x7f0eac19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0eaaffd640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac108cf0 0x7f0eac19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0eaaffd640 1 -- 172.21.15.112:0/1295572036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eac1083a0 msgr2=0x7f0eac19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0eaaffd640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eac1083a0 0x7f0eac19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0eaaffd640 1 -- 172.21.15.112:0/1295572036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac1a03e0 con 0x7f0eac102a60 2024-09-21T03:34:17.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0ea3fff640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eac1083a0 0x7f0eac19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:17.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0eaaffd640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eac102a60 0x7f0eac19a470 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00e9f0 tx=0x7f0e9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0ea8ff9640 1 -- 172.21.15.112:0/1295572036 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00cd80 con 0x7f0eac102a60 2024-09-21T03:34:17.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0ea8ff9640 1 -- 172.21.15.112:0/1295572036 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e9c0108e0 con 0x7f0eac102a60 2024-09-21T03:34:17.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0eb1455640 1 -- 172.21.15.112:0/1295572036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac1a06d0 con 0x7f0eac102a60 2024-09-21T03:34:17.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0ea8ff9640 1 -- 172.21.15.112:0/1295572036 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c018980 con 0x7f0eac102a60 2024-09-21T03:34:17.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.053+0000 7f0eb1455640 1 -- 172.21.15.112:0/1295572036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac1a6e30 con 0x7f0eac102a60 2024-09-21T03:34:17.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.055+0000 7f0eb79c9740 1 Processor -- start 2024-09-21T03:34:17.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0eb79c9740 1 -- start start 2024-09-21T03:34:17.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258730350 0x560258ad7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 0x560258ad7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56025870f670 0x560258ad7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560258a4de60 con 0x560258707490 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560258a3d290 con 0x560258730350 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560258a410a0 con 0x56025870f670 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0eab7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258730350 0x560258ad7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0ea3fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56025870f670 0x560258ad7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0eab7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258730350 0x560258ad7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:54638/0 (socket says 172.21.15.112:54638) 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0eab7fe640 1 -- 172.21.15.112:0/4109488991 learned_addr learned my addr 172.21.15.112:0/4109488991 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.056+0000 7f0ea8ff9640 1 -- 172.21.15.112:0/1295572036 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0e9c018b20 con 0x7f0eac102a60 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0eaaffd640 1 --2- 172.21.15.112:0/4109488991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 0x560258ad7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0eab7fe640 1 -- 172.21.15.112:0/4109488991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56025870f670 msgr2=0x560258ad7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/4109488991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56025870f670 0x560258ad7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0eab7fe640 1 -- 172.21.15.112:0/4109488991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 msgr2=0x560258ad7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/4109488991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 0x560258ad7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0eab7fe640 1 -- 172.21.15.112:0/4109488991 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56025879b930 con 0x560258730350 2024-09-21T03:34:17.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0ea3fff640 1 --2- 172.21.15.112:0/4109488991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56025870f670 0x560258ad7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0eaaffd640 1 --2- 172.21.15.112:0/4109488991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 0x560258ad7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/4109488991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258730350 0x560258ad7430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40039f0 tx=0x7f0ea4003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb909d9a18772048 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0ea17fa640 1 -- 172.21.15.112:0/4109488991 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea401a470 con 0x560258730350 2024-09-21T03:34:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0ea17fa640 1 -- 172.21.15.112:0/4109488991 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea4015070 con 0x560258730350 2024-09-21T03:34:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0ea17fa640 1 -- 172.21.15.112:0/4109488991 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40107e0 con 0x560258730350 2024-09-21T03:34:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.057+0000 7f0ea8ff9640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e980b6680 0x7f0e980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0eaaffd640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e980b6680 0x7f0e980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0ea8ff9640 1 -- 172.21.15.112:0/1295572036 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f0e9c0e79f0 con 0x7f0eac102a60 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0eb79c9740 1 -- 172.21.15.112:0/4109488991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258730350 msgr2=0x560258ad7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0eaaffd640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e980b6680 0x7f0e980b8b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0e5250 tx=0x7f0e9c000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/4109488991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258730350 0x560258ad7430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40039f0 tx=0x7f0ea4003eb0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0eb79c9740 1 -- 172.21.15.112:0/4109488991 shutdown_connections 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/4109488991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56025870f670 0x560258ad7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/4109488991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 0x560258ad7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/4109488991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258730350 0x560258ad7430 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0eb79c9740 1 -- 172.21.15.112:0/4109488991 >> 172.21.15.112:0/4109488991 conn(0x560258724740 msgr2=0x560258ad5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0eb79c9740 1 -- 172.21.15.112:0/4109488991 shutdown_connections 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.058+0000 7f0eb79c9740 1 -- 172.21.15.112:0/4109488991 wait complete. 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eb79c9740 1 -- start start 2024-09-21T03:34:17.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eb79c9740 1 Processor -- start 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258707490 0x560258ae3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56025870f670 0x560258ae3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258730350 0x560258aeef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56025889e890 con 0x56025870f670 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5602589d7d40 con 0x560258707490 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560258ae5c10 con 0x560258730350 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eb79c9740 10 client.0 _ll_register_callbacks cb 0x560258a3bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0ea3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56025870f670 0x560258ae3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eab7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258707490 0x560258ae3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0ea3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56025870f670 0x560258ae3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39930/0 (socket says 172.21.15.112:39930) 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eab7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258707490 0x560258ae3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:54652/0 (socket says 172.21.15.112:54652) 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0ea3fff640 1 -- 172.21.15.112:0/2220998749 learned_addr learned my addr 172.21.15.112:0/2220998749 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.059+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258730350 0x560258aeef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.060+0000 7f0ea3fff640 1 -- 172.21.15.112:0/2220998749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258730350 msgr2=0x560258aeef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.060+0000 7f0ea3fff640 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258730350 0x560258aeef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.060+0000 7f0ea3fff640 1 -- 172.21.15.112:0/2220998749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258707490 msgr2=0x560258ae3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.060+0000 7f0ea3fff640 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258707490 0x560258ae3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.060+0000 7f0ea3fff640 1 -- 172.21.15.112:0/2220998749 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560258aef4c0 con 0x56025870f670 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.060+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258707490 0x560258ae3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.060+0000 7f0ea3fff640 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56025870f670 0x560258ae3540 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009900 tx=0x7f0e94031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5be840666cf07e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.060+0000 7f0e927fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.060+0000 7f0e927fc640 1 -- 172.21.15.112:0/2220998749 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94035040 con 0x56025870f670 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.060+0000 7f0e927fc640 1 -- 172.21.15.112:0/2220998749 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f0e94004240 con 0x56025870f670 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.060+0000 7f0eb79c9740 1 -- 172.21.15.112:0/2220998749 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea4015070 con 0x56025870f670 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0e927fc640 1 -- 172.21.15.112:0/2220998749 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e94004440 con 0x56025870f670 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0eb79c9740 10 client.16458 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0eb79c9740 1 -- 172.21.15.112:0/2220998749 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560258af1d20 con 0x56025870f670 2024-09-21T03:34:17.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0e927fc640 1 -- 172.21.15.112:0/2220998749 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea4015070 con 0x56025870f670 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0e927fc640 1 -- 172.21.15.112:0/2220998749 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f0e94004b50 con 0x56025870f670 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0eb79c9740 10 client.16458 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0eb79c9740 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0eb79c9740 20 client.16458 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5602586eecd0 0x560258af51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0eb79c9740 4 client.16458 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0eb79c9740 1 -- 172.21.15.112:0/2220998749 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560258a4fdb0 con 0x5602586eecd0 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.061+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5602586eecd0 0x560258af51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.062+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5602586eecd0 0x560258af51f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734beac12d16ac3 server_cookie=d308cad580594b2c in_seq=0 out_seq=0 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.062+0000 7f0e927fc640 10 client.16458 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0e927fc640 1 -- 172.21.15.112:0/2220998749 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560258a4fdb0 con 0x5602586eecd0 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0e927fc640 10 client.16458 handle_command_reply: tid=0 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 1 client.16458 shutdown 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 1 -- 172.21.15.112:0/2220998749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56025870f670 msgr2=0x560258ae3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56025870f670 0x560258ae3540 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009900 tx=0x7f0e94031810 comp rx=0 tx=0).stop 2024-09-21T03:34:17.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 1 -- 172.21.15.112:0/2220998749 shutdown_connections 2024-09-21T03:34:17.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5602586eecd0 0x560258af51f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258730350 0x560258aeef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56025870f670 0x560258ae3540 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0e927fc640 0 client.16458 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:17.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/2220998749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560258707490 0x560258ae3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 1 -- 172.21.15.112:0/2220998749 >> 172.21.15.112:0/2220998749 conn(0x560258724740 msgr2=0x560258ada5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:17.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 1 -- 172.21.15.112:0/2220998749 shutdown_connections 2024-09-21T03:34:17.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 1 -- 172.21.15.112:0/2220998749 wait complete. 2024-09-21T03:34:17.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.063+0000 7f0eb79c9740 20 client.16458 trim_cache size 0 max 16384 2024-09-21T03:34:17.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eb79c9740 1 Processor -- start 2024-09-21T03:34:17.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eb79c9740 1 -- start start 2024-09-21T03:34:17.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258707490 0x560258aeee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 0x560258add550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258730350 0x560258ae1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1b1e70 con 0x560258730350 2024-09-21T03:34:17.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1af710 con 0x56025870f670 2024-09-21T03:34:17.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1ad560 con 0x560258707490 2024-09-21T03:34:17.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258730350 0x560258ae1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258730350 0x560258ae1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39942/0 (socket says 172.21.15.112:39942) 2024-09-21T03:34:17.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eab7fe640 1 -- 172.21.15.112:0/141937429 learned_addr learned my addr 172.21.15.112:0/141937429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:17.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/141937429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258707490 0x560258aeee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0ea3fff640 1 --2- 172.21.15.112:0/141937429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 0x560258add550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eab7fe640 1 -- 172.21.15.112:0/141937429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258707490 msgr2=0x560258aeee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:17.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/141937429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258707490 0x560258aeee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eab7fe640 1 -- 172.21.15.112:0/141937429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 msgr2=0x560258add550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:17.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.081+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/141937429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 0x560258add550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.082+0000 7f0eab7fe640 1 -- 172.21.15.112:0/141937429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560258ae2750 con 0x560258730350 2024-09-21T03:34:17.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.082+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/141937429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258730350 0x560258ae1680 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4014040 tx=0x7f0ea4010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772338c1763d3c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.082+0000 7f0ea0ff9640 1 -- 172.21.15.112:0/141937429 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4023040 con 0x560258730350 2024-09-21T03:34:17.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.082+0000 7f0ea0ff9640 1 -- 172.21.15.112:0/141937429 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea4015040 con 0x560258730350 2024-09-21T03:34:17.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.082+0000 7f0ea0ff9640 1 -- 172.21.15.112:0/141937429 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40104b0 con 0x560258730350 2024-09-21T03:34:17.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.083+0000 7f0eb79c9740 1 -- 172.21.15.112:0/141937429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258730350 msgr2=0x560258ae1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.083+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/141937429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258730350 0x560258ae1680 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4014040 tx=0x7f0ea4010440 comp rx=0 tx=0).stop 2024-09-21T03:34:17.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.083+0000 7f0eb79c9740 1 -- 172.21.15.112:0/141937429 shutdown_connections 2024-09-21T03:34:17.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.083+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/141937429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258730350 0x560258ae1680 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.083+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/141937429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 0x560258add550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.083+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/141937429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258707490 0x560258aeee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.083+0000 7f0eb79c9740 1 -- 172.21.15.112:0/141937429 >> 172.21.15.112:0/141937429 conn(0x560258724740 msgr2=0x560258addf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:17.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.083+0000 7f0eb79c9740 1 -- 172.21.15.112:0/141937429 shutdown_connections 2024-09-21T03:34:17.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.083+0000 7f0eb79c9740 1 -- 172.21.15.112:0/141937429 wait complete. 2024-09-21T03:34:17.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.083+0000 7f0eb79c9740 1 -- start start 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.083+0000 7f0eb79c9740 1 Processor -- start 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 0x560258ae74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 0x560258ae7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0eb79c9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258730350 0x560258b5ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1adbd0 con 0x560258707490 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1af040 con 0x56025870f670 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0eb79c9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1ae2a0 con 0x560258730350 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0eb79c9740 10 client.0 _ll_register_callbacks cb 0x7f0e94009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0eab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 0x560258ae74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0ea3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 0x560258ae7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0ea3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 0x560258ae7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:54668/0 (socket says 172.21.15.112:54668) 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0ea3fff640 1 -- 172.21.15.112:0/558064455 learned_addr learned my addr 172.21.15.112:0/558064455 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258730350 0x560258b5ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0ea3fff640 1 -- 172.21.15.112:0/558064455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258730350 msgr2=0x560258b5ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0ea3fff640 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258730350 0x560258b5ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0ea3fff640 1 -- 172.21.15.112:0/558064455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 msgr2=0x560258ae74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0ea3fff640 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 0x560258ae74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0ea3fff640 1 -- 172.21.15.112:0/558064455 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560258b5f130 con 0x56025870f670 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 0x560258ae74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0ea3fff640 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 0x560258ae7a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e94003640 tx=0x7f0e94004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04facbf9c71a614 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.084+0000 7f0e927fc640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0e927fc640 1 -- 172.21.15.112:0/558064455 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94035050 con 0x56025870f670 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0eb79c9740 1 -- 172.21.15.112:0/558064455 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea4015040 con 0x56025870f670 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0e927fc640 1 -- 172.21.15.112:0/558064455 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f0e94005410 con 0x56025870f670 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0e927fc640 1 -- 172.21.15.112:0/558064455 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e94061050 con 0x56025870f670 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0eb79c9740 10 client.26030 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0eb79c9740 1 -- 172.21.15.112:0/558064455 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560258b60360 con 0x56025870f670 2024-09-21T03:34:17.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0e927fc640 1 -- 172.21.15.112:0/558064455 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea4015040 con 0x56025870f670 2024-09-21T03:34:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0e927fc640 1 -- 172.21.15.112:0/558064455 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f0e940040a0 con 0x56025870f670 2024-09-21T03:34:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0eb79c9740 10 client.26030 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0eb79c9740 10 client.26030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0eb79c9740 20 client.26030 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5602586c4080 0x560258b62b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0eb79c9740 4 client.26030 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:34:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0eb79c9740 1 -- 172.21.15.112:0/558064455 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560258a4b860 con 0x5602586c4080 2024-09-21T03:34:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.085+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5602586c4080 0x560258b62b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.086+0000 7f0eab7fe640 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5602586c4080 0x560258b62b80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60a4c1a43ee0505 server_cookie=41ba58225af829d8 in_seq=0 out_seq=0 2024-09-21T03:34:17.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.086+0000 7f0e927fc640 10 client.26030 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:17.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0e927fc640 1 -- 172.21.15.112:0/558064455 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560258a4b860 con 0x5602586c4080 2024-09-21T03:34:17.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0e927fc640 10 client.26030 handle_command_reply: tid=0 2024-09-21T03:34:17.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0eb79c9740 1 client.26030 shutdown 2024-09-21T03:34:17.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0eb79c9740 1 -- 172.21.15.112:0/558064455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 msgr2=0x560258ae7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 0x560258ae7a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e94003640 tx=0x7f0e94004cf0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0eb79c9740 1 -- 172.21.15.112:0/558064455 shutdown_connections 2024-09-21T03:34:17.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5602586c4080 0x560258b62b80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560258730350 0x560258b5ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0e927fc640 0 client.26030 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:17.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56025870f670 0x560258ae7a30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0eb79c9740 1 --2- 172.21.15.112:0/558064455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560258707490 0x560258ae74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0eb79c9740 1 -- 172.21.15.112:0/558064455 >> 172.21.15.112:0/558064455 conn(0x560258724740 msgr2=0x560258addf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:17.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0eb79c9740 1 -- 172.21.15.112:0/558064455 shutdown_connections 2024-09-21T03:34:17.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.087+0000 7f0eb79c9740 1 -- 172.21.15.112:0/558064455 wait complete. 2024-09-21T03:34:17.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb79c9740 20 client.26030 trim_cache size 0 max 16384 2024-09-21T03:34:17.093 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:34:17.093 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:34:17.094 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "7f611a1d-7b3b-4b63-add4-85afb713d24a", 2024-09-21T03:34:17.094 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:34:17.094 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:34:17.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 -- 172.21.15.112:0/1295572036 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e980b6680 msgr2=0x7f0e980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e980b6680 0x7f0e980b8b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0e5250 tx=0x7f0e9c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 -- 172.21.15.112:0/1295572036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eac102a60 msgr2=0x7f0eac19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:17.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eac102a60 0x7f0eac19a470 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00e9f0 tx=0x7f0e9c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 -- 172.21.15.112:0/1295572036 shutdown_connections 2024-09-21T03:34:17.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0e980b6680 0x7f0e980b8b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eac108cf0 0x7f0eac19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eac1083a0 0x7f0eac19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 --2- 172.21.15.112:0/1295572036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0eac102a60 0x7f0eac19a470 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:17.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 -- 172.21.15.112:0/1295572036 >> 172.21.15.112:0/1295572036 conn(0x7f0eac079730 msgr2=0x7f0eac104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:17.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 -- 172.21.15.112:0/1295572036 shutdown_connections 2024-09-21T03:34:17.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:17.088+0000 7f0eb1455640 1 -- 172.21.15.112:0/1295572036 wait complete. 2024-09-21T03:34:17.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 523808 10.94 MB/sec execute 137 sec latency 44.342 ms 2024-09-21T03:34:17.764 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:17 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:34:17.764 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:17 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:17.764 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:17 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:34:17.764 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:17 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:34:17.764 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:17 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:17.789 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:34:17.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:17 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:34:17.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:17 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:17.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:17 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:34:17.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:17 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:34:17.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:17 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:17.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:17 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:34:17.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:17 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:17.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:17 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:34:17.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:17 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:34:17.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:17 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:18.071 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:18.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 526083 10.96 MB/sec execute 138 sec latency 16.255 ms 2024-09-21T03:34:18.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.602+0000 7f71a04b7640 1 -- 172.21.15.112:0/834106797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7198108d20 msgr2=0x7f719810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.602+0000 7f71a04b7640 1 --2- 172.21.15.112:0/834106797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7198108d20 0x7f719810f550 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7190009ca0 tx=0x7f719002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.602+0000 7f71a04b7640 1 -- 172.21.15.112:0/834106797 shutdown_connections 2024-09-21T03:34:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.602+0000 7f71a04b7640 1 --2- 172.21.15.112:0/834106797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7198108d20 0x7f719810f550 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.602+0000 7f71a04b7640 1 --2- 172.21.15.112:0/834106797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71981083d0 0x7f71981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.602+0000 7f71a04b7640 1 --2- 172.21.15.112:0/834106797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7198102ae0 0x7f7198102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.602+0000 7f71a04b7640 1 -- 172.21.15.112:0/834106797 >> 172.21.15.112:0/834106797 conn(0x7f7198079790 msgr2=0x7f7198079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.602+0000 7f71a04b7640 1 -- 172.21.15.112:0/834106797 shutdown_connections 2024-09-21T03:34:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.602+0000 7f71a04b7640 1 -- 172.21.15.112:0/834106797 wait complete. 2024-09-21T03:34:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.603+0000 7f71a04b7640 1 Processor -- start 2024-09-21T03:34:19.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.603+0000 7f71a04b7640 1 -- start start 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.603+0000 7f71a04b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7198102ae0 0x7f719819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.603+0000 7f71a04b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71981083d0 0x7f719819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.603+0000 7f71a04b7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7198108d20 0x7f71981a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.603+0000 7f71a04b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71981129e0 con 0x7f71981083d0 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.603+0000 7f71a04b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7198112860 con 0x7f7198102ae0 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.603+0000 7f71a04b7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7198112b60 con 0x7f7198108d20 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719da52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71981083d0 0x7f719819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719da52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71981083d0 0x7f719819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39972/0 (socket says 172.21.15.112:39972) 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719da52640 1 -- 172.21.15.112:0/3250812789 learned_addr learned my addr 172.21.15.112:0/3250812789 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719ea54640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7198108d20 0x7f71981a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719e253640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7198102ae0 0x7f719819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719da52640 1 -- 172.21.15.112:0/3250812789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7198108d20 msgr2=0x7f71981a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719da52640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7198108d20 0x7f71981a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719da52640 1 -- 172.21.15.112:0/3250812789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7198102ae0 msgr2=0x7f719819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719da52640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7198102ae0 0x7f719819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719da52640 1 -- 172.21.15.112:0/3250812789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71981a2620 con 0x7f71981083d0 2024-09-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719ea54640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7198108d20 0x7f71981a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719e253640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7198102ae0 0x7f719819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f719da52640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71981083d0 0x7f719819cc50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f719400e9c0 tx=0x7f719400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f718f7fe640 1 -- 172.21.15.112:0/3250812789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719400ccc0 con 0x7f71981083d0 2024-09-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f718f7fe640 1 -- 172.21.15.112:0/3250812789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71940054d0 con 0x7f71981083d0 2024-09-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f71a04b7640 1 -- 172.21.15.112:0/3250812789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71981a2910 con 0x7f71981083d0 2024-09-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f718f7fe640 1 -- 172.21.15.112:0/3250812789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194010430 con 0x7f71981083d0 2024-09-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.604+0000 7f71a04b7640 1 -- 172.21.15.112:0/3250812789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71981a9070 con 0x7f71981083d0 2024-09-21T03:34:19.202 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:18 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:34:19.202 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:18 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:19.202 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:18 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:19.202 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:18 smithi187 ceph-mon[34603]: pgmap v477: 97 pgs: 97 active+clean; 344 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 246 op/s 2024-09-21T03:34:19.203 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:18 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:34:19.203 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:18 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:19.203 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:18 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:19.203 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:18 smithi203 ceph-mon[34723]: pgmap v477: 97 pgs: 97 active+clean; 344 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 246 op/s 2024-09-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.607+0000 7f718f7fe640 1 -- 172.21.15.112:0/3250812789 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7194005020 con 0x7f71981083d0 2024-09-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.607+0000 7f71a6a2b740 1 Processor -- start 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f71a6a2b740 1 -- start start 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec05649070 0x55ec059f0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec056201b0 0x55ec059f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05628390 0x55ec059f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ec05966e50 con 0x55ec05649070 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ec05956280 con 0x55ec05628390 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ec0595a090 con 0x55ec056201b0 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f719ea54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec05649070 0x55ec059f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f719e253640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec056201b0 0x55ec059f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f719ea54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec05649070 0x55ec059f0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39976/0 (socket says 172.21.15.112:39976) 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f719ea54640 1 -- 172.21.15.112:0/559990354 learned_addr learned my addr 172.21.15.112:0/559990354 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f719da52640 1 --2- 172.21.15.112:0/559990354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05628390 0x55ec059f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f718f7fe640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71840b6680 0x7f71840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f719e253640 1 -- 172.21.15.112:0/559990354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05628390 msgr2=0x55ec059f0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f719e253640 1 --2- 172.21.15.112:0/559990354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05628390 0x55ec059f0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f719e253640 1 -- 172.21.15.112:0/559990354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec05649070 msgr2=0x55ec059f0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f719e253640 1 --2- 172.21.15.112:0/559990354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec05649070 0x55ec059f0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f719e253640 1 -- 172.21.15.112:0/559990354 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec056b4650 con 0x55ec056201b0 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.608+0000 7f719ea54640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71840b6680 0x7f71840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.609+0000 7f718f7fe640 1 -- 172.21.15.112:0/3250812789 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f71940e7af0 con 0x7f71981083d0 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.609+0000 7f719e253640 1 --2- 172.21.15.112:0/559990354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec056201b0 0x55ec059f07b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f718800bc80 tx=0x7f718800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afafea8ccde2b529 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.609+0000 7f718cd9a640 1 -- 172.21.15.112:0/559990354 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188015070 con 0x55ec056201b0 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.609+0000 7f718cd9a640 1 -- 172.21.15.112:0/559990354 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718801a910 con 0x55ec056201b0 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.609+0000 7f718cd9a640 1 -- 172.21.15.112:0/559990354 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71880118e0 con 0x55ec056201b0 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.610+0000 7f71a6a2b740 1 -- 172.21.15.112:0/559990354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec056201b0 msgr2=0x55ec059f07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.610+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/559990354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec056201b0 0x55ec059f07b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f718800bc80 tx=0x7f718800f730 comp rx=0 tx=0).stop 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.610+0000 7f719ea54640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71840b6680 0x7f71840b8b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7190035000 tx=0x7f71900095c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.610+0000 7f71a6a2b740 1 -- 172.21.15.112:0/559990354 shutdown_connections 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.610+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/559990354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05628390 0x55ec059f0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.610+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/559990354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec056201b0 0x55ec059f07b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.610+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/559990354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec05649070 0x55ec059f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.610+0000 7f71a6a2b740 1 -- 172.21.15.112:0/559990354 >> 172.21.15.112:0/559990354 conn(0x55ec0563d460 msgr2=0x55ec059ee160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.610+0000 7f71a6a2b740 1 -- 172.21.15.112:0/559990354 shutdown_connections 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.610+0000 7f71a6a2b740 1 -- 172.21.15.112:0/559990354 wait complete. 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f71a6a2b740 1 -- start start 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f71a6a2b740 1 Processor -- start 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059fb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec059fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec059fc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ec057b7700 con 0x55ec056201b0 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ec058f1110 con 0x55ec05649070 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ec059fea50 con 0x55ec05628390 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f71a6a2b740 10 client.0 _ll_register_callbacks cb 0x55ec0590a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f719e253640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f719e253640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059fb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39984/0 (socket says 172.21.15.112:39984) 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f719e253640 1 -- 172.21.15.112:0/900442825 learned_addr learned my addr 172.21.15.112:0/900442825 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f719da52640 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec059fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.611+0000 7f719ea54640 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec059fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f719e253640 1 -- 172.21.15.112:0/900442825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 msgr2=0x55ec059fbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f719e253640 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec059fbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f719e253640 1 -- 172.21.15.112:0/900442825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 msgr2=0x55ec059fc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f719e253640 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec059fc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f719e253640 1 -- 172.21.15.112:0/900442825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0591ec40 con 0x55ec056201b0 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f719da52640 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec059fbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f719ea54640 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec059fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f719e253640 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059fb890 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7188011480 tx=0x7f718800b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c364e6b51e6728 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f7181ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f7181ffb640 1 -- 172.21.15.112:0/900442825 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188015070 con 0x55ec056201b0 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f7181ffb640 1 -- 172.21.15.112:0/900442825 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f718801a6f0 con 0x55ec056201b0 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f71a6a2b740 1 -- 172.21.15.112:0/900442825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718801a910 con 0x55ec056201b0 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f7181ffb640 1 -- 172.21.15.112:0/900442825 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f718801aac0 con 0x55ec056201b0 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f71a6a2b740 10 client.16476 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f71a6a2b740 1 -- 172.21.15.112:0/900442825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec05a0a690 con 0x55ec056201b0 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.612+0000 7f7181ffb640 1 -- 172.21.15.112:0/900442825 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718801a910 con 0x55ec056201b0 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.613+0000 7f7181ffb640 1 -- 172.21.15.112:0/900442825 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f7188005400 con 0x55ec056201b0 2024-09-21T03:34:19.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.613+0000 7f71a6a2b740 10 client.16476 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.613+0000 7f71a6a2b740 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.613+0000 7f71a6a2b740 20 client.16476 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.613+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ec056079f0 0x55ec05a0d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.613+0000 7f71a6a2b740 4 client.16476 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.613+0000 7f71a6a2b740 1 -- 172.21.15.112:0/900442825 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec05968da0 con 0x55ec056079f0 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.613+0000 7f719da52640 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ec056079f0 0x55ec05a0d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.613+0000 7f7181ffb640 10 client.16476 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.613+0000 7f719da52640 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ec056079f0 0x55ec05a0d9f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5df3190a8e3fc9a server_cookie=5d92f9c8424f340a in_seq=0 out_seq=0 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.614+0000 7f7181ffb640 1 -- 172.21.15.112:0/900442825 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ec05968da0 con 0x55ec056079f0 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.614+0000 7f7181ffb640 10 client.16476 handle_command_reply: tid=0 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.614+0000 7f71a6a2b740 1 client.16476 shutdown 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f71a6a2b740 1 -- 172.21.15.112:0/900442825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 msgr2=0x55ec059fb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059fb890 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7188011480 tx=0x7f718800b9f0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f71a6a2b740 1 -- 172.21.15.112:0/900442825 shutdown_connections 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ec056079f0 0x55ec05a0d9f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec059fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec059fbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f7181ffb640 0 client.16476 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/900442825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059fb890 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f71a6a2b740 1 -- 172.21.15.112:0/900442825 >> 172.21.15.112:0/900442825 conn(0x55ec0563d460 msgr2=0x55ec059f3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f71a6a2b740 1 -- 172.21.15.112:0/900442825 shutdown_connections 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f71a6a2b740 1 -- 172.21.15.112:0/900442825 wait complete. 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.615+0000 7f71a6a2b740 20 client.16476 trim_cache size 0 max 16384 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f71a6a2b740 1 Processor -- start 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f71a6a2b740 1 -- start start 2024-09-21T03:34:19.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec059fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec059f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71981b4190 con 0x55ec056201b0 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71981b1a30 con 0x55ec05649070 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f71981af880 con 0x55ec05628390 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f719e253640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f719e253640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39998/0 (socket says 172.21.15.112:39998) 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f719ea54640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec059f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f719e253640 1 -- 172.21.15.112:0/1316608630 learned_addr learned my addr 172.21.15.112:0/1316608630 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.636+0000 7f719da52640 1 --2- 172.21.15.112:0/1316608630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec059fd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.637+0000 7f719e253640 1 -- 172.21.15.112:0/1316608630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 msgr2=0x55ec059fd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.637+0000 7f719e253640 1 --2- 172.21.15.112:0/1316608630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec059fd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.637+0000 7f719e253640 1 -- 172.21.15.112:0/1316608630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 msgr2=0x55ec059f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.637+0000 7f719e253640 1 --2- 172.21.15.112:0/1316608630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec059f8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.637+0000 7f719e253640 1 -- 172.21.15.112:0/1316608630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec059fcb50 con 0x55ec056201b0 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.637+0000 7f719ea54640 1 --2- 172.21.15.112:0/1316608630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec059f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.637+0000 7f719e253640 1 --2- 172.21.15.112:0/1316608630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059f1c60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7188004830 tx=0x7f718800bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1636089112010559 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.637+0000 7f7183fff640 1 -- 172.21.15.112:0/1316608630 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188015040 con 0x55ec056201b0 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.637+0000 7f7183fff640 1 -- 172.21.15.112:0/1316608630 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718800bde0 con 0x55ec056201b0 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.637+0000 7f7183fff640 1 -- 172.21.15.112:0/1316608630 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718801ad30 con 0x55ec056201b0 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.638+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1316608630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 msgr2=0x55ec059f1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.638+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/1316608630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059f1c60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7188004830 tx=0x7f718800bb80 comp rx=0 tx=0).stop 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.638+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1316608630 shutdown_connections 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.638+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/1316608630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec059f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.638+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/1316608630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec059fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.638+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/1316608630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec059f1c60 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.638+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1316608630 >> 172.21.15.112:0/1316608630 conn(0x55ec0563d460 msgr2=0x55ec059fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:19.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.638+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1316608630 shutdown_connections 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.638+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1316608630 wait complete. 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.638+0000 7f71a6a2b740 1 -- start start 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.638+0000 7f71a6a2b740 1 Processor -- start 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec05a01810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec05a01db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f71a6a2b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec05a6f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71981afef0 con 0x55ec056201b0 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71981b1360 con 0x55ec05649070 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f71a6a2b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f71981b05c0 con 0x55ec05628390 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f71a6a2b740 10 client.0 _ll_register_callbacks cb 0x7f7190009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719e253640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec05a01810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719da52640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec05a01db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719e253640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec05a01810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40006/0 (socket says 172.21.15.112:40006) 2024-09-21T03:34:19.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719e253640 1 -- 172.21.15.112:0/1641220289 learned_addr learned my addr 172.21.15.112:0/1641220289 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719ea54640 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec05a6f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719e253640 1 -- 172.21.15.112:0/1641220289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 msgr2=0x55ec05a01db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719e253640 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec05a01db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719e253640 1 -- 172.21.15.112:0/1641220289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 msgr2=0x55ec05a6f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719e253640 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec05a6f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719e253640 1 -- 172.21.15.112:0/1641220289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec05a6f890 con 0x55ec056201b0 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719da52640 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec05a01db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f719e253640 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec05a01810 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f718801aa60 tx=0x7f718800b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a252a05733f4226 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.639+0000 7f7181ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f7181ffb640 1 -- 172.21.15.112:0/1641220289 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188004f60 con 0x55ec056201b0 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f7181ffb640 1 -- 172.21.15.112:0/1641220289 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f7188005100 con 0x55ec056201b0 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1641220289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718800bde0 con 0x55ec056201b0 2024-09-21T03:34:19.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f7181ffb640 1 -- 172.21.15.112:0/1641220289 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7188005340 con 0x55ec056201b0 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f71a6a2b740 10 client.16482 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1641220289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec05a77dc0 con 0x55ec056201b0 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f7181ffb640 1 -- 172.21.15.112:0/1641220289 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718800bde0 con 0x55ec056201b0 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f7181ffb640 1 -- 172.21.15.112:0/1641220289 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f7188023400 con 0x55ec056201b0 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f71a6a2b740 10 client.16482 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f71a6a2b740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f71a6a2b740 20 client.16482 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ec055dcda0 0x55ec05a7a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f71a6a2b740 4 client.16482 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1641220289 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ec05964850 con 0x55ec055dcda0 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.640+0000 7f719da52640 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ec055dcda0 0x55ec05a7a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f7181ffb640 10 client.16482 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f719da52640 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ec055dcda0 0x55ec05a7a430 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8726d5a0708d3812 server_cookie=d7dfcc756c54a863 in_seq=0 out_seq=0 2024-09-21T03:34:19.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f7181ffb640 1 -- 172.21.15.112:0/1641220289 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x55ec05964850 con 0x55ec055dcda0 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f7181ffb640 10 client.16482 handle_command_reply: tid=0 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f71a6a2b740 1 client.16482 shutdown 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1641220289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 msgr2=0x55ec05a01810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec05a01810 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f718801aa60 tx=0x7f718800b3a0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1641220289 shutdown_connections 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ec055dcda0 0x55ec05a7a430 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec05649070 0x55ec05a6f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ec05628390 0x55ec05a01db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f7181ffb640 0 client.16482 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f71a6a2b740 1 --2- 172.21.15.112:0/1641220289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ec056201b0 0x55ec05a01810 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1641220289 >> 172.21.15.112:0/1641220289 conn(0x55ec0563d460 msgr2=0x55ec059f1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1641220289 shutdown_connections 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.641+0000 7f71a6a2b740 1 -- 172.21.15.112:0/1641220289 wait complete. 2024-09-21T03:34:19.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a6a2b740 20 client.16482 trim_cache size 0 max 16384 2024-09-21T03:34:19.226 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:34:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:34:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:34:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:34:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "7f611a1d-7b3b-4b63-add4-85afb713d24a", 2024-09-21T03:34:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:34:19.226 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:19.227 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:34:19.228 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:19.229 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:19.230 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:19.231 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:34:19.232 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:34:19.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 -- 172.21.15.112:0/3250812789 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71840b6680 msgr2=0x7f71840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71840b6680 0x7f71840b8b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7190035000 tx=0x7f71900095c0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 -- 172.21.15.112:0/3250812789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71981083d0 msgr2=0x7f719819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:19.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71981083d0 0x7f719819cc50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f719400e9c0 tx=0x7f719400ee80 comp rx=0 tx=0).stop 2024-09-21T03:34:19.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 -- 172.21.15.112:0/3250812789 shutdown_connections 2024-09-21T03:34:19.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71840b6680 0x7f71840b8b90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7198108d20 0x7f71981a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71981083d0 0x7f719819cc50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 --2- 172.21.15.112:0/3250812789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7198102ae0 0x7f719819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:19.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 -- 172.21.15.112:0/3250812789 >> 172.21.15.112:0/3250812789 conn(0x7f7198079790 msgr2=0x7f719806bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:19.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 -- 172.21.15.112:0/3250812789 shutdown_connections 2024-09-21T03:34:19.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:18.642+0000 7f71a04b7640 1 -- 172.21.15.112:0/3250812789 wait complete. 2024-09-21T03:34:19.234 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:18 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:34:19.234 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:18 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:19.234 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:18 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:34:19.234 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:18 smithi112 ceph-mon[31442]: pgmap v477: 97 pgs: 97 active+clean; 344 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 246 op/s 2024-09-21T03:34:19.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 527855 10.92 MB/sec execute 139 sec latency 20.838 ms 2024-09-21T03:34:19.298 INFO:tasks.cephfs.filesystem:scrub status for tag:7f611a1d-7b3b-4b63-add4-85afb713d24a - {'path': '/', 'tag': '7f611a1d-7b3b-4b63-add4-85afb713d24a', 'options': 'recursive,force'} 2024-09-21T03:34:20.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 529859 10.94 MB/sec execute 140 sec latency 10.611 ms 2024-09-21T03:34:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:20 smithi112 ceph-mon[31442]: pgmap v478: 97 pgs: 97 active+clean; 346 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 261 op/s 2024-09-21T03:34:20.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:20 smithi203 ceph-mon[34723]: pgmap v478: 97 pgs: 97 active+clean; 346 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 261 op/s 2024-09-21T03:34:20.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:20 smithi187 ceph-mon[34603]: pgmap v478: 97 pgs: 97 active+clean; 346 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 261 op/s 2024-09-21T03:34:21.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 532106 10.94 MB/sec execute 141 sec latency 26.297 ms 2024-09-21T03:34:22.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 533678 10.92 MB/sec execute 142 sec latency 35.990 ms 2024-09-21T03:34:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:22 smithi112 ceph-mon[31442]: pgmap v479: 97 pgs: 97 active+clean; 350 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 256 op/s 2024-09-21T03:34:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:22 smithi203 ceph-mon[34723]: pgmap v479: 97 pgs: 97 active+clean; 350 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 256 op/s 2024-09-21T03:34:22.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:22 smithi187 ceph-mon[34603]: pgmap v479: 97 pgs: 97 active+clean; 350 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 256 op/s 2024-09-21T03:34:23.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 536439 10.97 MB/sec execute 143 sec latency 16.944 ms 2024-09-21T03:34:24.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 537988 10.94 MB/sec execute 144 sec latency 14.512 ms 2024-09-21T03:34:24.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:24 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:34:24.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:24 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:34:24.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:24 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:34:24.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:24 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:34:24.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:24 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:34:24.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:24 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:34:25.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 540357 10.96 MB/sec execute 145 sec latency 13.877 ms 2024-09-21T03:34:25.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:25 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:34:25.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:25 smithi203 ceph-mon[34723]: pgmap v480: 97 pgs: 97 active+clean; 352 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 228 op/s 2024-09-21T03:34:25.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:25 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:34:25.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:25 smithi187 ceph-mon[34603]: pgmap v480: 97 pgs: 97 active+clean; 352 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 228 op/s 2024-09-21T03:34:25.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:25 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:34:25.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:25 smithi112 ceph-mon[31442]: pgmap v480: 97 pgs: 97 active+clean; 352 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 228 op/s 2024-09-21T03:34:26.237 INFO:tasks.workunit.client.0.smithi112.stdout: 1 542434 10.93 MB/sec execute 146 sec latency 18.534 ms 2024-09-21T03:34:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:26 smithi112 ceph-mon[31442]: pgmap v481: 97 pgs: 97 active+clean; 357 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-09-21T03:34:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:26 smithi203 ceph-mon[34723]: pgmap v481: 97 pgs: 97 active+clean; 357 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-09-21T03:34:26.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:26 smithi187 ceph-mon[34603]: pgmap v481: 97 pgs: 97 active+clean; 357 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-09-21T03:34:27.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 544074 10.94 MB/sec execute 147 sec latency 60.471 ms 2024-09-21T03:34:28.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 546488 10.95 MB/sec execute 148 sec latency 26.168 ms 2024-09-21T03:34:28.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:28 smithi112 ceph-mon[31442]: pgmap v482: 97 pgs: 97 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-09-21T03:34:28.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:28 smithi203 ceph-mon[34723]: pgmap v482: 97 pgs: 97 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-09-21T03:34:28.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:28 smithi187 ceph-mon[34603]: pgmap v482: 97 pgs: 97 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-09-21T03:34:29.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 547996 10.93 MB/sec execute 149 sec latency 16.621 ms 2024-09-21T03:34:30.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 550823 10.97 MB/sec execute 150 sec latency 17.895 ms 2024-09-21T03:34:30.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:30 smithi112 ceph-mon[31442]: pgmap v483: 97 pgs: 97 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-09-21T03:34:30.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:30 smithi203 ceph-mon[34723]: pgmap v483: 97 pgs: 97 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-09-21T03:34:30.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:30 smithi187 ceph-mon[34603]: pgmap v483: 97 pgs: 97 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-09-21T03:34:31.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 552272 10.94 MB/sec execute 151 sec latency 17.287 ms 2024-09-21T03:34:32.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 554486 10.96 MB/sec execute 152 sec latency 41.425 ms 2024-09-21T03:34:32.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:32 smithi112 ceph-mon[31442]: pgmap v484: 97 pgs: 97 active+clean; 366 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 302 op/s 2024-09-21T03:34:32.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:32 smithi203 ceph-mon[34723]: pgmap v484: 97 pgs: 97 active+clean; 366 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 302 op/s 2024-09-21T03:34:32.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:32 smithi187 ceph-mon[34603]: pgmap v484: 97 pgs: 97 active+clean; 366 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 302 op/s 2024-09-21T03:34:33.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 556344 10.92 MB/sec execute 153 sec latency 19.287 ms 2024-09-21T03:34:34.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 558279 10.94 MB/sec execute 154 sec latency 11.500 ms 2024-09-21T03:34:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:34 smithi112 ceph-mon[31442]: pgmap v485: 97 pgs: 97 active+clean; 367 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-09-21T03:34:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:34 smithi203 ceph-mon[34723]: pgmap v485: 97 pgs: 97 active+clean; 367 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-09-21T03:34:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:34 smithi187 ceph-mon[34603]: pgmap v485: 97 pgs: 97 active+clean; 367 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 251 op/s 2024-09-21T03:34:35.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 560597 10.95 MB/sec execute 155 sec latency 18.684 ms 2024-09-21T03:34:36.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 562197 10.93 MB/sec execute 156 sec latency 18.320 ms 2024-09-21T03:34:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:36 smithi112 ceph-mon[31442]: pgmap v486: 97 pgs: 97 active+clean; 371 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-09-21T03:34:36.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:36 smithi203 ceph-mon[34723]: pgmap v486: 97 pgs: 97 active+clean; 371 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-09-21T03:34:36.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:36 smithi187 ceph-mon[34603]: pgmap v486: 97 pgs: 97 active+clean; 371 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-09-21T03:34:37.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 564779 10.96 MB/sec execute 157 sec latency 34.622 ms 2024-09-21T03:34:38.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 566026 10.92 MB/sec execute 158 sec latency 33.307 ms 2024-09-21T03:34:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:38 smithi112 ceph-mon[31442]: pgmap v487: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-09-21T03:34:38.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:38 smithi203 ceph-mon[34723]: pgmap v487: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-09-21T03:34:38.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:38 smithi187 ceph-mon[34603]: pgmap v487: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-09-21T03:34:39.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 568573 10.95 MB/sec execute 159 sec latency 23.386 ms 2024-09-21T03:34:40.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 570003 10.92 MB/sec execute 160 sec latency 18.596 ms 2024-09-21T03:34:40.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:40 smithi112 ceph-mon[31442]: pgmap v488: 97 pgs: 97 active+clean; 379 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-09-21T03:34:40.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:40 smithi203 ceph-mon[34723]: pgmap v488: 97 pgs: 97 active+clean; 379 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-09-21T03:34:40.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:40 smithi187 ceph-mon[34603]: pgmap v488: 97 pgs: 97 active+clean; 379 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-09-21T03:34:41.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 572272 10.94 MB/sec execute 161 sec latency 13.971 ms 2024-09-21T03:34:42.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 574029 10.91 MB/sec execute 162 sec latency 46.471 ms 2024-09-21T03:34:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:42 smithi112 ceph-mon[31442]: pgmap v489: 97 pgs: 97 active+clean; 379 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2024-09-21T03:34:42.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:42 smithi203 ceph-mon[34723]: pgmap v489: 97 pgs: 97 active+clean; 379 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2024-09-21T03:34:42.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:42 smithi187 ceph-mon[34603]: pgmap v489: 97 pgs: 97 active+clean; 379 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2024-09-21T03:34:43.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 575889 10.92 MB/sec execute 163 sec latency 41.046 ms 2024-09-21T03:34:44.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 578213 10.92 MB/sec execute 164 sec latency 18.074 ms 2024-09-21T03:34:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:44 smithi112 ceph-mon[31442]: pgmap v490: 97 pgs: 97 active+clean; 379 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-21T03:34:44.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:44 smithi203 ceph-mon[34723]: pgmap v490: 97 pgs: 97 active+clean; 379 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-21T03:34:44.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:44 smithi187 ceph-mon[34603]: pgmap v490: 97 pgs: 97 active+clean; 379 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-21T03:34:45.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 579804 10.91 MB/sec execute 165 sec latency 12.523 ms 2024-09-21T03:34:46.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 582611 10.95 MB/sec execute 166 sec latency 15.543 ms 2024-09-21T03:34:46.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:46 smithi112 ceph-mon[31442]: pgmap v491: 97 pgs: 97 active+clean; 388 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-09-21T03:34:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:46 smithi203 ceph-mon[34723]: pgmap v491: 97 pgs: 97 active+clean; 388 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-09-21T03:34:46.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:46 smithi187 ceph-mon[34603]: pgmap v491: 97 pgs: 97 active+clean; 388 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 305 op/s 2024-09-21T03:34:47.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 584023 10.92 MB/sec execute 167 sec latency 33.383 ms 2024-09-21T03:34:48.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 586412 10.94 MB/sec execute 168 sec latency 49.994 ms 2024-09-21T03:34:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:48 smithi112 ceph-mon[31442]: pgmap v492: 97 pgs: 97 active+clean; 388 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2024-09-21T03:34:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:48 smithi203 ceph-mon[34723]: pgmap v492: 97 pgs: 97 active+clean; 388 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2024-09-21T03:34:48.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:48 smithi187 ceph-mon[34603]: pgmap v492: 97 pgs: 97 active+clean; 388 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2024-09-21T03:34:49.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 588509 10.92 MB/sec execute 169 sec latency 19.081 ms 2024-09-21T03:34:49.298 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:34:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:50.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 590191 10.92 MB/sec execute 170 sec latency 77.994 ms 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.341+0000 7f3966449640 1 -- 172.21.15.112:0/3278666673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3960105300 msgr2=0x7f396010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.341+0000 7f3966449640 1 --2- 172.21.15.112:0/3278666673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3960105300 0x7f396010d850 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3954009900 tx=0x7f395402a410 comp rx=0 tx=0).stop 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.341+0000 7f3966449640 1 -- 172.21.15.112:0/3278666673 shutdown_connections 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.341+0000 7f3966449640 1 --2- 172.21.15.112:0/3278666673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f396010de10 0x7f3960110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.341+0000 7f3966449640 1 --2- 172.21.15.112:0/3278666673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3960105300 0x7f396010d850 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.341+0000 7f3966449640 1 --2- 172.21.15.112:0/3278666673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39601049b0 0x7f3960104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.341+0000 7f3966449640 1 -- 172.21.15.112:0/3278666673 >> 172.21.15.112:0/3278666673 conn(0x7f3960079730 msgr2=0x7f3960079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.341+0000 7f3966449640 1 -- 172.21.15.112:0/3278666673 shutdown_connections 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.341+0000 7f3966449640 1 -- 172.21.15.112:0/3278666673 wait complete. 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.342+0000 7f3966449640 1 Processor -- start 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.342+0000 7f3966449640 1 -- start start 2024-09-21T03:34:50.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f3966449640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39601049b0 0x7f396019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f3966449640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396010de10 0x7f396019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f3966449640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f396019fdc0 0x7f39601a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f3966449640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3960113a70 con 0x7f396010de10 2024-09-21T03:34:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f3966449640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39601138f0 con 0x7f39601049b0 2024-09-21T03:34:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f3966449640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3960113bf0 con 0x7f396019fdc0 2024-09-21T03:34:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f395f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396010de10 0x7f396019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f395ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39601049b0 0x7f396019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f395f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396010de10 0x7f396019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56840/0 (socket says 172.21.15.112:56840) 2024-09-21T03:34:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f395f7fe640 1 -- 172.21.15.112:0/571723345 learned_addr learned my addr 172.21.15.112:0/571723345 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f39649e6640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f396019fdc0 0x7f39601a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f395ffff640 1 -- 172.21.15.112:0/571723345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f396019fdc0 msgr2=0x7f39601a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f395ffff640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f396019fdc0 0x7f39601a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f395ffff640 1 -- 172.21.15.112:0/571723345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396010de10 msgr2=0x7f396019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f395ffff640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396010de10 0x7f396019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f395ffff640 1 -- 172.21.15.112:0/571723345 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39601a09a0 con 0x7f39601049b0 2024-09-21T03:34:50.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.343+0000 7f395ffff640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39601049b0 0x7f396019a640 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f395000e9c0 tx=0x7f395000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:50.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.344+0000 7f395d7fa640 1 -- 172.21.15.112:0/571723345 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395000cd80 con 0x7f39601049b0 2024-09-21T03:34:50.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.344+0000 7f3966449640 1 -- 172.21.15.112:0/571723345 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39601a6f40 con 0x7f39601049b0 2024-09-21T03:34:50.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.344+0000 7f395d7fa640 1 -- 172.21.15.112:0/571723345 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39500108e0 con 0x7f39601049b0 2024-09-21T03:34:50.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.344+0000 7f395d7fa640 1 -- 172.21.15.112:0/571723345 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39500187f0 con 0x7f39601049b0 2024-09-21T03:34:50.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.344+0000 7f3966449640 1 -- 172.21.15.112:0/571723345 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39601a72e0 con 0x7f39601049b0 2024-09-21T03:34:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.347+0000 7f396c9bd740 1 Processor -- start 2024-09-21T03:34:50.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.347+0000 7f396c9bd740 1 -- start start 2024-09-21T03:34:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.347+0000 7f395d7fa640 1 -- 172.21.15.112:0/571723345 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f395001f020 con 0x7f39601049b0 2024-09-21T03:34:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.347+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 0x55d8923b7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.347+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fe71b0 0x55d8923b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.347+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d891fef390 0x55d8923b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.347+0000 7f396c9bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d89232de50 con 0x55d892010070 2024-09-21T03:34:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.347+0000 7f396c9bd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d89231d280 con 0x55d891fef390 2024-09-21T03:34:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.347+0000 7f396c9bd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d892321090 con 0x55d891fe71b0 2024-09-21T03:34:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f39649e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 0x55d8923b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f395ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fe71b0 0x55d8923b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f395f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d891fef390 0x55d8923b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f39649e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 0x55d8923b7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56850/0 (socket says 172.21.15.112:56850) 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f39649e6640 1 -- 172.21.15.112:0/963517786 learned_addr learned my addr 172.21.15.112:0/963517786 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f395f7fe640 1 -- 172.21.15.112:0/963517786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fe71b0 msgr2=0x55d8923b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f395f7fe640 1 --2- 172.21.15.112:0/963517786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fe71b0 0x55d8923b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f395f7fe640 1 -- 172.21.15.112:0/963517786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 msgr2=0x55d8923b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f395f7fe640 1 --2- 172.21.15.112:0/963517786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 0x55d8923b7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f395f7fe640 1 -- 172.21.15.112:0/963517786 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89207b650 con 0x55d891fef390 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f395f7fe640 1 --2- 172.21.15.112:0/963517786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d891fef390 0x55d8923b7cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3954005940 tx=0x7f3954032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731cab2cf1e34054 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f395d7fa640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39480b6680 0x7f39480b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.348+0000 7f39467fc640 1 -- 172.21.15.112:0/963517786 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954040070 con 0x55d891fef390 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.349+0000 7f39649e6640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39480b6680 0x7f39480b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.349+0000 7f39467fc640 1 -- 172.21.15.112:0/963517786 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3954004030 con 0x55d891fef390 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.349+0000 7f39467fc640 1 -- 172.21.15.112:0/963517786 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954035040 con 0x55d891fef390 2024-09-21T03:34:50.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.349+0000 7f395d7fa640 1 -- 172.21.15.112:0/571723345 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f39500e7690 con 0x7f39601049b0 2024-09-21T03:34:50.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.350+0000 7f396c9bd740 1 -- 172.21.15.112:0/963517786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d891fef390 msgr2=0x55d8923b7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.350+0000 7f39649e6640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39480b6680 0x7f39480b8b90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39580116b0 tx=0x7f3958009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:50.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.350+0000 7f396c9bd740 1 --2- 172.21.15.112:0/963517786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d891fef390 0x55d8923b7cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3954005940 tx=0x7f3954032b20 comp rx=0 tx=0).stop 2024-09-21T03:34:50.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.350+0000 7f396c9bd740 1 -- 172.21.15.112:0/963517786 shutdown_connections 2024-09-21T03:34:50.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.350+0000 7f396c9bd740 1 --2- 172.21.15.112:0/963517786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d891fef390 0x55d8923b7cf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.350+0000 7f396c9bd740 1 --2- 172.21.15.112:0/963517786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fe71b0 0x55d8923b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.350+0000 7f396c9bd740 1 --2- 172.21.15.112:0/963517786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 0x55d8923b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.350+0000 7f396c9bd740 1 -- 172.21.15.112:0/963517786 >> 172.21.15.112:0/963517786 conn(0x55d892004460 msgr2=0x55d8923b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:50.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.350+0000 7f396c9bd740 1 -- 172.21.15.112:0/963517786 shutdown_connections 2024-09-21T03:34:50.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.350+0000 7f396c9bd740 1 -- 172.21.15.112:0/963517786 wait complete. 2024-09-21T03:34:50.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.350+0000 7f396c9bd740 1 -- start start 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f396c9bd740 1 Processor -- start 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fe71b0 0x55d8923c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fef390 0x55d8923c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 0x55d8923cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f396c9bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d89217e700 con 0x55d891fe71b0 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f396c9bd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d8922b8110 con 0x55d892010070 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f396c9bd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d8923c59f0 con 0x55d891fef390 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f396c9bd740 10 client.0 _ll_register_callbacks cb 0x55d8922d1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f395ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fe71b0 0x55d8923c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f395f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fef390 0x55d8923c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f395ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fe71b0 0x55d8923c2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56856/0 (socket says 172.21.15.112:56856) 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f395ffff640 1 -- 172.21.15.112:0/710231203 learned_addr learned my addr 172.21.15.112:0/710231203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:50.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f39649e6640 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 0x55d8923cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f395ffff640 1 -- 172.21.15.112:0/710231203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fef390 msgr2=0x55d8923c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:50.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f395ffff640 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fef390 0x55d8923c2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f395ffff640 1 -- 172.21.15.112:0/710231203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 msgr2=0x55d8923cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:50.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f395ffff640 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 0x55d8923cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f395ffff640 1 -- 172.21.15.112:0/710231203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8922e5c40 con 0x55d891fe71b0 2024-09-21T03:34:50.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f39649e6640 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 0x55d8923cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:50.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.351+0000 7f395f7fe640 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fef390 0x55d8923c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:50.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f395ffff640 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fe71b0 0x55d8923c2990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f395000c6a0 tx=0x7f3950010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9ef62573e90762 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f393f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f393f7fe640 1 -- 172.21.15.112:0/710231203 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39500ad310 con 0x55d891fe71b0 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f393f7fe640 1 -- 172.21.15.112:0/710231203 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f39500b6ad0 con 0x55d891fe71b0 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f396c9bd740 1 -- 172.21.15.112:0/710231203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3954004030 con 0x55d891fe71b0 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f393f7fe640 1 -- 172.21.15.112:0/710231203 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39500accb0 con 0x55d891fe71b0 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f396c9bd740 10 client.16494 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f396c9bd740 1 -- 172.21.15.112:0/710231203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d8923d17c0 con 0x55d891fe71b0 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f393f7fe640 1 -- 172.21.15.112:0/710231203 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3954004030 con 0x55d891fe71b0 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f393f7fe640 1 -- 172.21.15.112:0/710231203 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f39500e7b40 con 0x55d891fe71b0 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f396c9bd740 10 client.16494 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f396c9bd740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f396c9bd740 20 client.16494 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f396c9bd740 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d891fce9f0 0x55d8923d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f396c9bd740 4 client.16494 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:50.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.352+0000 7f396c9bd740 1 -- 172.21.15.112:0/710231203 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d89232fda0 con 0x55d891fce9f0 2024-09-21T03:34:50.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.353+0000 7f395f7fe640 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d891fce9f0 0x55d8923d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.353+0000 7f393f7fe640 10 client.16494 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:50.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.353+0000 7f395f7fe640 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d891fce9f0 0x55d8923d4d70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcd263043d7ebbad server_cookie=98c5c0a7dac210d0 in_seq=0 out_seq=0 2024-09-21T03:34:50.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.354+0000 7f393f7fe640 1 -- 172.21.15.112:0/710231203 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d89232fda0 con 0x55d891fce9f0 2024-09-21T03:34:50.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.354+0000 7f393f7fe640 10 client.16494 handle_command_reply: tid=0 2024-09-21T03:34:50.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.354+0000 7f396c9bd740 1 client.16494 shutdown 2024-09-21T03:34:50.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.354+0000 7f396c9bd740 1 -- 172.21.15.112:0/710231203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fe71b0 msgr2=0x55d8923c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.354+0000 7f396c9bd740 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fe71b0 0x55d8923c2990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f395000c6a0 tx=0x7f3950010d10 comp rx=0 tx=0).stop 2024-09-21T03:34:50.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.354+0000 7f396c9bd740 1 -- 172.21.15.112:0/710231203 shutdown_connections 2024-09-21T03:34:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.354+0000 7f396c9bd740 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d891fce9f0 0x55d8923d4d70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.354+0000 7f396c9bd740 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 0x55d8923cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.354+0000 7f396c9bd740 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fef390 0x55d8923c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.354+0000 7f396c9bd740 1 --2- 172.21.15.112:0/710231203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fe71b0 0x55d8923c2990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.354+0000 7f393f7fe640 0 client.16494 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.355+0000 7f396c9bd740 1 -- 172.21.15.112:0/710231203 >> 172.21.15.112:0/710231203 conn(0x55d892004460 msgr2=0x55d8923ba450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.355+0000 7f396c9bd740 1 -- 172.21.15.112:0/710231203 shutdown_connections 2024-09-21T03:34:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.355+0000 7f396c9bd740 1 -- 172.21.15.112:0/710231203 wait complete. 2024-09-21T03:34:50.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.355+0000 7f396c9bd740 20 client.16494 trim_cache size 0 max 16384 2024-09-21T03:34:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.376+0000 7f396c9bd740 1 Processor -- start 2024-09-21T03:34:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.376+0000 7f396c9bd740 1 -- start start 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d891fe71b0 0x55d8923b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fef390 0x55d8923cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 0x55d8923c4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f396c9bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39601b2550 con 0x55d892010070 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f396c9bd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39601afdf0 con 0x55d891fe71b0 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f396c9bd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f39601adc40 con 0x55d891fef390 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f39649e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 0x55d8923c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f39649e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 0x55d8923c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56864/0 (socket says 172.21.15.112:56864) 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f39649e6640 1 -- 172.21.15.112:0/3145141006 learned_addr learned my addr 172.21.15.112:0/3145141006 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f395f7fe640 1 --2- 172.21.15.112:0/3145141006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fef390 0x55d8923cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f395ffff640 1 --2- 172.21.15.112:0/3145141006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d891fe71b0 0x55d8923b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f39649e6640 1 -- 172.21.15.112:0/3145141006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fef390 msgr2=0x55d8923cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f39649e6640 1 --2- 172.21.15.112:0/3145141006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fef390 0x55d8923cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f39649e6640 1 -- 172.21.15.112:0/3145141006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d891fe71b0 msgr2=0x55d8923b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f39649e6640 1 --2- 172.21.15.112:0/3145141006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d891fe71b0 0x55d8923b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f39649e6640 1 -- 172.21.15.112:0/3145141006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8923b5ad0 con 0x55d892010070 2024-09-21T03:34:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.377+0000 7f39649e6640 1 --2- 172.21.15.112:0/3145141006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 0x55d8923c4360 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3958011480 tx=0x7f3958016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3047e510bf2bb3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.378+0000 7f3945ffb640 1 -- 172.21.15.112:0/3145141006 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958003a30 con 0x55d892010070 2024-09-21T03:34:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.378+0000 7f3945ffb640 1 -- 172.21.15.112:0/3145141006 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3958003bd0 con 0x55d892010070 2024-09-21T03:34:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.378+0000 7f3945ffb640 1 -- 172.21.15.112:0/3145141006 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395801f620 con 0x55d892010070 2024-09-21T03:34:50.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.378+0000 7f396c9bd740 1 -- 172.21.15.112:0/3145141006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 msgr2=0x55d8923c4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.378+0000 7f396c9bd740 1 --2- 172.21.15.112:0/3145141006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 0x55d8923c4360 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3958011480 tx=0x7f3958016000 comp rx=0 tx=0).stop 2024-09-21T03:34:50.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.379+0000 7f396c9bd740 1 -- 172.21.15.112:0/3145141006 shutdown_connections 2024-09-21T03:34:50.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.379+0000 7f396c9bd740 1 --2- 172.21.15.112:0/3145141006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d892010070 0x55d8923c4360 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.379+0000 7f396c9bd740 1 --2- 172.21.15.112:0/3145141006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fef390 0x55d8923cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.379+0000 7f396c9bd740 1 --2- 172.21.15.112:0/3145141006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d891fe71b0 0x55d8923b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.379+0000 7f396c9bd740 1 -- 172.21.15.112:0/3145141006 >> 172.21.15.112:0/3145141006 conn(0x55d892004460 msgr2=0x55d8923c3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.379+0000 7f396c9bd740 1 -- 172.21.15.112:0/3145141006 shutdown_connections 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.379+0000 7f396c9bd740 1 -- 172.21.15.112:0/3145141006 wait complete. 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.379+0000 7f396c9bd740 1 -- start start 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.379+0000 7f396c9bd740 1 Processor -- start 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fe71b0 0x55d8923cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fef390 0x55d8923c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f396c9bd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 0x55d8923c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f396c9bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39601ae2b0 con 0x55d891fef390 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f396c9bd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39601af720 con 0x55d892010070 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f396c9bd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f39601ae980 con 0x55d891fe71b0 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f395ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fe71b0 0x55d8923cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f395f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fef390 0x55d8923c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f396c9bd740 10 client.0 _ll_register_callbacks cb 0x7f3950009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f395f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fef390 0x55d8923c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56878/0 (socket says 172.21.15.112:56878) 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f395ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fe71b0 0x55d8923cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35358/0 (socket says 172.21.15.112:35358) 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f395f7fe640 1 -- 172.21.15.112:0/1880728368 learned_addr learned my addr 172.21.15.112:0/1880728368 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f39649e6640 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 0x55d8923c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f395f7fe640 1 -- 172.21.15.112:0/1880728368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fe71b0 msgr2=0x55d8923cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f395f7fe640 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fe71b0 0x55d8923cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f395f7fe640 1 -- 172.21.15.112:0/1880728368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 msgr2=0x55d8923c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f395f7fe640 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 0x55d8923c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f395f7fe640 1 -- 172.21.15.112:0/1880728368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8923c75a0 con 0x55d891fef390 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f39649e6640 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 0x55d8923c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f395f7fe640 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fef390 0x55d8923c68d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3954033000 tx=0x7f3954000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e794125d3636a58f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f393f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.380+0000 7f393f7fe640 1 -- 172.21.15.112:0/1880728368 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954040040 con 0x55d891fef390 2024-09-21T03:34:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.381+0000 7f393f7fe640 1 -- 172.21.15.112:0/1880728368 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f39540314a0 con 0x55d891fef390 2024-09-21T03:34:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.381+0000 7f396c9bd740 1 -- 172.21.15.112:0/1880728368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3958003bd0 con 0x55d891fef390 2024-09-21T03:34:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.381+0000 7f393f7fe640 1 -- 172.21.15.112:0/1880728368 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f395405d050 con 0x55d891fef390 2024-09-21T03:34:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.381+0000 7f396c9bd740 10 client.16500 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.381+0000 7f396c9bd740 1 -- 172.21.15.112:0/1880728368 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d89243f6f0 con 0x55d891fef390 2024-09-21T03:34:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.382+0000 7f393f7fe640 1 -- 172.21.15.112:0/1880728368 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3958003bd0 con 0x55d891fef390 2024-09-21T03:34:50.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.382+0000 7f393f7fe640 1 -- 172.21.15.112:0/1880728368 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f3954003680 con 0x55d891fef390 2024-09-21T03:34:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.382+0000 7f396c9bd740 10 client.16500 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.382+0000 7f396c9bd740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.382+0000 7f396c9bd740 20 client.16500 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.382+0000 7f396c9bd740 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d891fa3da0 0x55d892441b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.382+0000 7f396c9bd740 4 client.16500 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:34:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.382+0000 7f396c9bd740 1 -- 172.21.15.112:0/1880728368 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d89232b850 con 0x55d891fa3da0 2024-09-21T03:34:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.383+0000 7f395ffff640 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d891fa3da0 0x55d892441b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.383+0000 7f393f7fe640 10 client.16500 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.383+0000 7f395ffff640 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d891fa3da0 0x55d892441b00 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6cefda2c3d963a server_cookie=f15e54a50a98b06a in_seq=0 out_seq=0 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f393f7fe640 1 -- 172.21.15.112:0/1880728368 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x55d89232b850 con 0x55d891fa3da0 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f393f7fe640 10 client.16500 handle_command_reply: tid=0 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 1 client.16500 shutdown 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 1 -- 172.21.15.112:0/1880728368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fef390 msgr2=0x55d8923c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fef390 0x55d8923c68d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3954033000 tx=0x7f3954000bc0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 1 -- 172.21.15.112:0/1880728368 shutdown_connections 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d891fa3da0 0x55d892441b00 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d892010070 0x55d8923c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d891fef390 0x55d8923c68d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 1 --2- 172.21.15.112:0/1880728368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d891fe71b0 0x55d8923cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 1 -- 172.21.15.112:0/1880728368 >> 172.21.15.112:0/1880728368 conn(0x55d892004460 msgr2=0x55d8923b8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f393f7fe640 0 client.16500 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:50.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 1 -- 172.21.15.112:0/1880728368 shutdown_connections 2024-09-21T03:34:50.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 1 -- 172.21.15.112:0/1880728368 wait complete. 2024-09-21T03:34:50.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.384+0000 7f396c9bd740 20 client.16500 trim_cache size 0 max 16384 2024-09-21T03:34:50.390 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:34:50.390 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:34:50.390 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:34:50.390 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:34:50.390 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:34:50.390 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:34:50.390 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:50.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:50.392 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:50.393 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:34:50.394 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:50.395 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:50.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:34:50.396 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:34:50.396 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:34:50.396 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:34:50.396 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:34:50.396 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:34:50.396 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:34:50.396 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:34:50.396 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:34:50.396 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:34:50.396 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:34:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 -- 172.21.15.112:0/571723345 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39480b6680 msgr2=0x7f39480b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39480b6680 0x7f39480b8b90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39580116b0 tx=0x7f3958009500 comp rx=0 tx=0).stop 2024-09-21T03:34:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 -- 172.21.15.112:0/571723345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39601049b0 msgr2=0x7f396019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39601049b0 0x7f396019a640 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f395000e9c0 tx=0x7f395000ee80 comp rx=0 tx=0).stop 2024-09-21T03:34:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 -- 172.21.15.112:0/571723345 shutdown_connections 2024-09-21T03:34:50.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f39480b6680 0x7f39480b8b90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f396019fdc0 0x7f39601a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f396010de10 0x7f396019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 --2- 172.21.15.112:0/571723345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39601049b0 0x7f396019a640 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:50.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 -- 172.21.15.112:0/571723345 >> 172.21.15.112:0/571723345 conn(0x7f3960079730 msgr2=0x7f39600ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:50.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 -- 172.21.15.112:0/571723345 shutdown_connections 2024-09-21T03:34:50.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:50.385+0000 7f3966449640 1 -- 172.21.15.112:0/571723345 wait complete. 2024-09-21T03:34:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:50 smithi112 ceph-mon[31442]: pgmap v493: 97 pgs: 97 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-09-21T03:34:50.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:50 smithi203 ceph-mon[34723]: pgmap v493: 97 pgs: 97 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-09-21T03:34:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:50 smithi187 ceph-mon[34603]: pgmap v493: 97 pgs: 97 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-09-21T03:34:51.217 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:34:51.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:34:51.238 INFO:tasks.workunit.client.0.smithi112.stdout: 1 592552 10.93 MB/sec execute 171 sec latency 18.928 ms 2024-09-21T03:34:51.487 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:52.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.995+0000 7f2f538e9640 1 -- 172.21.15.112:0/2569068283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c103b70 msgr2=0x7f2f4c103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:52.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.995+0000 7f2f538e9640 1 --2- 172.21.15.112:0/2569068283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c103b70 0x7f2f4c103f80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009900 tx=0x7f2f4802a410 comp rx=0 tx=0).stop 2024-09-21T03:34:52.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.995+0000 7f2f538e9640 1 -- 172.21.15.112:0/2569068283 shutdown_connections 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.995+0000 7f2f538e9640 1 --2- 172.21.15.112:0/2569068283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4c109e00 0x7f2f4c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.995+0000 7f2f538e9640 1 --2- 172.21.15.112:0/2569068283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f4c1094b0 0x7f2f4c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.995+0000 7f2f538e9640 1 --2- 172.21.15.112:0/2569068283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c103b70 0x7f2f4c103f80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.995+0000 7f2f538e9640 1 -- 172.21.15.112:0/2569068283 >> 172.21.15.112:0/2569068283 conn(0x7f2f4c0ff5e0 msgr2=0x7f2f4c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.995+0000 7f2f538e9640 1 -- 172.21.15.112:0/2569068283 shutdown_connections 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.995+0000 7f2f538e9640 1 -- 172.21.15.112:0/2569068283 wait complete. 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.996+0000 7f2f538e9640 1 Processor -- start 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.996+0000 7f2f538e9640 1 -- start start 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.996+0000 7f2f538e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f4c103b70 0x7f2f4c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.996+0000 7f2f538e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c1094b0 0x7f2f4c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.996+0000 7f2f538e9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4c109e00 0x7f2f4c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:52.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.996+0000 7f2f538e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c113ac0 con 0x7f2f4c1094b0 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.996+0000 7f2f538e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c113940 con 0x7f2f4c103b70 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.996+0000 7f2f538e9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c113c40 con 0x7f2f4c109e00 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.996+0000 7f2f50e84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c1094b0 0x7f2f4c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f50e84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c1094b0 0x7f2f4c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56890/0 (socket says 172.21.15.112:56890) 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f50e84640 1 -- 172.21.15.112:0/442478616 learned_addr learned my addr 172.21.15.112:0/442478616 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f51e86640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4c109e00 0x7f2f4c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f51685640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f4c103b70 0x7f2f4c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f50e84640 1 -- 172.21.15.112:0/442478616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4c109e00 msgr2=0x7f2f4c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f50e84640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4c109e00 0x7f2f4c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f50e84640 1 -- 172.21.15.112:0/442478616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f4c103b70 msgr2=0x7f2f4c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f50e84640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f4c103b70 0x7f2f4c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f50e84640 1 -- 172.21.15.112:0/442478616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4c1a0420 con 0x7f2f4c1094b0 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f50e84640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c1094b0 0x7f2f4c19aa50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00e970 tx=0x7f2f3c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:52.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f427fc640 1 -- 172.21.15.112:0/442478616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00cd80 con 0x7f2f4c1094b0 2024-09-21T03:34:52.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f427fc640 1 -- 172.21.15.112:0/442478616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f3c0035f0 con 0x7f2f4c1094b0 2024-09-21T03:34:52.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f538e9640 1 -- 172.21.15.112:0/442478616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4c1a0710 con 0x7f2f4c1094b0 2024-09-21T03:34:52.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f427fc640 1 -- 172.21.15.112:0/442478616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c010690 con 0x7f2f4c1094b0 2024-09-21T03:34:52.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.997+0000 7f2f538e9640 1 -- 172.21.15.112:0/442478616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4c1a6e70 con 0x7f2f4c1094b0 2024-09-21T03:34:52.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:51.998+0000 7f2f538e9640 1 -- 172.21.15.112:0/442478616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f4c103f80 con 0x7f2f4c1094b0 2024-09-21T03:34:52.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.001+0000 7f2f427fc640 1 -- 172.21.15.112:0/442478616 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2f3c003aa0 con 0x7f2f4c1094b0 2024-09-21T03:34:52.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.001+0000 7f2f427fc640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f380b6700 0x7f2f380b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:52.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.002+0000 7f2f51685640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f380b6700 0x7f2f380b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:52.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.002+0000 7f2f427fc640 1 -- 172.21.15.112:0/442478616 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f2f3c0e7950 con 0x7f2f4c1094b0 2024-09-21T03:34:52.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.002+0000 7f2f51685640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f380b6700 0x7f2f380b8c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2f48031440 tx=0x7f2f48009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:52.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.005+0000 7f2f427fc640 1 -- 172.21.15.112:0/442478616 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f3c0adf00 con 0x7f2f4c1094b0 2024-09-21T03:34:52.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 594038 10.91 MB/sec execute 172 sec latency 36.260 ms 2024-09-21T03:34:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.270+0000 7f2f538e9640 1 -- 172.21.15.112:0/442478616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f4c19b5e0 con 0x7f2f4c1094b0 2024-09-21T03:34:52.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.271+0000 7f2f427fc640 1 -- 172.21.15.112:0/442478616 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11930 (secure 0 0 0) 0x7f2f4c19b5e0 con 0x7f2f4c1094b0 2024-09-21T03:34:52.273 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:34:52.273 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"btime":"2024-09-21T03:33:42:353851+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:33:42.303719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:34:52.273 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-09-21T03:34:52.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 -- 172.21.15.112:0/442478616 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f380b6700 msgr2=0x7f2f380b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:52.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f380b6700 0x7f2f380b8c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2f48031440 tx=0x7f2f48009560 comp rx=0 tx=0).stop 2024-09-21T03:34:52.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 -- 172.21.15.112:0/442478616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c1094b0 msgr2=0x7f2f4c19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:52.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c1094b0 0x7f2f4c19aa50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00e970 tx=0x7f2f3c00ee30 comp rx=0 tx=0).stop 2024-09-21T03:34:52.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 -- 172.21.15.112:0/442478616 shutdown_connections 2024-09-21T03:34:52.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f380b6700 0x7f2f380b8c10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4c109e00 0x7f2f4c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f4c1094b0 0x7f2f4c19aa50 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 --2- 172.21.15.112:0/442478616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f4c103b70 0x7f2f4c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:52.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 -- 172.21.15.112:0/442478616 >> 172.21.15.112:0/442478616 conn(0x7f2f4c0ff5e0 msgr2=0x7f2f4c101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:52.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 -- 172.21.15.112:0/442478616 shutdown_connections 2024-09-21T03:34:52.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:52.273+0000 7f2f538e9640 1 -- 172.21.15.112:0/442478616 wait complete. 2024-09-21T03:34:52.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:52 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/442478616' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:34:52.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:52 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/442478616' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:34:52.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:52 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/442478616' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:34:53.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:34:53.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 596534 10.93 MB/sec execute 173 sec latency 38.380 ms 2024-09-21T03:34:53.393 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:53.653 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:53 smithi112 ceph-mon[31442]: pgmap v494: 97 pgs: 97 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2024-09-21T03:34:53.660 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:53 smithi203 ceph-mon[34723]: pgmap v494: 97 pgs: 97 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2024-09-21T03:34:53.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:53 smithi187 ceph-mon[34603]: pgmap v494: 97 pgs: 97 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.886+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2341227884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2681049b0 msgr2=0x7fc268104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.886+0000 7fc26cb3a640 1 --2- 172.21.15.112:0/2341227884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2681049b0 0x7fc268104dc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc258009900 tx=0x7fc25802a410 comp rx=0 tx=0).stop 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.886+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2341227884 shutdown_connections 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.886+0000 7fc26cb3a640 1 --2- 172.21.15.112:0/2341227884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc26810de10 0x7fc268110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.886+0000 7fc26cb3a640 1 --2- 172.21.15.112:0/2341227884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc268105300 0x7fc26810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.886+0000 7fc26cb3a640 1 --2- 172.21.15.112:0/2341227884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2681049b0 0x7fc268104dc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.886+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2341227884 >> 172.21.15.112:0/2341227884 conn(0x7fc268079730 msgr2=0x7fc268079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.887+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2341227884 shutdown_connections 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.887+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2341227884 wait complete. 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.887+0000 7fc26cb3a640 1 Processor -- start 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc26cb3a640 1 -- start start 2024-09-21T03:34:53.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc26cb3a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2681049b0 0x7fc26819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc26cb3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268105300 0x7fc26819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc26cb3a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc26810de10 0x7fc26819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc26cb3a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc268113a90 con 0x7fc268105300 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc26cb3a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc268113910 con 0x7fc26810de10 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc26cb3a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc268113c10 con 0x7fc2681049b0 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc25dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268105300 0x7fc26819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc25dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268105300 0x7fc26819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56908/0 (socket says 172.21.15.112:56908) 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2813541459 learned_addr learned my addr 172.21.15.112:0/2813541459 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc26659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2681049b0 0x7fc26819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc266d9d640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc26810de10 0x7fc26819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2813541459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2681049b0 msgr2=0x7fc26819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2681049b0 0x7fc26819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2813541459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc26810de10 msgr2=0x7fc26819fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc26810de10 0x7fc26819fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2813541459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2681a0470 con 0x7fc268105300 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc266d9d640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc26810de10 0x7fc26819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.888+0000 7fc26659c640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2681049b0 0x7fc26819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.889+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268105300 0x7fc26819ab60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc25000e9c0 tx=0x7fc25000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.889+0000 7fc25ffff640 1 -- 172.21.15.112:0/2813541459 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25000ccc0 con 0x7fc268105300 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.889+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2813541459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2681a0760 con 0x7fc268105300 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.889+0000 7fc25ffff640 1 -- 172.21.15.112:0/2813541459 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2500054d0 con 0x7fc268105300 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.889+0000 7fc25ffff640 1 -- 172.21.15.112:0/2813541459 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250010430 con 0x7fc268105300 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.889+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2813541459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2681a6f80 con 0x7fc268105300 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.892+0000 7fc2730ae740 1 Processor -- start 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.892+0000 7fc2730ae740 1 -- start start 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.892+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e27070 0x55fed81ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.892+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.892+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 0x55fed81cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.892+0000 7fc2730ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fed8144ea0 con 0x55fed7dfe1b0 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.892+0000 7fc2730ae740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fed81342d0 con 0x55fed7e06390 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.892+0000 7fc2730ae740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fed81380e0 con 0x55fed7e27070 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.892+0000 7fc25ffff640 1 -- 172.21.15.112:0/2813541459 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc2500105d0 con 0x7fc268105300 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc26659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc266d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e27070 0x55fed81ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc26659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56916/0 (socket says 172.21.15.112:56916) 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc26659c640 1 -- 172.21.15.112:0/568584371 learned_addr learned my addr 172.21.15.112:0/568584371 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/568584371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 0x55fed81cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc266d9d640 1 -- 172.21.15.112:0/568584371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 msgr2=0x55fed81cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc266d9d640 1 --2- 172.21.15.112:0/568584371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 0x55fed81cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc266d9d640 1 -- 172.21.15.112:0/568584371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 msgr2=0x55fed81cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc266d9d640 1 --2- 172.21.15.112:0/568584371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc266d9d640 1 -- 172.21.15.112:0/568584371 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed7e92650 con 0x55fed7e27070 2024-09-21T03:34:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc26659c640 1 --2- 172.21.15.112:0/568584371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc266d9d640 1 --2- 172.21.15.112:0/568584371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e27070 0x55fed81ce6a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc260010f70 tx=0x7fc260007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1653a95543d60bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc25cd99640 1 -- 172.21.15.112:0/568584371 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260004b20 con 0x55fed7e27070 2024-09-21T03:34:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc25cd99640 1 -- 172.21.15.112:0/568584371 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc260004cc0 con 0x55fed7e27070 2024-09-21T03:34:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc25cd99640 1 -- 172.21.15.112:0/568584371 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26001b710 con 0x55fed7e27070 2024-09-21T03:34:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.893+0000 7fc25ffff640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2540b66b0 0x7fc2540b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.894+0000 7fc26659c640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2540b66b0 0x7fc2540b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.894+0000 7fc25ffff640 1 -- 172.21.15.112:0/2813541459 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc2500e7ae0 con 0x7fc268105300 2024-09-21T03:34:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.894+0000 7fc2730ae740 1 -- 172.21.15.112:0/568584371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e27070 msgr2=0x55fed81ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.894+0000 7fc26659c640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2540b66b0 0x7fc2540b8bc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55fed81cd320 tx=0x7fc258031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.894+0000 7fc2730ae740 1 --2- 172.21.15.112:0/568584371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e27070 0x55fed81ce6a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc260010f70 tx=0x7fc260007930 comp rx=0 tx=0).stop 2024-09-21T03:34:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.894+0000 7fc2730ae740 1 -- 172.21.15.112:0/568584371 shutdown_connections 2024-09-21T03:34:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.894+0000 7fc2730ae740 1 --2- 172.21.15.112:0/568584371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 0x55fed81cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.894+0000 7fc2730ae740 1 --2- 172.21.15.112:0/568584371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.894+0000 7fc2730ae740 1 --2- 172.21.15.112:0/568584371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e27070 0x55fed81ce6a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.894+0000 7fc2730ae740 1 -- 172.21.15.112:0/568584371 >> 172.21.15.112:0/568584371 conn(0x55fed7e1b460 msgr2=0x55fed81cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.895+0000 7fc2730ae740 1 -- 172.21.15.112:0/568584371 shutdown_connections 2024-09-21T03:34:53.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.895+0000 7fc2730ae740 1 -- 172.21.15.112:0/568584371 wait complete. 2024-09-21T03:34:53.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.895+0000 7fc2730ae740 1 -- start start 2024-09-21T03:34:53.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.895+0000 7fc2730ae740 1 Processor -- start 2024-09-21T03:34:53.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.895+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81d9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.895+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e06390 0x55fed81da280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.895+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e27070 0x55fed81e6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.895+0000 7fc2730ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fed7f95700 con 0x55fed7dfe1b0 2024-09-21T03:34:53.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.895+0000 7fc2730ae740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fed80cf160 con 0x55fed7e27070 2024-09-21T03:34:53.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.895+0000 7fc2730ae740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fed81dce80 con 0x55fed7e06390 2024-09-21T03:34:53.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.895+0000 7fc2730ae740 10 client.0 _ll_register_callbacks cb 0x55fed80e89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:53.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc26659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81d9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc26659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81d9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56926/0 (socket says 172.21.15.112:56926) 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc26659c640 1 -- 172.21.15.112:0/571318272 learned_addr learned my addr 172.21.15.112:0/571318272 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e06390 0x55fed81da280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc266d9d640 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e27070 0x55fed81e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc26659c640 1 -- 172.21.15.112:0/571318272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e06390 msgr2=0x55fed81da280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc26659c640 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e06390 0x55fed81da280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc26659c640 1 -- 172.21.15.112:0/571318272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e27070 msgr2=0x55fed81e6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc26659c640 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e27070 0x55fed81e6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc26659c640 1 -- 172.21.15.112:0/571318272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed80fcc90 con 0x55fed7dfe1b0 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc266d9d640 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e27070 0x55fed81e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e06390 0x55fed81da280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc26659c640 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81d9d40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc25802a930 tx=0x7fc258009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1c41cf13d8a675 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc24dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:53.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc24dffb640 1 -- 172.21.15.112:0/571318272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25804d030 con 0x55fed7dfe1b0 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc24dffb640 1 -- 172.21.15.112:0/571318272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc258004980 con 0x55fed7dfe1b0 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.896+0000 7fc2730ae740 1 -- 172.21.15.112:0/571318272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc260004cc0 con 0x55fed7dfe1b0 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc24dffb640 1 -- 172.21.15.112:0/571318272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc258035040 con 0x55fed7dfe1b0 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc2730ae740 10 client.16518 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc2730ae740 1 -- 172.21.15.112:0/571318272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fed81e8b70 con 0x55fed7dfe1b0 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc24dffb640 1 -- 172.21.15.112:0/571318272 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc260004cc0 con 0x55fed7dfe1b0 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc24dffb640 1 -- 172.21.15.112:0/571318272 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fc25803da90 con 0x55fed7dfe1b0 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc2730ae740 10 client.16518 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc2730ae740 10 client.16518 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc2730ae740 20 client.16518 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc2730ae740 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fed7de59f0 0x55fed81ec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc2730ae740 4 client.16518 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc2730ae740 1 -- 172.21.15.112:0/571318272 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fed8146df0 con 0x55fed7de59f0 2024-09-21T03:34:53.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.897+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fed7de59f0 0x55fed81ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.898+0000 7fc24dffb640 10 client.16518 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.898+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fed7de59f0 0x55fed81ec040 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42d681c71687da server_cookie=63e7f02f90f56af4 in_seq=0 out_seq=0 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.899+0000 7fc24dffb640 1 -- 172.21.15.112:0/571318272 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fed8146df0 con 0x55fed7de59f0 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.899+0000 7fc24dffb640 10 client.16518 handle_command_reply: tid=0 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.899+0000 7fc2730ae740 1 client.16518 shutdown 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc2730ae740 1 -- 172.21.15.112:0/571318272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 msgr2=0x55fed81d9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc2730ae740 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81d9d40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc25802a930 tx=0x7fc258009260 comp rx=0 tx=0).stop 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc2730ae740 1 -- 172.21.15.112:0/571318272 shutdown_connections 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc2730ae740 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fed7de59f0 0x55fed81ec040 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc2730ae740 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e27070 0x55fed81e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc2730ae740 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e06390 0x55fed81da280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc2730ae740 1 --2- 172.21.15.112:0/571318272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81d9d40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc24dffb640 0 client.16518 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc2730ae740 1 -- 172.21.15.112:0/571318272 >> 172.21.15.112:0/571318272 conn(0x55fed7e1b460 msgr2=0x55fed81d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc2730ae740 1 -- 172.21.15.112:0/571318272 shutdown_connections 2024-09-21T03:34:53.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc2730ae740 1 -- 172.21.15.112:0/571318272 wait complete. 2024-09-21T03:34:53.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.900+0000 7fc2730ae740 20 client.16518 trim_cache size 0 max 16384 2024-09-21T03:34:53.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.913+0000 7fc2730ae740 1 Processor -- start 2024-09-21T03:34:53.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.913+0000 7fc2730ae740 1 -- start start 2024-09-21T03:34:53.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7dfe1b0 0x55fed81d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7e06390 0x55fed81db970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e27070 0x55fed81cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc2730ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681afc90 con 0x55fed7e06390 2024-09-21T03:34:53.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc2730ae740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681ad530 con 0x55fed7e27070 2024-09-21T03:34:53.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc2730ae740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681ab380 con 0x55fed7dfe1b0 2024-09-21T03:34:53.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc25dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7e06390 0x55fed81db970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc25dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7e06390 0x55fed81db970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56942/0 (socket says 172.21.15.112:56942) 2024-09-21T03:34:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2507402865 learned_addr learned my addr 172.21.15.112:0/2507402865 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc266d9d640 1 --2- 172.21.15.112:0/2507402865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e27070 0x55fed81cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc26659c640 1 --2- 172.21.15.112:0/2507402865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7dfe1b0 0x55fed81d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2507402865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7dfe1b0 msgr2=0x55fed81d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/2507402865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7dfe1b0 0x55fed81d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2507402865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e27070 msgr2=0x55fed81cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/2507402865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e27070 0x55fed81cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2507402865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed81db000 con 0x55fed7e06390 2024-09-21T03:34:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.914+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/2507402865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7e06390 0x55fed81db970 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc2500e5890 tx=0x7fc250014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfcec0f97e8cd0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.915+0000 7fc24ffff640 1 -- 172.21.15.112:0/2507402865 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25000ce60 con 0x55fed7e06390 2024-09-21T03:34:53.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.915+0000 7fc24ffff640 1 -- 172.21.15.112:0/2507402865 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2500ad4a0 con 0x55fed7e06390 2024-09-21T03:34:53.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.915+0000 7fc24ffff640 1 -- 172.21.15.112:0/2507402865 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2500a9fe0 con 0x55fed7e06390 2024-09-21T03:34:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.916+0000 7fc2730ae740 1 -- 172.21.15.112:0/2507402865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7e06390 msgr2=0x55fed81db970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.916+0000 7fc2730ae740 1 --2- 172.21.15.112:0/2507402865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7e06390 0x55fed81db970 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc2500e5890 tx=0x7fc250014000 comp rx=0 tx=0).stop 2024-09-21T03:34:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.916+0000 7fc2730ae740 1 -- 172.21.15.112:0/2507402865 shutdown_connections 2024-09-21T03:34:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.916+0000 7fc2730ae740 1 --2- 172.21.15.112:0/2507402865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e27070 0x55fed81cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.916+0000 7fc2730ae740 1 --2- 172.21.15.112:0/2507402865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7e06390 0x55fed81db970 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.916+0000 7fc2730ae740 1 --2- 172.21.15.112:0/2507402865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7dfe1b0 0x55fed81d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.916+0000 7fc2730ae740 1 -- 172.21.15.112:0/2507402865 >> 172.21.15.112:0/2507402865 conn(0x55fed7e1b460 msgr2=0x55fed81db260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.916+0000 7fc2730ae740 1 -- 172.21.15.112:0/2507402865 shutdown_connections 2024-09-21T03:34:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.916+0000 7fc2730ae740 1 -- 172.21.15.112:0/2507402865 wait complete. 2024-09-21T03:34:53.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.916+0000 7fc2730ae740 1 -- start start 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.916+0000 7fc2730ae740 1 Processor -- start 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 0x55fed81ddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc2730ae740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e27070 0x55fed81de150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc2730ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681ab9f0 con 0x55fed7dfe1b0 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc2730ae740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681ace60 con 0x55fed7e06390 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc2730ae740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc2681ac0c0 con 0x55fed7e27070 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc25dd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 0x55fed81ddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc26659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc266d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e27070 0x55fed81de150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc25dd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 0x55fed81ddc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:55802/0 (socket says 172.21.15.112:55802) 2024-09-21T03:34:53.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc2730ae740 10 client.0 _ll_register_callbacks cb 0x7fc2580315c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2443269288 learned_addr learned my addr 172.21.15.112:0/2443269288 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2443269288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e27070 msgr2=0x55fed81de150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/2443269288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e27070 0x55fed81de150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2443269288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 msgr2=0x55fed81e2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/2443269288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81e2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc25dd9b640 1 -- 172.21.15.112:0/2443269288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed81de790 con 0x55fed7e06390 2024-09-21T03:34:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc25dd9b640 1 --2- 172.21.15.112:0/2443269288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 0x55fed81ddc10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2500a9f60 tx=0x7fc250009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb92689d03de5828 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.917+0000 7fc24dffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:34:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc24dffb640 1 -- 172.21.15.112:0/2443269288 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2500a9100 con 0x55fed7e06390 2024-09-21T03:34:53.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc2730ae740 1 -- 172.21.15.112:0/2443269288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2500ad4a0 con 0x55fed7e06390 2024-09-21T03:34:53.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc24dffb640 1 -- 172.21.15.112:0/2443269288 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fc2500acbf0 con 0x55fed7e06390 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc24dffb640 1 -- 172.21.15.112:0/2443269288 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2500a9fe0 con 0x55fed7e06390 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc2730ae740 10 client.26087 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc2730ae740 1 -- 172.21.15.112:0/2443269288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fed8256840 con 0x55fed7e06390 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc24dffb640 1 -- 172.21.15.112:0/2443269288 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2500ad4a0 con 0x55fed7e06390 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc24dffb640 1 -- 172.21.15.112:0/2443269288 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fc250005000 con 0x55fed7e06390 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc2730ae740 10 client.26087 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc2730ae740 10 client.26087 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc2730ae740 20 client.26087 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc2730ae740 1 --2- 172.21.15.112:0/2443269288 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fed7dbada0 0x55fed8258ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc2730ae740 4 client.26087 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.918+0000 7fc2730ae740 1 -- 172.21.15.112:0/2443269288 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fed81428a0 con 0x55fed7dbada0 2024-09-21T03:34:53.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc26659c640 1 --2- 172.21.15.112:0/2443269288 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fed7dbada0 0x55fed8258ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:53.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc24dffb640 10 client.26087 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:34:53.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc26659c640 1 --2- 172.21.15.112:0/2443269288 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fed7dbada0 0x55fed8258ef0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=734f3ebae2039194 server_cookie=44378308201a44e2 in_seq=0 out_seq=0 2024-09-21T03:34:53.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc24dffb640 1 -- 172.21.15.112:0/2443269288 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fed81428a0 con 0x55fed7dbada0 2024-09-21T03:34:53.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc24dffb640 10 client.26087 handle_command_reply: tid=0 2024-09-21T03:34:53.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc2730ae740 1 client.26087 shutdown 2024-09-21T03:34:53.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc2730ae740 1 -- 172.21.15.112:0/2443269288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 msgr2=0x55fed81ddc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc2730ae740 1 --2- 172.21.15.112:0/2443269288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 0x55fed81ddc10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2500a9f60 tx=0x7fc250009310 comp rx=0 tx=0).stop 2024-09-21T03:34:53.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc2730ae740 1 -- 172.21.15.112:0/2443269288 shutdown_connections 2024-09-21T03:34:53.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc2730ae740 1 --2- 172.21.15.112:0/2443269288 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55fed7dbada0 0x55fed8258ef0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc2730ae740 1 --2- 172.21.15.112:0/2443269288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fed7e27070 0x55fed81de150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc2730ae740 1 --2- 172.21.15.112:0/2443269288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fed7e06390 0x55fed81ddc10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc24dffb640 0 client.26087 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:34:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc2730ae740 1 --2- 172.21.15.112:0/2443269288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fed7dfe1b0 0x55fed81e2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.919+0000 7fc2730ae740 1 -- 172.21.15.112:0/2443269288 >> 172.21.15.112:0/2443269288 conn(0x55fed7e1b460 msgr2=0x55fed81cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc2730ae740 1 -- 172.21.15.112:0/2443269288 shutdown_connections 2024-09-21T03:34:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc2730ae740 1 -- 172.21.15.112:0/2443269288 wait complete. 2024-09-21T03:34:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc2730ae740 20 client.26087 trim_cache size 0 max 16384 2024-09-21T03:34:53.930 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:34:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2813541459 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2540b66b0 msgr2=0x7fc2540b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc26cb3a640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2540b66b0 0x7fc2540b8bc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55fed81cd320 tx=0x7fc258031400 comp rx=0 tx=0).stop 2024-09-21T03:34:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2813541459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268105300 msgr2=0x7fc26819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc26cb3a640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268105300 0x7fc26819ab60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc25000e9c0 tx=0x7fc25000ee80 comp rx=0 tx=0).stop 2024-09-21T03:34:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2813541459 shutdown_connections 2024-09-21T03:34:53.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc26cb3a640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc2540b66b0 0x7fc2540b8bc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc26cb3a640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc26810de10 0x7fc26819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc26cb3a640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc268105300 0x7fc26819ab60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc26cb3a640 1 --2- 172.21.15.112:0/2813541459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2681049b0 0x7fc26819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:53.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.920+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2813541459 >> 172.21.15.112:0/2813541459 conn(0x7fc268079730 msgr2=0x7fc2681008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:53.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.921+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2813541459 shutdown_connections 2024-09-21T03:34:53.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:53.921+0000 7fc26cb3a640 1 -- 172.21.15.112:0/2813541459 wait complete. 2024-09-21T03:34:54.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 598096 10.92 MB/sec execute 174 sec latency 28.269 ms 2024-09-21T03:34:54.728 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:34:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:54 smithi112 ceph-mon[31442]: pgmap v495: 97 pgs: 97 active+clean; 400 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 287 op/s 2024-09-21T03:34:54.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:54 smithi203 ceph-mon[34723]: pgmap v495: 97 pgs: 97 active+clean; 400 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 287 op/s 2024-09-21T03:34:54.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:54 smithi187 ceph-mon[34603]: pgmap v495: 97 pgs: 97 active+clean; 400 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 287 op/s 2024-09-21T03:34:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:55.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 600546 10.94 MB/sec execute 175 sec latency 16.544 ms 2024-09-21T03:34:55.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.493+0000 7fb349e48640 1 -- 172.21.15.112:0/370539860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3441094b0 msgr2=0x7fb3441098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.493+0000 7fb349e48640 1 --2- 172.21.15.112:0/370539860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3441094b0 0x7fb3441098c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009900 tx=0x7fb32c02a410 comp rx=0 tx=0).stop 2024-09-21T03:34:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.493+0000 7fb349e48640 1 -- 172.21.15.112:0/370539860 shutdown_connections 2024-09-21T03:34:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.493+0000 7fb349e48640 1 --2- 172.21.15.112:0/370539860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb344109e00 0x7fb344110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.493+0000 7fb349e48640 1 --2- 172.21.15.112:0/370539860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3441094b0 0x7fb3441098c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.493+0000 7fb349e48640 1 --2- 172.21.15.112:0/370539860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb344103bc0 0x7fb344103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.493+0000 7fb349e48640 1 -- 172.21.15.112:0/370539860 >> 172.21.15.112:0/370539860 conn(0x7fb34407b910 msgr2=0x7fb34407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.494+0000 7fb349e48640 1 -- 172.21.15.112:0/370539860 shutdown_connections 2024-09-21T03:34:55.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.494+0000 7fb349e48640 1 -- 172.21.15.112:0/370539860 wait complete. 2024-09-21T03:34:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.494+0000 7fb349e48640 1 Processor -- start 2024-09-21T03:34:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.494+0000 7fb349e48640 1 -- start start 2024-09-21T03:34:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb349e48640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb344103bc0 0x7fb344075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb349e48640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3441094b0 0x7fb344075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb349e48640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb344109e00 0x7fb344076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb349e48640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb344113ac0 con 0x7fb3441094b0 2024-09-21T03:34:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb349e48640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb344113940 con 0x7fb344103bc0 2024-09-21T03:34:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb349e48640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb344113c40 con 0x7fb344109e00 2024-09-21T03:34:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb33bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3441094b0 0x7fb344075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb33bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3441094b0 0x7fb344075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56348/0 (socket says 172.21.15.112:56348) 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb33bfff640 1 -- 172.21.15.112:0/1696924101 learned_addr learned my addr 172.21.15.112:0/1696924101 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb343fff640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb344109e00 0x7fb344076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb3437fe640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb344103bc0 0x7fb344075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb33bfff640 1 -- 172.21.15.112:0/1696924101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb344109e00 msgr2=0x7fb344076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb33bfff640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb344109e00 0x7fb344076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb33bfff640 1 -- 172.21.15.112:0/1696924101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb344103bc0 msgr2=0x7fb344075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb33bfff640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb344103bc0 0x7fb344075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb33bfff640 1 -- 172.21.15.112:0/1696924101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3441a8790 con 0x7fb3441094b0 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb343fff640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb344109e00 0x7fb344076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.495+0000 7fb33bfff640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3441094b0 0x7fb344075c50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb32c02a930 tx=0x7fb32c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.496+0000 7fb3417fa640 1 -- 172.21.15.112:0/1696924101 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c03f030 con 0x7fb3441094b0 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.496+0000 7fb349e48640 1 -- 172.21.15.112:0/1696924101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3441a8a20 con 0x7fb3441094b0 2024-09-21T03:34:55.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.496+0000 7fb3417fa640 1 -- 172.21.15.112:0/1696924101 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb32c004490 con 0x7fb3441094b0 2024-09-21T03:34:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.496+0000 7fb3417fa640 1 -- 172.21.15.112:0/1696924101 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c035070 con 0x7fb3441094b0 2024-09-21T03:34:55.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.496+0000 7fb349e48640 1 -- 172.21.15.112:0/1696924101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3441a8f00 con 0x7fb3441094b0 2024-09-21T03:34:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.498+0000 7fb3503bc740 1 Processor -- start 2024-09-21T03:34:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.498+0000 7fb3503bc740 1 -- start start 2024-09-21T03:34:55.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3ba070 0x55c1ac7612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac761800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb3417fa640 1 -- 172.21.15.112:0/1696924101 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb32c042030 con 0x7fb3441094b0 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 0x55c1ac761d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb3503bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac6d7ea0 con 0x55c1ac3ba070 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb3503bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac6c72d0 con 0x55c1ac3911b0 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb3503bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac6cb0e0 con 0x55c1ac399390 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb343fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3ba070 0x55c1ac7612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb343fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3ba070 0x55c1ac7612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56356/0 (socket says 172.21.15.112:56356) 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb343fff640 1 -- 172.21.15.112:0/1378720138 learned_addr learned my addr 172.21.15.112:0/1378720138 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb33bfff640 1 --2- 172.21.15.112:0/1378720138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 0x55c1ac761d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb3437fe640 1 --2- 172.21.15.112:0/1378720138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac761800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb343fff640 1 -- 172.21.15.112:0/1378720138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 msgr2=0x55c1ac761d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb343fff640 1 --2- 172.21.15.112:0/1378720138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 0x55c1ac761d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb343fff640 1 -- 172.21.15.112:0/1378720138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 msgr2=0x55c1ac761800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb343fff640 1 --2- 172.21.15.112:0/1378720138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac761800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb343fff640 1 -- 172.21.15.112:0/1378720138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1ac425650 con 0x55c1ac3ba070 2024-09-21T03:34:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb3437fe640 1 --2- 172.21.15.112:0/1378720138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac761800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb343fff640 1 --2- 172.21.15.112:0/1378720138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3ba070 0x55c1ac7612c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb33c011790 tx=0x7fb33c011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c214bce4b91bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.499+0000 7fb339ffb640 1 -- 172.21.15.112:0/1378720138 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c003c30 con 0x55c1ac3ba070 2024-09-21T03:34:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.500+0000 7fb339ffb640 1 -- 172.21.15.112:0/1378720138 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb33c00fe50 con 0x55c1ac3ba070 2024-09-21T03:34:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.500+0000 7fb3417fa640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3300b6680 0x7fb3300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.500+0000 7fb339ffb640 1 -- 172.21.15.112:0/1378720138 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c004840 con 0x55c1ac3ba070 2024-09-21T03:34:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.500+0000 7fb3437fe640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3300b6680 0x7fb3300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.500+0000 7fb3417fa640 1 -- 172.21.15.112:0/1696924101 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fb32c107650 con 0x7fb3441094b0 2024-09-21T03:34:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3503bc740 1 -- 172.21.15.112:0/1378720138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3ba070 msgr2=0x55c1ac7612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3437fe640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3300b6680 0x7fb3300b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55c1ac75ff40 tx=0x7fb334005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3503bc740 1 --2- 172.21.15.112:0/1378720138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3ba070 0x55c1ac7612c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb33c011790 tx=0x7fb33c011c50 comp rx=0 tx=0).stop 2024-09-21T03:34:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3503bc740 1 -- 172.21.15.112:0/1378720138 shutdown_connections 2024-09-21T03:34:55.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3503bc740 1 --2- 172.21.15.112:0/1378720138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 0x55c1ac761d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3503bc740 1 --2- 172.21.15.112:0/1378720138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac761800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3503bc740 1 --2- 172.21.15.112:0/1378720138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3ba070 0x55c1ac7612c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3503bc740 1 -- 172.21.15.112:0/1378720138 >> 172.21.15.112:0/1378720138 conn(0x55c1ac3ae460 msgr2=0x55c1ac75f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3503bc740 1 -- 172.21.15.112:0/1378720138 shutdown_connections 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3503bc740 1 -- 172.21.15.112:0/1378720138 wait complete. 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3503bc740 1 -- start start 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.501+0000 7fb3503bc740 1 Processor -- start 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3911b0 0x55c1ac76c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 0x55c1ac76cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3ba070 0x55c1ac778d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb3503bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac528700 con 0x55c1ac3911b0 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb3503bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac662160 con 0x55c1ac3ba070 2024-09-21T03:34:55.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb3503bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c1ac76faa0 con 0x55c1ac399390 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb3503bc740 10 client.0 _ll_register_callbacks cb 0x55c1ac67b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb33bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 0x55c1ac76cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb343fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3ba070 0x55c1ac778d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb3437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3911b0 0x55c1ac76c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb343fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3ba070 0x55c1ac778d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:38720/0 (socket says 172.21.15.112:38720) 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb33bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 0x55c1ac76cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:53400/0 (socket says 172.21.15.112:53400) 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb343fff640 1 -- 172.21.15.112:0/3055758692 learned_addr learned my addr 172.21.15.112:0/3055758692 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb343fff640 1 -- 172.21.15.112:0/3055758692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 msgr2=0x55c1ac76cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb343fff640 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 0x55c1ac76cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb343fff640 1 -- 172.21.15.112:0/3055758692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3911b0 msgr2=0x55c1ac76c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb343fff640 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3911b0 0x55c1ac76c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb343fff640 1 -- 172.21.15.112:0/3055758692 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1ac68fc90 con 0x55c1ac3ba070 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.502+0000 7fb33bfff640 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 0x55c1ac76cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:55.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.503+0000 7fb343fff640 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3ba070 0x55c1ac778d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb33c004460 tx=0x7fb33c00fbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48d6046c92221e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.503+0000 7fb322ffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.503+0000 7fb322ffd640 1 -- 172.21.15.112:0/3055758692 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c01f040 con 0x55c1ac3ba070 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.503+0000 7fb322ffd640 1 -- 172.21.15.112:0/3055758692 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fb33c011250 con 0x55c1ac3ba070 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.503+0000 7fb3503bc740 1 -- 172.21.15.112:0/3055758692 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb33c00fe50 con 0x55c1ac3ba070 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.503+0000 7fb322ffd640 1 -- 172.21.15.112:0/3055758692 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb33c032050 con 0x55c1ac3ba070 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.503+0000 7fb3503bc740 10 client.26099 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.503+0000 7fb3503bc740 1 -- 172.21.15.112:0/3055758692 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1ac77b790 con 0x55c1ac3ba070 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.503+0000 7fb322ffd640 1 -- 172.21.15.112:0/3055758692 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb33c029070 con 0x55c1ac3ba070 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.504+0000 7fb322ffd640 1 -- 172.21.15.112:0/3055758692 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fb33c018070 con 0x55c1ac3ba070 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.504+0000 7fb3503bc740 10 client.26099 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.504+0000 7fb3503bc740 10 client.26099 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.504+0000 7fb3503bc740 20 client.26099 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.504+0000 7fb3503bc740 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c1ac3789f0 0x55c1ac77eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.504+0000 7fb3503bc740 4 client.26099 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.504+0000 7fb3503bc740 1 -- 172.21.15.112:0/3055758692 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb33c00fe50 con 0x55c1ac3789f0 2024-09-21T03:34:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.504+0000 7fb3437fe640 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c1ac3789f0 0x55c1ac77eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.504+0000 7fb3437fe640 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c1ac3789f0 0x55c1ac77eba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b2b7c689b3c0ee0 server_cookie=1de2300c61f8a96b in_seq=0 out_seq=0 2024-09-21T03:34:55.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.504+0000 7fb322ffd640 10 client.26099 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb322ffd640 1 -- 172.21.15.112:0/3055758692 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fb33c00fe50 con 0x55c1ac3789f0 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb322ffd640 10 client.26099 handle_command_reply: tid=0 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb3503bc740 1 client.26099 shutdown 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb3503bc740 1 -- 172.21.15.112:0/3055758692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3ba070 msgr2=0x55c1ac778d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb3503bc740 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3ba070 0x55c1ac778d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb33c004460 tx=0x7fb33c00fbd0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb3503bc740 1 -- 172.21.15.112:0/3055758692 shutdown_connections 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb3503bc740 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c1ac3789f0 0x55c1ac77eba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb3503bc740 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3ba070 0x55c1ac778d90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb3503bc740 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac399390 0x55c1ac76cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb3503bc740 1 --2- 172.21.15.112:0/3055758692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac3911b0 0x55c1ac76c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb322ffd640 0 client.26099 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:34:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb3503bc740 1 -- 172.21.15.112:0/3055758692 >> 172.21.15.112:0/3055758692 conn(0x55c1ac3ae460 msgr2=0x55c1ac764470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb3503bc740 1 -- 172.21.15.112:0/3055758692 shutdown_connections 2024-09-21T03:34:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.505+0000 7fb3503bc740 1 -- 172.21.15.112:0/3055758692 wait complete. 2024-09-21T03:34:55.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.506+0000 7fb3503bc740 20 client.26099 trim_cache size 0 max 16384 2024-09-21T03:34:55.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb3503bc740 1 Processor -- start 2024-09-21T03:34:55.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb3503bc740 1 -- start start 2024-09-21T03:34:55.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 0x55c1ac75f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac778c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 0x55c1ac7643c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb3503bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3441b4100 con 0x55c1ac399390 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb3503bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb3441b19a0 con 0x55c1ac3911b0 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb3503bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb3441af7f0 con 0x55c1ac3789f0 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb343fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 0x55c1ac7643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb343fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 0x55c1ac7643c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56382/0 (socket says 172.21.15.112:56382) 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb3437fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 0x55c1ac75f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb343fff640 1 -- 172.21.15.112:0/3714493993 learned_addr learned my addr 172.21.15.112:0/3714493993 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb33bfff640 1 --2- 172.21.15.112:0/3714493993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac778c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb33bfff640 1 -- 172.21.15.112:0/3714493993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 msgr2=0x55c1ac75f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb33bfff640 1 --2- 172.21.15.112:0/3714493993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 0x55c1ac75f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb33bfff640 1 -- 172.21.15.112:0/3714493993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 msgr2=0x55c1ac7643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb33bfff640 1 --2- 172.21.15.112:0/3714493993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 0x55c1ac7643c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb33bfff640 1 -- 172.21.15.112:0/3714493993 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1ac76dc80 con 0x55c1ac3911b0 2024-09-21T03:34:55.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb3437fe640 1 --2- 172.21.15.112:0/3714493993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 0x55c1ac75f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:55.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.520+0000 7fb343fff640 1 --2- 172.21.15.112:0/3714493993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 0x55c1ac7643c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:55.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.521+0000 7fb33bfff640 1 --2- 172.21.15.112:0/3714493993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac778c90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb32c104e00 tx=0x7fb32c047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c53605a8a68636fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.521+0000 7fb3397fa640 1 -- 172.21.15.112:0/3714493993 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c049040 con 0x55c1ac3911b0 2024-09-21T03:34:55.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.521+0000 7fb3397fa640 1 -- 172.21.15.112:0/3714493993 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb32c0cd0c0 con 0x55c1ac3911b0 2024-09-21T03:34:55.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.521+0000 7fb3397fa640 1 -- 172.21.15.112:0/3714493993 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c0d6f20 con 0x55c1ac3911b0 2024-09-21T03:34:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.522+0000 7fb3503bc740 1 -- 172.21.15.112:0/3714493993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 msgr2=0x55c1ac778c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.522+0000 7fb3503bc740 1 --2- 172.21.15.112:0/3714493993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac778c90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb32c104e00 tx=0x7fb32c047000 comp rx=0 tx=0).stop 2024-09-21T03:34:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.522+0000 7fb3503bc740 1 -- 172.21.15.112:0/3714493993 shutdown_connections 2024-09-21T03:34:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.522+0000 7fb3503bc740 1 --2- 172.21.15.112:0/3714493993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 0x55c1ac7643c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.522+0000 7fb3503bc740 1 --2- 172.21.15.112:0/3714493993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac778c90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.522+0000 7fb3503bc740 1 --2- 172.21.15.112:0/3714493993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 0x55c1ac75f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.522+0000 7fb3503bc740 1 -- 172.21.15.112:0/3714493993 >> 172.21.15.112:0/3714493993 conn(0x55c1ac3ae460 msgr2=0x55c1ac75eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.522+0000 7fb3503bc740 1 -- 172.21.15.112:0/3714493993 shutdown_connections 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.522+0000 7fb3503bc740 1 -- 172.21.15.112:0/3714493993 wait complete. 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.522+0000 7fb3503bc740 1 -- start start 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.522+0000 7fb3503bc740 1 Processor -- start 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 0x55c1ac770e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac7713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3503bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 0x55c1ac771910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3503bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3441afe60 con 0x55c1ac399390 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3503bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb3441b12d0 con 0x55c1ac3911b0 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3503bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb3441b0530 con 0x55c1ac3789f0 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3503bc740 10 client.0 _ll_register_callbacks cb 0x7fb32c10d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3437fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 0x55c1ac770e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb33bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac7713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3437fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 0x55c1ac770e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:53420/0 (socket says 172.21.15.112:53420) 2024-09-21T03:34:55.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3437fe640 1 -- 172.21.15.112:0/2754352526 learned_addr learned my addr 172.21.15.112:0/2754352526 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb343fff640 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 0x55c1ac771910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3437fe640 1 -- 172.21.15.112:0/2754352526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 msgr2=0x55c1ac7713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3437fe640 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac7713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3437fe640 1 -- 172.21.15.112:0/2754352526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 msgr2=0x55c1ac771910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3437fe640 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 0x55c1ac771910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3437fe640 1 -- 172.21.15.112:0/2754352526 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1ac7e8680 con 0x55c1ac3789f0 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb33bfff640 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac7713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb3437fe640 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 0x55c1ac770e90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55c1ac7636f0 tx=0x7fb3340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d2444893b98ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.523+0000 7fb322ffd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb322ffd640 1 -- 172.21.15.112:0/2754352526 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33402b400 con 0x55c1ac3789f0 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb322ffd640 1 -- 172.21.15.112:0/2754352526 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fb33402b5a0 con 0x55c1ac3789f0 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb3503bc740 1 -- 172.21.15.112:0/2754352526 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb32c0cd0c0 con 0x55c1ac3789f0 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb322ffd640 1 -- 172.21.15.112:0/2754352526 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb33403e050 con 0x55c1ac3789f0 2024-09-21T03:34:55.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb3503bc740 10 client.26110 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb3503bc740 1 -- 172.21.15.112:0/2754352526 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1ac7e9950 con 0x55c1ac3789f0 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb322ffd640 1 -- 172.21.15.112:0/2754352526 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb32c0cd0c0 con 0x55c1ac3789f0 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb322ffd640 1 -- 172.21.15.112:0/2754352526 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fb334030750 con 0x55c1ac3789f0 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb3503bc740 10 client.26110 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb3503bc740 10 client.26110 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb3503bc740 20 client.26110 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb3503bc740 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c1ac3ba070 0x55c1ac7ec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb3503bc740 4 client.26110 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.524+0000 7fb3503bc740 1 -- 172.21.15.112:0/2754352526 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1ac6d9df0 con 0x55c1ac3ba070 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.525+0000 7fb33bfff640 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c1ac3ba070 0x55c1ac7ec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.525+0000 7fb322ffd640 10 client.26110 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.525+0000 7fb33bfff640 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c1ac3ba070 0x55c1ac7ec1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a498f63bda5021e server_cookie=9ce2fcc580f88e0f in_seq=0 out_seq=0 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.525+0000 7fb322ffd640 1 -- 172.21.15.112:0/2754352526 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1ac6d9df0 con 0x55c1ac3ba070 2024-09-21T03:34:55.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.525+0000 7fb322ffd640 10 client.26110 handle_command_reply: tid=0 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.525+0000 7fb3503bc740 1 client.26110 shutdown 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.525+0000 7fb3503bc740 1 -- 172.21.15.112:0/2754352526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 msgr2=0x55c1ac770e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.525+0000 7fb3503bc740 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 0x55c1ac770e90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55c1ac7636f0 tx=0x7fb3340046c0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb3503bc740 1 -- 172.21.15.112:0/2754352526 shutdown_connections 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb3503bc740 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55c1ac3ba070 0x55c1ac7ec1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb3503bc740 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1ac399390 0x55c1ac771910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb3503bc740 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1ac3911b0 0x55c1ac7713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb322ffd640 0 client.26110 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb3503bc740 1 --2- 172.21.15.112:0/2754352526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1ac3789f0 0x55c1ac770e90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb3503bc740 1 -- 172.21.15.112:0/2754352526 >> 172.21.15.112:0/2754352526 conn(0x55c1ac3ae460 msgr2=0x55c1ac7643c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb3503bc740 1 -- 172.21.15.112:0/2754352526 shutdown_connections 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb3503bc740 1 -- 172.21.15.112:0/2754352526 wait complete. 2024-09-21T03:34:55.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb3503bc740 20 client.26110 trim_cache size 0 max 16384 2024-09-21T03:34:55.533 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:34:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb349e48640 1 -- 172.21.15.112:0/1696924101 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3300b6680 msgr2=0x7fb3300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb349e48640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3300b6680 0x7fb3300b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55c1ac75ff40 tx=0x7fb334005f90 comp rx=0 tx=0).stop 2024-09-21T03:34:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb349e48640 1 -- 172.21.15.112:0/1696924101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3441094b0 msgr2=0x7fb344075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb349e48640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3441094b0 0x7fb344075c50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb32c02a930 tx=0x7fb32c031a20 comp rx=0 tx=0).stop 2024-09-21T03:34:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb349e48640 1 -- 172.21.15.112:0/1696924101 shutdown_connections 2024-09-21T03:34:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb349e48640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb3300b6680 0x7fb3300b8b90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb349e48640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb344109e00 0x7fb344076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb349e48640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3441094b0 0x7fb344075c50 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb349e48640 1 --2- 172.21.15.112:0/1696924101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb344103bc0 0x7fb344075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.526+0000 7fb349e48640 1 -- 172.21.15.112:0/1696924101 >> 172.21.15.112:0/1696924101 conn(0x7fb34407b910 msgr2=0x7fb344107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.527+0000 7fb349e48640 1 -- 172.21.15.112:0/1696924101 shutdown_connections 2024-09-21T03:34:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:55.527+0000 7fb349e48640 1 -- 172.21.15.112:0/1696924101 wait complete. 2024-09-21T03:34:56.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 601959 10.91 MB/sec execute 176 sec latency 80.734 ms 2024-09-21T03:34:56.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:34:56.641 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:56.762 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:56 smithi112 ceph-mon[31442]: pgmap v496: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 319 op/s 2024-09-21T03:34:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:56 smithi203 ceph-mon[34723]: pgmap v496: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 319 op/s 2024-09-21T03:34:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:56 smithi187 ceph-mon[34603]: pgmap v496: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 319 op/s 2024-09-21T03:34:57.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.188+0000 7fa2a5666640 1 -- 172.21.15.112:0/421093632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0109e00 msgr2=0x7fa2a0110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:57.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.188+0000 7fa2a5666640 1 --2- 172.21.15.112:0/421093632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0109e00 0x7fa2a0110630 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa298009ca0 tx=0x7fa29802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.188+0000 7fa2a5666640 1 -- 172.21.15.112:0/421093632 shutdown_connections 2024-09-21T03:34:57.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.188+0000 7fa2a5666640 1 --2- 172.21.15.112:0/421093632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0109e00 0x7fa2a0110630 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.188+0000 7fa2a5666640 1 --2- 172.21.15.112:0/421093632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2a01094b0 0x7fa2a01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.188+0000 7fa2a5666640 1 --2- 172.21.15.112:0/421093632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a0101f70 0x7fa2a0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.188+0000 7fa2a5666640 1 -- 172.21.15.112:0/421093632 >> 172.21.15.112:0/421093632 conn(0x7fa2a0077c60 msgr2=0x7fa2a00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:57.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.188+0000 7fa2a5666640 1 -- 172.21.15.112:0/421093632 shutdown_connections 2024-09-21T03:34:57.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.188+0000 7fa2a5666640 1 -- 172.21.15.112:0/421093632 wait complete. 2024-09-21T03:34:57.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.189+0000 7fa2a5666640 1 Processor -- start 2024-09-21T03:34:57.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.189+0000 7fa2a5666640 1 -- start start 2024-09-21T03:34:57.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2a5666640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2a0101f70 0x7fa2a019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2a5666640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01094b0 0x7fa2a019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2a5666640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0109e00 0x7fa2a019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2a5666640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0113ac0 con 0x7fa2a0109e00 2024-09-21T03:34:57.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2a5666640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0113940 con 0x7fa2a0101f70 2024-09-21T03:34:57.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2a5666640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0113c40 con 0x7fa2a01094b0 2024-09-21T03:34:57.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa29f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0109e00 0x7fa2a019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2967fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01094b0 0x7fa2a019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa29f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0109e00 0x7fa2a019fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56416/0 (socket says 172.21.15.112:56416) 2024-09-21T03:34:57.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa29f7fe640 1 -- 172.21.15.112:0/2608807778 learned_addr learned my addr 172.21.15.112:0/2608807778 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:57.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2967fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01094b0 0x7fa2a019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:53444/0 (socket says 172.21.15.112:53444) 2024-09-21T03:34:57.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa29effd640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2a0101f70 0x7fa2a019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2967fc640 1 -- 172.21.15.112:0/2608807778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2a0101f70 msgr2=0x7fa2a019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:57.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2967fc640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2a0101f70 0x7fa2a019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2967fc640 1 -- 172.21.15.112:0/2608807778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0109e00 msgr2=0x7fa2a019fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:57.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2967fc640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0109e00 0x7fa2a019fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2967fc640 1 -- 172.21.15.112:0/2608807778 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a01a0180 con 0x7fa2a01094b0 2024-09-21T03:34:57.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.190+0000 7fa2967fc640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01094b0 0x7fa2a019a7b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa288005fd0 tx=0x7fa28800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:57.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.191+0000 7fa29cff9640 1 -- 172.21.15.112:0/2608807778 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288009c00 con 0x7fa2a01094b0 2024-09-21T03:34:57.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.191+0000 7fa2a5666640 1 -- 172.21.15.112:0/2608807778 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a01a0470 con 0x7fa2a01094b0 2024-09-21T03:34:57.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.191+0000 7fa29cff9640 1 -- 172.21.15.112:0/2608807778 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa288009da0 con 0x7fa2a01094b0 2024-09-21T03:34:57.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.191+0000 7fa29cff9640 1 -- 172.21.15.112:0/2608807778 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288003e10 con 0x7fa2a01094b0 2024-09-21T03:34:57.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.191+0000 7fa2a5666640 1 -- 172.21.15.112:0/2608807778 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a01a6bd0 con 0x7fa2a01094b0 2024-09-21T03:34:57.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.193+0000 7fa2abbda740 1 Processor -- start 2024-09-21T03:34:57.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.193+0000 7fa2abbda740 1 -- start start 2024-09-21T03:34:57.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da6842070 0x563da6be92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da68191b0 0x563da6be9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6be9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa2abbda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563da6b5fea0 con 0x563da6842070 2024-09-21T03:34:57.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa2abbda740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563da6b4f2d0 con 0x563da68191b0 2024-09-21T03:34:57.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa2abbda740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563da6b530e0 con 0x563da6821390 2024-09-21T03:34:57.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa29effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da68191b0 0x563da6be9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa29effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da68191b0 0x563da6be9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:38776/0 (socket says 172.21.15.112:38776) 2024-09-21T03:34:57.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa29effd640 1 -- 172.21.15.112:0/428460059 learned_addr learned my addr 172.21.15.112:0/428460059 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:57.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa29cff9640 1 -- 172.21.15.112:0/2608807778 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa288004030 con 0x7fa2a01094b0 2024-09-21T03:34:57.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa29f7fe640 1 --2- 172.21.15.112:0/428460059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da6842070 0x563da6be92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.194+0000 7fa29effd640 1 -- 172.21.15.112:0/428460059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 msgr2=0x563da6be9d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:34:57.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.195+0000 7fa29effd640 1 --2- 172.21.15.112:0/428460059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6be9d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.195+0000 7fa29effd640 1 -- 172.21.15.112:0/428460059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da6842070 msgr2=0x563da6be92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:57.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.195+0000 7fa29effd640 1 --2- 172.21.15.112:0/428460059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da6842070 0x563da6be92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.195+0000 7fa29effd640 1 -- 172.21.15.112:0/428460059 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da68ad650 con 0x563da68191b0 2024-09-21T03:34:57.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.195+0000 7fa29f7fe640 1 --2- 172.21.15.112:0/428460059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da6842070 0x563da6be92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:34:57.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.195+0000 7fa29effd640 1 --2- 172.21.15.112:0/428460059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da68191b0 0x563da6be9800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa290009d20 tx=0x7fa290011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba5562248de9d85f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:57.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.195+0000 7fa2957fa640 1 -- 172.21.15.112:0/428460059 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29001e070 con 0x563da68191b0 2024-09-21T03:34:57.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.195+0000 7fa2957fa640 1 -- 172.21.15.112:0/428460059 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa290019ae0 con 0x563da68191b0 2024-09-21T03:34:57.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.195+0000 7fa2957fa640 1 -- 172.21.15.112:0/428460059 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29000eac0 con 0x563da68191b0 2024-09-21T03:34:57.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.196+0000 7fa29cff9640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa28c0b6680 0x7fa28c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.196+0000 7fa29f7fe640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa28c0b6680 0x7fa28c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.196+0000 7fa29cff9640 1 -- 172.21.15.112:0/2608807778 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa2880e74e0 con 0x7fa2a01094b0 2024-09-21T03:34:57.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.196+0000 7fa29f7fe640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa28c0b6680 0x7fa28c0b8b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa298004af0 tx=0x7fa2980098c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.196+0000 7fa2abbda740 1 -- 172.21.15.112:0/428460059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da68191b0 msgr2=0x563da6be9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.196+0000 7fa2abbda740 1 --2- 172.21.15.112:0/428460059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da68191b0 0x563da6be9800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa290009d20 tx=0x7fa290011810 comp rx=0 tx=0).stop 2024-09-21T03:34:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.197+0000 7fa2abbda740 1 -- 172.21.15.112:0/428460059 shutdown_connections 2024-09-21T03:34:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.197+0000 7fa2abbda740 1 --2- 172.21.15.112:0/428460059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6be9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.197+0000 7fa2abbda740 1 --2- 172.21.15.112:0/428460059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da68191b0 0x563da6be9800 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.197+0000 7fa2abbda740 1 --2- 172.21.15.112:0/428460059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da6842070 0x563da6be92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.197+0000 7fa2abbda740 1 -- 172.21.15.112:0/428460059 >> 172.21.15.112:0/428460059 conn(0x563da6836460 msgr2=0x563da6be71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.197+0000 7fa2abbda740 1 -- 172.21.15.112:0/428460059 shutdown_connections 2024-09-21T03:34:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.197+0000 7fa2abbda740 1 -- 172.21.15.112:0/428460059 wait complete. 2024-09-21T03:34:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.197+0000 7fa2abbda740 1 -- start start 2024-09-21T03:34:57.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.197+0000 7fa2abbda740 1 Processor -- start 2024-09-21T03:34:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6bf4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6842070 0x563da6c00d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2abbda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563da69b0700 con 0x563da68191b0 2024-09-21T03:34:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2abbda740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563da6aea160 con 0x563da6842070 2024-09-21T03:34:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa29effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2abbda740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563da6bf7aa0 con 0x563da6821390 2024-09-21T03:34:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa29effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56432/0 (socket says 172.21.15.112:56432) 2024-09-21T03:34:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa29effd640 1 -- 172.21.15.112:0/2947087861 learned_addr learned my addr 172.21.15.112:0/2947087861 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2967fc640 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6bf4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa29f7fe640 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6842070 0x563da6c00d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2abbda740 10 client.0 _ll_register_callbacks cb 0x563da6b039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa29effd640 1 -- 172.21.15.112:0/2947087861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 msgr2=0x563da6bf4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa29effd640 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6bf4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa29effd640 1 -- 172.21.15.112:0/2947087861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6842070 msgr2=0x563da6c00d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa29effd640 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6842070 0x563da6c00d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa29effd640 1 -- 172.21.15.112:0/2947087861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da6b17c90 con 0x563da68191b0 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2967fc640 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6bf4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa29effd640 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf4a30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa29000e760 tx=0x7fa290004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0ad49745ec6fb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2867fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2867fc640 1 -- 172.21.15.112:0/2947087861 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290027020 con 0x563da68191b0 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2abbda740 1 -- 172.21.15.112:0/2947087861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa290019ae0 con 0x563da68191b0 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.198+0000 7fa2867fc640 1 -- 172.21.15.112:0/2947087861 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa290019630 con 0x563da68191b0 2024-09-21T03:34:57.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2867fc640 1 -- 172.21.15.112:0/2947087861 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa290032050 con 0x563da68191b0 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2abbda740 10 client.16551 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2abbda740 1 -- 172.21.15.112:0/2947087861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563da6c03760 con 0x563da68191b0 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2867fc640 1 -- 172.21.15.112:0/2947087861 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa290019ae0 con 0x563da68191b0 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2867fc640 1 -- 172.21.15.112:0/2947087861 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fa29000eba0 con 0x563da68191b0 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2abbda740 10 client.16551 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2abbda740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2abbda740 20 client.16551 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563da68009f0 0x563da6c06ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2abbda740 4 client.16551 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2abbda740 1 -- 172.21.15.112:0/2947087861 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563da6b61df0 con 0x563da68009f0 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.199+0000 7fa2967fc640 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563da68009f0 0x563da6c06ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.200+0000 7fa2967fc640 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563da68009f0 0x563da6c06ac0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28cf1ad9c6d5f2c server_cookie=e61aeccdcaf95a11 in_seq=0 out_seq=0 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.200+0000 7fa2867fc640 10 client.16551 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:57.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2867fc640 1 -- 172.21.15.112:0/2947087861 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563da6b61df0 con 0x563da68009f0 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2867fc640 10 client.16551 handle_command_reply: tid=0 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2abbda740 1 client.16551 shutdown 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2abbda740 1 -- 172.21.15.112:0/2947087861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 msgr2=0x563da6bf4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf4a30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa29000e760 tx=0x7fa290004270 comp rx=0 tx=0).stop 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2abbda740 1 -- 172.21.15.112:0/2947087861 shutdown_connections 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563da68009f0 0x563da6c06ac0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6842070 0x563da6c00d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6bf4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2947087861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf4a30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2abbda740 1 -- 172.21.15.112:0/2947087861 >> 172.21.15.112:0/2947087861 conn(0x563da6836460 msgr2=0x563da6bec4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.201+0000 7fa2867fc640 0 client.16551 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.202+0000 7fa2abbda740 1 -- 172.21.15.112:0/2947087861 shutdown_connections 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.202+0000 7fa2abbda740 1 -- 172.21.15.112:0/2947087861 wait complete. 2024-09-21T03:34:57.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.202+0000 7fa2abbda740 20 client.16551 trim_cache size 0 max 16384 2024-09-21T03:34:57.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.215+0000 7fa2abbda740 1 Processor -- start 2024-09-21T03:34:57.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.215+0000 7fa2abbda740 1 -- start start 2024-09-21T03:34:57.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6c00c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6842070 0x563da6bf1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa2abbda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01b1ea0 con 0x563da68191b0 2024-09-21T03:34:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa2abbda740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01af740 con 0x563da6842070 2024-09-21T03:34:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa2abbda740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01ad590 con 0x563da6821390 2024-09-21T03:34:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa29effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa29effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56444/0 (socket says 172.21.15.112:56444) 2024-09-21T03:34:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa29effd640 1 -- 172.21.15.112:0/1509968168 learned_addr learned my addr 172.21.15.112:0/1509968168 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa2967fc640 1 --2- 172.21.15.112:0/1509968168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6c00c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa29f7fe640 1 --2- 172.21.15.112:0/1509968168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6842070 0x563da6bf1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa29effd640 1 -- 172.21.15.112:0/1509968168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 msgr2=0x563da6c00c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa29effd640 1 --2- 172.21.15.112:0/1509968168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6c00c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa29effd640 1 -- 172.21.15.112:0/1509968168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6842070 msgr2=0x563da6bf1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:57.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa29effd640 1 --2- 172.21.15.112:0/1509968168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6842070 0x563da6bf1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa29effd640 1 -- 172.21.15.112:0/1509968168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da6bf5cf0 con 0x563da68191b0 2024-09-21T03:34:57.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.216+0000 7fa29f7fe640 1 --2- 172.21.15.112:0/1509968168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6842070 0x563da6bf1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:57.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.217+0000 7fa29effd640 1 --2- 172.21.15.112:0/1509968168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf6660 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa2900058c0 tx=0x7fa290005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec1f6996542988b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:57.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.217+0000 7fa294ff9640 1 -- 172.21.15.112:0/1509968168 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29003b070 con 0x563da68191b0 2024-09-21T03:34:57.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.217+0000 7fa294ff9640 1 -- 172.21.15.112:0/1509968168 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa29001f040 con 0x563da68191b0 2024-09-21T03:34:57.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.217+0000 7fa294ff9640 1 -- 172.21.15.112:0/1509968168 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290027070 con 0x563da68191b0 2024-09-21T03:34:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 -- 172.21.15.112:0/1509968168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 msgr2=0x563da6bf6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 --2- 172.21.15.112:0/1509968168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf6660 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa2900058c0 tx=0x7fa290005850 comp rx=0 tx=0).stop 2024-09-21T03:34:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 -- 172.21.15.112:0/1509968168 shutdown_connections 2024-09-21T03:34:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 --2- 172.21.15.112:0/1509968168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6842070 0x563da6bf1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 --2- 172.21.15.112:0/1509968168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6821390 0x563da6c00c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 --2- 172.21.15.112:0/1509968168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bf6660 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 -- 172.21.15.112:0/1509968168 >> 172.21.15.112:0/1509968168 conn(0x563da6836460 msgr2=0x563da6bf5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 -- 172.21.15.112:0/1509968168 shutdown_connections 2024-09-21T03:34:57.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 -- 172.21.15.112:0/1509968168 wait complete. 2024-09-21T03:34:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 -- start start 2024-09-21T03:34:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 Processor -- start 2024-09-21T03:34:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.218+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bfd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6821390 0x563da6bf8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa2abbda740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6842070 0x563da6bf9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa2abbda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01adc00 con 0x563da68191b0 2024-09-21T03:34:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa2abbda740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01af070 con 0x563da6821390 2024-09-21T03:34:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa2abbda740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01ae2d0 con 0x563da6842070 2024-09-21T03:34:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa2abbda740 10 client.0 _ll_register_callbacks cb 0x7fa29001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa29effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bfd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa2967fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6821390 0x563da6bf8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa29effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bfd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56452/0 (socket says 172.21.15.112:56452) 2024-09-21T03:34:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa29effd640 1 -- 172.21.15.112:0/2856751839 learned_addr learned my addr 172.21.15.112:0/2856751839 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa29f7fe640 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6842070 0x563da6bf9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa29effd640 1 -- 172.21.15.112:0/2856751839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6842070 msgr2=0x563da6bf9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa29effd640 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6842070 0x563da6bf9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa29effd640 1 -- 172.21.15.112:0/2856751839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6821390 msgr2=0x563da6bf8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa29effd640 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6821390 0x563da6bf8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa29effd640 1 -- 172.21.15.112:0/2856751839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da6bf98c0 con 0x563da68191b0 2024-09-21T03:34:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa29effd640 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bfd600 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa290004bc0 tx=0x7fa290005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee503fcbf401d13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa2867fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:34:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa2867fc640 1 -- 172.21.15.112:0/2856751839 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29003b030 con 0x563da68191b0 2024-09-21T03:34:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.219+0000 7fa2867fc640 1 -- 172.21.15.112:0/2856751839 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7fa290019a40 con 0x563da68191b0 2024-09-21T03:34:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2abbda740 1 -- 172.21.15.112:0/2856751839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa29001f040 con 0x563da68191b0 2024-09-21T03:34:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2867fc640 1 -- 172.21.15.112:0/2856751839 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa290027070 con 0x563da68191b0 2024-09-21T03:34:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2abbda740 10 client.16557 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2abbda740 1 -- 172.21.15.112:0/2856751839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563da6c71320 con 0x563da68191b0 2024-09-21T03:34:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2867fc640 1 -- 172.21.15.112:0/2856751839 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa29001f040 con 0x563da68191b0 2024-09-21T03:34:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2867fc640 1 -- 172.21.15.112:0/2856751839 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fa290004c90 con 0x563da68191b0 2024-09-21T03:34:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2abbda740 10 client.16557 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2abbda740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:57.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2abbda740 20 client.16557 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:57.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563da67d5da0 0x563da6c73a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:57.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2abbda740 4 client.16557 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:34:57.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2abbda740 1 -- 172.21.15.112:0/2856751839 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563da6b5d8a0 con 0x563da67d5da0 2024-09-21T03:34:57.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.220+0000 7fa2967fc640 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563da67d5da0 0x563da6c73a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:57.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2867fc640 10 client.16557 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:57.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2967fc640 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563da67d5da0 0x563da6c73a50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4113c711eecb5ae server_cookie=f3f1c6223c4ab9d6 in_seq=0 out_seq=0 2024-09-21T03:34:57.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2867fc640 1 -- 172.21.15.112:0/2856751839 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563da6b5d8a0 con 0x563da67d5da0 2024-09-21T03:34:57.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2867fc640 10 client.16557 handle_command_reply: tid=0 2024-09-21T03:34:57.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 1 client.16557 shutdown 2024-09-21T03:34:57.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 1 -- 172.21.15.112:0/2856751839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 msgr2=0x563da6bfd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:57.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bfd600 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa290004bc0 tx=0x7fa290005d30 comp rx=0 tx=0).stop 2024-09-21T03:34:57.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 1 -- 172.21.15.112:0/2856751839 shutdown_connections 2024-09-21T03:34:57.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563da67d5da0 0x563da6c73a50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563da6842070 0x563da6bf9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563da6821390 0x563da6bf8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 1 --2- 172.21.15.112:0/2856751839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563da68191b0 0x563da6bfd600 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 1 -- 172.21.15.112:0/2856751839 >> 172.21.15.112:0/2856751839 conn(0x563da6836460 msgr2=0x563da6becb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:57.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2867fc640 0 client.16557 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:57.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 1 -- 172.21.15.112:0/2856751839 shutdown_connections 2024-09-21T03:34:57.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 1 -- 172.21.15.112:0/2856751839 wait complete. 2024-09-21T03:34:57.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.221+0000 7fa2abbda740 20 client.16557 trim_cache size 0 max 16384 2024-09-21T03:34:57.236 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:34:57.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 -- 172.21.15.112:0/2608807778 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa28c0b6680 msgr2=0x7fa28c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:57.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa28c0b6680 0x7fa28c0b8b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa298004af0 tx=0x7fa2980098c0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 -- 172.21.15.112:0/2608807778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01094b0 msgr2=0x7fa2a019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:57.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01094b0 0x7fa2a019a7b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa288005fd0 tx=0x7fa28800dde0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 -- 172.21.15.112:0/2608807778 shutdown_connections 2024-09-21T03:34:57.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa28c0b6680 0x7fa28c0b8b90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2a0109e00 0x7fa2a019fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2a01094b0 0x7fa2a019a7b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 --2- 172.21.15.112:0/2608807778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2a0101f70 0x7fa2a019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:57.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 -- 172.21.15.112:0/2608807778 >> 172.21.15.112:0/2608807778 conn(0x7fa2a0077c60 msgr2=0x7fa2a0106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:57.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 -- 172.21.15.112:0/2608807778 shutdown_connections 2024-09-21T03:34:57.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:57.222+0000 7fa2a5666640 1 -- 172.21.15.112:0/2608807778 wait complete. 2024-09-21T03:34:57.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 604248 10.92 MB/sec execute 177 sec latency 12.948 ms 2024-09-21T03:34:58.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 606234 10.90 MB/sec execute 178 sec latency 51.891 ms 2024-09-21T03:34:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:34:58 smithi112 ceph-mon[31442]: pgmap v497: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-09-21T03:34:58.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:34:58 smithi203 ceph-mon[34723]: pgmap v497: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-09-21T03:34:58.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:34:58 smithi187 ceph-mon[34603]: pgmap v497: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-09-21T03:34:59.120 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:34:59.121 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:34:59.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 608086 10.91 MB/sec execute 179 sec latency 18.922 ms 2024-09-21T03:34:59.397 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:34:59.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.962+0000 7f3e5ab61640 1 -- 172.21.15.112:0/3923386720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e5410de30 msgr2=0x7f3e54110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.962+0000 7f3e5ab61640 1 --2- 172.21.15.112:0/3923386720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e5410de30 0x7f3e54110340 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009900 tx=0x7f3e4c02a410 comp rx=0 tx=0).stop 2024-09-21T03:34:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.962+0000 7f3e5ab61640 1 -- 172.21.15.112:0/3923386720 shutdown_connections 2024-09-21T03:34:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.962+0000 7f3e5ab61640 1 --2- 172.21.15.112:0/3923386720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e5410de30 0x7f3e54110340 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.962+0000 7f3e5ab61640 1 --2- 172.21.15.112:0/3923386720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e541052b0 0x7f3e5410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.962+0000 7f3e5ab61640 1 --2- 172.21.15.112:0/3923386720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e54104880 0x7f3e54104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.962+0000 7f3e5ab61640 1 -- 172.21.15.112:0/3923386720 >> 172.21.15.112:0/3923386720 conn(0x7f3e540797e0 msgr2=0x7f3e54079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.962+0000 7f3e5ab61640 1 -- 172.21.15.112:0/3923386720 shutdown_connections 2024-09-21T03:34:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.962+0000 7f3e5ab61640 1 -- 172.21.15.112:0/3923386720 wait complete. 2024-09-21T03:34:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.962+0000 7f3e5ab61640 1 Processor -- start 2024-09-21T03:34:59.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.962+0000 7f3e5ab61640 1 -- start start 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e5ab61640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e54104880 0x7f3e5419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e5ab61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e541052b0 0x7f3e5419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e5ab61640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e5410de30 0x7f3e5419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e5ab61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54113ad0 con 0x7f3e541052b0 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e5ab61640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54113950 con 0x7f3e5410de30 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e5ab61640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54113c50 con 0x7f3e54104880 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e541052b0 0x7f3e5419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e590fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e5410de30 0x7f3e5419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e541052b0 0x7f3e5419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56468/0 (socket says 172.21.15.112:56468) 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e4bfff640 1 -- 172.21.15.112:0/750880692 learned_addr learned my addr 172.21.15.112:0/750880692 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e588fd640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e54104880 0x7f3e5419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e4bfff640 1 -- 172.21.15.112:0/750880692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e54104880 msgr2=0x7f3e5419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e54104880 0x7f3e5419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e4bfff640 1 -- 172.21.15.112:0/750880692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e5410de30 msgr2=0x7f3e5419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e5410de30 0x7f3e5419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e4bfff640 1 -- 172.21.15.112:0/750880692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e541a03d0 con 0x7f3e541052b0 2024-09-21T03:34:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e590fe640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e5410de30 0x7f3e5419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.963+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e541052b0 0x7f3e5419aa00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400e9c0 tx=0x7f3e4400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.964+0000 7f3e49ffb640 1 -- 172.21.15.112:0/750880692 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400ccc0 con 0x7f3e541052b0 2024-09-21T03:34:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.964+0000 7f3e49ffb640 1 -- 172.21.15.112:0/750880692 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e440054d0 con 0x7f3e541052b0 2024-09-21T03:34:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.964+0000 7f3e5ab61640 1 -- 172.21.15.112:0/750880692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e541a06c0 con 0x7f3e541052b0 2024-09-21T03:34:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.964+0000 7f3e49ffb640 1 -- 172.21.15.112:0/750880692 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44010430 con 0x7f3e541052b0 2024-09-21T03:34:59.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.964+0000 7f3e5ab61640 1 -- 172.21.15.112:0/750880692 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e541a6e20 con 0x7f3e541052b0 2024-09-21T03:34:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e610d5740 1 Processor -- start 2024-09-21T03:34:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e49ffb640 1 -- 172.21.15.112:0/750880692 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3e44005020 con 0x7f3e541052b0 2024-09-21T03:34:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e610d5740 1 -- start start 2024-09-21T03:34:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f58847b350 0x55f588822430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f588452490 0x55f588822970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f588822eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e610d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f588798e60 con 0x55f58845a670 2024-09-21T03:34:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e610d5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f588788290 con 0x55f58847b350 2024-09-21T03:34:59.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e610d5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f58878c0a0 con 0x55f588452490 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e588fd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f588452490 0x55f588822970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e590fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f58847b350 0x55f588822430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e588fd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f588452490 0x55f588822970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:53516/0 (socket says 172.21.15.112:53516) 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e588fd640 1 -- 172.21.15.112:0/2786454409 learned_addr learned my addr 172.21.15.112:0/2786454409 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.967+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/2786454409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f588822eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e588fd640 1 -- 172.21.15.112:0/2786454409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f58847b350 msgr2=0x55f588822430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e588fd640 1 --2- 172.21.15.112:0/2786454409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f58847b350 0x55f588822430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e588fd640 1 -- 172.21.15.112:0/2786454409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 msgr2=0x55f588822eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e588fd640 1 --2- 172.21.15.112:0/2786454409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f588822eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e588fd640 1 -- 172.21.15.112:0/2786454409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5884e6930 con 0x55f588452490 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e588fd640 1 --2- 172.21.15.112:0/2786454409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f588452490 0x55f588822970 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3e50011470 tx=0x7f3e50011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc122a50c8eace95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e49ffb640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e3c0b6680 0x7f3e3c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e3affd640 1 -- 172.21.15.112:0/2786454409 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5001e070 con 0x55f588452490 2024-09-21T03:34:59.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e3affd640 1 -- 172.21.15.112:0/2786454409 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e50003c30 con 0x55f588452490 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e3affd640 1 -- 172.21.15.112:0/2786454409 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000f8a0 con 0x55f588452490 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e590fe640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e3c0b6680 0x7f3e3c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.968+0000 7f3e49ffb640 1 -- 172.21.15.112:0/750880692 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3e440e7a40 con 0x7f3e541052b0 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e590fe640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e3c0b6680 0x7f3e3c0b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c006010 tx=0x7f3e4c0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e610d5740 1 -- 172.21.15.112:0/2786454409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f588452490 msgr2=0x55f588822970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e610d5740 1 --2- 172.21.15.112:0/2786454409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f588452490 0x55f588822970 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3e50011470 tx=0x7f3e50011930 comp rx=0 tx=0).stop 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e610d5740 1 -- 172.21.15.112:0/2786454409 shutdown_connections 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e610d5740 1 --2- 172.21.15.112:0/2786454409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f588822eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e610d5740 1 --2- 172.21.15.112:0/2786454409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f588452490 0x55f588822970 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e610d5740 1 --2- 172.21.15.112:0/2786454409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f58847b350 0x55f588822430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e610d5740 1 -- 172.21.15.112:0/2786454409 >> 172.21.15.112:0/2786454409 conn(0x55f58846f740 msgr2=0x55f588820320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e610d5740 1 -- 172.21.15.112:0/2786454409 shutdown_connections 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e610d5740 1 -- 172.21.15.112:0/2786454409 wait complete. 2024-09-21T03:34:59.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e610d5740 1 -- start start 2024-09-21T03:34:59.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.970+0000 7f3e610d5740 1 Processor -- start 2024-09-21T03:34:59.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f58882e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f58882e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f58883e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e610d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f5885e9890 con 0x55f58845a670 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e610d5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f588722d40 con 0x55f588452490 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e610d5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f588835020 con 0x55f58847b350 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e610d5740 10 client.0 _ll_register_callbacks cb 0x55f588786c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f58882e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e588fd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f58882e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e590fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f58883e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e588fd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f58882e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:38850/0 (socket says 172.21.15.112:38850) 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e588fd640 1 -- 172.21.15.112:0/4141460899 learned_addr learned my addr 172.21.15.112:0/4141460899 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:59.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e588fd640 1 -- 172.21.15.112:0/4141460899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 msgr2=0x55f58883e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e588fd640 1 --2- 172.21.15.112:0/4141460899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f58883e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e588fd640 1 -- 172.21.15.112:0/4141460899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 msgr2=0x55f58882e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e588fd640 1 --2- 172.21.15.112:0/4141460899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f58882e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.971+0000 7f3e588fd640 1 -- 172.21.15.112:0/4141460899 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f58883e930 con 0x55f588452490 2024-09-21T03:34:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.972+0000 7f3e588fd640 1 --2- 172.21.15.112:0/4141460899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f58882e010 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e50011440 tx=0x7f3e50005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaedc8b196719454 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.972+0000 7f3e33fff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:34:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.972+0000 7f3e33fff640 1 -- 172.21.15.112:0/4141460899 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50027020 con 0x55f588452490 2024-09-21T03:34:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.972+0000 7f3e610d5740 1 -- 172.21.15.112:0/4141460899 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e50003c30 con 0x55f588452490 2024-09-21T03:34:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.972+0000 7f3e33fff640 1 -- 172.21.15.112:0/4141460899 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3e50004000 con 0x55f588452490 2024-09-21T03:34:59.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.972+0000 7f3e33fff640 1 -- 172.21.15.112:0/4141460899 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e50032050 con 0x55f588452490 2024-09-21T03:34:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.972+0000 7f3e610d5740 10 client.26135 fetch_fsmap learned FSMap version 29 2024-09-21T03:34:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.972+0000 7f3e610d5740 1 -- 172.21.15.112:0/4141460899 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f588841260 con 0x55f588452490 2024-09-21T03:34:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.972+0000 7f3e33fff640 1 -- 172.21.15.112:0/4141460899 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e50003c30 con 0x55f588452490 2024-09-21T03:34:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.973+0000 7f3e33fff640 1 -- 172.21.15.112:0/4141460899 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f3e5000fc80 con 0x55f588452490 2024-09-21T03:34:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.973+0000 7f3e610d5740 10 client.26135 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:34:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.973+0000 7f3e610d5740 10 client.26135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:34:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.973+0000 7f3e610d5740 20 client.26135 populate_metadata read hostname 'smithi112' 2024-09-21T03:34:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.973+0000 7f3e610d5740 1 --2- 172.21.15.112:0/4141460899 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f588439cd0 0x55f588844620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.973+0000 7f3e610d5740 4 client.26135 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:34:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.973+0000 7f3e610d5740 1 -- 172.21.15.112:0/4141460899 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f58879adb0 con 0x55f588439cd0 2024-09-21T03:34:59.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.973+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/4141460899 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f588439cd0 0x55f588844620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e33fff640 10 client.26135 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/4141460899 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f588439cd0 0x55f588844620 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc30a9cdcd2cdfc server_cookie=d7ff39948a411079 in_seq=0 out_seq=0 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e33fff640 1 -- 172.21.15.112:0/4141460899 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f58879adb0 con 0x55f588439cd0 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e33fff640 10 client.26135 handle_command_reply: tid=0 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e610d5740 1 client.26135 shutdown 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e610d5740 1 -- 172.21.15.112:0/4141460899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 msgr2=0x55f58882e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e610d5740 1 --2- 172.21.15.112:0/4141460899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f58882e010 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e50011440 tx=0x7f3e50005bd0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e610d5740 1 -- 172.21.15.112:0/4141460899 shutdown_connections 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e610d5740 1 --2- 172.21.15.112:0/4141460899 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f588439cd0 0x55f588844620 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e610d5740 1 --2- 172.21.15.112:0/4141460899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f58883e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e610d5740 1 --2- 172.21.15.112:0/4141460899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f58882e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e33fff640 0 client.26135 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:34:59.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.974+0000 7f3e610d5740 1 --2- 172.21.15.112:0/4141460899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f58882e010 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.975+0000 7f3e610d5740 1 -- 172.21.15.112:0/4141460899 >> 172.21.15.112:0/4141460899 conn(0x55f58846f740 msgr2=0x55f5888255c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:34:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.975+0000 7f3e610d5740 1 -- 172.21.15.112:0/4141460899 shutdown_connections 2024-09-21T03:34:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.975+0000 7f3e610d5740 1 -- 172.21.15.112:0/4141460899 wait complete. 2024-09-21T03:34:59.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.975+0000 7f3e610d5740 20 client.26135 trim_cache size 0 max 16384 2024-09-21T03:34:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.994+0000 7f3e610d5740 1 Processor -- start 2024-09-21T03:34:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.994+0000 7f3e610d5740 1 -- start start 2024-09-21T03:34:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.994+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f58883e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.994+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f5888274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.994+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f588827a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:34:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.994+0000 7f3e610d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541afd50 con 0x55f58845a670 2024-09-21T03:34:59.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.994+0000 7f3e610d5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541ad5f0 con 0x55f588452490 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.994+0000 7f3e610d5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541ab460 con 0x55f58847b350 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.994+0000 7f3e4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f5888274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f5888274c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56496/0 (socket says 172.21.15.112:56496) 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e4bfff640 1 -- 172.21.15.112:0/3033206978 learned_addr learned my addr 172.21.15.112:0/3033206978 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e588fd640 1 --2- 172.21.15.112:0/3033206978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f58883e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e590fe640 1 --2- 172.21.15.112:0/3033206978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f588827a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e4bfff640 1 -- 172.21.15.112:0/3033206978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 msgr2=0x55f588827a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/3033206978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f588827a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e4bfff640 1 -- 172.21.15.112:0/3033206978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 msgr2=0x55f58883e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/3033206978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f58883e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e4bfff640 1 -- 172.21.15.112:0/3033206978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f588820860 con 0x55f58845a670 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e590fe640 1 --2- 172.21.15.112:0/3033206978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f588827a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e588fd640 1 --2- 172.21.15.112:0/3033206978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f58883e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:34:59.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/3033206978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f5888274c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3e44004650 tx=0x7f3e440d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb80d1e5bedc8cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:34:59.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e3a7fc640 1 -- 172.21.15.112:0/3033206978 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400ce60 con 0x55f58845a670 2024-09-21T03:34:59.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e3a7fc640 1 -- 172.21.15.112:0/3033206978 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e44005310 con 0x55f58845a670 2024-09-21T03:34:59.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.995+0000 7f3e3a7fc640 1 -- 172.21.15.112:0/3033206978 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440b21d0 con 0x55f58845a670 2024-09-21T03:35:00.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.996+0000 7f3e610d5740 1 -- 172.21.15.112:0/3033206978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 msgr2=0x55f5888274c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.996+0000 7f3e610d5740 1 --2- 172.21.15.112:0/3033206978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f5888274c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3e44004650 tx=0x7f3e440d1040 comp rx=0 tx=0).stop 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.996+0000 7f3e610d5740 1 -- 172.21.15.112:0/3033206978 shutdown_connections 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.996+0000 7f3e610d5740 1 --2- 172.21.15.112:0/3033206978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f588827a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.996+0000 7f3e610d5740 1 --2- 172.21.15.112:0/3033206978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f5888274c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.996+0000 7f3e610d5740 1 --2- 172.21.15.112:0/3033206978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f58883e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.996+0000 7f3e610d5740 1 -- 172.21.15.112:0/3033206978 >> 172.21.15.112:0/3033206978 conn(0x55f58846f740 msgr2=0x55f588823e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.996+0000 7f3e610d5740 1 -- 172.21.15.112:0/3033206978 shutdown_connections 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.996+0000 7f3e610d5740 1 -- 172.21.15.112:0/3033206978 wait complete. 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.997+0000 7f3e610d5740 1 -- start start 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.997+0000 7f3e610d5740 1 Processor -- start 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.997+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f588836b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.997+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f588837150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:00.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.997+0000 7f3e610d5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f5888a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.997+0000 7f3e610d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541abab0 con 0x55f58845a670 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.997+0000 7f3e610d5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541acf20 con 0x55f588452490 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.997+0000 7f3e610d5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e541ac180 con 0x55f58847b350 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e610d5740 10 client.0 _ll_register_callbacks cb 0x7f3e5001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f588837150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f588837150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56502/0 (socket says 172.21.15.112:56502) 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e4bfff640 1 -- 172.21.15.112:0/3402160680 learned_addr learned my addr 172.21.15.112:0/3402160680 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e590fe640 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f5888a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e588fd640 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f588836b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e4bfff640 1 -- 172.21.15.112:0/3402160680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 msgr2=0x55f5888a7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f5888a7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e4bfff640 1 -- 172.21.15.112:0/3402160680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 msgr2=0x55f588836b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:00.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f588836b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e4bfff640 1 -- 172.21.15.112:0/3402160680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5888a8500 con 0x55f58845a670 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e588fd640 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f588836b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e4bfff640 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f588837150 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400c6a0 tx=0x7f3e44009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcec2c9df75120a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e33fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e33fff640 1 -- 172.21.15.112:0/3402160680 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440a9060 con 0x55f58845a670 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e33fff640 1 -- 172.21.15.112:0/3402160680 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3e440acb50 con 0x55f58845a670 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.998+0000 7f3e610d5740 1 -- 172.21.15.112:0/3402160680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f588796860 con 0x55f58845a670 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e33fff640 1 -- 172.21.15.112:0/3402160680 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e4400ce60 con 0x55f58845a670 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e610d5740 10 client.16575 fetch_fsmap learned FSMap version 29 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e610d5740 1 -- 172.21.15.112:0/3402160680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f5888af810 con 0x55f58845a670 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e33fff640 1 -- 172.21.15.112:0/3402160680 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f588796860 con 0x55f58845a670 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e33fff640 1 -- 172.21.15.112:0/3402160680 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f3e440ac5c0 con 0x55f58845a670 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e610d5740 10 client.16575 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:35:00.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e610d5740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:35:00.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e610d5740 20 client.16575 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:00.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e610d5740 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f58840f080 0x55f5888b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:00.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e610d5740 4 client.16575 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:35:00.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e610d5740 1 -- 172.21.15.112:0/3402160680 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f588793c90 con 0x55f58840f080 2024-09-21T03:35:00.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:34:59.999+0000 7f3e588fd640 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f58840f080 0x55f5888b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:00.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.000+0000 7f3e33fff640 10 client.16575 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:00.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.000+0000 7f3e588fd640 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f58840f080 0x55f5888b1f50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b60a32d7d712a0a server_cookie=31fad2e1cc6b51e5 in_seq=0 out_seq=0 2024-09-21T03:35:00.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.003+0000 7f3e33fff640 1 -- 172.21.15.112:0/3402160680 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f588793c90 con 0x55f58840f080 2024-09-21T03:35:00.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e33fff640 10 client.16575 handle_command_reply: tid=0 2024-09-21T03:35:00.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 1 client.16575 shutdown 2024-09-21T03:35:00.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 1 -- 172.21.15.112:0/3402160680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 msgr2=0x55f588837150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:00.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f588837150 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400c6a0 tx=0x7f3e44009730 comp rx=0 tx=0).stop 2024-09-21T03:35:00.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 1 -- 172.21.15.112:0/3402160680 shutdown_connections 2024-09-21T03:35:00.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f58840f080 0x55f5888b1f50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f58847b350 0x55f5888a7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f58845a670 0x55f588837150 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 1 --2- 172.21.15.112:0/3402160680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f588452490 0x55f588836b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 1 -- 172.21.15.112:0/3402160680 >> 172.21.15.112:0/3402160680 conn(0x55f58846f740 msgr2=0x55f588823bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:00.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e33fff640 0 client.16575 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:00.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 1 -- 172.21.15.112:0/3402160680 shutdown_connections 2024-09-21T03:35:00.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 1 -- 172.21.15.112:0/3402160680 wait complete. 2024-09-21T03:35:00.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.004+0000 7f3e610d5740 20 client.16575 trim_cache size 0 max 16384 2024-09-21T03:35:00.007 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:35:00.008 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:35:00.008 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "148c9e2b-f634-4315-ad67-9877174aed84", 2024-09-21T03:35:00.008 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:35:00.008 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:35:00.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 -- 172.21.15.112:0/750880692 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e3c0b6680 msgr2=0x7f3e3c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:00.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e3c0b6680 0x7f3e3c0b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c006010 tx=0x7f3e4c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 -- 172.21.15.112:0/750880692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e541052b0 msgr2=0x7f3e5419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:00.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e541052b0 0x7f3e5419aa00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400e9c0 tx=0x7f3e4400ee80 comp rx=0 tx=0).stop 2024-09-21T03:35:00.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 -- 172.21.15.112:0/750880692 shutdown_connections 2024-09-21T03:35:00.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e3c0b6680 0x7f3e3c0b8b90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e5410de30 0x7f3e5419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e541052b0 0x7f3e5419aa00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 --2- 172.21.15.112:0/750880692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e54104880 0x7f3e5419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:00.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 -- 172.21.15.112:0/750880692 >> 172.21.15.112:0/750880692 conn(0x7f3e540797e0 msgr2=0x7f3e540ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:00.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 -- 172.21.15.112:0/750880692 shutdown_connections 2024-09-21T03:35:00.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:00.005+0000 7f3e5ab61640 1 -- 172.21.15.112:0/750880692 wait complete. 2024-09-21T03:35:00.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 610336 10.92 MB/sec execute 180 sec latency 65.193 ms 2024-09-21T03:35:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:00 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:35:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:00 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:00 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:35:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:00 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:35:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:00 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:00 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:35:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:00 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:00 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:35:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:00 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:00 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:00 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:35:00.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:00 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:00 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:35:00.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:00 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:35:00.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:00 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:00 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:35:00.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:00 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:00 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:35:00.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:00 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:00 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:00 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:35:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:00 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:00 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:35:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:00 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:35:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:00 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:00 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:35:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:00 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:00 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:35:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:00 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:00 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:00.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:35:01.082 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:01.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 611363 10.89 MB/sec execute 181 sec latency 32.174 ms 2024-09-21T03:35:01.386 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:01 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:35:01.386 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:01 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:01.386 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:01 smithi112 ceph-mon[31442]: pgmap v498: 97 pgs: 97 active+clean; 407 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 299 op/s 2024-09-21T03:35:01.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:01 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:35:01.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:01 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:01.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:01 smithi203 ceph-mon[34723]: pgmap v498: 97 pgs: 97 active+clean; 407 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 299 op/s 2024-09-21T03:35:01.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:01 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:35:01.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:01 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:01.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:01 smithi187 ceph-mon[34603]: pgmap v498: 97 pgs: 97 active+clean; 407 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 299 op/s 2024-09-21T03:35:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.603+0000 7f3a96fe0640 1 -- 172.21.15.112:0/1970252387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a90105af0 msgr2=0x7f3a9010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.603+0000 7f3a96fe0640 1 --2- 172.21.15.112:0/1970252387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a90105af0 0x7f3a9010f560 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009ca0 tx=0x7f3a8802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.603+0000 7f3a96fe0640 1 -- 172.21.15.112:0/1970252387 shutdown_connections 2024-09-21T03:35:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.603+0000 7f3a96fe0640 1 --2- 172.21.15.112:0/1970252387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a90105af0 0x7f3a9010f560 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.603+0000 7f3a96fe0640 1 --2- 172.21.15.112:0/1970252387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a901051a0 0x7f3a901055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.603+0000 7f3a96fe0640 1 --2- 172.21.15.112:0/1970252387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a90103190 0x7f3a901035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.603+0000 7f3a96fe0640 1 -- 172.21.15.112:0/1970252387 >> 172.21.15.112:0/1970252387 conn(0x7f3a90077cc0 msgr2=0x7f3a90078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.603+0000 7f3a96fe0640 1 -- 172.21.15.112:0/1970252387 shutdown_connections 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.603+0000 7f3a96fe0640 1 -- 172.21.15.112:0/1970252387 wait complete. 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.603+0000 7f3a96fe0640 1 Processor -- start 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.603+0000 7f3a96fe0640 1 -- start start 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a96fe0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a90103190 0x7f3a90199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a96fe0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901051a0 0x7f3a90199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a96fe0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a90105af0 0x7f3a9019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a96fe0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901129f0 con 0x7f3a901051a0 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a96fe0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a90112870 con 0x7f3a90105af0 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a96fe0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a90112b70 con 0x7f3a90103190 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a8ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901051a0 0x7f3a90199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a8ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901051a0 0x7f3a90199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56532/0 (socket says 172.21.15.112:56532) 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a94d7c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a90103190 0x7f3a90199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a8ffff640 1 -- 172.21.15.112:0/2996665354 learned_addr learned my addr 172.21.15.112:0/2996665354 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a9557d640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a90105af0 0x7f3a9019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a8ffff640 1 -- 172.21.15.112:0/2996665354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a90103190 msgr2=0x7f3a90199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a90103190 0x7f3a90199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a8ffff640 1 -- 172.21.15.112:0/2996665354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a90105af0 msgr2=0x7f3a9019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a90105af0 0x7f3a9019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a8ffff640 1 -- 172.21.15.112:0/2996665354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9019f2f0 con 0x7f3a901051a0 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a90103190 0x7f3a90199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a9557d640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a90105af0 0x7f3a9019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901051a0 0x7f3a90199920 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00e9c0 tx=0x7f3a7c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.604+0000 7f3a8dffb640 1 -- 172.21.15.112:0/2996665354 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c00ccc0 con 0x7f3a901051a0 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.605+0000 7f3a8dffb640 1 -- 172.21.15.112:0/2996665354 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a7c0054d0 con 0x7f3a901051a0 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.605+0000 7f3a96fe0640 1 -- 172.21.15.112:0/2996665354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9019f5e0 con 0x7f3a901051a0 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.605+0000 7f3a8dffb640 1 -- 172.21.15.112:0/2996665354 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c010430 con 0x7f3a901051a0 2024-09-21T03:35:01.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.605+0000 7f3a96fe0640 1 -- 172.21.15.112:0/2996665354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a901a5d40 con 0x7f3a901051a0 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a9d554740 1 Processor -- start 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a9d554740 1 -- start start 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a8dffb640 1 -- 172.21.15.112:0/2996665354 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3a7c005020 con 0x7f3a901051a0 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba137070 0x5582ba4de270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba10e1b0 0x5582ba4de7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba116390 0x5582ba4decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a9d554740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5582ba454e50 con 0x5582ba137070 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a9d554740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5582ba444280 con 0x5582ba10e1b0 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a9d554740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5582ba448090 con 0x5582ba116390 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a9557d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba137070 0x5582ba4de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a94d7c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba10e1b0 0x5582ba4de7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.608+0000 7f3a9557d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba137070 0x5582ba4de270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56544/0 (socket says 172.21.15.112:56544) 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a9557d640 1 -- 172.21.15.112:0/3302344363 learned_addr learned my addr 172.21.15.112:0/3302344363 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:01.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/3302344363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba116390 0x5582ba4decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a9557d640 1 -- 172.21.15.112:0/3302344363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba116390 msgr2=0x5582ba4decf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a9557d640 1 --2- 172.21.15.112:0/3302344363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba116390 0x5582ba4decf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a9557d640 1 -- 172.21.15.112:0/3302344363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba10e1b0 msgr2=0x5582ba4de7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a9557d640 1 --2- 172.21.15.112:0/3302344363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba10e1b0 0x5582ba4de7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a9557d640 1 -- 172.21.15.112:0/3302344363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ba1a2650 con 0x5582ba137070 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/3302344363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba10e1b0 0x5582ba4de7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a9557d640 1 --2- 172.21.15.112:0/3302344363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba137070 0x5582ba4de270 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009470 tx=0x7f3a880094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea907c89bdaebdd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a8dffb640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a800b6680 0x7f3a800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a7affd640 1 -- 172.21.15.112:0/3302344363 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88031cf0 con 0x5582ba137070 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a800b6680 0x7f3a800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a7affd640 1 -- 172.21.15.112:0/3302344363 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a88036040 con 0x5582ba137070 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a7affd640 1 -- 172.21.15.112:0/3302344363 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a880042a0 con 0x5582ba137070 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.609+0000 7f3a8dffb640 1 -- 172.21.15.112:0/2996665354 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3a7c0e7af0 con 0x7f3a901051a0 2024-09-21T03:35:01.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.610+0000 7f3a9d554740 1 -- 172.21.15.112:0/3302344363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba137070 msgr2=0x5582ba4de270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.610+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a800b6680 0x7f3a800b8b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009fd0 tx=0x7f3a84005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.610+0000 7f3a9d554740 1 --2- 172.21.15.112:0/3302344363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba137070 0x5582ba4de270 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009470 tx=0x7f3a880094a0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.610+0000 7f3a9d554740 1 -- 172.21.15.112:0/3302344363 shutdown_connections 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.610+0000 7f3a9d554740 1 --2- 172.21.15.112:0/3302344363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba116390 0x5582ba4decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.610+0000 7f3a9d554740 1 --2- 172.21.15.112:0/3302344363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba10e1b0 0x5582ba4de7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.610+0000 7f3a9d554740 1 --2- 172.21.15.112:0/3302344363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba137070 0x5582ba4de270 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.610+0000 7f3a9d554740 1 -- 172.21.15.112:0/3302344363 >> 172.21.15.112:0/3302344363 conn(0x5582ba12b460 msgr2=0x5582ba4dc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.610+0000 7f3a9d554740 1 -- 172.21.15.112:0/3302344363 shutdown_connections 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.610+0000 7f3a9d554740 1 -- 172.21.15.112:0/3302344363 wait complete. 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.611+0000 7f3a9d554740 1 -- start start 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.611+0000 7f3a9d554740 1 Processor -- start 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.611+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.611+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba116390 0x5582ba4ec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.611+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba137070 0x5582ba4ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.611+0000 7f3a9d554740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5582ba2a5700 con 0x5582ba10e1b0 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.611+0000 7f3a9d554740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5582ba3df110 con 0x5582ba116390 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.611+0000 7f3a9d554740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5582ba4eeb50 con 0x5582ba137070 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.611+0000 7f3a9d554740 10 client.0 _ll_register_callbacks cb 0x5582ba3f8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a94d7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a94d7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ebaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56560/0 (socket says 172.21.15.112:56560) 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a94d7c640 1 -- 172.21.15.112:0/770354339 learned_addr learned my addr 172.21.15.112:0/770354339 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba116390 0x5582ba4ec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a9557d640 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba137070 0x5582ba4ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a94d7c640 1 -- 172.21.15.112:0/770354339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba137070 msgr2=0x5582ba4ec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba137070 0x5582ba4ec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a94d7c640 1 -- 172.21.15.112:0/770354339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba116390 msgr2=0x5582ba4ec030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba116390 0x5582ba4ec030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a94d7c640 1 -- 172.21.15.112:0/770354339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ba40cc40 con 0x5582ba10e1b0 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ebaf0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3a84012e50 tx=0x7f3a84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235ac609ab97950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a6ffff640 1 -- 172.21.15.112:0/770354339 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400fe20 con 0x5582ba10e1b0 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a6ffff640 1 -- 172.21.15.112:0/770354339 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3a84011b50 con 0x5582ba10e1b0 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a9d554740 1 -- 172.21.15.112:0/770354339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a88036040 con 0x5582ba10e1b0 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a6ffff640 1 -- 172.21.15.112:0/770354339 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a84011d50 con 0x5582ba10e1b0 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a9d554740 10 client.16587 fetch_fsmap learned FSMap version 29 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.612+0000 7f3a9d554740 1 -- 172.21.15.112:0/770354339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582ba4fa8c0 con 0x5582ba10e1b0 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.613+0000 7f3a6ffff640 1 -- 172.21.15.112:0/770354339 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a88036040 con 0x5582ba10e1b0 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.613+0000 7f3a6ffff640 1 -- 172.21.15.112:0/770354339 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f3a84027070 con 0x5582ba10e1b0 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.613+0000 7f3a9d554740 10 client.16587 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:35:01.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.613+0000 7f3a9d554740 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:35:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.613+0000 7f3a9d554740 20 client.16587 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.613+0000 7f3a9d554740 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5582ba0f59f0 0x5582ba4fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.613+0000 7f3a9d554740 4 client.16587 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.613+0000 7f3a9d554740 1 -- 172.21.15.112:0/770354339 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582ba456da0 con 0x5582ba0f59f0 2024-09-21T03:35:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.613+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5582ba0f59f0 0x5582ba4fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a6ffff640 10 client.16587 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5582ba0f59f0 0x5582ba4fde80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc560487155acde server_cookie=a9561ca78390b9ce in_seq=0 out_seq=0 2024-09-21T03:35:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a6ffff640 1 -- 172.21.15.112:0/770354339 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582ba456da0 con 0x5582ba0f59f0 2024-09-21T03:35:01.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a6ffff640 10 client.16587 handle_command_reply: tid=0 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a9d554740 1 client.16587 shutdown 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a9d554740 1 -- 172.21.15.112:0/770354339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 msgr2=0x5582ba4ebaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a9d554740 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ebaf0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3a84012e50 tx=0x7f3a84004730 comp rx=0 tx=0).stop 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a9d554740 1 -- 172.21.15.112:0/770354339 shutdown_connections 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a9d554740 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5582ba0f59f0 0x5582ba4fde80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a9d554740 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba137070 0x5582ba4ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a9d554740 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba116390 0x5582ba4ec030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a6ffff640 0 client.16587 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a9d554740 1 --2- 172.21.15.112:0/770354339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ebaf0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.614+0000 7f3a9d554740 1 -- 172.21.15.112:0/770354339 >> 172.21.15.112:0/770354339 conn(0x5582ba12b460 msgr2=0x5582ba4da700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.615+0000 7f3a9d554740 1 -- 172.21.15.112:0/770354339 shutdown_connections 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.615+0000 7f3a9d554740 1 -- 172.21.15.112:0/770354339 wait complete. 2024-09-21T03:35:01.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.615+0000 7f3a9d554740 20 client.16587 trim_cache size 0 max 16384 2024-09-21T03:35:01.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a9d554740 1 Processor -- start 2024-09-21T03:35:01.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a9d554740 1 -- start start 2024-09-21T03:35:01.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ed9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba116390 0x5582ba4df9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba137070 0x5582ba4e8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a9d554740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901afd50 con 0x5582ba10e1b0 2024-09-21T03:35:01.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a9d554740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901ad5f0 con 0x5582ba116390 2024-09-21T03:35:01.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a9d554740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901ab440 con 0x5582ba137070 2024-09-21T03:35:01.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a94d7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ed9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a94d7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ed9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56568/0 (socket says 172.21.15.112:56568) 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a94d7c640 1 -- 172.21.15.112:0/2031438667 learned_addr learned my addr 172.21.15.112:0/2031438667 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a9557d640 1 --2- 172.21.15.112:0/2031438667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba137070 0x5582ba4e8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/2031438667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba116390 0x5582ba4df9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a94d7c640 1 -- 172.21.15.112:0/2031438667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba137070 msgr2=0x5582ba4e8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/2031438667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba137070 0x5582ba4e8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a94d7c640 1 -- 172.21.15.112:0/2031438667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba116390 msgr2=0x5582ba4df9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/2031438667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba116390 0x5582ba4df9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.633+0000 7f3a94d7c640 1 -- 172.21.15.112:0/2031438667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ba4e0790 con 0x5582ba10e1b0 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.634+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/2031438667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ed9d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3a840092e0 tx=0x7f3a8400f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157b21c737e131db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.634+0000 7f3a7a7fc640 1 -- 172.21.15.112:0/2031438667 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84035070 con 0x5582ba10e1b0 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.634+0000 7f3a7a7fc640 1 -- 172.21.15.112:0/2031438667 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a840123f0 con 0x5582ba10e1b0 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.634+0000 7f3a7a7fc640 1 -- 172.21.15.112:0/2031438667 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84028760 con 0x5582ba10e1b0 2024-09-21T03:35:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.635+0000 7f3a9d554740 1 -- 172.21.15.112:0/2031438667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 msgr2=0x5582ba4ed9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.635+0000 7f3a9d554740 1 --2- 172.21.15.112:0/2031438667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ed9d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3a840092e0 tx=0x7f3a8400f840 comp rx=0 tx=0).stop 2024-09-21T03:35:01.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.635+0000 7f3a9d554740 1 -- 172.21.15.112:0/2031438667 shutdown_connections 2024-09-21T03:35:01.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.635+0000 7f3a9d554740 1 --2- 172.21.15.112:0/2031438667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba137070 0x5582ba4e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.635+0000 7f3a9d554740 1 --2- 172.21.15.112:0/2031438667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba116390 0x5582ba4df9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.635+0000 7f3a9d554740 1 --2- 172.21.15.112:0/2031438667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba10e1b0 0x5582ba4ed9d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.635+0000 7f3a9d554740 1 -- 172.21.15.112:0/2031438667 >> 172.21.15.112:0/2031438667 conn(0x5582ba12b460 msgr2=0x5582ba4df2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.635+0000 7f3a9d554740 1 -- 172.21.15.112:0/2031438667 shutdown_connections 2024-09-21T03:35:01.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.635+0000 7f3a9d554740 1 -- 172.21.15.112:0/2031438667 wait complete. 2024-09-21T03:35:01.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.635+0000 7f3a9d554740 1 -- start start 2024-09-21T03:35:01.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.635+0000 7f3a9d554740 1 Processor -- start 2024-09-21T03:35:01.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba10e1b0 0x5582ba4f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba116390 0x5582ba4f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a9d554740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba137070 0x5582ba4f2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a9d554740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901abab0 con 0x5582ba116390 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a9d554740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901acf20 con 0x5582ba137070 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a9d554740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901ac180 con 0x5582ba10e1b0 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a9d554740 10 client.0 _ll_register_callbacks cb 0x7f3a84009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a8ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba116390 0x5582ba4f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a8ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba116390 0x5582ba4f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56578/0 (socket says 172.21.15.112:56578) 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a8ffff640 1 -- 172.21.15.112:0/2007449481 learned_addr learned my addr 172.21.15.112:0/2007449481 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba10e1b0 0x5582ba4f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a9557d640 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba137070 0x5582ba4f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a8ffff640 1 -- 172.21.15.112:0/2007449481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba10e1b0 msgr2=0x5582ba4f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba10e1b0 0x5582ba4f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a8ffff640 1 -- 172.21.15.112:0/2007449481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba137070 msgr2=0x5582ba4f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba137070 0x5582ba4f2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a8ffff640 1 -- 172.21.15.112:0/2007449481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ba4f2960 con 0x5582ba116390 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba10e1b0 0x5582ba4f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a8ffff640 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba116390 0x5582ba4f1b80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0e5300 tx=0x7f3a7c0ad8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ede0b3b2bef236d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.636+0000 7f3a6ffff640 1 -- 172.21.15.112:0/2007449481 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0b2070 con 0x5582ba116390 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a6ffff640 1 -- 172.21.15.112:0/2007449481 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8426+0+0 (secure 0 0 0) 0x7f3a7c0e5520 con 0x5582ba116390 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a9d554740 1 -- 172.21.15.112:0/2007449481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a840123f0 con 0x5582ba116390 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a6ffff640 1 -- 172.21.15.112:0/2007449481 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a7c0a9110 con 0x5582ba116390 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a9d554740 10 client.16593 fetch_fsmap learned FSMap version 29 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a9d554740 1 -- 172.21.15.112:0/2007449481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582ba5688f0 con 0x5582ba116390 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a6ffff640 1 -- 172.21.15.112:0/2007449481 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a840123f0 con 0x5582ba116390 2024-09-21T03:35:01.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a6ffff640 1 -- 172.21.15.112:0/2007449481 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f3a7c0a9c10 con 0x5582ba116390 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a9d554740 10 client.16593 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a9d554740 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a9d554740 20 client.16593 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a9d554740 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5582ba0cada0 0x5582ba56b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a9d554740 4 client.16593 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a9d554740 1 -- 172.21.15.112:0/2007449481 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582ba452850 con 0x5582ba0cada0 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.637+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5582ba0cada0 0x5582ba56b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.638+0000 7f3a6ffff640 10 client.16593 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.638+0000 7f3a94d7c640 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5582ba0cada0 0x5582ba56b020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6452ee4f70174f0 server_cookie=f3cd9aa1af932366 in_seq=0 out_seq=0 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.638+0000 7f3a6ffff640 1 -- 172.21.15.112:0/2007449481 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x5582ba452850 con 0x5582ba0cada0 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.638+0000 7f3a6ffff640 10 client.16593 handle_command_reply: tid=0 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.638+0000 7f3a9d554740 1 client.16593 shutdown 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.638+0000 7f3a9d554740 1 -- 172.21.15.112:0/2007449481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba116390 msgr2=0x5582ba4f1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.638+0000 7f3a9d554740 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba116390 0x5582ba4f1b80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0e5300 tx=0x7f3a7c0ad8a0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.638+0000 7f3a9d554740 1 -- 172.21.15.112:0/2007449481 shutdown_connections 2024-09-21T03:35:01.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.638+0000 7f3a9d554740 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5582ba0cada0 0x5582ba56b020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.638+0000 7f3a9d554740 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582ba137070 0x5582ba4f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.638+0000 7f3a9d554740 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582ba116390 0x5582ba4f1b80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.639+0000 7f3a9d554740 1 --2- 172.21.15.112:0/2007449481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582ba10e1b0 0x5582ba4f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.639+0000 7f3a9d554740 1 -- 172.21.15.112:0/2007449481 >> 172.21.15.112:0/2007449481 conn(0x5582ba12b460 msgr2=0x5582ba4ed6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.639+0000 7f3a6ffff640 0 client.16593 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:01.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.639+0000 7f3a9d554740 1 -- 172.21.15.112:0/2007449481 shutdown_connections 2024-09-21T03:35:01.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.639+0000 7f3a9d554740 1 -- 172.21.15.112:0/2007449481 wait complete. 2024-09-21T03:35:01.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.639+0000 7f3a9d554740 20 client.16593 trim_cache size 0 max 16384 2024-09-21T03:35:01.646 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "148c9e2b-f634-4315-ad67-9877174aed84", 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.647 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:35:01.648 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:35:01.649 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:01.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:01.651 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:01.652 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:35:01.653 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:35:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.639+0000 7f3a96fe0640 1 -- 172.21.15.112:0/2996665354 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a800b6680 msgr2=0x7f3a800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.639+0000 7f3a96fe0640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a800b6680 0x7f3a800b8b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009fd0 tx=0x7f3a84005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.639+0000 7f3a96fe0640 1 -- 172.21.15.112:0/2996665354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901051a0 msgr2=0x7f3a90199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:01.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.640+0000 7f3a96fe0640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901051a0 0x7f3a90199920 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00e9c0 tx=0x7f3a7c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:35:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.640+0000 7f3a96fe0640 1 -- 172.21.15.112:0/2996665354 shutdown_connections 2024-09-21T03:35:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.640+0000 7f3a96fe0640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a800b6680 0x7f3a800b8b90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.640+0000 7f3a96fe0640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a90105af0 0x7f3a9019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.640+0000 7f3a96fe0640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a901051a0 0x7f3a90199920 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.640+0000 7f3a96fe0640 1 --2- 172.21.15.112:0/2996665354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a90103190 0x7f3a90199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.640+0000 7f3a96fe0640 1 -- 172.21.15.112:0/2996665354 >> 172.21.15.112:0/2996665354 conn(0x7f3a90077cc0 msgr2=0x7f3a90106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.640+0000 7f3a96fe0640 1 -- 172.21.15.112:0/2996665354 shutdown_connections 2024-09-21T03:35:01.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:01.640+0000 7f3a96fe0640 1 -- 172.21.15.112:0/2996665354 wait complete. 2024-09-21T03:35:02.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 613452 10.88 MB/sec execute 182 sec latency 49.012 ms 2024-09-21T03:35:02.588 INFO:tasks.cephfs.filesystem:scrub status for tag:148c9e2b-f634-4315-ad67-9877174aed84 - {'path': '/', 'tag': '148c9e2b-f634-4315-ad67-9877174aed84', 'options': 'recursive,force'} 2024-09-21T03:35:02.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:02 smithi112 ceph-mon[31442]: pgmap v499: 97 pgs: 97 active+clean; 410 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 269 op/s 2024-09-21T03:35:02.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:02 smithi203 ceph-mon[34723]: pgmap v499: 97 pgs: 97 active+clean; 410 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 269 op/s 2024-09-21T03:35:02.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:02 smithi187 ceph-mon[34603]: pgmap v499: 97 pgs: 97 active+clean; 410 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 269 op/s 2024-09-21T03:35:03.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 615056 10.88 MB/sec execute 183 sec latency 25.731 ms 2024-09-21T03:35:04.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 617609 10.89 MB/sec execute 184 sec latency 25.020 ms 2024-09-21T03:35:05.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 619204 10.87 MB/sec execute 185 sec latency 21.276 ms 2024-09-21T03:35:05.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:05 smithi203 ceph-mon[34723]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:35:05.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:05 smithi203 ceph-mon[34723]: pgmap v501: 97 pgs: 97 active+clean; 411 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 247 op/s 2024-09-21T03:35:05.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:05 smithi187 ceph-mon[34603]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:35:05.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:05 smithi187 ceph-mon[34603]: pgmap v501: 97 pgs: 97 active+clean; 411 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 247 op/s 2024-09-21T03:35:05.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:05 smithi112 ceph-mon[31442]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T03:35:05.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:05 smithi112 ceph-mon[31442]: pgmap v501: 97 pgs: 97 active+clean; 411 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 247 op/s 2024-09-21T03:35:06.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 621768 10.91 MB/sec execute 186 sec latency 17.948 ms 2024-09-21T03:35:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:06 smithi112 ceph-mon[31442]: pgmap v502: 97 pgs: 97 active+clean; 417 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 315 op/s 2024-09-21T03:35:06.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:06 smithi203 ceph-mon[34723]: pgmap v502: 97 pgs: 97 active+clean; 417 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 315 op/s 2024-09-21T03:35:06.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:06 smithi187 ceph-mon[34603]: pgmap v502: 97 pgs: 97 active+clean; 417 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 315 op/s 2024-09-21T03:35:07.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 623195 10.88 MB/sec execute 187 sec latency 55.787 ms 2024-09-21T03:35:08.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 625393 10.89 MB/sec execute 188 sec latency 34.449 ms 2024-09-21T03:35:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:08 smithi112 ceph-mon[31442]: pgmap v503: 97 pgs: 97 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 344 op/s 2024-09-21T03:35:08.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:08 smithi203 ceph-mon[34723]: pgmap v503: 97 pgs: 97 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 344 op/s 2024-09-21T03:35:08.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:08 smithi187 ceph-mon[34603]: pgmap v503: 97 pgs: 97 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 344 op/s 2024-09-21T03:35:09.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 627168 10.87 MB/sec execute 189 sec latency 17.852 ms 2024-09-21T03:35:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:09 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:35:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:09 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:35:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:09 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:35:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:09 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:35:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:09 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:35:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:09 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:35:09.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:09 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:35:09.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:09 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:35:09.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:09 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:35:10.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 629095 10.88 MB/sec execute 190 sec latency 13.973 ms 2024-09-21T03:35:10.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:10 smithi112 ceph-mon[31442]: pgmap v504: 97 pgs: 97 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 293 op/s 2024-09-21T03:35:10.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:10 smithi203 ceph-mon[34723]: pgmap v504: 97 pgs: 97 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 293 op/s 2024-09-21T03:35:10.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:10 smithi187 ceph-mon[34603]: pgmap v504: 97 pgs: 97 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 293 op/s 2024-09-21T03:35:11.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 631175 10.86 MB/sec execute 191 sec latency 26.201 ms 2024-09-21T03:35:12.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 632833 10.87 MB/sec execute 192 sec latency 14.933 ms 2024-09-21T03:35:12.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:12 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:12.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:12 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:12.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:12 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:35:13.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 634762 10.85 MB/sec execute 193 sec latency 47.482 ms 2024-09-21T03:35:13.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:13 smithi112 ceph-mon[31442]: pgmap v505: 97 pgs: 97 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 308 op/s 2024-09-21T03:35:13.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:13.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:13.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:35:13.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:13 smithi203 ceph-mon[34723]: pgmap v505: 97 pgs: 97 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 308 op/s 2024-09-21T03:35:13.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:13.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:13.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:35:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:13 smithi187 ceph-mon[34603]: pgmap v505: 97 pgs: 97 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 308 op/s 2024-09-21T03:35:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:35:13.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:35:13.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:35:14.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 636433 10.85 MB/sec execute 194 sec latency 18.349 ms 2024-09-21T03:35:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:14 smithi112 ceph-mon[31442]: pgmap v506: 97 pgs: 97 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 301 op/s 2024-09-21T03:35:14.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:14 smithi203 ceph-mon[34723]: pgmap v506: 97 pgs: 97 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 301 op/s 2024-09-21T03:35:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:14 smithi187 ceph-mon[34603]: pgmap v506: 97 pgs: 97 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 301 op/s 2024-09-21T03:35:15.239 INFO:tasks.workunit.client.0.smithi112.stdout: 1 638981 10.87 MB/sec execute 195 sec latency 17.181 ms 2024-09-21T03:35:16.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:15 smithi112 ceph-mon[31442]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:35:16.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:15 smithi203 ceph-mon[34723]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:35:16.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:15 smithi187 ceph-mon[34603]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T03:35:16.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 640516 10.85 MB/sec execute 196 sec latency 17.875 ms 2024-09-21T03:35:17.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:16 smithi112 ceph-mon[31442]: pgmap v508: 97 pgs: 97 active+clean; 435 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 276 op/s 2024-09-21T03:35:17.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:16 smithi203 ceph-mon[34723]: pgmap v508: 97 pgs: 97 active+clean; 435 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 276 op/s 2024-09-21T03:35:17.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:16 smithi187 ceph-mon[34603]: pgmap v508: 97 pgs: 97 active+clean; 435 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 276 op/s 2024-09-21T03:35:17.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 643047 10.88 MB/sec execute 197 sec latency 17.743 ms 2024-09-21T03:35:18.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 644394 10.85 MB/sec execute 198 sec latency 35.187 ms 2024-09-21T03:35:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:18 smithi112 ceph-mon[31442]: pgmap v509: 97 pgs: 97 active+clean; 437 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-09-21T03:35:18.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:18 smithi203 ceph-mon[34723]: pgmap v509: 97 pgs: 97 active+clean; 437 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-09-21T03:35:18.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:18 smithi187 ceph-mon[34603]: pgmap v509: 97 pgs: 97 active+clean; 437 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-09-21T03:35:19.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 646725 10.87 MB/sec execute 199 sec latency 14.109 ms 2024-09-21T03:35:20.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 648719 10.85 MB/sec execute 200 sec latency 16.971 ms 2024-09-21T03:35:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:20 smithi112 ceph-mon[31442]: pgmap v510: 97 pgs: 97 active+clean; 446 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-09-21T03:35:20.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:20 smithi203 ceph-mon[34723]: pgmap v510: 97 pgs: 97 active+clean; 446 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-09-21T03:35:20.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:20 smithi187 ceph-mon[34603]: pgmap v510: 97 pgs: 97 active+clean; 446 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-09-21T03:35:21.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 650484 10.85 MB/sec execute 201 sec latency 12.497 ms 2024-09-21T03:35:22.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 652769 10.86 MB/sec execute 202 sec latency 22.252 ms 2024-09-21T03:35:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:22 smithi112 ceph-mon[31442]: pgmap v511: 97 pgs: 97 active+clean; 446 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 280 op/s 2024-09-21T03:35:22.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:22 smithi203 ceph-mon[34723]: pgmap v511: 97 pgs: 97 active+clean; 446 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 280 op/s 2024-09-21T03:35:22.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:22 smithi187 ceph-mon[34603]: pgmap v511: 97 pgs: 97 active+clean; 446 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 280 op/s 2024-09-21T03:35:23.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 654306 10.85 MB/sec execute 203 sec latency 39.911 ms 2024-09-21T03:35:24.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 657115 10.88 MB/sec execute 204 sec latency 23.662 ms 2024-09-21T03:35:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:24 smithi112 ceph-mon[31442]: pgmap v512: 97 pgs: 97 active+clean; 446 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2024-09-21T03:35:24.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:24 smithi203 ceph-mon[34723]: pgmap v512: 97 pgs: 97 active+clean; 446 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2024-09-21T03:35:24.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:24 smithi187 ceph-mon[34603]: pgmap v512: 97 pgs: 97 active+clean; 446 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2024-09-21T03:35:25.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 658605 10.86 MB/sec execute 205 sec latency 16.596 ms 2024-09-21T03:35:26.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 660950 10.87 MB/sec execute 206 sec latency 12.144 ms 2024-09-21T03:35:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:26 smithi112 ceph-mon[31442]: pgmap v513: 97 pgs: 97 active+clean; 455 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 324 op/s 2024-09-21T03:35:26.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:26 smithi203 ceph-mon[34723]: pgmap v513: 97 pgs: 97 active+clean; 455 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 324 op/s 2024-09-21T03:35:26.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:26 smithi187 ceph-mon[34603]: pgmap v513: 97 pgs: 97 active+clean; 455 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 324 op/s 2024-09-21T03:35:27.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 663126 10.86 MB/sec execute 207 sec latency 19.324 ms 2024-09-21T03:35:28.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 664828 10.86 MB/sec execute 208 sec latency 45.420 ms 2024-09-21T03:35:28.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:28 smithi112 ceph-mon[31442]: pgmap v514: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-21T03:35:28.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:28 smithi203 ceph-mon[34723]: pgmap v514: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-21T03:35:28.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:28 smithi187 ceph-mon[34603]: pgmap v514: 97 pgs: 97 active+clean; 455 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-21T03:35:29.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 667562 10.89 MB/sec execute 209 sec latency 21.121 ms 2024-09-21T03:35:30.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 669007 10.87 MB/sec execute 210 sec latency 17.287 ms 2024-09-21T03:35:30.789 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:30 smithi112 ceph-mon[31442]: pgmap v515: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 324 op/s 2024-09-21T03:35:30.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:30 smithi203 ceph-mon[34723]: pgmap v515: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 324 op/s 2024-09-21T03:35:30.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:30 smithi187 ceph-mon[34603]: pgmap v515: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 324 op/s 2024-09-21T03:35:31.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 671387 10.88 MB/sec execute 211 sec latency 61.996 ms 2024-09-21T03:35:32.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 672832 10.86 MB/sec execute 212 sec latency 17.889 ms 2024-09-21T03:35:32.589 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:35:32.868 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:33.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 675051 10.87 MB/sec execute 213 sec latency 36.073 ms 2024-09-21T03:35:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.425+0000 7f148ffd6640 1 -- 172.21.15.112:0/1189154490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488108d10 msgr2=0x7f148810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.425+0000 7f148ffd6640 1 --2- 172.21.15.112:0/1189154490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488108d10 0x7f148810f540 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1480009ca0 tx=0x7f148002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.425+0000 7f148ffd6640 1 -- 172.21.15.112:0/1189154490 shutdown_connections 2024-09-21T03:35:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.425+0000 7f148ffd6640 1 --2- 172.21.15.112:0/1189154490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488108d10 0x7f148810f540 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.425+0000 7f148ffd6640 1 --2- 172.21.15.112:0/1189154490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14881083c0 0x7f14881087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.425+0000 7f148ffd6640 1 --2- 172.21.15.112:0/1189154490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488102a80 0x7f1488102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.425+0000 7f148ffd6640 1 -- 172.21.15.112:0/1189154490 >> 172.21.15.112:0/1189154490 conn(0x7f1488079740 msgr2=0x7f1488079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.426+0000 7f148ffd6640 1 -- 172.21.15.112:0/1189154490 shutdown_connections 2024-09-21T03:35:33.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.426+0000 7f148ffd6640 1 -- 172.21.15.112:0/1189154490 wait complete. 2024-09-21T03:35:33.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.426+0000 7f148ffd6640 1 Processor -- start 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.426+0000 7f148ffd6640 1 -- start start 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148ffd6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488102a80 0x7f148819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148ffd6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14881083c0 0x7f148819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148ffd6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488108d10 0x7f148819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148ffd6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1488112980 con 0x7f1488108d10 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148ffd6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1488112800 con 0x7f1488102a80 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148ffd6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1488112b00 con 0x7f14881083c0 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148dd72640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488102a80 0x7f148819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148d571640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14881083c0 0x7f148819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148e573640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488108d10 0x7f148819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148d571640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14881083c0 0x7f148819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52956/0 (socket says 172.21.15.112:52956) 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148dd72640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488102a80 0x7f148819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33386/0 (socket says 172.21.15.112:33386) 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148d571640 1 -- 172.21.15.112:0/48293655 learned_addr learned my addr 172.21.15.112:0/48293655 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:33.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148e573640 1 -- 172.21.15.112:0/48293655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14881083c0 msgr2=0x7f148819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148e573640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14881083c0 0x7f148819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148e573640 1 -- 172.21.15.112:0/48293655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488102a80 msgr2=0x7f148819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148e573640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488102a80 0x7f148819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148e573640 1 -- 172.21.15.112:0/48293655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14881a0390 con 0x7f1488108d10 2024-09-21T03:35:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.427+0000 7f148e573640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488108d10 0x7f148819fc60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f148002ade0 tx=0x7f14800316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.428+0000 7f147effd640 1 -- 172.21.15.112:0/48293655 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480003b70 con 0x7f1488108d10 2024-09-21T03:35:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.428+0000 7f147effd640 1 -- 172.21.15.112:0/48293655 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1480003d10 con 0x7f1488108d10 2024-09-21T03:35:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.428+0000 7f148ffd6640 1 -- 172.21.15.112:0/48293655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14881a0620 con 0x7f1488108d10 2024-09-21T03:35:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.428+0000 7f147effd640 1 -- 172.21.15.112:0/48293655 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148002f9e0 con 0x7f1488108d10 2024-09-21T03:35:33.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.428+0000 7f148ffd6640 1 -- 172.21.15.112:0/48293655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881a6de0 con 0x7f1488108d10 2024-09-21T03:35:33.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.431+0000 7f149654a740 1 Processor -- start 2024-09-21T03:35:33.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.431+0000 7f147effd640 1 -- 172.21.15.112:0/48293655 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f148002fb80 con 0x7f1488108d10 2024-09-21T03:35:33.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f149654a740 1 -- start start 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f149654a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139d5070 0x562b13d7c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f149654a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 0x562b13d7c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f149654a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 0x562b13d7ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f149654a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562b13cf2e50 con 0x562b139ac1b0 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f149654a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562b13ce2280 con 0x562b139d5070 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f149654a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b13ce6090 con 0x562b139b4390 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f148dd72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 0x562b13d7c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f148dd72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 0x562b13d7c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46608/0 (socket says 172.21.15.112:46608) 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f148dd72640 1 -- 172.21.15.112:0/2540571725 learned_addr learned my addr 172.21.15.112:0/2540571725 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f148d571640 1 --2- 172.21.15.112:0/2540571725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 0x562b13d7ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f148d571640 1 --2- 172.21.15.112:0/2540571725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139d5070 0x562b13d7c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f147effd640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14740b6680 0x7f14740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f148dd72640 1 -- 172.21.15.112:0/2540571725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 msgr2=0x562b13d7ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:33.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f148dd72640 1 --2- 172.21.15.112:0/2540571725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 0x562b13d7ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f148dd72640 1 -- 172.21.15.112:0/2540571725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139d5070 msgr2=0x562b13d7c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:33.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f148dd72640 1 --2- 172.21.15.112:0/2540571725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139d5070 0x562b13d7c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.432+0000 7f148dd72640 1 -- 172.21.15.112:0/2540571725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b13a40650 con 0x562b139ac1b0 2024-09-21T03:35:33.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.433+0000 7f147effd640 1 -- 172.21.15.112:0/48293655 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f1480108810 con 0x7f1488108d10 2024-09-21T03:35:33.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.433+0000 7f148e573640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14740b6680 0x7f14740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.433+0000 7f148dd72640 1 --2- 172.21.15.112:0/2540571725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 0x562b13d7c7b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f148400bd10 tx=0x7f14840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be30bb4069eaf26c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:33.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.433+0000 7f1473fff640 1 -- 172.21.15.112:0/2540571725 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484004030 con 0x562b139ac1b0 2024-09-21T03:35:33.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.433+0000 7f1473fff640 1 -- 172.21.15.112:0/2540571725 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14840035f0 con 0x562b139ac1b0 2024-09-21T03:35:33.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.433+0000 7f1473fff640 1 -- 172.21.15.112:0/2540571725 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484011640 con 0x562b139ac1b0 2024-09-21T03:35:33.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f149654a740 1 -- 172.21.15.112:0/2540571725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 msgr2=0x562b13d7c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f149654a740 1 --2- 172.21.15.112:0/2540571725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 0x562b13d7c7b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f148400bd10 tx=0x7f14840107a0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f148e573640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14740b6680 0x7f14740b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f14800cac60 tx=0x7f1480005c30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:33.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f149654a740 1 -- 172.21.15.112:0/2540571725 shutdown_connections 2024-09-21T03:35:33.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f149654a740 1 --2- 172.21.15.112:0/2540571725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 0x562b13d7ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f149654a740 1 --2- 172.21.15.112:0/2540571725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 0x562b13d7c7b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f149654a740 1 --2- 172.21.15.112:0/2540571725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139d5070 0x562b13d7c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f149654a740 1 -- 172.21.15.112:0/2540571725 >> 172.21.15.112:0/2540571725 conn(0x562b139c9460 msgr2=0x562b13d7a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:33.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f149654a740 1 -- 172.21.15.112:0/2540571725 shutdown_connections 2024-09-21T03:35:33.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f149654a740 1 -- 172.21.15.112:0/2540571725 wait complete. 2024-09-21T03:35:33.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f149654a740 1 -- start start 2024-09-21T03:35:33.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.434+0000 7f149654a740 1 Processor -- start 2024-09-21T03:35:33.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.435+0000 7f149654a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 0x562b13d878f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.435+0000 7f149654a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 0x562b13d87e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.435+0000 7f149654a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139d5070 0x562b13d88390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.435+0000 7f149654a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562b13b43700 con 0x562b139ac1b0 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.435+0000 7f149654a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562b13c7d110 con 0x562b139d5070 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.435+0000 7f149654a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b13d8aa50 con 0x562b139b4390 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.435+0000 7f149654a740 10 client.0 _ll_register_callbacks cb 0x562b13c96990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.435+0000 7f148d571640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 0x562b13d878f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.435+0000 7f148dd72640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 0x562b13d87e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.435+0000 7f148dd72640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 0x562b13d87e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52962/0 (socket says 172.21.15.112:52962) 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.435+0000 7f148dd72640 1 -- 172.21.15.112:0/1969195208 learned_addr learned my addr 172.21.15.112:0/1969195208 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f148dd72640 1 -- 172.21.15.112:0/1969195208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139d5070 msgr2=0x562b13d88390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f148dd72640 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139d5070 0x562b13d88390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f148dd72640 1 -- 172.21.15.112:0/1969195208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 msgr2=0x562b13d878f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f148dd72640 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 0x562b13d878f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f148dd72640 1 -- 172.21.15.112:0/1969195208 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b13caac40 con 0x562b139b4390 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f148d571640 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 0x562b13d878f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f148dd72640 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 0x562b13d87e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1484000c00 tx=0x7f148400ba60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0ca7ad317642c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f14717fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f14717fa640 1 -- 172.21.15.112:0/1969195208 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484029060 con 0x562b139b4390 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f14717fa640 1 -- 172.21.15.112:0/1969195208 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f1484004330 con 0x562b139b4390 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f149654a740 1 -- 172.21.15.112:0/1969195208 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14840035f0 con 0x562b139b4390 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f14717fa640 1 -- 172.21.15.112:0/1969195208 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f14840037a0 con 0x562b139b4390 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f149654a740 10 client.26173 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.436+0000 7f149654a740 1 -- 172.21.15.112:0/1969195208 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b13d96700 con 0x562b139b4390 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.437+0000 7f14717fa640 1 -- 172.21.15.112:0/1969195208 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14840035f0 con 0x562b139b4390 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.437+0000 7f14717fa640 1 -- 172.21.15.112:0/1969195208 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f14840046c0 con 0x562b139b4390 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.437+0000 7f149654a740 10 client.26173 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.437+0000 7f149654a740 10 client.26173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:35:33.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.437+0000 7f149654a740 20 client.26173 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.437+0000 7f149654a740 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562b139939f0 0x562b13d9a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.437+0000 7f149654a740 4 client.26173 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.437+0000 7f149654a740 1 -- 172.21.15.112:0/1969195208 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b13cf4da0 con 0x562b139939f0 2024-09-21T03:35:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.437+0000 7f148d571640 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562b139939f0 0x562b13d9a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.438+0000 7f14717fa640 10 client.26173 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:33.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.438+0000 7f148d571640 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562b139939f0 0x562b13d9a000 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504b4e34becad1a2 server_cookie=f2c958b2b69b07eb in_seq=0 out_seq=0 2024-09-21T03:35:33.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f14717fa640 1 -- 172.21.15.112:0/1969195208 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b13cf4da0 con 0x562b139939f0 2024-09-21T03:35:33.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f14717fa640 10 client.26173 handle_command_reply: tid=0 2024-09-21T03:35:33.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 1 client.26173 shutdown 2024-09-21T03:35:33.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 1 -- 172.21.15.112:0/1969195208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 msgr2=0x562b13d87e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 0x562b13d87e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1484000c00 tx=0x7f148400ba60 comp rx=0 tx=0).stop 2024-09-21T03:35:33.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 1 -- 172.21.15.112:0/1969195208 shutdown_connections 2024-09-21T03:35:33.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562b139939f0 0x562b13d9a000 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139d5070 0x562b13d88390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139b4390 0x562b13d87e30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 1 --2- 172.21.15.112:0/1969195208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139ac1b0 0x562b13d878f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f14717fa640 0 client.26173 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:33.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 1 -- 172.21.15.112:0/1969195208 >> 172.21.15.112:0/1969195208 conn(0x562b139c9460 msgr2=0x562b13d7f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:33.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 1 -- 172.21.15.112:0/1969195208 shutdown_connections 2024-09-21T03:35:33.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 1 -- 172.21.15.112:0/1969195208 wait complete. 2024-09-21T03:35:33.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.439+0000 7f149654a740 20 client.26173 trim_cache size 0 max 16384 2024-09-21T03:35:33.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.459+0000 7f149654a740 1 Processor -- start 2024-09-21T03:35:33.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.459+0000 7f149654a740 1 -- start start 2024-09-21T03:35:33.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.459+0000 7f149654a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139ac1b0 0x562b13d84ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.459+0000 7f149654a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d85020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.459+0000 7f149654a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139d5070 0x562b13d7d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.459+0000 7f149654a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14881b1f00 con 0x562b139b4390 2024-09-21T03:35:33.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.459+0000 7f149654a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14881af7d0 con 0x562b139ac1b0 2024-09-21T03:35:33.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.459+0000 7f149654a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14881ad640 con 0x562b139d5070 2024-09-21T03:35:33.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f148dd72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d85020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f148dd72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d85020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46626/0 (socket says 172.21.15.112:46626) 2024-09-21T03:35:33.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f148dd72640 1 -- 172.21.15.112:0/583418994 learned_addr learned my addr 172.21.15.112:0/583418994 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:33.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f148dd72640 1 -- 172.21.15.112:0/583418994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139d5070 msgr2=0x562b13d7d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:33.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f148dd72640 1 --2- 172.21.15.112:0/583418994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139d5070 0x562b13d7d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f148dd72640 1 -- 172.21.15.112:0/583418994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139ac1b0 msgr2=0x562b13d84ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:33.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f148dd72640 1 --2- 172.21.15.112:0/583418994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139ac1b0 0x562b13d84ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f148dd72640 1 -- 172.21.15.112:0/583418994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b13d93c70 con 0x562b139b4390 2024-09-21T03:35:33.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f148dd72640 1 --2- 172.21.15.112:0/583418994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d85020 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1484004ac0 tx=0x7f14840058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd895b2c936a388 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:33.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f14737fe640 1 -- 172.21.15.112:0/583418994 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484029070 con 0x562b139b4390 2024-09-21T03:35:33.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f14737fe640 1 -- 172.21.15.112:0/583418994 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f148400bb60 con 0x562b139b4390 2024-09-21T03:35:33.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.460+0000 7f14737fe640 1 -- 172.21.15.112:0/583418994 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484003940 con 0x562b139b4390 2024-09-21T03:35:33.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.462+0000 7f149654a740 1 -- 172.21.15.112:0/583418994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 msgr2=0x562b13d85020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.462+0000 7f149654a740 1 --2- 172.21.15.112:0/583418994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d85020 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1484004ac0 tx=0x7f14840058d0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.462+0000 7f149654a740 1 -- 172.21.15.112:0/583418994 shutdown_connections 2024-09-21T03:35:33.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.462+0000 7f149654a740 1 --2- 172.21.15.112:0/583418994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139d5070 0x562b13d7d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.462+0000 7f149654a740 1 --2- 172.21.15.112:0/583418994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d85020 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.462+0000 7f149654a740 1 --2- 172.21.15.112:0/583418994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139ac1b0 0x562b13d84ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.462+0000 7f149654a740 1 -- 172.21.15.112:0/583418994 >> 172.21.15.112:0/583418994 conn(0x562b139c9460 msgr2=0x562b13d86170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:33.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.462+0000 7f149654a740 1 -- 172.21.15.112:0/583418994 shutdown_connections 2024-09-21T03:35:33.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.462+0000 7f149654a740 1 -- 172.21.15.112:0/583418994 wait complete. 2024-09-21T03:35:33.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.462+0000 7f149654a740 1 -- start start 2024-09-21T03:35:33.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.462+0000 7f149654a740 1 Processor -- start 2024-09-21T03:35:33.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f149654a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139ac1b0 0x562b13d90700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f149654a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d8ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f149654a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139d5070 0x562b13d8b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f149654a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14881adc90 con 0x562b139b4390 2024-09-21T03:35:33.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f149654a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14881af100 con 0x562b139ac1b0 2024-09-21T03:35:33.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f149654a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14881ae360 con 0x562b139d5070 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f149654a740 10 client.0 _ll_register_callbacks cb 0x7f1478009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f148dd72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d8ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f148dd72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d8ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46634/0 (socket says 172.21.15.112:46634) 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f148dd72640 1 -- 172.21.15.112:0/387549591 learned_addr learned my addr 172.21.15.112:0/387549591 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f148d571640 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139d5070 0x562b13d8b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.463+0000 7f148d571640 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139ac1b0 0x562b13d90700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f148dd72640 1 -- 172.21.15.112:0/387549591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139d5070 msgr2=0x562b13d8b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f148dd72640 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139d5070 0x562b13d8b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f148dd72640 1 -- 172.21.15.112:0/387549591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139ac1b0 msgr2=0x562b13d90700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f148dd72640 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139ac1b0 0x562b13d90700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f148dd72640 1 -- 172.21.15.112:0/387549591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b13e029f0 con 0x562b139b4390 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f148d571640 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139d5070 0x562b13d8b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:33.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f148dd72640 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d8ad10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f148400b7f0 tx=0x7f1484003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e308a48f0e069e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f14717fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f14717fa640 1 -- 172.21.15.112:0/387549591 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484029070 con 0x562b139b4390 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f14717fa640 1 -- 172.21.15.112:0/387549591 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f14840053f0 con 0x562b139b4390 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f149654a740 1 -- 172.21.15.112:0/387549591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f148400bb60 con 0x562b139b4390 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f14717fa640 1 -- 172.21.15.112:0/387549591 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1484044050 con 0x562b139b4390 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f149654a740 10 client.16611 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.464+0000 7f149654a740 1 -- 172.21.15.112:0/387549591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b13e03b90 con 0x562b139b4390 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.465+0000 7f14717fa640 1 -- 172.21.15.112:0/387549591 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f148400bb60 con 0x562b139b4390 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.465+0000 7f14717fa640 1 -- 172.21.15.112:0/387549591 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f148402e6e0 con 0x562b139b4390 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.465+0000 7f149654a740 10 client.16611 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.465+0000 7f149654a740 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:35:33.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.465+0000 7f149654a740 20 client.16611 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:33.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.465+0000 7f149654a740 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562b13968da0 0x562b13e061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:33.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.465+0000 7f149654a740 4 client.16611 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:35:33.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.465+0000 7f149654a740 1 -- 172.21.15.112:0/387549591 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b13cf0850 con 0x562b13968da0 2024-09-21T03:35:33.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.465+0000 7f148d571640 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562b13968da0 0x562b13e061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:33.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.465+0000 7f14717fa640 10 client.16611 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:33.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f148d571640 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562b13968da0 0x562b13e061e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc7d3da892c522fb server_cookie=a31fad9e5d02be2f in_seq=0 out_seq=0 2024-09-21T03:35:33.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f14717fa640 1 -- 172.21.15.112:0/387549591 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x562b13cf0850 con 0x562b13968da0 2024-09-21T03:35:33.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f14717fa640 10 client.16611 handle_command_reply: tid=0 2024-09-21T03:35:33.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 1 client.16611 shutdown 2024-09-21T03:35:33.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 1 -- 172.21.15.112:0/387549591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 msgr2=0x562b13d8ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d8ad10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f148400b7f0 tx=0x7f1484003730 comp rx=0 tx=0).stop 2024-09-21T03:35:33.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 1 -- 172.21.15.112:0/387549591 shutdown_connections 2024-09-21T03:35:33.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562b13968da0 0x562b13e061e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b139d5070 0x562b13d8b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f14717fa640 0 client.16611 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:33.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b139b4390 0x562b13d8ad10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 1 --2- 172.21.15.112:0/387549591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b139ac1b0 0x562b13d90700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 1 -- 172.21.15.112:0/387549591 >> 172.21.15.112:0/387549591 conn(0x562b139c9460 msgr2=0x562b13d7d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:33.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 1 -- 172.21.15.112:0/387549591 shutdown_connections 2024-09-21T03:35:33.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 1 -- 172.21.15.112:0/387549591 wait complete. 2024-09-21T03:35:33.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.466+0000 7f149654a740 20 client.16611 trim_cache size 0 max 16384 2024-09-21T03:35:33.475 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:35:33.475 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:35:33.475 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:35:33.475 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:35:33.475 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:35:33.475 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:35:33.475 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:33.475 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:33.475 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:33.476 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:33.477 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:33.478 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:33.478 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:33.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:33.478 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:35:33.478 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:35:33.478 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:33.478 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:33.478 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:33.478 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:35:33.479 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:33.480 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:35:33.481 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:35:33.482 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:35:33.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 -- 172.21.15.112:0/48293655 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14740b6680 msgr2=0x7f14740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14740b6680 0x7f14740b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f14800cac60 tx=0x7f1480005c30 comp rx=0 tx=0).stop 2024-09-21T03:35:33.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 -- 172.21.15.112:0/48293655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488108d10 msgr2=0x7f148819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:33.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488108d10 0x7f148819fc60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f148002ade0 tx=0x7f14800316f0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 -- 172.21.15.112:0/48293655 shutdown_connections 2024-09-21T03:35:33.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f14740b6680 0x7f14740b8b90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488108d10 0x7f148819fc60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14881083c0 0x7f148819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 --2- 172.21.15.112:0/48293655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488102a80 0x7f148819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:33.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 -- 172.21.15.112:0/48293655 >> 172.21.15.112:0/48293655 conn(0x7f1488079740 msgr2=0x7f148806bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:33.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 -- 172.21.15.112:0/48293655 shutdown_connections 2024-09-21T03:35:33.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:33.467+0000 7f148ffd6640 1 -- 172.21.15.112:0/48293655 wait complete. 2024-09-21T03:35:33.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:33 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:35:33.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:33 smithi203 ceph-mon[34723]: pgmap v516: 97 pgs: 97 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 252 op/s 2024-09-21T03:35:33.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:33 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:35:33.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:33 smithi187 ceph-mon[34603]: pgmap v516: 97 pgs: 97 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 252 op/s 2024-09-21T03:35:33.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:33 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:35:33.785 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:33 smithi112 ceph-mon[31442]: pgmap v516: 97 pgs: 97 active+clean; 465 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 252 op/s 2024-09-21T03:35:34.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 676470 10.85 MB/sec execute 214 sec latency 22.239 ms 2024-09-21T03:35:34.349 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:35:34.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:35:34.618 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:34.740 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:34 smithi112 ceph-mon[31442]: pgmap v517: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 240 op/s 2024-09-21T03:35:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:34 smithi203 ceph-mon[34723]: pgmap v517: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 240 op/s 2024-09-21T03:35:34.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:34 smithi187 ceph-mon[34603]: pgmap v517: 97 pgs: 97 active+clean; 464 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 240 op/s 2024-09-21T03:35:35.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.151+0000 7fe67c787640 1 -- 172.21.15.112:0/2102814915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe674105270 msgr2=0x7fe67410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.151+0000 7fe67c787640 1 --2- 172.21.15.112:0/2102814915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe674105270 0x7fe67410d8b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe664009900 tx=0x7fe66402a410 comp rx=0 tx=0).stop 2024-09-21T03:35:35.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.151+0000 7fe67c787640 1 -- 172.21.15.112:0/2102814915 shutdown_connections 2024-09-21T03:35:35.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.151+0000 7fe67c787640 1 --2- 172.21.15.112:0/2102814915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67410ddf0 0x7fe674110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.151+0000 7fe67c787640 1 --2- 172.21.15.112:0/2102814915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe674105270 0x7fe67410d8b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.151+0000 7fe67c787640 1 --2- 172.21.15.112:0/2102814915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe674104840 0x7fe674104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.151+0000 7fe67c787640 1 -- 172.21.15.112:0/2102814915 >> 172.21.15.112:0/2102814915 conn(0x7fe674079730 msgr2=0x7fe674079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:35.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.152+0000 7fe67c787640 1 -- 172.21.15.112:0/2102814915 shutdown_connections 2024-09-21T03:35:35.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.152+0000 7fe67c787640 1 -- 172.21.15.112:0/2102814915 wait complete. 2024-09-21T03:35:35.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67c787640 1 Processor -- start 2024-09-21T03:35:35.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67c787640 1 -- start start 2024-09-21T03:35:35.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67c787640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe674104840 0x7fe67419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67c787640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe674105270 0x7fe67419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67c787640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67410ddf0 0x7fe67419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67c787640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe674113a70 con 0x7fe674105270 2024-09-21T03:35:35.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67c787640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741138f0 con 0x7fe67410ddf0 2024-09-21T03:35:35.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67c787640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe674113bf0 con 0x7fe674104840 2024-09-21T03:35:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67a523640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe674104840 0x7fe67419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe679d22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe674105270 0x7fe67419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67a523640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe674104840 0x7fe67419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35112/0 (socket says 172.21.15.112:35112) 2024-09-21T03:35:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67a523640 1 -- 172.21.15.112:0/2310389951 learned_addr learned my addr 172.21.15.112:0/2310389951 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.153+0000 7fe67ad24640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67410ddf0 0x7fe67419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.154+0000 7fe67a523640 1 -- 172.21.15.112:0/2310389951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67410ddf0 msgr2=0x7fe67419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.154+0000 7fe67a523640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67410ddf0 0x7fe67419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.154+0000 7fe67a523640 1 -- 172.21.15.112:0/2310389951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe674105270 msgr2=0x7fe67419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.154+0000 7fe67a523640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe674105270 0x7fe67419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.154+0000 7fe67a523640 1 -- 172.21.15.112:0/2310389951 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6741a04b0 con 0x7fe674104840 2024-09-21T03:35:35.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.154+0000 7fe67ad24640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67410ddf0 0x7fe67419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:35.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.154+0000 7fe679d22640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe674105270 0x7fe67419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:35.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.154+0000 7fe67a523640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe674104840 0x7fe67419a540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe67000eb70 tx=0x7fe67000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:35.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.154+0000 7fe66b7fe640 1 -- 172.21.15.112:0/2310389951 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670003b70 con 0x7fe674104840 2024-09-21T03:35:35.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.154+0000 7fe67c787640 1 -- 172.21.15.112:0/2310389951 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6741a07a0 con 0x7fe674104840 2024-09-21T03:35:35.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.154+0000 7fe66b7fe640 1 -- 172.21.15.112:0/2310389951 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe67000ee70 con 0x7fe674104840 2024-09-21T03:35:35.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.155+0000 7fe66b7fe640 1 -- 172.21.15.112:0/2310389951 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670010650 con 0x7fe674104840 2024-09-21T03:35:35.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.155+0000 7fe67c787640 1 -- 172.21.15.112:0/2310389951 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6741a6f00 con 0x7fe674104840 2024-09-21T03:35:35.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.156+0000 7fe67c787640 1 -- 172.21.15.112:0/2310389951 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67406b500 con 0x7fe674104840 2024-09-21T03:35:35.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.157+0000 7fe66b7fe640 1 -- 172.21.15.112:0/2310389951 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe670004000 con 0x7fe674104840 2024-09-21T03:35:35.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.158+0000 7fe66b7fe640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6600b6680 0x7fe6600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:35.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.158+0000 7fe679d22640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6600b6680 0x7fe6600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:35.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.158+0000 7fe66b7fe640 1 -- 172.21.15.112:0/2310389951 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7fe6700e7e50 con 0x7fe674104840 2024-09-21T03:35:35.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.158+0000 7fe679d22640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6600b6680 0x7fe6600b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe664009230 tx=0x7fe664031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:35.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.163+0000 7fe66b7fe640 1 -- 172.21.15.112:0/2310389951 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6700ae3d0 con 0x7fe674104840 2024-09-21T03:35:35.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 678687 10.86 MB/sec execute 215 sec latency 46.748 ms 2024-09-21T03:35:35.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.408+0000 7fe67c787640 1 -- 172.21.15.112:0/2310389951 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6741a7100 con 0x7fe674104840 2024-09-21T03:35:35.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.409+0000 7fe66b7fe640 1 -- 172.21.15.112:0/2310389951 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7fe6700b27b0 con 0x7fe674104840 2024-09-21T03:35:35.411 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:35:35.411 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"btime":"2024-09-21T03:35:32:357323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:35:32.307199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:35:35.412 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-09-21T03:35:35.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.411+0000 7fe67c787640 1 -- 172.21.15.112:0/2310389951 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6600b6680 msgr2=0x7fe6600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.411+0000 7fe67c787640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6600b6680 0x7fe6600b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe664009230 tx=0x7fe664031400 comp rx=0 tx=0).stop 2024-09-21T03:35:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.411+0000 7fe67c787640 1 -- 172.21.15.112:0/2310389951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe674104840 msgr2=0x7fe67419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.411+0000 7fe67c787640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe674104840 0x7fe67419a540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe67000eb70 tx=0x7fe67000c6a0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.412+0000 7fe67c787640 1 -- 172.21.15.112:0/2310389951 shutdown_connections 2024-09-21T03:35:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.412+0000 7fe67c787640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe6600b6680 0x7fe6600b8b90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.412+0000 7fe67c787640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe67410ddf0 0x7fe67419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.412+0000 7fe67c787640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe674105270 0x7fe67419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.412+0000 7fe67c787640 1 --2- 172.21.15.112:0/2310389951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe674104840 0x7fe67419a540 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:35.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.412+0000 7fe67c787640 1 -- 172.21.15.112:0/2310389951 >> 172.21.15.112:0/2310389951 conn(0x7fe674079730 msgr2=0x7fe6741068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.412+0000 7fe67c787640 1 -- 172.21.15.112:0/2310389951 shutdown_connections 2024-09-21T03:35:35.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:35.412+0000 7fe67c787640 1 -- 172.21.15.112:0/2310389951 wait complete. 2024-09-21T03:35:35.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:35 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2310389951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:35:35.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:35 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2310389951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:35:35.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:35 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2310389951' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:35:36.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 680658 10.84 MB/sec execute 216 sec latency 18.731 ms 2024-09-21T03:35:36.272 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:35:36.544 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:36 smithi112 ceph-mon[31442]: pgmap v518: 97 pgs: 97 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 329 op/s 2024-09-21T03:35:36.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:36 smithi203 ceph-mon[34723]: pgmap v518: 97 pgs: 97 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 329 op/s 2024-09-21T03:35:36.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:36 smithi187 ceph-mon[34603]: pgmap v518: 97 pgs: 97 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 329 op/s 2024-09-21T03:35:37.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.065+0000 7f917ca3a640 1 -- 172.21.15.112:0/506303874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178102a60 msgr2=0x7f9178102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:37.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.065+0000 7f917ca3a640 1 --2- 172.21.15.112:0/506303874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178102a60 0x7f9178102e70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9168009900 tx=0x7f916802a410 comp rx=0 tx=0).stop 2024-09-21T03:35:37.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.065+0000 7f917ca3a640 1 -- 172.21.15.112:0/506303874 shutdown_connections 2024-09-21T03:35:37.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.065+0000 7f917ca3a640 1 --2- 172.21.15.112:0/506303874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9178108cf0 0x7f917810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.065+0000 7f917ca3a640 1 --2- 172.21.15.112:0/506303874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91781083a0 0x7f91781087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.065+0000 7f917ca3a640 1 --2- 172.21.15.112:0/506303874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178102a60 0x7f9178102e70 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.065+0000 7f917ca3a640 1 -- 172.21.15.112:0/506303874 >> 172.21.15.112:0/506303874 conn(0x7f9178079730 msgr2=0x7f9178079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:37.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.065+0000 7f917ca3a640 1 -- 172.21.15.112:0/506303874 shutdown_connections 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.065+0000 7f917ca3a640 1 -- 172.21.15.112:0/506303874 wait complete. 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.066+0000 7f917ca3a640 1 Processor -- start 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.066+0000 7f917ca3a640 1 -- start start 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.066+0000 7f917ca3a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9178102a60 0x7f917819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.066+0000 7f917ca3a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91781083a0 0x7f917819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f917ca3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178108cf0 0x7f917819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f917ca3a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9178112960 con 0x7f9178108cf0 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f917ca3a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f91781127e0 con 0x7f9178102a60 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f917ca3a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9178112ae0 con 0x7f91781083a0 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f9175d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91781083a0 0x7f917819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f9176d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178108cf0 0x7f917819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f9175d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91781083a0 0x7f917819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35136/0 (socket says 172.21.15.112:35136) 2024-09-21T03:35:37.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f9175d9b640 1 -- 172.21.15.112:0/2810479666 learned_addr learned my addr 172.21.15.112:0/2810479666 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:37.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f917659c640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9178102a60 0x7f917819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f9176d9d640 1 -- 172.21.15.112:0/2810479666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91781083a0 msgr2=0x7f917819b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:37.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f9176d9d640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91781083a0 0x7f917819b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f9176d9d640 1 -- 172.21.15.112:0/2810479666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9178102a60 msgr2=0x7f917819b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:37.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f9176d9d640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9178102a60 0x7f917819b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f9176d9d640 1 -- 172.21.15.112:0/2810479666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91781a64c0 con 0x7f9178108cf0 2024-09-21T03:35:37.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f9176d9d640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178108cf0 0x7f917819bcb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9170009fd0 tx=0x7f917000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:37.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.067+0000 7f91677fe640 1 -- 172.21.15.112:0/2810479666 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917000ce20 con 0x7f9178108cf0 2024-09-21T03:35:37.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.068+0000 7f917ca3a640 1 -- 172.21.15.112:0/2810479666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9178199080 con 0x7f9178108cf0 2024-09-21T03:35:37.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.068+0000 7f91677fe640 1 -- 172.21.15.112:0/2810479666 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9170005020 con 0x7f9178108cf0 2024-09-21T03:35:37.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.068+0000 7f91677fe640 1 -- 172.21.15.112:0/2810479666 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170010690 con 0x7f9178108cf0 2024-09-21T03:35:37.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.068+0000 7f917ca3a640 1 -- 172.21.15.112:0/2810479666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9178199610 con 0x7f9178108cf0 2024-09-21T03:35:37.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.070+0000 7f9182fae740 1 Processor -- start 2024-09-21T03:35:37.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.071+0000 7f91677fe640 1 -- 172.21.15.112:0/2810479666 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9170010830 con 0x7f9178108cf0 2024-09-21T03:35:37.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.071+0000 7f9182fae740 1 -- start start 2024-09-21T03:35:37.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9182fae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 0x55adeb95d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9182fae740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 0x55adeb95dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9182fae740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb95e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9182fae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55adeb8d3ea0 con 0x55adeb5b6070 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9182fae740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55adeb8c32d0 con 0x55adeb595390 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9182fae740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55adeb8c70e0 con 0x55adeb58d1b0 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9175d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 0x55adeb95d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9175d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 0x55adeb95d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35908/0 (socket says 172.21.15.112:35908) 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9175d9b640 1 -- 172.21.15.112:0/3979176639 learned_addr learned my addr 172.21.15.112:0/3979176639 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f917659c640 1 --2- 172.21.15.112:0/3979176639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 0x55adeb95dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9175d9b640 1 --2- 172.21.15.112:0/3979176639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb95e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f91677fe640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91600b6730 0x7f91600b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9175d9b640 1 -- 172.21.15.112:0/3979176639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 msgr2=0x55adeb95dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:37.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9175d9b640 1 --2- 172.21.15.112:0/3979176639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 0x55adeb95dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9175d9b640 1 -- 172.21.15.112:0/3979176639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 msgr2=0x55adeb95e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:37.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9175d9b640 1 --2- 172.21.15.112:0/3979176639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb95e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.072+0000 7f9175d9b640 1 -- 172.21.15.112:0/3979176639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adeb621650 con 0x55adeb5b6070 2024-09-21T03:35:37.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.073+0000 7f9176d9d640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91600b6730 0x7f91600b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.073+0000 7f9175d9b640 1 --2- 172.21.15.112:0/3979176639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 0x55adeb95d6a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f916c00fb80 tx=0x7f916c00fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc08ffd2f5bda17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:37.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.073+0000 7f91677fe640 1 -- 172.21.15.112:0/2810479666 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f91700e7c60 con 0x7f9178108cf0 2024-09-21T03:35:37.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.073+0000 7f9164ff9640 1 -- 172.21.15.112:0/3979176639 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c00f020 con 0x55adeb5b6070 2024-09-21T03:35:37.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.073+0000 7f9164ff9640 1 -- 172.21.15.112:0/3979176639 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f916c005020 con 0x55adeb5b6070 2024-09-21T03:35:37.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.073+0000 7f9164ff9640 1 -- 172.21.15.112:0/3979176639 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c004ad0 con 0x55adeb5b6070 2024-09-21T03:35:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.074+0000 7f9182fae740 1 -- 172.21.15.112:0/3979176639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 msgr2=0x55adeb95d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.074+0000 7f9182fae740 1 --2- 172.21.15.112:0/3979176639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 0x55adeb95d6a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f916c00fb80 tx=0x7f916c00fdc0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.074+0000 7f9176d9d640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91600b6730 0x7f91600b8c40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f91700e5490 tx=0x7f91700051d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:37.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.074+0000 7f9182fae740 1 -- 172.21.15.112:0/3979176639 shutdown_connections 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.074+0000 7f9182fae740 1 --2- 172.21.15.112:0/3979176639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb95e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.074+0000 7f9182fae740 1 --2- 172.21.15.112:0/3979176639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 0x55adeb95dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.074+0000 7f9182fae740 1 --2- 172.21.15.112:0/3979176639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 0x55adeb95d6a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.074+0000 7f9182fae740 1 -- 172.21.15.112:0/3979176639 >> 172.21.15.112:0/3979176639 conn(0x55adeb5aa460 msgr2=0x55adeb95b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.074+0000 7f9182fae740 1 -- 172.21.15.112:0/3979176639 shutdown_connections 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.074+0000 7f9182fae740 1 -- 172.21.15.112:0/3979176639 wait complete. 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9182fae740 1 -- start start 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9182fae740 1 Processor -- start 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9182fae740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 0x55adeb968de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9182fae740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb969320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9182fae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 0x55adeb975170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9182fae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55adeb724700 con 0x55adeb5b6070 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9182fae740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55adeb85e160 con 0x55adeb595390 2024-09-21T03:35:37.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9182fae740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55adeb96be80 con 0x55adeb58d1b0 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9182fae740 10 client.0 _ll_register_callbacks cb 0x55adeb8779e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9175d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 0x55adeb968de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9175d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 0x55adeb968de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35154/0 (socket says 172.21.15.112:35154) 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9175d9b640 1 -- 172.21.15.112:0/2720579200 learned_addr learned my addr 172.21.15.112:0/2720579200 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f9175d9b640 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 0x55adeb975170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.075+0000 7f917659c640 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb969320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f9175d9b640 1 -- 172.21.15.112:0/2720579200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 msgr2=0x55adeb969320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f9175d9b640 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb969320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f9175d9b640 1 -- 172.21.15.112:0/2720579200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 msgr2=0x55adeb975170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f9175d9b640 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 0x55adeb975170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f9175d9b640 1 -- 172.21.15.112:0/2720579200 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adeb88bc90 con 0x55adeb58d1b0 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f9175d9b640 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 0x55adeb968de0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f916c00fb50 tx=0x7f916c00f890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4f752bc7ed7ba84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f915dffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:35:37.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f915dffb640 1 -- 172.21.15.112:0/2720579200 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c0053e0 con 0x55adeb58d1b0 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f915dffb640 1 -- 172.21.15.112:0/2720579200 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f916c005580 con 0x55adeb58d1b0 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f9182fae740 1 -- 172.21.15.112:0/2720579200 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f916c005020 con 0x55adeb58d1b0 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f915dffb640 1 -- 172.21.15.112:0/2720579200 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f916c02d440 con 0x55adeb58d1b0 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f9182fae740 10 client.26194 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.076+0000 7f9182fae740 1 -- 172.21.15.112:0/2720579200 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55adeb977c10 con 0x55adeb58d1b0 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.077+0000 7f915dffb640 1 -- 172.21.15.112:0/2720579200 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f916c005020 con 0x55adeb58d1b0 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.077+0000 7f915dffb640 1 -- 172.21.15.112:0/2720579200 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f916c01b070 con 0x55adeb58d1b0 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.077+0000 7f9182fae740 10 client.26194 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.077+0000 7f9182fae740 10 client.26194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.077+0000 7f9182fae740 20 client.26194 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.077+0000 7f9182fae740 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55adeb5749f0 0x55adeb97afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.077+0000 7f9182fae740 4 client.26194 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.077+0000 7f9182fae740 1 -- 172.21.15.112:0/2720579200 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adeb8d5df0 con 0x55adeb5749f0 2024-09-21T03:35:37.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.077+0000 7f917659c640 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55adeb5749f0 0x55adeb97afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.077+0000 7f915dffb640 10 client.26194 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:35:37.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.078+0000 7f917659c640 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55adeb5749f0 0x55adeb97afd0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7cced6f7ebf7a74 server_cookie=741eadbac3c6a869 in_seq=0 out_seq=0 2024-09-21T03:35:37.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.078+0000 7f915dffb640 1 -- 172.21.15.112:0/2720579200 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55adeb8d5df0 con 0x55adeb5749f0 2024-09-21T03:35:37.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.078+0000 7f915dffb640 10 client.26194 handle_command_reply: tid=0 2024-09-21T03:35:37.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.078+0000 7f9182fae740 1 client.26194 shutdown 2024-09-21T03:35:37.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f9182fae740 1 -- 172.21.15.112:0/2720579200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 msgr2=0x55adeb968de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:37.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f9182fae740 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 0x55adeb968de0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f916c00fb50 tx=0x7f916c00f890 comp rx=0 tx=0).stop 2024-09-21T03:35:37.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f9182fae740 1 -- 172.21.15.112:0/2720579200 shutdown_connections 2024-09-21T03:35:37.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f9182fae740 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55adeb5749f0 0x55adeb97afd0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f9182fae740 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb5b6070 0x55adeb975170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f9182fae740 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb969320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f915dffb640 0 client.26194 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:35:37.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f9182fae740 1 --2- 172.21.15.112:0/2720579200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb58d1b0 0x55adeb968de0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f9182fae740 1 -- 172.21.15.112:0/2720579200 >> 172.21.15.112:0/2720579200 conn(0x55adeb5aa460 msgr2=0x55adeb960880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:37.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f9182fae740 1 -- 172.21.15.112:0/2720579200 shutdown_connections 2024-09-21T03:35:37.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f9182fae740 1 -- 172.21.15.112:0/2720579200 wait complete. 2024-09-21T03:35:37.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.079+0000 7f9182fae740 20 client.26194 trim_cache size 0 max 16384 2024-09-21T03:35:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f9182fae740 1 Processor -- start 2024-09-21T03:35:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f9182fae740 1 -- start start 2024-09-21T03:35:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f9182fae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 0x55adeb95ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f9182fae740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb965fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f9182fae740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb5b6070 0x55adeb966510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f9182fae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91781afc80 con 0x55adeb58d1b0 2024-09-21T03:35:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f9182fae740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f91781ad550 con 0x55adeb595390 2024-09-21T03:35:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f9182fae740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f91781ab3c0 con 0x55adeb5b6070 2024-09-21T03:35:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f9175d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 0x55adeb95ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f9175d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 0x55adeb95ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35928/0 (socket says 172.21.15.112:35928) 2024-09-21T03:35:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f9175d9b640 1 -- 172.21.15.112:0/2041579321 learned_addr learned my addr 172.21.15.112:0/2041579321 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.093+0000 7f917659c640 1 --2- 172.21.15.112:0/2041579321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb965fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.094+0000 7f9175d9b640 1 --2- 172.21.15.112:0/2041579321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb5b6070 0x55adeb966510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.094+0000 7f917659c640 1 -- 172.21.15.112:0/2041579321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb5b6070 msgr2=0x55adeb966510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.094+0000 7f917659c640 1 --2- 172.21.15.112:0/2041579321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb5b6070 0x55adeb966510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.094+0000 7f917659c640 1 -- 172.21.15.112:0/2041579321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 msgr2=0x55adeb95ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.094+0000 7f917659c640 1 --2- 172.21.15.112:0/2041579321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 0x55adeb95ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.094+0000 7f917659c640 1 -- 172.21.15.112:0/2041579321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adeb9672e0 con 0x55adeb595390 2024-09-21T03:35:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.094+0000 7f9175d9b640 1 --2- 172.21.15.112:0/2041579321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 0x55adeb95ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.094+0000 7f917659c640 1 --2- 172.21.15.112:0/2041579321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb965fd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f917806b9c0 tx=0x7f9168031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=754188ed6e04ddac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.094+0000 7f915ffff640 1 -- 172.21.15.112:0/2041579321 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168035040 con 0x55adeb595390 2024-09-21T03:35:37.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.094+0000 7f915ffff640 1 -- 172.21.15.112:0/2041579321 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91680054f0 con 0x55adeb595390 2024-09-21T03:35:37.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.094+0000 7f915ffff640 1 -- 172.21.15.112:0/2041579321 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168004c80 con 0x55adeb595390 2024-09-21T03:35:37.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.095+0000 7f9182fae740 1 -- 172.21.15.112:0/2041579321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 msgr2=0x55adeb965fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:37.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.095+0000 7f9182fae740 1 --2- 172.21.15.112:0/2041579321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb965fd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f917806b9c0 tx=0x7f9168031400 comp rx=0 tx=0).stop 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.095+0000 7f9182fae740 1 -- 172.21.15.112:0/2041579321 shutdown_connections 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.095+0000 7f9182fae740 1 --2- 172.21.15.112:0/2041579321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb5b6070 0x55adeb966510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.095+0000 7f9182fae740 1 --2- 172.21.15.112:0/2041579321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb965fd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.095+0000 7f9182fae740 1 --2- 172.21.15.112:0/2041579321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 0x55adeb95ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.095+0000 7f9182fae740 1 -- 172.21.15.112:0/2041579321 >> 172.21.15.112:0/2041579321 conn(0x55adeb5aa460 msgr2=0x55adeb95e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.095+0000 7f9182fae740 1 -- 172.21.15.112:0/2041579321 shutdown_connections 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.095+0000 7f9182fae740 1 -- 172.21.15.112:0/2041579321 wait complete. 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9182fae740 1 -- start start 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9182fae740 1 Processor -- start 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9182fae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 0x55adeb971600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9182fae740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb96ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9182fae740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb5b6070 0x55adeb96f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9182fae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91781aba10 con 0x55adeb58d1b0 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9182fae740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f91781ace80 con 0x55adeb595390 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9182fae740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f91781ac0e0 con 0x55adeb5b6070 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9182fae740 10 client.0 _ll_register_callbacks cb 0x7f9168009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9175d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 0x55adeb971600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9175d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 0x55adeb971600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35934/0 (socket says 172.21.15.112:35934) 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9175d9b640 1 -- 172.21.15.112:0/208929198 learned_addr learned my addr 172.21.15.112:0/208929198 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f917659c640 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb96ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f9175d9b640 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb5b6070 0x55adeb96f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f917659c640 1 -- 172.21.15.112:0/208929198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb5b6070 msgr2=0x55adeb96f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f917659c640 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb5b6070 0x55adeb96f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f917659c640 1 -- 172.21.15.112:0/208929198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 msgr2=0x55adeb971600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:37.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f917659c640 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 0x55adeb971600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.096+0000 7f917659c640 1 -- 172.21.15.112:0/208929198 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adeb96f550 con 0x55adeb595390 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.097+0000 7f917659c640 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb96ead0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9168000c00 tx=0x7f916802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=791bf8ee5279c4ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.097+0000 7f915dffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.097+0000 7f915dffb640 1 -- 172.21.15.112:0/208929198 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168054070 con 0x55adeb595390 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.097+0000 7f915dffb640 1 -- 172.21.15.112:0/208929198 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f9168004300 con 0x55adeb595390 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.097+0000 7f9182fae740 1 -- 172.21.15.112:0/208929198 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91680054f0 con 0x55adeb595390 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.097+0000 7f915dffb640 1 -- 172.21.15.112:0/208929198 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9168035040 con 0x55adeb595390 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.097+0000 7f9182fae740 10 client.26195 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.097+0000 7f9182fae740 1 -- 172.21.15.112:0/208929198 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55adeb9e5500 con 0x55adeb595390 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.097+0000 7f915dffb640 1 -- 172.21.15.112:0/208929198 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91680054f0 con 0x55adeb595390 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.097+0000 7f915dffb640 1 -- 172.21.15.112:0/208929198 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9168003a80 con 0x55adeb595390 2024-09-21T03:35:37.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f9182fae740 10 client.26195 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f9182fae740 10 client.26195 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f9182fae740 20 client.26195 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f9182fae740 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55adeb549da0 0x55adeb9e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f9182fae740 4 client.26195 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f9182fae740 1 -- 172.21.15.112:0/208929198 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55adeb8d18a0 con 0x55adeb549da0 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f9175d9b640 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55adeb549da0 0x55adeb9e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f915dffb640 10 client.26195 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f9175d9b640 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55adeb549da0 0x55adeb9e7b70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec93d4ea88b35017 server_cookie=e4f0ba93dd13261b in_seq=0 out_seq=0 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f915dffb640 1 -- 172.21.15.112:0/208929198 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55adeb8d18a0 con 0x55adeb549da0 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f915dffb640 10 client.26195 handle_command_reply: tid=0 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.098+0000 7f9182fae740 1 client.26195 shutdown 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f9182fae740 1 -- 172.21.15.112:0/208929198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 msgr2=0x55adeb96ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f9182fae740 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb96ead0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9168000c00 tx=0x7f916802a410 comp rx=0 tx=0).stop 2024-09-21T03:35:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f9182fae740 1 -- 172.21.15.112:0/208929198 shutdown_connections 2024-09-21T03:35:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f9182fae740 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55adeb549da0 0x55adeb9e7b70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f9182fae740 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55adeb5b6070 0x55adeb96f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f9182fae740 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adeb595390 0x55adeb96ead0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f915dffb640 0 client.26195 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:35:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f9182fae740 1 --2- 172.21.15.112:0/208929198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55adeb58d1b0 0x55adeb971600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f9182fae740 1 -- 172.21.15.112:0/208929198 >> 172.21.15.112:0/208929198 conn(0x55adeb5aa460 msgr2=0x55adeb96ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f9182fae740 1 -- 172.21.15.112:0/208929198 shutdown_connections 2024-09-21T03:35:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f9182fae740 1 -- 172.21.15.112:0/208929198 wait complete. 2024-09-21T03:35:37.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f9182fae740 20 client.26195 trim_cache size 0 max 16384 2024-09-21T03:35:37.108 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:35:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f917ca3a640 1 -- 172.21.15.112:0/2810479666 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91600b6730 msgr2=0x7f91600b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.099+0000 7f917ca3a640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91600b6730 0x7f91600b8c40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f91700e5490 tx=0x7f91700051d0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.100+0000 7f917ca3a640 1 -- 172.21.15.112:0/2810479666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178108cf0 msgr2=0x7f917819bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.100+0000 7f917ca3a640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178108cf0 0x7f917819bcb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9170009fd0 tx=0x7f917000ef10 comp rx=0 tx=0).stop 2024-09-21T03:35:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.100+0000 7f917ca3a640 1 -- 172.21.15.112:0/2810479666 shutdown_connections 2024-09-21T03:35:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.100+0000 7f917ca3a640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f91600b6730 0x7f91600b8c40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.100+0000 7f917ca3a640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9178108cf0 0x7f917819bcb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.100+0000 7f917ca3a640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91781083a0 0x7f917819b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.100+0000 7f917ca3a640 1 --2- 172.21.15.112:0/2810479666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9178102a60 0x7f917819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.100+0000 7f917ca3a640 1 -- 172.21.15.112:0/2810479666 >> 172.21.15.112:0/2810479666 conn(0x7f9178079730 msgr2=0x7f9178101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.100+0000 7f917ca3a640 1 -- 172.21.15.112:0/2810479666 shutdown_connections 2024-09-21T03:35:37.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:37.100+0000 7f917ca3a640 1 -- 172.21.15.112:0/2810479666 wait complete. 2024-09-21T03:35:37.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 682442 10.85 MB/sec execute 217 sec latency 16.242 ms 2024-09-21T03:35:38.178 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:35:38.240 INFO:tasks.workunit.client.0.smithi112.stdout: 1 684848 10.86 MB/sec execute 218 sec latency 26.062 ms 2024-09-21T03:35:38.470 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.008+0000 7f41f1e2d640 1 -- 172.21.15.112:0/963108551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec10bff0 msgr2=0x7f41ec10c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.008+0000 7f41f1e2d640 1 --2- 172.21.15.112:0/963108551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec10bff0 0x7f41ec10c400 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00fb10 tx=0x7f41dc02e590 comp rx=0 tx=0).stop 2024-09-21T03:35:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.008+0000 7f41f1e2d640 1 -- 172.21.15.112:0/963108551 shutdown_connections 2024-09-21T03:35:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.008+0000 7f41f1e2d640 1 --2- 172.21.15.112:0/963108551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec115720 0x7f41ec117b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.008+0000 7f41f1e2d640 1 --2- 172.21.15.112:0/963108551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ec112dc0 0x7f41ec1151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.008+0000 7f41f1e2d640 1 --2- 172.21.15.112:0/963108551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec10bff0 0x7f41ec10c400 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.008+0000 7f41f1e2d640 1 -- 172.21.15.112:0/963108551 >> 172.21.15.112:0/963108551 conn(0x7f41ec0712f0 msgr2=0x7f41ec073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.008+0000 7f41f1e2d640 1 -- 172.21.15.112:0/963108551 shutdown_connections 2024-09-21T03:35:39.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.008+0000 7f41f1e2d640 1 -- 172.21.15.112:0/963108551 wait complete. 2024-09-21T03:35:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.009+0000 7f41f1e2d640 1 Processor -- start 2024-09-21T03:35:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.009+0000 7f41f1e2d640 1 -- start start 2024-09-21T03:35:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.010+0000 7f41f1e2d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec10bff0 0x7f41ec1a9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.010+0000 7f41f1e2d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ec112dc0 0x7f41ec1a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.010+0000 7f41f1e2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec115720 0x7f41ec1aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.010+0000 7f41f1e2d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec11a120 con 0x7f41ec115720 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.010+0000 7f41f1e2d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec119fa0 con 0x7f41ec10bff0 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.010+0000 7f41f1e2d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec11a2a0 con 0x7f41ec112dc0 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.010+0000 7f41ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec115720 0x7f41ec1aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.010+0000 7f41ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec115720 0x7f41ec1aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35952/0 (socket says 172.21.15.112:35952) 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.010+0000 7f41ebfff640 1 -- 172.21.15.112:0/3073249615 learned_addr learned my addr 172.21.15.112:0/3073249615 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.010+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec10bff0 0x7f41ec1a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.010+0000 7f41eaffd640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ec112dc0 0x7f41ec1a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.011+0000 7f41ebfff640 1 -- 172.21.15.112:0/3073249615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ec112dc0 msgr2=0x7f41ec1a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.011+0000 7f41ebfff640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ec112dc0 0x7f41ec1a98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.011+0000 7f41ebfff640 1 -- 172.21.15.112:0/3073249615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec10bff0 msgr2=0x7f41ec1a9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.011+0000 7f41ebfff640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec10bff0 0x7f41ec1a9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.011+0000 7f41ebfff640 1 -- 172.21.15.112:0/3073249615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec1af290 con 0x7f41ec115720 2024-09-21T03:35:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.011+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec10bff0 0x7f41ec1a9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.011+0000 7f41ebfff640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec115720 0x7f41ec1aeb60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f41e4007bd0 tx=0x7f41e400f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.011+0000 7f41e8ff9640 1 -- 172.21.15.112:0/3073249615 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4019030 con 0x7f41ec115720 2024-09-21T03:35:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.011+0000 7f41e8ff9640 1 -- 172.21.15.112:0/3073249615 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41e400fc10 con 0x7f41ec115720 2024-09-21T03:35:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.012+0000 7f41f1e2d640 1 -- 172.21.15.112:0/3073249615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ec1af580 con 0x7f41ec115720 2024-09-21T03:35:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.012+0000 7f41f1e2d640 1 -- 172.21.15.112:0/3073249615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ec1b5ce0 con 0x7f41ec115720 2024-09-21T03:35:39.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.012+0000 7f41e8ff9640 1 -- 172.21.15.112:0/3073249615 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4005880 con 0x7f41ec115720 2024-09-21T03:35:39.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.015+0000 7f41e8ff9640 1 -- 172.21.15.112:0/3073249615 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f41e401c070 con 0x7f41ec115720 2024-09-21T03:35:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.016+0000 7f41f83a1740 1 Processor -- start 2024-09-21T03:35:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.016+0000 7f41e8ff9640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41d40b6700 0x7f41d40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.016+0000 7f41f83a1740 1 -- start start 2024-09-21T03:35:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.016+0000 7f41e8ff9640 1 -- 172.21.15.112:0/3073249615 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f41e40e79f0 con 0x7f41ec115720 2024-09-21T03:35:39.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.017+0000 7f41eaffd640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41d40b6700 0x7f41d40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.017+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.017+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.017+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.017+0000 7f41f83a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dda305aea0 con 0x55dda2d3d070 2024-09-21T03:35:39.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.017+0000 7f41f83a1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dda304a2d0 con 0x55dda2d141b0 2024-09-21T03:35:39.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.017+0000 7f41f83a1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55dda304e0e0 con 0x55dda2d1c390 2024-09-21T03:35:39.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.018+0000 7f41eaffd640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41d40b6700 0x7f41d40b8c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f41ec114b50 tx=0x7f41e0009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:39.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.018+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.018+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30e46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35960/0 (socket says 172.21.15.112:35960) 2024-09-21T03:35:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.018+0000 7f41eb7fe640 1 -- 172.21.15.112:0/2806923819 learned_addr learned my addr 172.21.15.112:0/2806923819 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.018+0000 7f41eaffd640 1 --2- 172.21.15.112:0/2806923819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.018+0000 7f41eb7fe640 1 -- 172.21.15.112:0/2806923819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 msgr2=0x55dda30e5120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:35:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.018+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/2806923819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30e5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.018+0000 7f41eb7fe640 1 -- 172.21.15.112:0/2806923819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 msgr2=0x55dda30e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.018+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/2806923819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.018+0000 7f41eb7fe640 1 -- 172.21.15.112:0/2806923819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dda2da8650 con 0x55dda2d3d070 2024-09-21T03:35:39.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.019+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/2806923819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30e46a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00fb10 tx=0x7f41dc004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcffb3fcbb436fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.019+0000 7f41d1ffb640 1 -- 172.21.15.112:0/2806923819 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc043070 con 0x55dda2d3d070 2024-09-21T03:35:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.020+0000 7f41d1ffb640 1 -- 172.21.15.112:0/2806923819 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41dc00e5c0 con 0x55dda2d3d070 2024-09-21T03:35:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.020+0000 7f41d1ffb640 1 -- 172.21.15.112:0/2806923819 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00ec10 con 0x55dda2d3d070 2024-09-21T03:35:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.021+0000 7f41f83a1740 1 -- 172.21.15.112:0/2806923819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 msgr2=0x55dda30e46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.021+0000 7f41f83a1740 1 --2- 172.21.15.112:0/2806923819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30e46a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00fb10 tx=0x7f41dc004800 comp rx=0 tx=0).stop 2024-09-21T03:35:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.021+0000 7f41f83a1740 1 -- 172.21.15.112:0/2806923819 shutdown_connections 2024-09-21T03:35:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.021+0000 7f41f83a1740 1 --2- 172.21.15.112:0/2806923819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.021+0000 7f41f83a1740 1 --2- 172.21.15.112:0/2806923819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.021+0000 7f41f83a1740 1 --2- 172.21.15.112:0/2806923819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30e46a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.021+0000 7f41f83a1740 1 -- 172.21.15.112:0/2806923819 >> 172.21.15.112:0/2806923819 conn(0x55dda2d31460 msgr2=0x55dda30e2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.021+0000 7f41f83a1740 1 -- 172.21.15.112:0/2806923819 shutdown_connections 2024-09-21T03:35:39.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.021+0000 7f41f83a1740 1 -- 172.21.15.112:0/2806923819 wait complete. 2024-09-21T03:35:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.022+0000 7f41f83a1740 1 -- start start 2024-09-21T03:35:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.022+0000 7f41f83a1740 1 Processor -- start 2024-09-21T03:35:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.022+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30efe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.022+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30f0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30fc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41f83a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dda2eab700 con 0x55dda2d3d070 2024-09-21T03:35:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41f83a1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dda2fe5160 con 0x55dda2d141b0 2024-09-21T03:35:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30efe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34010/0 (socket says 172.21.15.112:34010) 2024-09-21T03:35:39.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41f83a1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55dda30f2e70 con 0x55dda2d1c390 2024-09-21T03:35:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41eb7fe640 1 -- 172.21.15.112:0/115821576 learned_addr learned my addr 172.21.15.112:0/115821576 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41ebfff640 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41f83a1740 10 client.0 _ll_register_callbacks cb 0x55dda2ffe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41eaffd640 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41eb7fe640 1 -- 172.21.15.112:0/115821576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 msgr2=0x55dda30f0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30f0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41eb7fe640 1 -- 172.21.15.112:0/115821576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 msgr2=0x55dda30fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30fc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41eb7fe640 1 -- 172.21.15.112:0/115821576 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dda3012c90 con 0x55dda2d141b0 2024-09-21T03:35:39.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.023+0000 7f41eaffd640 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:35:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.024+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30efe10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00fc30 tx=0x7f41dc00ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba993d145ab5a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.024+0000 7f41caffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:35:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.024+0000 7f41caffd640 1 -- 172.21.15.112:0/115821576 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc043070 con 0x55dda2d141b0 2024-09-21T03:35:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.024+0000 7f41caffd640 1 -- 172.21.15.112:0/115821576 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f41dc00e3c0 con 0x55dda2d141b0 2024-09-21T03:35:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.024+0000 7f41f83a1740 1 -- 172.21.15.112:0/115821576 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41dc00e5c0 con 0x55dda2d141b0 2024-09-21T03:35:39.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.024+0000 7f41caffd640 1 -- 172.21.15.112:0/115821576 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f41dc048450 con 0x55dda2d141b0 2024-09-21T03:35:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.024+0000 7f41f83a1740 10 client.26207 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.025+0000 7f41f83a1740 1 -- 172.21.15.112:0/115821576 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dda30febf0 con 0x55dda2d141b0 2024-09-21T03:35:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.025+0000 7f41caffd640 1 -- 172.21.15.112:0/115821576 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41dc00e5c0 con 0x55dda2d141b0 2024-09-21T03:35:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.025+0000 7f41caffd640 1 -- 172.21.15.112:0/115821576 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f41dc03f430 con 0x55dda2d141b0 2024-09-21T03:35:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.025+0000 7f41f83a1740 10 client.26207 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.025+0000 7f41f83a1740 10 client.26207 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:35:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.025+0000 7f41f83a1740 20 client.26207 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.025+0000 7f41f83a1740 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55dda2cfb9f0 0x55dda3101f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.025+0000 7f41f83a1740 4 client.26207 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.025+0000 7f41f83a1740 1 -- 172.21.15.112:0/115821576 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dda305cdf0 con 0x55dda2cfb9f0 2024-09-21T03:35:39.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.026+0000 7f41eaffd640 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55dda2cfb9f0 0x55dda3101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.027+0000 7f41eaffd640 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55dda2cfb9f0 0x55dda3101f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79e62adf847a8863 server_cookie=4879b056b86a8d71 in_seq=0 out_seq=0 2024-09-21T03:35:39.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.027+0000 7f41caffd640 10 client.26207 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:35:39.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.028+0000 7f41caffd640 1 -- 172.21.15.112:0/115821576 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dda305cdf0 con 0x55dda2cfb9f0 2024-09-21T03:35:39.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.028+0000 7f41caffd640 10 client.26207 handle_command_reply: tid=0 2024-09-21T03:35:39.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.028+0000 7f41f83a1740 1 client.26207 shutdown 2024-09-21T03:35:39.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.029+0000 7f41f83a1740 1 -- 172.21.15.112:0/115821576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 msgr2=0x55dda30efe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.029+0000 7f41f83a1740 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30efe10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00fc30 tx=0x7f41dc00ebe0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.029+0000 7f41f83a1740 1 -- 172.21.15.112:0/115821576 shutdown_connections 2024-09-21T03:35:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.029+0000 7f41f83a1740 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55dda2cfb9f0 0x55dda3101f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.029+0000 7f41f83a1740 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30fc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.029+0000 7f41caffd640 0 client.26207 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:35:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.029+0000 7f41f83a1740 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.029+0000 7f41f83a1740 1 --2- 172.21.15.112:0/115821576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30efe10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.029+0000 7f41f83a1740 1 -- 172.21.15.112:0/115821576 >> 172.21.15.112:0/115821576 conn(0x55dda2d31460 msgr2=0x55dda30e7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.029+0000 7f41f83a1740 1 -- 172.21.15.112:0/115821576 shutdown_connections 2024-09-21T03:35:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.030+0000 7f41f83a1740 1 -- 172.21.15.112:0/115821576 wait complete. 2024-09-21T03:35:39.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.030+0000 7f41f83a1740 20 client.26207 trim_cache size 0 max 16384 2024-09-21T03:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.047+0000 7f41f83a1740 1 Processor -- start 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.047+0000 7f41f83a1740 1 -- start start 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.047+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.047+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.047+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30ed000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41f83a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1c0e50 con 0x55dda2d3d070 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41f83a1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1be720 con 0x55dda2d141b0 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41f83a1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1bc590 con 0x55dda2d1c390 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41eaffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41eaffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30e7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35224/0 (socket says 172.21.15.112:35224) 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41eaffd640 1 -- 172.21.15.112:0/3066064066 learned_addr learned my addr 172.21.15.112:0/3066064066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/3066064066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41ebfff640 1 --2- 172.21.15.112:0/3066064066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41eaffd640 1 -- 172.21.15.112:0/3066064066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 msgr2=0x55dda30eed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41eaffd640 1 --2- 172.21.15.112:0/3066064066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30eed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41eaffd640 1 -- 172.21.15.112:0/3066064066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 msgr2=0x55dda30ed000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41eaffd640 1 --2- 172.21.15.112:0/3066064066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30ed000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41eaffd640 1 -- 172.21.15.112:0/3066064066 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dda30f10d0 con 0x55dda2d1c390 2024-09-21T03:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41ebfff640 1 --2- 172.21.15.112:0/3066064066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41eaffd640 1 --2- 172.21.15.112:0/3066064066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30e7eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55dda30e3320 tx=0x7f41e0015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d052d50f9afbeb28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.048+0000 7f41d17fa640 1 -- 172.21.15.112:0/3066064066 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0015990 con 0x55dda2d1c390 2024-09-21T03:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.049+0000 7f41d17fa640 1 -- 172.21.15.112:0/3066064066 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41e0021420 con 0x55dda2d1c390 2024-09-21T03:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.049+0000 7f41d17fa640 1 -- 172.21.15.112:0/3066064066 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0013450 con 0x55dda2d1c390 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.049+0000 7f41f83a1740 1 -- 172.21.15.112:0/3066064066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 msgr2=0x55dda30e7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.049+0000 7f41f83a1740 1 --2- 172.21.15.112:0/3066064066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30e7eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55dda30e3320 tx=0x7f41e0015000 comp rx=0 tx=0).stop 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.050+0000 7f41f83a1740 1 -- 172.21.15.112:0/3066064066 shutdown_connections 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.050+0000 7f41f83a1740 1 --2- 172.21.15.112:0/3066064066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d3d070 0x55dda30ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.050+0000 7f41f83a1740 1 --2- 172.21.15.112:0/3066064066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d1c390 0x55dda30e7eb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.050+0000 7f41f83a1740 1 --2- 172.21.15.112:0/3066064066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d141b0 0x55dda30eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.050+0000 7f41f83a1740 1 -- 172.21.15.112:0/3066064066 >> 172.21.15.112:0/3066064066 conn(0x55dda2d31460 msgr2=0x55dda30e77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.050+0000 7f41f83a1740 1 -- 172.21.15.112:0/3066064066 shutdown_connections 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.050+0000 7f41f83a1740 1 -- 172.21.15.112:0/3066064066 wait complete. 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.050+0000 7f41f83a1740 1 -- start start 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.050+0000 7f41f83a1740 1 Processor -- start 2024-09-21T03:35:39.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d141b0 0x55dda3102f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d1c390 0x55dda31034d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41f83a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d3d070 0x55dda3103a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41f83a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1bcbe0 con 0x55dda2d141b0 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d141b0 0x55dda3102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d141b0 0x55dda3102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35984/0 (socket says 172.21.15.112:35984) 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41eb7fe640 1 -- 172.21.15.112:0/416026980 learned_addr learned my addr 172.21.15.112:0/416026980 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41ebfff640 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d3d070 0x55dda3103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41eaffd640 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d1c390 0x55dda31034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41f83a1740 1 -- 172.21.15.112:0/416026980 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1be050 con 0x55dda2d1c390 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41f83a1740 1 -- 172.21.15.112:0/416026980 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec1bd2b0 con 0x55dda2d3d070 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.051+0000 7f41f83a1740 10 client.0 _ll_register_callbacks cb 0x7f41dc004750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41eb7fe640 1 -- 172.21.15.112:0/416026980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d3d070 msgr2=0x55dda3103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d3d070 0x55dda3103a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41eb7fe640 1 -- 172.21.15.112:0/416026980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d1c390 msgr2=0x55dda31034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d1c390 0x55dda31034d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41eb7fe640 1 -- 172.21.15.112:0/416026980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dda316b700 con 0x55dda2d141b0 2024-09-21T03:35:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41ebfff640 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d3d070 0x55dda3103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:39.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41eaffd640 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d1c390 0x55dda31034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41eb7fe640 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d141b0 0x55dda3102f90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f41dc02eab0 tx=0x7f41dc03f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1176a3c6c21a8017 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41caffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41caffd640 1 -- 172.21.15.112:0/416026980 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc05f070 con 0x55dda2d141b0 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41caffd640 1 -- 172.21.15.112:0/416026980 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f41dc03fc80 con 0x55dda2d141b0 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41f83a1740 1 -- 172.21.15.112:0/416026980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41e0021420 con 0x55dda2d141b0 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.052+0000 7f41caffd640 1 -- 172.21.15.112:0/416026980 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f41dc043070 con 0x55dda2d141b0 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.053+0000 7f41f83a1740 10 client.16650 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.053+0000 7f41f83a1740 1 -- 172.21.15.112:0/416026980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dda316c920 con 0x55dda2d141b0 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.053+0000 7f41caffd640 1 -- 172.21.15.112:0/416026980 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41e0021420 con 0x55dda2d141b0 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.053+0000 7f41caffd640 1 -- 172.21.15.112:0/416026980 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f41dc062030 con 0x55dda2d141b0 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.053+0000 7f41f83a1740 10 client.16650 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.053+0000 7f41f83a1740 10 client.16650 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:35:39.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.053+0000 7f41f83a1740 20 client.16650 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.053+0000 7f41f83a1740 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55dda2cd0da0 0x55dda316f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.053+0000 7f41f83a1740 4 client.16650 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.053+0000 7f41f83a1740 1 -- 172.21.15.112:0/416026980 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dda30588a0 con 0x55dda2cd0da0 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.054+0000 7f41eaffd640 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55dda2cd0da0 0x55dda316f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.054+0000 7f41eaffd640 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55dda2cd0da0 0x55dda316f0a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f442338dfb885dbb server_cookie=27a8b51ecb73a24b in_seq=0 out_seq=0 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.054+0000 7f41caffd640 10 client.16650 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.054+0000 7f41caffd640 1 -- 172.21.15.112:0/416026980 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dda30588a0 con 0x55dda2cd0da0 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.054+0000 7f41caffd640 10 client.16650 handle_command_reply: tid=0 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.054+0000 7f41f83a1740 1 client.16650 shutdown 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f83a1740 1 -- 172.21.15.112:0/416026980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d141b0 msgr2=0x55dda3102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f83a1740 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d141b0 0x55dda3102f90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f41dc02eab0 tx=0x7f41dc03f660 comp rx=0 tx=0).stop 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f83a1740 1 -- 172.21.15.112:0/416026980 shutdown_connections 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f83a1740 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55dda2cd0da0 0x55dda316f0a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f83a1740 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dda2d3d070 0x55dda3103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f83a1740 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dda2d1c390 0x55dda31034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f83a1740 1 --2- 172.21.15.112:0/416026980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dda2d141b0 0x55dda3102f90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f83a1740 1 -- 172.21.15.112:0/416026980 >> 172.21.15.112:0/416026980 conn(0x55dda2d31460 msgr2=0x55dda30e56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41caffd640 0 client.16650 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:35:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f83a1740 1 -- 172.21.15.112:0/416026980 shutdown_connections 2024-09-21T03:35:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f83a1740 1 -- 172.21.15.112:0/416026980 wait complete. 2024-09-21T03:35:39.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f83a1740 20 client.16650 trim_cache size 0 max 16384 2024-09-21T03:35:39.063 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:35:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.055+0000 7f41f1e2d640 1 -- 172.21.15.112:0/3073249615 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41d40b6700 msgr2=0x7f41d40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.056+0000 7f41f1e2d640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41d40b6700 0x7f41d40b8c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f41ec114b50 tx=0x7f41e0009110 comp rx=0 tx=0).stop 2024-09-21T03:35:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.056+0000 7f41f1e2d640 1 -- 172.21.15.112:0/3073249615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec115720 msgr2=0x7f41ec1aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.056+0000 7f41f1e2d640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec115720 0x7f41ec1aeb60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f41e4007bd0 tx=0x7f41e400f500 comp rx=0 tx=0).stop 2024-09-21T03:35:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.056+0000 7f41f1e2d640 1 -- 172.21.15.112:0/3073249615 shutdown_connections 2024-09-21T03:35:39.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.056+0000 7f41f1e2d640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f41d40b6700 0x7f41d40b8c10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.056+0000 7f41f1e2d640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41ec115720 0x7f41ec1aeb60 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.056+0000 7f41f1e2d640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ec112dc0 0x7f41ec1a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.056+0000 7f41f1e2d640 1 --2- 172.21.15.112:0/3073249615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec10bff0 0x7f41ec1a9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:39.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.056+0000 7f41f1e2d640 1 -- 172.21.15.112:0/3073249615 >> 172.21.15.112:0/3073249615 conn(0x7f41ec0712f0 msgr2=0x7f41ec115b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:39.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.056+0000 7f41f1e2d640 1 -- 172.21.15.112:0/3073249615 shutdown_connections 2024-09-21T03:35:39.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:39.056+0000 7f41f1e2d640 1 -- 172.21.15.112:0/3073249615 wait complete. 2024-09-21T03:35:39.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 686008 10.84 MB/sec execute 219 sec latency 38.661 ms 2024-09-21T03:35:39.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:39 smithi112 ceph-mon[31442]: pgmap v519: 97 pgs: 97 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2024-09-21T03:35:39.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:35:39.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:39 smithi203 ceph-mon[34723]: pgmap v519: 97 pgs: 97 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2024-09-21T03:35:39.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:39 smithi187 ceph-mon[34603]: pgmap v519: 97 pgs: 97 active+clean; 472 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2024-09-21T03:35:40.154 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:40.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 688573 10.85 MB/sec execute 220 sec latency 18.882 ms 2024-09-21T03:35:40.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.673+0000 7f07c3ea4640 1 -- 172.21.15.112:0/754550654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07bc102030 msgr2=0x7f07bc102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:40.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.673+0000 7f07c3ea4640 1 --2- 172.21.15.112:0/754550654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07bc102030 0x7f07bc102440 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009900 tx=0x7f07b802a410 comp rx=0 tx=0).stop 2024-09-21T03:35:40.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.673+0000 7f07c3ea4640 1 -- 172.21.15.112:0/754550654 shutdown_connections 2024-09-21T03:35:40.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.673+0000 7f07c3ea4640 1 --2- 172.21.15.112:0/754550654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07bc106bd0 0x7f07bc110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.673+0000 7f07c3ea4640 1 --2- 172.21.15.112:0/754550654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07bc106280 0x7f07bc106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.673+0000 7f07c3ea4640 1 --2- 172.21.15.112:0/754550654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07bc102030 0x7f07bc102440 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.673+0000 7f07c3ea4640 1 -- 172.21.15.112:0/754550654 >> 172.21.15.112:0/754550654 conn(0x7f07bc077c60 msgr2=0x7f07bc0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.673+0000 7f07c3ea4640 1 -- 172.21.15.112:0/754550654 shutdown_connections 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.673+0000 7f07c3ea4640 1 -- 172.21.15.112:0/754550654 wait complete. 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.673+0000 7f07c3ea4640 1 Processor -- start 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c3ea4640 1 -- start start 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c3ea4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07bc102030 0x7f07bc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c3ea4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07bc106280 0x7f07bc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c3ea4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07bc106bd0 0x7f07bc19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c3ea4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc113aa0 con 0x7f07bc106280 2024-09-21T03:35:40.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c3ea4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc113920 con 0x7f07bc106bd0 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c3ea4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc113c20 con 0x7f07bc102030 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c143f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07bc106280 0x7f07bc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c143f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07bc106280 0x7f07bc19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35990/0 (socket says 172.21.15.112:35990) 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c143f640 1 -- 172.21.15.112:0/3161288812 learned_addr learned my addr 172.21.15.112:0/3161288812 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c1c40640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07bc102030 0x7f07bc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c2441640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07bc106bd0 0x7f07bc19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c143f640 1 -- 172.21.15.112:0/3161288812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07bc102030 msgr2=0x7f07bc19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c143f640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07bc102030 0x7f07bc19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c143f640 1 -- 172.21.15.112:0/3161288812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07bc106bd0 msgr2=0x7f07bc19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c143f640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07bc106bd0 0x7f07bc19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.674+0000 7f07c143f640 1 -- 172.21.15.112:0/3161288812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc1a0420 con 0x7f07bc106280 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.675+0000 7f07c143f640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07bc106280 0x7f07bc19aa50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00e970 tx=0x7f07ac00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:40.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.675+0000 7f07b2ffd640 1 -- 172.21.15.112:0/3161288812 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00cd80 con 0x7f07bc106280 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.675+0000 7f07c3ea4640 1 -- 172.21.15.112:0/3161288812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07bc1a0710 con 0x7f07bc106280 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.675+0000 7f07b2ffd640 1 -- 172.21.15.112:0/3161288812 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07ac0035f0 con 0x7f07bc106280 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.675+0000 7f07b2ffd640 1 -- 172.21.15.112:0/3161288812 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00cd80 con 0x7f07bc106280 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.675+0000 7f07c3ea4640 1 -- 172.21.15.112:0/3161288812 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07bc1a6e70 con 0x7f07bc106280 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.678+0000 7f07ca418740 1 Processor -- start 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.678+0000 7f07b2ffd640 1 -- 172.21.15.112:0/3161288812 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f07ac004000 con 0x7f07bc106280 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.678+0000 7f07ca418740 1 -- start start 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07ca418740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf5f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07ca418740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 0x5595ddf5fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07ca418740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf60120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07ca418740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5595dded5ea0 con 0x5595ddb8f1b0 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07ca418740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5595ddec52d0 con 0x5595ddb97390 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07ca418740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5595ddec90e0 con 0x5595ddbb8070 2024-09-21T03:35:40.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07c2441640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf5f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07c2441640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf5f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35252/0 (socket says 172.21.15.112:35252) 2024-09-21T03:35:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07c2441640 1 -- 172.21.15.112:0/3175285137 learned_addr learned my addr 172.21.15.112:0/3175285137 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07c1c40640 1 --2- 172.21.15.112:0/3175285137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 0x5595ddf5fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07c143f640 1 --2- 172.21.15.112:0/3175285137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf60120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07c2441640 1 -- 172.21.15.112:0/3175285137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 msgr2=0x5595ddf60120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07c2441640 1 --2- 172.21.15.112:0/3175285137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf60120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07c2441640 1 -- 172.21.15.112:0/3175285137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 msgr2=0x5595ddf5fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07c2441640 1 --2- 172.21.15.112:0/3175285137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 0x5595ddf5fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.679+0000 7f07c2441640 1 -- 172.21.15.112:0/3175285137 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595ddc23650 con 0x5595ddbb8070 2024-09-21T03:35:40.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.680+0000 7f07b2ffd640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07a80b6880 0x7f07a80b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.680+0000 7f07c2441640 1 --2- 172.21.15.112:0/3175285137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf5f6a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07b4010370 tx=0x7f07b4010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960cd38f6ff9fb70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:40.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.680+0000 7f07c1c40640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07a80b6880 0x7f07a80b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.680+0000 7f07a7fff640 1 -- 172.21.15.112:0/3175285137 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4016070 con 0x5595ddbb8070 2024-09-21T03:35:40.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.680+0000 7f07b2ffd640 1 -- 172.21.15.112:0/3161288812 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f07ac0e7590 con 0x7f07bc106280 2024-09-21T03:35:40.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.680+0000 7f07a7fff640 1 -- 172.21.15.112:0/3175285137 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b4004d20 con 0x5595ddbb8070 2024-09-21T03:35:40.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.680+0000 7f07a7fff640 1 -- 172.21.15.112:0/3175285137 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b40053c0 con 0x5595ddbb8070 2024-09-21T03:35:40.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.681+0000 7f07c1c40640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07a80b6880 0x7f07a80b8d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f07b8004770 tx=0x7f07b80058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:40.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.681+0000 7f07ca418740 1 -- 172.21.15.112:0/3175285137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 msgr2=0x5595ddf5f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.681+0000 7f07ca418740 1 --2- 172.21.15.112:0/3175285137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf5f6a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07b4010370 tx=0x7f07b4010830 comp rx=0 tx=0).stop 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.681+0000 7f07ca418740 1 -- 172.21.15.112:0/3175285137 shutdown_connections 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.681+0000 7f07ca418740 1 --2- 172.21.15.112:0/3175285137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf60120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.681+0000 7f07ca418740 1 --2- 172.21.15.112:0/3175285137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 0x5595ddf5fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.681+0000 7f07ca418740 1 --2- 172.21.15.112:0/3175285137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf5f6a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.681+0000 7f07ca418740 1 -- 172.21.15.112:0/3175285137 >> 172.21.15.112:0/3175285137 conn(0x5595ddbac460 msgr2=0x5595ddf5d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.682+0000 7f07ca418740 1 -- 172.21.15.112:0/3175285137 shutdown_connections 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.682+0000 7f07ca418740 1 -- 172.21.15.112:0/3175285137 wait complete. 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.682+0000 7f07ca418740 1 -- start start 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.682+0000 7f07ca418740 1 Processor -- start 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07ca418740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 0x5595ddf6ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07ca418740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf6b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07ca418740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf77170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07ca418740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5595ddd26700 con 0x5595ddb8f1b0 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07ca418740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5595dde60160 con 0x5595ddb97390 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07ca418740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5595ddf6de80 con 0x5595ddbb8070 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07ca418740 10 client.0 _ll_register_callbacks cb 0x5595dde799e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c1c40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 0x5595ddf6ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c2441640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf77170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c1c40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 0x5595ddf6ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35998/0 (socket says 172.21.15.112:35998) 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c1c40640 1 -- 172.21.15.112:0/3109410307 learned_addr learned my addr 172.21.15.112:0/3109410307 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c2441640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf77170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35254/0 (socket says 172.21.15.112:35254) 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c143f640 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf6b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c1c40640 1 -- 172.21.15.112:0/3109410307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 msgr2=0x5595ddf77170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c1c40640 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf77170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c1c40640 1 -- 172.21.15.112:0/3109410307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 msgr2=0x5595ddf6b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:40.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c1c40640 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf6b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c1c40640 1 -- 172.21.15.112:0/3109410307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595dde8dc90 con 0x5595ddb8f1b0 2024-09-21T03:35:40.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07c1c40640 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 0x5595ddf6ada0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f07bc06ae30 tx=0x7f07b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b1f9433800a474 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:40.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.683+0000 7f07a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:35:40.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07a57fa640 1 -- 172.21.15.112:0/3109410307 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b804c030 con 0x5595ddb8f1b0 2024-09-21T03:35:40.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07a57fa640 1 -- 172.21.15.112:0/3109410307 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f07b8004250 con 0x5595ddb8f1b0 2024-09-21T03:35:40.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07ca418740 1 -- 172.21.15.112:0/3109410307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b4004d20 con 0x5595ddb8f1b0 2024-09-21T03:35:40.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07a57fa640 1 -- 172.21.15.112:0/3109410307 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07b8035040 con 0x5595ddb8f1b0 2024-09-21T03:35:40.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07ca418740 10 client.16662 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:40.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07ca418740 1 -- 172.21.15.112:0/3109410307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595ddf79bf0 con 0x5595ddb8f1b0 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07a57fa640 1 -- 172.21.15.112:0/3109410307 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b8004450 con 0x5595ddb8f1b0 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07a57fa640 1 -- 172.21.15.112:0/3109410307 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f07b805a680 con 0x5595ddb8f1b0 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07ca418740 10 client.16662 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07ca418740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07ca418740 20 client.16662 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07ca418740 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5595ddb769f0 0x5595ddf7d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07ca418740 4 client.16662 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.684+0000 7f07ca418740 1 -- 172.21.15.112:0/3109410307 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f07b4004d20 con 0x5595ddb769f0 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.685+0000 7f07c143f640 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5595ddb769f0 0x5595ddf7d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.685+0000 7f07a57fa640 10 client.16662 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.685+0000 7f07c143f640 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5595ddb769f0 0x5595ddf7d0f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37bb92251801ea29 server_cookie=6892dc1a6116e2ac in_seq=0 out_seq=0 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07a57fa640 1 -- 172.21.15.112:0/3109410307 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f07b4004d20 con 0x5595ddb769f0 2024-09-21T03:35:40.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07a57fa640 10 client.16662 handle_command_reply: tid=0 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07ca418740 1 client.16662 shutdown 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07ca418740 1 -- 172.21.15.112:0/3109410307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 msgr2=0x5595ddf6ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07ca418740 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 0x5595ddf6ada0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f07bc06ae30 tx=0x7f07b8009260 comp rx=0 tx=0).stop 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07ca418740 1 -- 172.21.15.112:0/3109410307 shutdown_connections 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07ca418740 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5595ddb769f0 0x5595ddf7d0f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07ca418740 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf77170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07ca418740 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf6b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07ca418740 1 --2- 172.21.15.112:0/3109410307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb8f1b0 0x5595ddf6ada0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07ca418740 1 -- 172.21.15.112:0/3109410307 >> 172.21.15.112:0/3109410307 conn(0x5595ddbac460 msgr2=0x5595ddf62830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.686+0000 7f07a57fa640 0 client.16662 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.687+0000 7f07ca418740 1 -- 172.21.15.112:0/3109410307 shutdown_connections 2024-09-21T03:35:40.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.687+0000 7f07ca418740 1 -- 172.21.15.112:0/3109410307 wait complete. 2024-09-21T03:35:40.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.687+0000 7f07ca418740 20 client.16662 trim_cache size 0 max 16384 2024-09-21T03:35:40.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.700+0000 7f07ca418740 1 Processor -- start 2024-09-21T03:35:40.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.700+0000 7f07ca418740 1 -- start start 2024-09-21T03:35:40.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07ca418740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb8f1b0 0x5595ddf60de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07ca418740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb97390 0x5595ddf67f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07ca418740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07ca418740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1b2010 con 0x5595ddb97390 2024-09-21T03:35:40.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07ca418740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1af8e0 con 0x5595ddb8f1b0 2024-09-21T03:35:40.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07ca418740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1ad750 con 0x5595ddbb8070 2024-09-21T03:35:40.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c143f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb97390 0x5595ddf67f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c2441640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c1c40640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb8f1b0 0x5595ddf60de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c1c40640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb8f1b0 0x5595ddf60de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34066/0 (socket says 172.21.15.112:34066) 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c2441640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf684d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35270/0 (socket says 172.21.15.112:35270) 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c1c40640 1 -- 172.21.15.112:0/1648423546 learned_addr learned my addr 172.21.15.112:0/1648423546 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c143f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb97390 0x5595ddf67f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36006/0 (socket says 172.21.15.112:36006) 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c1c40640 1 -- 172.21.15.112:0/1648423546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 msgr2=0x5595ddf684d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c1c40640 1 --2- 172.21.15.112:0/1648423546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf684d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c1c40640 1 -- 172.21.15.112:0/1648423546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb97390 msgr2=0x5595ddf67f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c1c40640 1 --2- 172.21.15.112:0/1648423546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb97390 0x5595ddf67f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c1c40640 1 -- 172.21.15.112:0/1648423546 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595ddf6c060 con 0x5595ddb8f1b0 2024-09-21T03:35:40.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07c1c40640 1 --2- 172.21.15.112:0/1648423546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb8f1b0 0x5595ddf60de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07b805ab20 tx=0x7f07b802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=743029e33b5d45b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:40.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.701+0000 7f07a77fe640 1 -- 172.21.15.112:0/1648423546 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b804c040 con 0x5595ddb8f1b0 2024-09-21T03:35:40.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.702+0000 7f07a77fe640 1 -- 172.21.15.112:0/1648423546 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b805f070 con 0x5595ddb8f1b0 2024-09-21T03:35:40.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.702+0000 7f07a77fe640 1 -- 172.21.15.112:0/1648423546 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8003b50 con 0x5595ddb8f1b0 2024-09-21T03:35:40.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.702+0000 7f07ca418740 1 -- 172.21.15.112:0/1648423546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb8f1b0 msgr2=0x5595ddf60de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:40.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.702+0000 7f07ca418740 1 --2- 172.21.15.112:0/1648423546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb8f1b0 0x5595ddf60de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07b805ab20 tx=0x7f07b802a410 comp rx=0 tx=0).stop 2024-09-21T03:35:40.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.703+0000 7f07ca418740 1 -- 172.21.15.112:0/1648423546 shutdown_connections 2024-09-21T03:35:40.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.703+0000 7f07ca418740 1 --2- 172.21.15.112:0/1648423546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddbb8070 0x5595ddf684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.703+0000 7f07ca418740 1 --2- 172.21.15.112:0/1648423546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddb97390 0x5595ddf67f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.703+0000 7f07ca418740 1 --2- 172.21.15.112:0/1648423546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb8f1b0 0x5595ddf60de0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.703+0000 7f07ca418740 1 -- 172.21.15.112:0/1648423546 >> 172.21.15.112:0/1648423546 conn(0x5595ddbac460 msgr2=0x5595ddf606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.703+0000 7f07ca418740 1 -- 172.21.15.112:0/1648423546 shutdown_connections 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.703+0000 7f07ca418740 1 -- 172.21.15.112:0/1648423546 wait complete. 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.703+0000 7f07ca418740 1 -- start start 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.703+0000 7f07ca418740 1 Processor -- start 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07ca418740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddb8f1b0 0x5595ddf711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07ca418740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07ca418740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddbb8070 0x5595ddf71c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07ca418740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1adda0 con 0x5595ddbb8070 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07ca418740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1af210 con 0x5595ddb97390 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07ca418740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1ae470 con 0x5595ddb8f1b0 2024-09-21T03:35:40.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07ca418740 10 client.0 _ll_register_callbacks cb 0x7f07b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c2441640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddbb8070 0x5595ddf71c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c1c40640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddb8f1b0 0x5595ddf711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c2441640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddbb8070 0x5595ddf71c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36018/0 (socket says 172.21.15.112:36018) 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c2441640 1 -- 172.21.15.112:0/2743519101 learned_addr learned my addr 172.21.15.112:0/2743519101 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c1c40640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddb8f1b0 0x5595ddf711b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35280/0 (socket says 172.21.15.112:35280) 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c143f640 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c2441640 1 -- 172.21.15.112:0/2743519101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddb8f1b0 msgr2=0x5595ddf711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c2441640 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddb8f1b0 0x5595ddf711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c2441640 1 -- 172.21.15.112:0/2743519101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 msgr2=0x5595ddf716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c2441640 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c2441640 1 -- 172.21.15.112:0/2743519101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595ddfe6b70 con 0x5595ddbb8070 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c1c40640 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddb8f1b0 0x5595ddf711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07c2441640 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddbb8070 0x5595ddf71c30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f07b4010370 tx=0x7f07b4009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3858bf1db3a615a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:40.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.704+0000 7f07a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.705+0000 7f07a57fa640 1 -- 172.21.15.112:0/2743519101 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b402a070 con 0x5595ddbb8070 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.705+0000 7f07a57fa640 1 -- 172.21.15.112:0/2743519101 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f07b4012710 con 0x5595ddbb8070 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.705+0000 7f07ca418740 1 -- 172.21.15.112:0/2743519101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b805f070 con 0x5595ddbb8070 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.705+0000 7f07a57fa640 1 -- 172.21.15.112:0/2743519101 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07b4036050 con 0x5595ddbb8070 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.705+0000 7f07ca418740 10 client.16668 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.705+0000 7f07ca418740 1 -- 172.21.15.112:0/2743519101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595ddfe7dd0 con 0x5595ddbb8070 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.705+0000 7f07a57fa640 1 -- 172.21.15.112:0/2743519101 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b805f070 con 0x5595ddbb8070 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.705+0000 7f07a57fa640 1 -- 172.21.15.112:0/2743519101 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f07b400f040 con 0x5595ddbb8070 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.705+0000 7f07ca418740 10 client.16668 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.705+0000 7f07ca418740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.705+0000 7f07ca418740 20 client.16668 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.706+0000 7f07ca418740 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5595ddb4bda0 0x5595ddfea200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.706+0000 7f07ca418740 4 client.16668 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:35:40.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.706+0000 7f07ca418740 1 -- 172.21.15.112:0/2743519101 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595dded7df0 con 0x5595ddb4bda0 2024-09-21T03:35:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.706+0000 7f07c1c40640 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5595ddb4bda0 0x5595ddfea200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.706+0000 7f07a57fa640 10 client.16668 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:40.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.706+0000 7f07c1c40640 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5595ddb4bda0 0x5595ddfea200 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62359b3dd1f1055 server_cookie=581f44d5f944c8d9 in_seq=0 out_seq=0 2024-09-21T03:35:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07a57fa640 1 -- 172.21.15.112:0/2743519101 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595dded7df0 con 0x5595ddb4bda0 2024-09-21T03:35:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07a57fa640 10 client.16668 handle_command_reply: tid=0 2024-09-21T03:35:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 1 client.16668 shutdown 2024-09-21T03:35:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 1 -- 172.21.15.112:0/2743519101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddbb8070 msgr2=0x5595ddf71c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddbb8070 0x5595ddf71c30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f07b4010370 tx=0x7f07b4009d70 comp rx=0 tx=0).stop 2024-09-21T03:35:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 1 -- 172.21.15.112:0/2743519101 shutdown_connections 2024-09-21T03:35:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5595ddb4bda0 0x5595ddfea200 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595ddbb8070 0x5595ddf71c30 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595ddb97390 0x5595ddf716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07a57fa640 0 client.16668 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:40.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 1 --2- 172.21.15.112:0/2743519101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5595ddb8f1b0 0x5595ddf711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 1 -- 172.21.15.112:0/2743519101 >> 172.21.15.112:0/2743519101 conn(0x5595ddbac460 msgr2=0x5595ddf62f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:40.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 1 -- 172.21.15.112:0/2743519101 shutdown_connections 2024-09-21T03:35:40.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 1 -- 172.21.15.112:0/2743519101 wait complete. 2024-09-21T03:35:40.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.707+0000 7f07ca418740 20 client.16668 trim_cache size 0 max 16384 2024-09-21T03:35:40.714 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:35:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 -- 172.21.15.112:0/3161288812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07a80b6880 msgr2=0x7f07a80b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07a80b6880 0x7f07a80b8d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f07b8004770 tx=0x7f07b80058d0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 -- 172.21.15.112:0/3161288812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07bc106280 msgr2=0x7f07bc19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07bc106280 0x7f07bc19aa50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00e970 tx=0x7f07ac00ee30 comp rx=0 tx=0).stop 2024-09-21T03:35:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 -- 172.21.15.112:0/3161288812 shutdown_connections 2024-09-21T03:35:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07a80b6880 0x7f07a80b8d90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07bc106bd0 0x7f07bc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07bc106280 0x7f07bc19aa50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 --2- 172.21.15.112:0/3161288812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07bc102030 0x7f07bc19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:40.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 -- 172.21.15.112:0/3161288812 >> 172.21.15.112:0/3161288812 conn(0x7f07bc077c60 msgr2=0x7f07bc107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:40.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 -- 172.21.15.112:0/3161288812 shutdown_connections 2024-09-21T03:35:40.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:40.708+0000 7f07c3ea4640 1 -- 172.21.15.112:0/3161288812 wait complete. 2024-09-21T03:35:41.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 689928 10.83 MB/sec execute 221 sec latency 61.427 ms 2024-09-21T03:35:41.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:41 smithi112 ceph-mon[31442]: pgmap v520: 97 pgs: 97 active+clean; 480 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 312 op/s 2024-09-21T03:35:41.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:41 smithi203 ceph-mon[34723]: pgmap v520: 97 pgs: 97 active+clean; 480 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 312 op/s 2024-09-21T03:35:41.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:41 smithi187 ceph-mon[34603]: pgmap v520: 97 pgs: 97 active+clean; 480 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 312 op/s 2024-09-21T03:35:42.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 692703 10.86 MB/sec execute 222 sec latency 19.739 ms 2024-09-21T03:35:42.483 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:35:42.483 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:35:42.744 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:43.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 694150 10.84 MB/sec execute 223 sec latency 19.548 ms 2024-09-21T03:35:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.244+0000 7fac63ea3640 1 -- 172.21.15.112:0/3165544558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5c102ae0 msgr2=0x7fac5c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.244+0000 7fac63ea3640 1 --2- 172.21.15.112:0/3165544558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5c102ae0 0x7fac5c102ef0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fac58009900 tx=0x7fac5802a410 comp rx=0 tx=0).stop 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.244+0000 7fac63ea3640 1 -- 172.21.15.112:0/3165544558 shutdown_connections 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.244+0000 7fac63ea3640 1 --2- 172.21.15.112:0/3165544558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5c108d20 0x7fac5c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.244+0000 7fac63ea3640 1 --2- 172.21.15.112:0/3165544558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5c1083d0 0x7fac5c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.244+0000 7fac63ea3640 1 --2- 172.21.15.112:0/3165544558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5c102ae0 0x7fac5c102ef0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.244+0000 7fac63ea3640 1 -- 172.21.15.112:0/3165544558 >> 172.21.15.112:0/3165544558 conn(0x7fac5c079760 msgr2=0x7fac5c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.244+0000 7fac63ea3640 1 -- 172.21.15.112:0/3165544558 shutdown_connections 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.244+0000 7fac63ea3640 1 -- 172.21.15.112:0/3165544558 wait complete. 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.245+0000 7fac63ea3640 1 Processor -- start 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.245+0000 7fac63ea3640 1 -- start start 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.245+0000 7fac63ea3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5c102ae0 0x7fac5c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.245+0000 7fac63ea3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5c1083d0 0x7fac5c19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.245+0000 7fac63ea3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5c108d20 0x7fac5c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.245+0000 7fac63ea3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1129e0 con 0x7fac5c1083d0 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.245+0000 7fac63ea3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c112860 con 0x7fac5c108d20 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.245+0000 7fac63ea3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c112b60 con 0x7fac5c102ae0 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.245+0000 7fac6143e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5c1083d0 0x7fac5c19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac6143e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5c1083d0 0x7fac5c19b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36038/0 (socket says 172.21.15.112:36038) 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac6143e640 1 -- 172.21.15.112:0/818377667 learned_addr learned my addr 172.21.15.112:0/818377667 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac62440640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5c108d20 0x7fac5c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac61c3f640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5c102ae0 0x7fac5c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac6143e640 1 -- 172.21.15.112:0/818377667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5c102ae0 msgr2=0x7fac5c19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac6143e640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5c102ae0 0x7fac5c19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac6143e640 1 -- 172.21.15.112:0/818377667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5c108d20 msgr2=0x7fac5c19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac6143e640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5c108d20 0x7fac5c19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac6143e640 1 -- 172.21.15.112:0/818377667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5c1a6570 con 0x7fac5c1083d0 2024-09-21T03:35:43.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac62440640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5c108d20 0x7fac5c19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:43.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac6143e640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5c1083d0 0x7fac5c19b7f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00e9a0 tx=0x7fac4c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:43.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac52ffd640 1 -- 172.21.15.112:0/818377667 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00cd80 con 0x7fac5c1083d0 2024-09-21T03:35:43.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac52ffd640 1 -- 172.21.15.112:0/818377667 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4c0035f0 con 0x7fac5c1083d0 2024-09-21T03:35:43.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac63ea3640 1 -- 172.21.15.112:0/818377667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5c1a6860 con 0x7fac5c1083d0 2024-09-21T03:35:43.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac52ffd640 1 -- 172.21.15.112:0/818377667 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c010690 con 0x7fac5c1083d0 2024-09-21T03:35:43.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.246+0000 7fac63ea3640 1 -- 172.21.15.112:0/818377667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5c1a6d70 con 0x7fac5c1083d0 2024-09-21T03:35:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.249+0000 7fac6a417740 1 Processor -- start 2024-09-21T03:35:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.249+0000 7fac6a417740 1 -- start start 2024-09-21T03:35:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.249+0000 7fac52ffd640 1 -- 172.21.15.112:0/818377667 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fac4c010830 con 0x7fac5c1083d0 2024-09-21T03:35:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac6a417740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210c5350 0x55ab2146c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac6a417740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 0x55ab2146c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac6a417740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210a4670 0x55ab2146ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac6a417740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ab213e2e60 con 0x55ab2109c490 2024-09-21T03:35:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac6a417740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ab213d2290 con 0x55ab210a4670 2024-09-21T03:35:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac6a417740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ab213d60a0 con 0x55ab210c5350 2024-09-21T03:35:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac62440640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210c5350 0x55ab2146c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac61c3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 0x55ab2146c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac62440640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210c5350 0x55ab2146c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35310/0 (socket says 172.21.15.112:35310) 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac62440640 1 -- 172.21.15.112:0/1522469879 learned_addr learned my addr 172.21.15.112:0/1522469879 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac6143e640 1 --2- 172.21.15.112:0/1522469879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210a4670 0x55ab2146ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac62440640 1 -- 172.21.15.112:0/1522469879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210a4670 msgr2=0x55ab2146ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac62440640 1 --2- 172.21.15.112:0/1522469879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210a4670 0x55ab2146ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac62440640 1 -- 172.21.15.112:0/1522469879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 msgr2=0x55ab2146c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac62440640 1 --2- 172.21.15.112:0/1522469879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 0x55ab2146c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac62440640 1 -- 172.21.15.112:0/1522469879 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab21130930 con 0x55ab210c5350 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac6143e640 1 --2- 172.21.15.112:0/1522469879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210a4670 0x55ab2146ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac62440640 1 --2- 172.21.15.112:0/1522469879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210c5350 0x55ab2146c430 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fac54011790 tx=0x7fac54011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e3e1ab6e5a1bb59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.250+0000 7fac52ffd640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fac480b6680 0x7fac480b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.251+0000 7fac47fff640 1 -- 172.21.15.112:0/1522469879 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54003c30 con 0x55ab210c5350 2024-09-21T03:35:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.251+0000 7fac61c3f640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fac480b6680 0x7fac480b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.251+0000 7fac52ffd640 1 -- 172.21.15.112:0/818377667 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7fac4c0e7980 con 0x7fac5c1083d0 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.251+0000 7fac47fff640 1 -- 172.21.15.112:0/1522469879 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac5400fe50 con 0x55ab210c5350 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.251+0000 7fac47fff640 1 -- 172.21.15.112:0/1522469879 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac540193e0 con 0x55ab210c5350 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac6a417740 1 -- 172.21.15.112:0/1522469879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210c5350 msgr2=0x55ab2146c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac61c3f640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fac480b6680 0x7fac480b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fac58005ab0 tx=0x7fac58005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac6a417740 1 --2- 172.21.15.112:0/1522469879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210c5350 0x55ab2146c430 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fac54011790 tx=0x7fac54011c50 comp rx=0 tx=0).stop 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac6a417740 1 -- 172.21.15.112:0/1522469879 shutdown_connections 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac6a417740 1 --2- 172.21.15.112:0/1522469879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210a4670 0x55ab2146ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac6a417740 1 --2- 172.21.15.112:0/1522469879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 0x55ab2146c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac6a417740 1 --2- 172.21.15.112:0/1522469879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210c5350 0x55ab2146c430 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac6a417740 1 -- 172.21.15.112:0/1522469879 >> 172.21.15.112:0/1522469879 conn(0x55ab210b9740 msgr2=0x55ab2146a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac6a417740 1 -- 172.21.15.112:0/1522469879 shutdown_connections 2024-09-21T03:35:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac6a417740 1 -- 172.21.15.112:0/1522469879 wait complete. 2024-09-21T03:35:43.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac6a417740 1 -- start start 2024-09-21T03:35:43.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.252+0000 7fac6a417740 1 Processor -- start 2024-09-21T03:35:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6a417740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 0x55ab21477f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6a417740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210a4670 0x55ab214784c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6a417740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210c5350 0x55ab21483f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6a417740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ab21233890 con 0x55ab2109c490 2024-09-21T03:35:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6a417740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ab2136cd40 con 0x55ab210c5350 2024-09-21T03:35:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6a417740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ab2147ac10 con 0x55ab210a4670 2024-09-21T03:35:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac61c3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 0x55ab21477f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac61c3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 0x55ab21477f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36070/0 (socket says 172.21.15.112:36070) 2024-09-21T03:35:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6143e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210a4670 0x55ab214784c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac61c3f640 1 -- 172.21.15.112:0/132243985 learned_addr learned my addr 172.21.15.112:0/132243985 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac62440640 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210c5350 0x55ab21483f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6a417740 10 client.0 _ll_register_callbacks cb 0x55ab213d0c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6143e640 1 -- 172.21.15.112:0/132243985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210c5350 msgr2=0x55ab21483f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6143e640 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210c5350 0x55ab21483f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6143e640 1 -- 172.21.15.112:0/132243985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 msgr2=0x55ab21477f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6143e640 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 0x55ab21477f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6143e640 1 -- 172.21.15.112:0/132243985 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab21484470 con 0x55ab210a4670 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac61c3f640 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 0x55ab21477f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac6143e640 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210a4670 0x55ab214784c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0e5100 tx=0x7fac4c003cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2373a80bee6d1d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.253+0000 7fac457fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac457fa640 1 -- 172.21.15.112:0/132243985 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c0a96a0 con 0x55ab210a4670 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac6a417740 1 -- 172.21.15.112:0/132243985 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac5400fe50 con 0x55ab210a4670 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac457fa640 1 -- 172.21.15.112:0/132243985 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7fac4c010b20 con 0x55ab210a4670 2024-09-21T03:35:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac457fa640 1 -- 172.21.15.112:0/132243985 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fac4c0acf70 con 0x55ab210a4670 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac6a417740 10 client.26239 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac6a417740 1 -- 172.21.15.112:0/132243985 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ab21486d60 con 0x55ab210a4670 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac457fa640 1 -- 172.21.15.112:0/132243985 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4c0d2070 con 0x55ab210a4670 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac457fa640 1 -- 172.21.15.112:0/132243985 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fac4c003fb0 con 0x55ab210a4670 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac6a417740 10 client.26239 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac6a417740 10 client.26239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac6a417740 20 client.26239 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac6a417740 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ab21083cd0 0x55ab2148a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac6a417740 4 client.26239 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.254+0000 7fac6a417740 1 -- 172.21.15.112:0/132243985 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab213e4db0 con 0x55ab21083cd0 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.255+0000 7fac61c3f640 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ab21083cd0 0x55ab2148a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.255+0000 7fac457fa640 10 client.26239 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.255+0000 7fac61c3f640 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ab21083cd0 0x55ab2148a790 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8ab8e5c8b8985e server_cookie=adc062692bfaecc4 in_seq=0 out_seq=0 2024-09-21T03:35:43.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.256+0000 7fac457fa640 1 -- 172.21.15.112:0/132243985 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab213e4db0 con 0x55ab21083cd0 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.256+0000 7fac457fa640 10 client.26239 handle_command_reply: tid=0 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.256+0000 7fac6a417740 1 client.26239 shutdown 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.256+0000 7fac6a417740 1 -- 172.21.15.112:0/132243985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210a4670 msgr2=0x55ab214784c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.256+0000 7fac6a417740 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210a4670 0x55ab214784c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0e5100 tx=0x7fac4c003cb0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.256+0000 7fac6a417740 1 -- 172.21.15.112:0/132243985 shutdown_connections 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.256+0000 7fac6a417740 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ab21083cd0 0x55ab2148a790 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.256+0000 7fac6a417740 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210c5350 0x55ab21483f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.256+0000 7fac6a417740 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab210a4670 0x55ab214784c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.256+0000 7fac6a417740 1 --2- 172.21.15.112:0/132243985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab2109c490 0x55ab21477f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.256+0000 7fac6a417740 1 -- 172.21.15.112:0/132243985 >> 172.21.15.112:0/132243985 conn(0x55ab210b9740 msgr2=0x55ab2146f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.257+0000 7fac457fa640 0 client.26239 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.257+0000 7fac6a417740 1 -- 172.21.15.112:0/132243985 shutdown_connections 2024-09-21T03:35:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.257+0000 7fac6a417740 1 -- 172.21.15.112:0/132243985 wait complete. 2024-09-21T03:35:43.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.257+0000 7fac6a417740 20 client.26239 trim_cache size 0 max 16384 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac6a417740 1 Processor -- start 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac6a417740 1 -- start start 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac6a417740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab2109c490 0x55ab21483e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac6a417740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210a4670 0x55ab21476600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac6a417740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210c5350 0x55ab21476b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac6a417740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1b1ea0 con 0x55ab210c5350 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac6a417740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1af770 con 0x55ab210a4670 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac6a417740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1ad5e0 con 0x55ab2109c490 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac62440640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210c5350 0x55ab21476b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac62440640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210c5350 0x55ab21476b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36080/0 (socket says 172.21.15.112:36080) 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac62440640 1 -- 172.21.15.112:0/2825671082 learned_addr learned my addr 172.21.15.112:0/2825671082 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac6143e640 1 --2- 172.21.15.112:0/2825671082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210a4670 0x55ab21476600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac61c3f640 1 --2- 172.21.15.112:0/2825671082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab2109c490 0x55ab21483e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac62440640 1 -- 172.21.15.112:0/2825671082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab2109c490 msgr2=0x55ab21483e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:43.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac62440640 1 --2- 172.21.15.112:0/2825671082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab2109c490 0x55ab21483e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac62440640 1 -- 172.21.15.112:0/2825671082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210a4670 msgr2=0x55ab21476600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:43.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac62440640 1 --2- 172.21.15.112:0/2825671082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210a4670 0x55ab21476600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.276+0000 7fac62440640 1 -- 172.21.15.112:0/2825671082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab2146f290 con 0x55ab210c5350 2024-09-21T03:35:43.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.277+0000 7fac62440640 1 --2- 172.21.15.112:0/2825671082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210c5350 0x55ab21476b40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fac5400f790 tx=0x7fac54004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0b4b10ecb17d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:43.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.277+0000 7fac477fe640 1 -- 172.21.15.112:0/2825671082 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5401f040 con 0x55ab210c5350 2024-09-21T03:35:43.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.277+0000 7fac477fe640 1 -- 172.21.15.112:0/2825671082 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac54011030 con 0x55ab210c5350 2024-09-21T03:35:43.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.277+0000 7fac477fe640 1 -- 172.21.15.112:0/2825671082 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54004590 con 0x55ab210c5350 2024-09-21T03:35:43.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.278+0000 7fac6a417740 1 -- 172.21.15.112:0/2825671082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210c5350 msgr2=0x55ab21476b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.278+0000 7fac6a417740 1 --2- 172.21.15.112:0/2825671082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210c5350 0x55ab21476b40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fac5400f790 tx=0x7fac54004290 comp rx=0 tx=0).stop 2024-09-21T03:35:43.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.278+0000 7fac6a417740 1 -- 172.21.15.112:0/2825671082 shutdown_connections 2024-09-21T03:35:43.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.278+0000 7fac6a417740 1 --2- 172.21.15.112:0/2825671082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210c5350 0x55ab21476b40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.278+0000 7fac6a417740 1 --2- 172.21.15.112:0/2825671082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210a4670 0x55ab21476600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.278+0000 7fac6a417740 1 --2- 172.21.15.112:0/2825671082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab2109c490 0x55ab21483e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.278+0000 7fac6a417740 1 -- 172.21.15.112:0/2825671082 >> 172.21.15.112:0/2825671082 conn(0x55ab210b9740 msgr2=0x55ab21472ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:43.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.278+0000 7fac6a417740 1 -- 172.21.15.112:0/2825671082 shutdown_connections 2024-09-21T03:35:43.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.278+0000 7fac6a417740 1 -- 172.21.15.112:0/2825671082 wait complete. 2024-09-21T03:35:43.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.278+0000 7fac6a417740 1 -- start start 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.278+0000 7fac6a417740 1 Processor -- start 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6a417740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab2109c490 0x55ab21480380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6a417740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210a4670 0x55ab2147b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6a417740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210c5350 0x55ab2147bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6a417740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1adc30 con 0x55ab210a4670 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6a417740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1af0a0 con 0x55ab210c5350 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6a417740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1ae300 con 0x55ab2109c490 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6a417740 10 client.0 _ll_register_callbacks cb 0x7fac4c00ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6143e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210a4670 0x55ab2147b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6143e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210a4670 0x55ab2147b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36088/0 (socket says 172.21.15.112:36088) 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac61c3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab2109c490 0x55ab21480380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6143e640 1 -- 172.21.15.112:0/2763716062 learned_addr learned my addr 172.21.15.112:0/2763716062 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac62440640 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210c5350 0x55ab2147bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6143e640 1 -- 172.21.15.112:0/2763716062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab2109c490 msgr2=0x55ab21480380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6143e640 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab2109c490 0x55ab21480380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6143e640 1 -- 172.21.15.112:0/2763716062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210c5350 msgr2=0x55ab2147bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6143e640 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210c5350 0x55ab2147bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac6143e640 1 -- 172.21.15.112:0/2763716062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab2147c2b0 con 0x55ab210a4670 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.279+0000 7fac61c3f640 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab2109c490 0x55ab21480380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac6143e640 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210a4670 0x55ab2147b830 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0ad790 tx=0x7fac4c0ac4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9ebbe236089b8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac457fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac457fa640 1 -- 172.21.15.112:0/2763716062 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c0ad150 con 0x55ab210a4670 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac457fa640 1 -- 172.21.15.112:0/2763716062 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7fac4c0aeb30 con 0x55ab210a4670 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac6a417740 1 -- 172.21.15.112:0/2763716062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac54011030 con 0x55ab210a4670 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac457fa640 1 -- 172.21.15.112:0/2763716062 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fac4c0cf050 con 0x55ab210a4670 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac6a417740 10 client.16686 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac6a417740 1 -- 172.21.15.112:0/2763716062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ab214f4e10 con 0x55ab210a4670 2024-09-21T03:35:43.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac457fa640 1 -- 172.21.15.112:0/2763716062 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4c0c8e50 con 0x55ab210a4670 2024-09-21T03:35:43.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac457fa640 1 -- 172.21.15.112:0/2763716062 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fac4c0b3470 con 0x55ab210a4670 2024-09-21T03:35:43.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac6a417740 10 client.16686 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:43.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac6a417740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:35:43.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac6a417740 20 client.16686 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:43.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac6a417740 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ab21059080 0x55ab214f7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:43.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac6a417740 4 client.16686 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:35:43.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.280+0000 7fac6a417740 1 -- 172.21.15.112:0/2763716062 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ab213e0860 con 0x55ab21059080 2024-09-21T03:35:43.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.281+0000 7fac61c3f640 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ab21059080 0x55ab214f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:43.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.281+0000 7fac457fa640 10 client.16686 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:43.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.281+0000 7fac61c3f640 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ab21059080 0x55ab214f7440 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f3735da26b9c73 server_cookie=496947e318394284 in_seq=0 out_seq=0 2024-09-21T03:35:43.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.282+0000 7fac457fa640 1 -- 172.21.15.112:0/2763716062 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ab213e0860 con 0x55ab21059080 2024-09-21T03:35:43.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.282+0000 7fac457fa640 10 client.16686 handle_command_reply: tid=0 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.282+0000 7fac6a417740 1 client.16686 shutdown 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.282+0000 7fac6a417740 1 -- 172.21.15.112:0/2763716062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210a4670 msgr2=0x55ab2147b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.282+0000 7fac6a417740 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210a4670 0x55ab2147b830 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0ad790 tx=0x7fac4c0ac4d0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.282+0000 7fac6a417740 1 -- 172.21.15.112:0/2763716062 shutdown_connections 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.282+0000 7fac6a417740 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ab21059080 0x55ab214f7440 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac457fa640 0 client.16686 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac6a417740 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ab210c5350 0x55ab2147bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac6a417740 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ab210a4670 0x55ab2147b830 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac6a417740 1 --2- 172.21.15.112:0/2763716062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ab2109c490 0x55ab21480380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac6a417740 1 -- 172.21.15.112:0/2763716062 >> 172.21.15.112:0/2763716062 conn(0x55ab210b9740 msgr2=0x55ab214761d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac6a417740 1 -- 172.21.15.112:0/2763716062 shutdown_connections 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac6a417740 1 -- 172.21.15.112:0/2763716062 wait complete. 2024-09-21T03:35:43.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac6a417740 20 client.16686 trim_cache size 0 max 16384 2024-09-21T03:35:43.289 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:35:43.289 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:35:43.289 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27", 2024-09-21T03:35:43.289 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:35:43.289 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac63ea3640 1 -- 172.21.15.112:0/818377667 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fac480b6680 msgr2=0x7fac480b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac63ea3640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fac480b6680 0x7fac480b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fac58005ab0 tx=0x7fac58005a20 comp rx=0 tx=0).stop 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac63ea3640 1 -- 172.21.15.112:0/818377667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5c1083d0 msgr2=0x7fac5c19b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac63ea3640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5c1083d0 0x7fac5c19b7f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00e9a0 tx=0x7fac4c00ee60 comp rx=0 tx=0).stop 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac63ea3640 1 -- 172.21.15.112:0/818377667 shutdown_connections 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac63ea3640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fac480b6680 0x7fac480b8b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac63ea3640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5c108d20 0x7fac5c19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac63ea3640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac5c1083d0 0x7fac5c19b7f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac63ea3640 1 --2- 172.21.15.112:0/818377667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5c102ae0 0x7fac5c19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.283+0000 7fac63ea3640 1 -- 172.21.15.112:0/818377667 >> 172.21.15.112:0/818377667 conn(0x7fac5c079760 msgr2=0x7fac5c101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.284+0000 7fac63ea3640 1 -- 172.21.15.112:0/818377667 shutdown_connections 2024-09-21T03:35:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:43.284+0000 7fac63ea3640 1 -- 172.21.15.112:0/818377667 wait complete. 2024-09-21T03:35:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:43 smithi112 ceph-mon[31442]: pgmap v521: 97 pgs: 97 active+clean; 481 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-09-21T03:35:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:43 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:35:43.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:43 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:43.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:43 smithi203 ceph-mon[34723]: pgmap v521: 97 pgs: 97 active+clean; 481 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-09-21T03:35:43.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:43 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:35:43.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:43 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:43.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:43 smithi187 ceph-mon[34603]: pgmap v521: 97 pgs: 97 active+clean; 481 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-09-21T03:35:43.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:43 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:35:43.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:43 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:44.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 696260 10.85 MB/sec execute 224 sec latency 61.431 ms 2024-09-21T03:35:44.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:35:44.519 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:35:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:44 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:35:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:44 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:44 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:35:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:44 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:35:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:44 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:44 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:44.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:44 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:35:44.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:44 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:44.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:44 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:35:44.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:44 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:35:44.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:44 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:44.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:44 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:44.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:44 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:35:44.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:44 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:44.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:44 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:35:44.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:44 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:35:44.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:44 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:44.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:44 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:35:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.127+0000 7f447db8a640 1 -- 172.21.15.112:0/3764329165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478105af0 msgr2=0x7f447810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.127+0000 7f447db8a640 1 --2- 172.21.15.112:0/3764329165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478105af0 0x7f447810f560 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4470009ca0 tx=0x7f447002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.127+0000 7f447db8a640 1 -- 172.21.15.112:0/3764329165 shutdown_connections 2024-09-21T03:35:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.127+0000 7f447db8a640 1 --2- 172.21.15.112:0/3764329165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478105af0 0x7f447810f560 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.127+0000 7f447db8a640 1 --2- 172.21.15.112:0/3764329165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44781051a0 0x7f44781055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.127+0000 7f447db8a640 1 --2- 172.21.15.112:0/3764329165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4478103120 0x7f4478103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.127+0000 7f447db8a640 1 -- 172.21.15.112:0/3764329165 >> 172.21.15.112:0/3764329165 conn(0x7f4478077c60 msgr2=0x7f44780780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.128+0000 7f447db8a640 1 -- 172.21.15.112:0/3764329165 shutdown_connections 2024-09-21T03:35:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.128+0000 7f447db8a640 1 -- 172.21.15.112:0/3764329165 wait complete. 2024-09-21T03:35:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.128+0000 7f447db8a640 1 Processor -- start 2024-09-21T03:35:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.128+0000 7f447db8a640 1 -- start start 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f447db8a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4478103120 0x7f447819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f447db8a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44781051a0 0x7f447819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f447db8a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478105af0 0x7f447819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f447db8a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44781129c0 con 0x7f4478105af0 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f447db8a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4478112840 con 0x7f4478103120 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f447db8a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4478112b40 con 0x7f44781051a0 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f4477fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478105af0 0x7f447819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44781051a0 0x7f447819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44781051a0 0x7f447819b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35380/0 (socket says 172.21.15.112:35380) 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f4477fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478105af0 0x7f447819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36108/0 (socket says 172.21.15.112:36108) 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f4476ffd640 1 -- 172.21.15.112:0/598607086 learned_addr learned my addr 172.21.15.112:0/598607086 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f44777fe640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4478103120 0x7f447819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f4476ffd640 1 -- 172.21.15.112:0/598607086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4478103120 msgr2=0x7f447819b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f4476ffd640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4478103120 0x7f447819b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f4476ffd640 1 -- 172.21.15.112:0/598607086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478105af0 msgr2=0x7f447819bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f4476ffd640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478105af0 0x7f447819bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f4476ffd640 1 -- 172.21.15.112:0/598607086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44781a6540 con 0x7f44781051a0 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.129+0000 7f4477fff640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478105af0 0x7f447819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.130+0000 7f4476ffd640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44781051a0 0x7f447819b7f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f446800d740 tx=0x7f446800dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.130+0000 7f4474ff9640 1 -- 172.21.15.112:0/598607086 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468017070 con 0x7f44781051a0 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.130+0000 7f447db8a640 1 -- 172.21.15.112:0/598607086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44781a6830 con 0x7f44781051a0 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.130+0000 7f4474ff9640 1 -- 172.21.15.112:0/598607086 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f446800de50 con 0x7f44781051a0 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.130+0000 7f4474ff9640 1 -- 172.21.15.112:0/598607086 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446800f620 con 0x7f44781051a0 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.130+0000 7f447db8a640 1 -- 172.21.15.112:0/598607086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44781a6dc0 con 0x7f44781051a0 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.133+0000 7f4474ff9640 1 -- 172.21.15.112:0/598607086 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f446801a070 con 0x7f44781051a0 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.133+0000 7f44840fe740 1 Processor -- start 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.133+0000 7f44840fe740 1 -- start start 2024-09-21T03:35:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.133+0000 7f44840fe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a759070 0x56094ab00270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.133+0000 7f44840fe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a7301b0 0x56094ab007b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.133+0000 7f44840fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a738390 0x56094ab00cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f44840fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56094aa76e50 con 0x56094a738390 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f44840fe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56094aa66280 con 0x56094a759070 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f44840fe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56094aa6a090 con 0x56094a7301b0 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f44777fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a7301b0 0x56094ab007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a738390 0x56094ab00cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f4477fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a759070 0x56094ab00270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f44777fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a7301b0 0x56094ab007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:35388/0 (socket says 172.21.15.112:35388) 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f44777fe640 1 -- 172.21.15.112:0/3078783422 learned_addr learned my addr 172.21.15.112:0/3078783422 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f4477fff640 1 -- 172.21.15.112:0/3078783422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a7301b0 msgr2=0x56094ab007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f4477fff640 1 --2- 172.21.15.112:0/3078783422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a7301b0 0x56094ab007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f4477fff640 1 -- 172.21.15.112:0/3078783422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a738390 msgr2=0x56094ab00cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f4477fff640 1 --2- 172.21.15.112:0/3078783422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a738390 0x56094ab00cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f4474ff9640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44600b66b0 0x7f44600b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f4477fff640 1 -- 172.21.15.112:0/3078783422 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094a7c4650 con 0x56094a759070 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.134+0000 7f44777fe640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44600b66b0 0x7f44600b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.135+0000 7f4474ff9640 1 -- 172.21.15.112:0/598607086 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f44680e7010 con 0x7f44781051a0 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.135+0000 7f4477fff640 1 --2- 172.21.15.112:0/3078783422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a759070 0x56094ab00270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4470005e30 tx=0x7f447002f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e209a83c480d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.135+0000 7f44777fe640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44600b66b0 0x7f44600b8bc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x56094aafeef0 tx=0x7f446c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.135+0000 7f445dffb640 1 -- 172.21.15.112:0/3078783422 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4470031650 con 0x56094a759070 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.135+0000 7f445dffb640 1 -- 172.21.15.112:0/3078783422 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4470036040 con 0x56094a759070 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.135+0000 7f445dffb640 1 -- 172.21.15.112:0/3078783422 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4470004400 con 0x56094a759070 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.136+0000 7f44840fe740 1 -- 172.21.15.112:0/3078783422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a759070 msgr2=0x56094ab00270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.136+0000 7f44840fe740 1 --2- 172.21.15.112:0/3078783422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a759070 0x56094ab00270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4470005e30 tx=0x7f447002f3c0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.136+0000 7f44840fe740 1 -- 172.21.15.112:0/3078783422 shutdown_connections 2024-09-21T03:35:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.136+0000 7f44840fe740 1 --2- 172.21.15.112:0/3078783422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a738390 0x56094ab00cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.136+0000 7f44840fe740 1 --2- 172.21.15.112:0/3078783422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a7301b0 0x56094ab007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.136+0000 7f44840fe740 1 --2- 172.21.15.112:0/3078783422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a759070 0x56094ab00270 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.136+0000 7f44840fe740 1 -- 172.21.15.112:0/3078783422 >> 172.21.15.112:0/3078783422 conn(0x56094a74d460 msgr2=0x56094aafe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.136+0000 7f44840fe740 1 -- 172.21.15.112:0/3078783422 shutdown_connections 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.136+0000 7f44840fe740 1 -- 172.21.15.112:0/3078783422 wait complete. 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f44840fe740 1 -- start start 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f44840fe740 1 Processor -- start 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f44840fe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7301b0 0x56094ab0b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f44840fe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094ab0bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f44840fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a759070 0x56094ab0c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f44840fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56094a8c7700 con 0x56094a759070 2024-09-21T03:35:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f44840fe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56094aa01110 con 0x56094a7301b0 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f44840fe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56094ab0e870 con 0x56094a738390 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f44840fe740 10 client.0 _ll_register_callbacks cb 0x56094aa1a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f4477fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a759070 0x56094ab0c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f44777fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7301b0 0x56094ab0b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f4477fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a759070 0x56094ab0c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36118/0 (socket says 172.21.15.112:36118) 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f4477fff640 1 -- 172.21.15.112:0/4231750631 learned_addr learned my addr 172.21.15.112:0/4231750631 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.137+0000 7f4476ffd640 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094ab0bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f44777fe640 1 -- 172.21.15.112:0/4231750631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 msgr2=0x56094ab0bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f44777fe640 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094ab0bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f44777fe640 1 -- 172.21.15.112:0/4231750631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a759070 msgr2=0x56094ab0c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f44777fe640 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a759070 0x56094ab0c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f44777fe640 1 -- 172.21.15.112:0/4231750631 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56094aa2ec40 con 0x56094a7301b0 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f4477fff640 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a759070 0x56094ab0c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:35:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f44777fe640 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7301b0 0x56094ab0b880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f446c01e9d0 tx=0x7f446c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=804eb1015b54fc71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f4456ffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f4456ffd640 1 -- 172.21.15.112:0/4231750631 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c02e060 con 0x56094a7301b0 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f4456ffd640 1 -- 172.21.15.112:0/4231750631 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f446c018030 con 0x56094a7301b0 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f44840fe740 1 -- 172.21.15.112:0/4231750631 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4470036040 con 0x56094a7301b0 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f4456ffd640 1 -- 172.21.15.112:0/4231750631 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f446c009e20 con 0x56094a7301b0 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f44840fe740 10 client.26261 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.138+0000 7f44840fe740 1 -- 172.21.15.112:0/4231750631 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56094ab1a5d0 con 0x56094a7301b0 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.139+0000 7f4456ffd640 1 -- 172.21.15.112:0/4231750631 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4470036040 con 0x56094a7301b0 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.139+0000 7f4456ffd640 1 -- 172.21.15.112:0/4231750631 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f446c01c560 con 0x56094a7301b0 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.139+0000 7f44840fe740 10 client.26261 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.139+0000 7f44840fe740 10 client.26261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.139+0000 7f44840fe740 20 client.26261 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.139+0000 7f44840fe740 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56094a7179f0 0x56094ab1db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.139+0000 7f44840fe740 4 client.26261 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.139+0000 7f44840fe740 1 -- 172.21.15.112:0/4231750631 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56094aa78da0 con 0x56094a7179f0 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.139+0000 7f4476ffd640 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56094a7179f0 0x56094ab1db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.140+0000 7f4456ffd640 10 client.26261 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.140+0000 7f4476ffd640 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56094a7179f0 0x56094ab1db30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f6ab6f5508023f server_cookie=f2b71aa85b95b94f in_seq=0 out_seq=0 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f4456ffd640 1 -- 172.21.15.112:0/4231750631 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56094aa78da0 con 0x56094a7179f0 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f4456ffd640 10 client.26261 handle_command_reply: tid=0 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 1 client.26261 shutdown 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 1 -- 172.21.15.112:0/4231750631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7301b0 msgr2=0x56094ab0b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7301b0 0x56094ab0b880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f446c01e9d0 tx=0x7f446c004730 comp rx=0 tx=0).stop 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 1 -- 172.21.15.112:0/4231750631 shutdown_connections 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56094a7179f0 0x56094ab1db30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f4456ffd640 0 client.26261 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a759070 0x56094ab0c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094ab0bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 1 --2- 172.21.15.112:0/4231750631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7301b0 0x56094ab0b880 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 1 -- 172.21.15.112:0/4231750631 >> 172.21.15.112:0/4231750631 conn(0x56094a74d460 msgr2=0x56094ab03400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 1 -- 172.21.15.112:0/4231750631 shutdown_connections 2024-09-21T03:35:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 1 -- 172.21.15.112:0/4231750631 wait complete. 2024-09-21T03:35:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.142+0000 7f44840fe740 20 client.26261 trim_cache size 0 max 16384 2024-09-21T03:35:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.160+0000 7f44840fe740 1 Processor -- start 2024-09-21T03:35:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.160+0000 7f44840fe740 1 -- start start 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.160+0000 7f44840fe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7179f0 0x56094ab0bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.160+0000 7f44840fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab17a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.160+0000 7f44840fe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094aafd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.160+0000 7f44840fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44781b1e90 con 0x56094a7301b0 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.160+0000 7f44840fe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44781af760 con 0x56094a7179f0 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.160+0000 7f44840fe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44781ad5d0 con 0x56094a738390 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.160+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab17a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.160+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab17a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36534/0 (socket says 172.21.15.112:36534) 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f4476ffd640 1 -- 172.21.15.112:0/4071854504 learned_addr learned my addr 172.21.15.112:0/4071854504 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f4477fff640 1 --2- 172.21.15.112:0/4071854504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094aafd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f44777fe640 1 --2- 172.21.15.112:0/4071854504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7179f0 0x56094ab0bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f4476ffd640 1 -- 172.21.15.112:0/4071854504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 msgr2=0x56094aafd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f4476ffd640 1 --2- 172.21.15.112:0/4071854504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094aafd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f4476ffd640 1 -- 172.21.15.112:0/4071854504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7179f0 msgr2=0x56094ab0bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f4476ffd640 1 --2- 172.21.15.112:0/4071854504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7179f0 0x56094ab0bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f4476ffd640 1 -- 172.21.15.112:0/4071854504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094ab09d80 con 0x56094a7301b0 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f4476ffd640 1 --2- 172.21.15.112:0/4071854504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab17a90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f44680e47f0 tx=0x7f44680d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb15455876bdc07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f445d7fa640 1 -- 172.21.15.112:0/4071854504 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468021040 con 0x56094a7301b0 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f445d7fa640 1 -- 172.21.15.112:0/4071854504 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44680aca50 con 0x56094a7301b0 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.161+0000 7f445d7fa640 1 -- 172.21.15.112:0/4071854504 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44680e4a10 con 0x56094a7301b0 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.162+0000 7f44840fe740 1 -- 172.21.15.112:0/4071854504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 msgr2=0x56094ab17a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.162+0000 7f44840fe740 1 --2- 172.21.15.112:0/4071854504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab17a90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f44680e47f0 tx=0x7f44680d0040 comp rx=0 tx=0).stop 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.162+0000 7f44840fe740 1 -- 172.21.15.112:0/4071854504 shutdown_connections 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.162+0000 7f44840fe740 1 --2- 172.21.15.112:0/4071854504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094aafd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.162+0000 7f44840fe740 1 --2- 172.21.15.112:0/4071854504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab17a90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.162+0000 7f44840fe740 1 --2- 172.21.15.112:0/4071854504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7179f0 0x56094ab0bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.162+0000 7f44840fe740 1 -- 172.21.15.112:0/4071854504 >> 172.21.15.112:0/4071854504 conn(0x56094a74d460 msgr2=0x56094ab0b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.162+0000 7f44840fe740 1 -- 172.21.15.112:0/4071854504 shutdown_connections 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.162+0000 7f44840fe740 1 -- 172.21.15.112:0/4071854504 wait complete. 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f44840fe740 1 -- start start 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f44840fe740 1 Processor -- start 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f44840fe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7179f0 0x56094ab14260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f44840fe740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab0f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f44840fe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094ab0fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f44840fe740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44781adc20 con 0x56094a7301b0 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f44840fe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44781af090 con 0x56094a7179f0 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f44840fe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44781ae2f0 con 0x56094a738390 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f44840fe740 10 client.0 _ll_register_callbacks cb 0x7f446c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab0f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab0f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36536/0 (socket says 172.21.15.112:36536) 2024-09-21T03:35:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f4476ffd640 1 -- 172.21.15.112:0/3014628794 learned_addr learned my addr 172.21.15.112:0/3014628794 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f4477fff640 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094ab0fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.163+0000 7f44777fe640 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7179f0 0x56094ab14260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f4476ffd640 1 -- 172.21.15.112:0/3014628794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 msgr2=0x56094ab0fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f4476ffd640 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094ab0fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f4476ffd640 1 -- 172.21.15.112:0/3014628794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7179f0 msgr2=0x56094ab14260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f4476ffd640 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7179f0 0x56094ab14260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f4476ffd640 1 -- 172.21.15.112:0/3014628794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56094ab10360 con 0x56094a7301b0 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f4476ffd640 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab0f710 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4468009610 tx=0x7f4468000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c73c7f1f8ca0762 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f4456ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f4456ffd640 1 -- 172.21.15.112:0/3014628794 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468021030 con 0x56094a7301b0 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f4456ffd640 1 -- 172.21.15.112:0/3014628794 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8470+0+0 (secure 0 0 0) 0x7f44680981e0 con 0x56094a7301b0 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f44840fe740 1 -- 172.21.15.112:0/3014628794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44680aca50 con 0x56094a7301b0 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f4456ffd640 1 -- 172.21.15.112:0/3014628794 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44680b1970 con 0x56094a7301b0 2024-09-21T03:35:45.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f44840fe740 10 client.16704 fetch_fsmap learned FSMap version 30 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f44840fe740 1 -- 172.21.15.112:0/3014628794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56094ab88400 con 0x56094a7301b0 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.164+0000 7f4456ffd640 1 -- 172.21.15.112:0/3014628794 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44680aca50 con 0x56094a7301b0 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.165+0000 7f4456ffd640 1 -- 172.21.15.112:0/3014628794 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f44680a8890 con 0x56094a7301b0 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.165+0000 7f44840fe740 10 client.16704 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.165+0000 7f44840fe740 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.165+0000 7f44840fe740 20 client.16704 populate_metadata read hostname 'smithi112' 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.165+0000 7f44840fe740 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56094a759070 0x56094ab8ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.165+0000 7f44840fe740 4 client.16704 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.165+0000 7f44840fe740 1 -- 172.21.15.112:0/3014628794 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56094aa74850 con 0x56094a759070 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.165+0000 7f44777fe640 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56094a759070 0x56094ab8ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.165+0000 7f4456ffd640 10 client.16704 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.165+0000 7f44777fe640 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56094a759070 0x56094ab8ab00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28864b5fef64969e server_cookie=cea230b0b9d1114 in_seq=0 out_seq=0 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f4456ffd640 1 -- 172.21.15.112:0/3014628794 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x56094aa74850 con 0x56094a759070 2024-09-21T03:35:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f4456ffd640 10 client.16704 handle_command_reply: tid=0 2024-09-21T03:35:45.210 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:35:45.210 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:35:45.210 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:35:45.210 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:35:45.210 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.210 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27", 2024-09-21T03:35:45.210 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:35:45.210 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.211 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:35:45.212 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.213 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.213 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.213 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.213 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.213 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:35:45.213 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:35:45.213 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.213 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.213 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.213 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.214 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:35:45.215 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:35:45.216 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.217 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:35:45.218 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:35:45.219 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:35:45.219 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:35:45.219 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:35:45.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f44840fe740 1 client.16704 shutdown 2024-09-21T03:35:45.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f44840fe740 1 -- 172.21.15.112:0/3014628794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 msgr2=0x56094ab0f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f44840fe740 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab0f710 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4468009610 tx=0x7f4468000bc0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f44840fe740 1 -- 172.21.15.112:0/3014628794 shutdown_connections 2024-09-21T03:35:45.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f44840fe740 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56094a759070 0x56094ab8ab00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f44840fe740 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56094a738390 0x56094ab0fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f44840fe740 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56094a7301b0 0x56094ab0f710 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f44840fe740 1 --2- 172.21.15.112:0/3014628794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56094a7179f0 0x56094ab14260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f44840fe740 1 -- 172.21.15.112:0/3014628794 >> 172.21.15.112:0/3014628794 conn(0x56094a74d460 msgr2=0x56094ab012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f4456ffd640 0 client.16704 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f44840fe740 1 -- 172.21.15.112:0/3014628794 shutdown_connections 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.166+0000 7f44840fe740 1 -- 172.21.15.112:0/3014628794 wait complete. 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.167+0000 7f44840fe740 20 client.16704 trim_cache size 0 max 16384 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.167+0000 7f447db8a640 1 -- 172.21.15.112:0/598607086 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44600b66b0 msgr2=0x7f44600b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.167+0000 7f447db8a640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44600b66b0 0x7f44600b8bc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x56094aafeef0 tx=0x7f446c005f90 comp rx=0 tx=0).stop 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.167+0000 7f447db8a640 1 -- 172.21.15.112:0/598607086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44781051a0 msgr2=0x7f447819b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.167+0000 7f447db8a640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44781051a0 0x7f447819b7f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f446800d740 tx=0x7f446800dc00 comp rx=0 tx=0).stop 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.167+0000 7f447db8a640 1 -- 172.21.15.112:0/598607086 shutdown_connections 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.167+0000 7f447db8a640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44600b66b0 0x7f44600b8bc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.167+0000 7f447db8a640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4478105af0 0x7f447819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.167+0000 7f447db8a640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44781051a0 0x7f447819b7f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.167+0000 7f447db8a640 1 --2- 172.21.15.112:0/598607086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4478103120 0x7f447819b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:35:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.167+0000 7f447db8a640 1 -- 172.21.15.112:0/598607086 >> 172.21.15.112:0/598607086 conn(0x7f4478077c60 msgr2=0x7f4478104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:35:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.168+0000 7f447db8a640 1 -- 172.21.15.112:0/598607086 shutdown_connections 2024-09-21T03:35:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:35:45.168+0000 7f447db8a640 1 -- 172.21.15.112:0/598607086 wait complete. 2024-09-21T03:35:45.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 697536 10.82 MB/sec execute 225 sec latency 49.174 ms 2024-09-21T03:35:45.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:45 smithi112 ceph-mon[31442]: pgmap v522: 97 pgs: 97 active+clean; 481 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-09-21T03:35:45.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:45 smithi203 ceph-mon[34723]: pgmap v522: 97 pgs: 97 active+clean; 481 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-09-21T03:35:45.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:45 smithi187 ceph-mon[34603]: pgmap v522: 97 pgs: 97 active+clean; 481 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-09-21T03:35:45.998 INFO:tasks.cephfs.filesystem:scrub status for tag:825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27 - {'path': '/', 'tag': '825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27', 'options': 'recursive,force'} 2024-09-21T03:35:46.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 699861 10.84 MB/sec execute 226 sec latency 13.318 ms 2024-09-21T03:35:47.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 701359 10.81 MB/sec execute 227 sec latency 21.678 ms 2024-09-21T03:35:47.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:47 smithi203 ceph-mon[34723]: pgmap v523: 97 pgs: 97 active+clean; 487 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 314 op/s 2024-09-21T03:35:47.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:47 smithi187 ceph-mon[34603]: pgmap v523: 97 pgs: 97 active+clean; 487 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 314 op/s 2024-09-21T03:35:48.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:47 smithi112 ceph-mon[31442]: pgmap v523: 97 pgs: 97 active+clean; 487 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 314 op/s 2024-09-21T03:35:48.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 703562 10.83 MB/sec execute 228 sec latency 13.176 ms 2024-09-21T03:35:49.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 705477 10.81 MB/sec execute 229 sec latency 45.689 ms 2024-09-21T03:35:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:49 smithi203 ceph-mon[34723]: pgmap v524: 97 pgs: 97 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-09-21T03:35:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:49 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:35:49.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:49 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:35:49.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:49 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:35:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:49 smithi187 ceph-mon[34603]: pgmap v524: 97 pgs: 97 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-09-21T03:35:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:49 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:35:49.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:49 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:35:49.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:49 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:35:50.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:49 smithi112 ceph-mon[31442]: pgmap v524: 97 pgs: 97 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-09-21T03:35:50.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:49 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:35:50.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:49 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:35:50.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:49 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:35:50.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 707266 10.82 MB/sec execute 230 sec latency 12.843 ms 2024-09-21T03:35:51.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 709670 10.82 MB/sec execute 231 sec latency 17.193 ms 2024-09-21T03:35:51.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:51 smithi203 ceph-mon[34723]: pgmap v525: 97 pgs: 97 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-09-21T03:35:51.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:51 smithi187 ceph-mon[34603]: pgmap v525: 97 pgs: 97 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-09-21T03:35:52.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:51 smithi112 ceph-mon[31442]: pgmap v525: 97 pgs: 97 active+clean; 488 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-09-21T03:35:52.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 711137 10.81 MB/sec execute 232 sec latency 16.771 ms 2024-09-21T03:35:53.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 713838 10.84 MB/sec execute 233 sec latency 14.794 ms 2024-09-21T03:35:53.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:53 smithi203 ceph-mon[34723]: pgmap v526: 97 pgs: 97 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 249 op/s 2024-09-21T03:35:53.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:53 smithi187 ceph-mon[34603]: pgmap v526: 97 pgs: 97 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 249 op/s 2024-09-21T03:35:54.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:53 smithi112 ceph-mon[31442]: pgmap v526: 97 pgs: 97 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 249 op/s 2024-09-21T03:35:54.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 714744 10.80 MB/sec execute 234 sec latency 46.234 ms 2024-09-21T03:35:55.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 717405 10.82 MB/sec execute 235 sec latency 18.270 ms 2024-09-21T03:35:55.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:55 smithi203 ceph-mon[34723]: pgmap v527: 97 pgs: 97 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-09-21T03:35:55.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:55 smithi187 ceph-mon[34603]: pgmap v527: 97 pgs: 97 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-09-21T03:35:56.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:55 smithi112 ceph-mon[31442]: pgmap v527: 97 pgs: 97 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-09-21T03:35:56.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 718809 10.81 MB/sec execute 236 sec latency 17.069 ms 2024-09-21T03:35:57.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 721150 10.82 MB/sec execute 237 sec latency 15.721 ms 2024-09-21T03:35:57.838 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:57 smithi112 ceph-mon[31442]: pgmap v528: 97 pgs: 97 active+clean; 504 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 320 op/s 2024-09-21T03:35:57.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:57 smithi203 ceph-mon[34723]: pgmap v528: 97 pgs: 97 active+clean; 504 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 320 op/s 2024-09-21T03:35:57.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:57 smithi187 ceph-mon[34603]: pgmap v528: 97 pgs: 97 active+clean; 504 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 320 op/s 2024-09-21T03:35:58.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 722772 10.80 MB/sec execute 238 sec latency 16.646 ms 2024-09-21T03:35:59.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 724788 10.81 MB/sec execute 239 sec latency 48.199 ms 2024-09-21T03:35:59.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:35:59 smithi203 ceph-mon[34723]: pgmap v529: 97 pgs: 97 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-09-21T03:35:59.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:35:59 smithi187 ceph-mon[34603]: pgmap v529: 97 pgs: 97 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-09-21T03:36:00.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:35:59 smithi112 ceph-mon[31442]: pgmap v529: 97 pgs: 97 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-09-21T03:36:00.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 726544 10.79 MB/sec execute 240 sec latency 23.171 ms 2024-09-21T03:36:01.241 INFO:tasks.workunit.client.0.smithi112.stdout: 1 728182 10.80 MB/sec execute 241 sec latency 26.746 ms 2024-09-21T03:36:01.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:01 smithi203 ceph-mon[34723]: pgmap v530: 97 pgs: 97 active+clean; 506 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 266 op/s 2024-09-21T03:36:01.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:01 smithi187 ceph-mon[34603]: pgmap v530: 97 pgs: 97 active+clean; 506 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 266 op/s 2024-09-21T03:36:02.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:01 smithi112 ceph-mon[31442]: pgmap v530: 97 pgs: 97 active+clean; 506 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 266 op/s 2024-09-21T03:36:02.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 729722 10.78 MB/sec execute 242 sec latency 15.957 ms 2024-09-21T03:36:03.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 731941 10.79 MB/sec execute 243 sec latency 13.695 ms 2024-09-21T03:36:03.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:03 smithi203 ceph-mon[34723]: pgmap v531: 97 pgs: 97 active+clean; 509 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 289 op/s 2024-09-21T03:36:03.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:03 smithi187 ceph-mon[34603]: pgmap v531: 97 pgs: 97 active+clean; 509 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 289 op/s 2024-09-21T03:36:04.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:03 smithi112 ceph-mon[31442]: pgmap v531: 97 pgs: 97 active+clean; 509 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 289 op/s 2024-09-21T03:36:04.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 733944 10.77 MB/sec execute 244 sec latency 30.196 ms 2024-09-21T03:36:05.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 735721 10.78 MB/sec execute 245 sec latency 20.580 ms 2024-09-21T03:36:05.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:05 smithi203 ceph-mon[34723]: pgmap v532: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-09-21T03:36:05.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:05 smithi203 ceph-mon[34723]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:36:05.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:05 smithi187 ceph-mon[34603]: pgmap v532: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-09-21T03:36:05.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:05 smithi187 ceph-mon[34603]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:36:06.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:05 smithi112 ceph-mon[31442]: pgmap v532: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-09-21T03:36:06.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:05 smithi112 ceph-mon[31442]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:36:06.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 738259 10.79 MB/sec execute 246 sec latency 18.865 ms 2024-09-21T03:36:07.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 739867 10.78 MB/sec execute 247 sec latency 12.115 ms 2024-09-21T03:36:07.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:07 smithi203 ceph-mon[34723]: pgmap v534: 97 pgs: 97 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 302 op/s 2024-09-21T03:36:07.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:07 smithi187 ceph-mon[34603]: pgmap v534: 97 pgs: 97 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 302 op/s 2024-09-21T03:36:08.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:07 smithi112 ceph-mon[31442]: pgmap v534: 97 pgs: 97 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 302 op/s 2024-09-21T03:36:08.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 742449 10.80 MB/sec execute 248 sec latency 15.856 ms 2024-09-21T03:36:09.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 744271 10.78 MB/sec execute 249 sec latency 47.597 ms 2024-09-21T03:36:09.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:09 smithi203 ceph-mon[34723]: pgmap v535: 97 pgs: 97 active+clean; 519 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-09-21T03:36:09.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:09 smithi187 ceph-mon[34603]: pgmap v535: 97 pgs: 97 active+clean; 519 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-09-21T03:36:10.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:09 smithi112 ceph-mon[31442]: pgmap v535: 97 pgs: 97 active+clean; 519 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-09-21T03:36:10.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 746213 10.79 MB/sec execute 250 sec latency 30.049 ms 2024-09-21T03:36:11.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 748570 10.80 MB/sec execute 251 sec latency 17.908 ms 2024-09-21T03:36:11.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:11 smithi203 ceph-mon[34723]: pgmap v536: 97 pgs: 97 active+clean; 527 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 337 op/s 2024-09-21T03:36:11.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:11 smithi187 ceph-mon[34603]: pgmap v536: 97 pgs: 97 active+clean; 527 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 337 op/s 2024-09-21T03:36:12.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:11 smithi112 ceph-mon[31442]: pgmap v536: 97 pgs: 97 active+clean; 527 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 337 op/s 2024-09-21T03:36:12.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 750010 10.78 MB/sec execute 252 sec latency 20.124 ms 2024-09-21T03:36:13.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 752594 10.80 MB/sec execute 253 sec latency 19.074 ms 2024-09-21T03:36:13.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: pgmap v537: 97 pgs: 97 active+clean; 531 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 335 op/s 2024-09-21T03:36:13.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:13.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi203 to 176.0M 2024-09-21T03:36:13.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-21T03:36:13.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: Adjusting osd_memory_target on smithi187 to 176.0M 2024-09-21T03:36:13.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: Unable to set osd_memory_target on smithi203 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-21T03:36:13.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: Unable to set osd_memory_target on smithi112 to 184556646: error parsing value: Value '184556646' is below minimum 939524096 2024-09-21T03:36:13.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: Unable to set osd_memory_target on smithi187 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-21T03:36:13.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:13.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:13.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:36:13.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: pgmap v537: 97 pgs: 97 active+clean; 531 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 335 op/s 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:13.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi203 to 176.0M 2024-09-21T03:36:13.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-21T03:36:13.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: Adjusting osd_memory_target on smithi187 to 176.0M 2024-09-21T03:36:13.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: Unable to set osd_memory_target on smithi203 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-21T03:36:13.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: Unable to set osd_memory_target on smithi112 to 184556646: error parsing value: Value '184556646' is below minimum 939524096 2024-09-21T03:36:13.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: Unable to set osd_memory_target on smithi187 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-21T03:36:13.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:13.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:13.931 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:36:14.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: pgmap v537: 97 pgs: 97 active+clean; 531 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 335 op/s 2024-09-21T03:36:14.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:36:14.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi203 to 176.0M 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi112 to 176.0M 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: Adjusting osd_memory_target on smithi187 to 176.0M 2024-09-21T03:36:14.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: Unable to set osd_memory_target on smithi203 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-21T03:36:14.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: Unable to set osd_memory_target on smithi112 to 184556646: error parsing value: Value '184556646' is below minimum 939524096 2024-09-21T03:36:14.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: Unable to set osd_memory_target on smithi187 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-21T03:36:14.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:36:14.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:36:14.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:36:14.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 753837 10.78 MB/sec execute 254 sec latency 32.214 ms 2024-09-21T03:36:15.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 756426 10.80 MB/sec execute 255 sec latency 22.046 ms 2024-09-21T03:36:15.999 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:36:16.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:15 smithi203 ceph-mon[34723]: pgmap v538: 97 pgs: 97 active+clean; 533 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 304 op/s 2024-09-21T03:36:16.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:15 smithi203 ceph-mon[34723]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:36:16.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:15 smithi187 ceph-mon[34603]: pgmap v538: 97 pgs: 97 active+clean; 533 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 304 op/s 2024-09-21T03:36:16.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:15 smithi187 ceph-mon[34603]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:36:16.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 757786 10.78 MB/sec execute 256 sec latency 20.084 ms 2024-09-21T03:36:16.263 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:16.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:15 smithi112 ceph-mon[31442]: pgmap v538: 97 pgs: 97 active+clean; 533 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 304 op/s 2024-09-21T03:36:16.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:15 smithi112 ceph-mon[31442]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:36:16.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.818+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/495840951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c410de10 msgr2=0x7fe9c4110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.818+0000 7fe9cb6b9640 1 --2- 172.21.15.112:0/495840951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c410de10 0x7fe9c4110300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009be0 tx=0x7fe9bc02a6c0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.818+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/495840951 shutdown_connections 2024-09-21T03:36:16.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.818+0000 7fe9cb6b9640 1 --2- 172.21.15.112:0/495840951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c410de10 0x7fe9c4110300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.818+0000 7fe9cb6b9640 1 --2- 172.21.15.112:0/495840951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c4105300 0x7fe9c410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.818+0000 7fe9cb6b9640 1 --2- 172.21.15.112:0/495840951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c41049b0 0x7fe9c4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.818+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/495840951 >> 172.21.15.112:0/495840951 conn(0x7fe9c4079730 msgr2=0x7fe9c4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.818+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/495840951 shutdown_connections 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.818+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/495840951 wait complete. 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9cb6b9640 1 Processor -- start 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9cb6b9640 1 -- start start 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9cb6b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c41049b0 0x7fe9c419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9cb6b9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c4105300 0x7fe9c419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9cb6b9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c410de10 0x7fe9c419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9cb6b9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c4113a70 con 0x7fe9c41049b0 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9cb6b9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41138f0 con 0x7fe9c410de10 2024-09-21T03:36:16.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9cb6b9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c4113bf0 con 0x7fe9c4105300 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9c9455640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c41049b0 0x7fe9c419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9c9455640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c41049b0 0x7fe9c419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48102/0 (socket says 172.21.15.112:48102) 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9c9455640 1 -- 172.21.15.112:0/3682886135 learned_addr learned my addr 172.21.15.112:0/3682886135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c4105300 0x7fe9c419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.819+0000 7fe9c9c56640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c410de10 0x7fe9c419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9c9455640 1 -- 172.21.15.112:0/3682886135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c4105300 msgr2=0x7fe9c419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9c9455640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c4105300 0x7fe9c419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9c9455640 1 -- 172.21.15.112:0/3682886135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c410de10 msgr2=0x7fe9c419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9c9455640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c410de10 0x7fe9c419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9c9455640 1 -- 172.21.15.112:0/3682886135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c41a03d0 con 0x7fe9c41049b0 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c4105300 0x7fe9c419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:16.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9c9455640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c41049b0 0x7fe9c419a460 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000e9f0 tx=0x7fe9c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9ba7fc640 1 -- 172.21.15.112:0/3682886135 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000ccc0 con 0x7fe9c41049b0 2024-09-21T03:36:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/3682886135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c41a06c0 con 0x7fe9c41049b0 2024-09-21T03:36:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9ba7fc640 1 -- 172.21.15.112:0/3682886135 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9c00035f0 con 0x7fe9c41049b0 2024-09-21T03:36:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/3682886135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c41025d0 con 0x7fe9c41049b0 2024-09-21T03:36:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.820+0000 7fe9ba7fc640 1 -- 172.21.15.112:0/3682886135 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000ce60 con 0x7fe9c41049b0 2024-09-21T03:36:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.823+0000 7fe9d1c2d740 1 Processor -- start 2024-09-21T03:36:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.823+0000 7fe9ba7fc640 1 -- 172.21.15.112:0/3682886135 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe9c0010690 con 0x7fe9c41049b0 2024-09-21T03:36:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.823+0000 7fe9d1c2d740 1 -- start start 2024-09-21T03:36:16.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 0x55ecb676b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 0x55ecb676bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb63a3390 0x55ecb676c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9d1c2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ecb66e1e50 con 0x55ecb63c4070 2024-09-21T03:36:16.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9d1c2d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ecb66d1280 con 0x55ecb63a3390 2024-09-21T03:36:16.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9d1c2d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ecb66d5090 con 0x55ecb639b1b0 2024-09-21T03:36:16.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9c9c56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 0x55ecb676b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9c9c56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 0x55ecb676b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48116/0 (socket says 172.21.15.112:48116) 2024-09-21T03:36:16.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9c9c56640 1 -- 172.21.15.112:0/3157219279 learned_addr learned my addr 172.21.15.112:0/3157219279 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:16.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9c9455640 1 --2- 172.21.15.112:0/3157219279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 0x55ecb676bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/3157219279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb63a3390 0x55ecb676c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9ba7fc640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b00b6680 0x7fe9b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9c9455640 1 -- 172.21.15.112:0/3157219279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb63a3390 msgr2=0x55ecb676c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:16.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9c9455640 1 --2- 172.21.15.112:0/3157219279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb63a3390 0x55ecb676c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9c9455640 1 -- 172.21.15.112:0/3157219279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 msgr2=0x55ecb676b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9c9455640 1 --2- 172.21.15.112:0/3157219279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 0x55ecb676b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9c9455640 1 -- 172.21.15.112:0/3157219279 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb642f650 con 0x55ecb639b1b0 2024-09-21T03:36:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.824+0000 7fe9c9c56640 1 --2- 172.21.15.112:0/3157219279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 0x55ecb676b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.825+0000 7fe9c9c56640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b00b6680 0x7fe9b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.825+0000 7fe9ba7fc640 1 -- 172.21.15.112:0/3682886135 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7fe9c00e7920 con 0x7fe9c41049b0 2024-09-21T03:36:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.825+0000 7fe9c9455640 1 --2- 172.21.15.112:0/3157219279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 0x55ecb676bb90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00e50a0 tx=0x7fe9c0021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f0965aea7c1079c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.825+0000 7fe9af7fe640 1 -- 172.21.15.112:0/3157219279 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00ad570 con 0x55ecb639b1b0 2024-09-21T03:36:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.825+0000 7fe9af7fe640 1 -- 172.21.15.112:0/3157219279 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9c00ad2b0 con 0x55ecb639b1b0 2024-09-21T03:36:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.825+0000 7fe9af7fe640 1 -- 172.21.15.112:0/3157219279 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00acf10 con 0x55ecb639b1b0 2024-09-21T03:36:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.825+0000 7fe9c9c56640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b00b6680 0x7fe9b00b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc006010 tx=0x7fe9bc009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.826+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/3157219279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 msgr2=0x55ecb676bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.826+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/3157219279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 0x55ecb676bb90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00e50a0 tx=0x7fe9c0021000 comp rx=0 tx=0).stop 2024-09-21T03:36:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.826+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/3157219279 shutdown_connections 2024-09-21T03:36:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.826+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/3157219279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb63a3390 0x55ecb676c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.826+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/3157219279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 0x55ecb676bb90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.826+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/3157219279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 0x55ecb676b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.826+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/3157219279 >> 172.21.15.112:0/3157219279 conn(0x55ecb63b8460 msgr2=0x55ecb6769540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.826+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/3157219279 shutdown_connections 2024-09-21T03:36:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.826+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/3157219279 wait complete. 2024-09-21T03:36:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.827+0000 7fe9d1c2d740 1 -- start start 2024-09-21T03:36:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.827+0000 7fe9d1c2d740 1 Processor -- start 2024-09-21T03:36:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 0x55ecb6776cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb63a3390 0x55ecb6777210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 0x55ecb6783060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c9455640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 0x55ecb6776cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c9455640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 0x55ecb6776cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:54302/0 (socket says 172.21.15.112:54302) 2024-09-21T03:36:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c9455640 1 -- 172.21.15.112:0/4280176041 learned_addr learned my addr 172.21.15.112:0/4280176041 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/4280176041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ecb6532700 con 0x55ecb63c4070 2024-09-21T03:36:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/4280176041 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ecb666c110 con 0x55ecb63a3390 2024-09-21T03:36:16.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/4280176041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ecb6779d70 con 0x55ecb639b1b0 2024-09-21T03:36:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb63a3390 0x55ecb6777210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c9c56640 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 0x55ecb6783060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c9455640 1 -- 172.21.15.112:0/4280176041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb63a3390 msgr2=0x55ecb6777210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c9455640 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb63a3390 0x55ecb6777210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c9455640 1 -- 172.21.15.112:0/4280176041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 msgr2=0x55ecb6783060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9d1c2d740 10 client.0 _ll_register_callbacks cb 0x55ecb6685990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c9455640 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 0x55ecb6783060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c9455640 1 -- 172.21.15.112:0/4280176041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb6699c40 con 0x55ecb639b1b0 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/4280176041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c00ad2b0 con 0x55ecb639b1b0 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb63a3390 0x55ecb6777210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9c9455640 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 0x55ecb6776cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009230 tx=0x7fe9c00acf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=816108a68d7c03e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.828+0000 7fe9acff9640 10 client.26275 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9acff9640 1 -- 172.21.15.112:0/4280176041 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0098020 con 0x55ecb639b1b0 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9acff9640 1 -- 172.21.15.112:0/4280176041 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7fe9c00a9a10 con 0x55ecb639b1b0 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9acff9640 1 -- 172.21.15.112:0/4280176041 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe9c00ad0f0 con 0x55ecb639b1b0 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9d1c2d740 10 client.26275 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/4280176041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55ecb67858a0 con 0x55ecb639b1b0 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9acff9640 1 -- 172.21.15.112:0/4280176041 <== mon.1 v2:172.21.15.203:3300/0 4 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe9c00b6d30 con 0x55ecb639b1b0 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9d1c2d740 10 client.26275 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9d1c2d740 10 client.26275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:36:16.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9d1c2d740 20 client.26275 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ecb63829f0 0x55ecb6788c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9d1c2d740 4 client.26275 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.829+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/4280176041 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecb66e3da0 con 0x55ecb63829f0 2024-09-21T03:36:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.830+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ecb63829f0 0x55ecb6788c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.830+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ecb63829f0 0x55ecb6788c00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feb367b26df0287e server_cookie=3807eb18bf987d00 in_seq=0 out_seq=0 2024-09-21T03:36:16.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.830+0000 7fe9acff9640 10 client.26275 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:16.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.831+0000 7fe9acff9640 1 -- 172.21.15.112:0/4280176041 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ecb66e3da0 con 0x55ecb63829f0 2024-09-21T03:36:16.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.831+0000 7fe9acff9640 10 client.26275 handle_command_reply: tid=0 2024-09-21T03:36:16.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.831+0000 7fe9d1c2d740 1 client.26275 shutdown 2024-09-21T03:36:16.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/4280176041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 msgr2=0x55ecb6776cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 0x55ecb6776cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009230 tx=0x7fe9c00acf10 comp rx=0 tx=0).stop 2024-09-21T03:36:16.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/4280176041 shutdown_connections 2024-09-21T03:36:16.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ecb63829f0 0x55ecb6788c00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63c4070 0x55ecb6783060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9acff9640 0 client.26275 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:16.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb63a3390 0x55ecb6777210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/4280176041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb639b1b0 0x55ecb6776cd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/4280176041 >> 172.21.15.112:0/4280176041 conn(0x55ecb63b8460 msgr2=0x55ecb676c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/4280176041 shutdown_connections 2024-09-21T03:36:16.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/4280176041 wait complete. 2024-09-21T03:36:16.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.832+0000 7fe9d1c2d740 20 client.26275 trim_cache size 0 max 16384 2024-09-21T03:36:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.864+0000 7fe9d1c2d740 1 Processor -- start 2024-09-21T03:36:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.864+0000 7fe9d1c2d740 1 -- start start 2024-09-21T03:36:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.864+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb63829f0 0x55ecb6778bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.864+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 0x55ecb6782fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.864+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63a3390 0x55ecb676cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9d1c2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41b41e0 con 0x55ecb63a3390 2024-09-21T03:36:16.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9d1c2d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41b1ab0 con 0x55ecb639b1b0 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9d1c2d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41af920 con 0x55ecb63829f0 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c8c54640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 0x55ecb6782fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c9c56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63a3390 0x55ecb676cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c8c54640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 0x55ecb6782fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:54568/0 (socket says 172.21.15.112:54568) 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c9c56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63a3390 0x55ecb676cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48128/0 (socket says 172.21.15.112:48128) 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c8c54640 1 -- 172.21.15.112:0/302550228 learned_addr learned my addr 172.21.15.112:0/302550228 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c9455640 1 --2- 172.21.15.112:0/302550228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb63829f0 0x55ecb6778bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c8c54640 1 -- 172.21.15.112:0/302550228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb63829f0 msgr2=0x55ecb6778bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/302550228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb63829f0 0x55ecb6778bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c8c54640 1 -- 172.21.15.112:0/302550228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63a3390 msgr2=0x55ecb676cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/302550228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63a3390 0x55ecb676cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c8c54640 1 -- 172.21.15.112:0/302550228 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb6769e30 con 0x55ecb639b1b0 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/302550228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 0x55ecb6782fd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe9c406ad30 tx=0x7fe9b4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48909a2e78b3adf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.865+0000 7fe9aeffd640 1 -- 172.21.15.112:0/302550228 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40120d0 con 0x55ecb639b1b0 2024-09-21T03:36:16.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.866+0000 7fe9aeffd640 1 -- 172.21.15.112:0/302550228 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b4012270 con 0x55ecb639b1b0 2024-09-21T03:36:16.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.866+0000 7fe9aeffd640 1 -- 172.21.15.112:0/302550228 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4004d00 con 0x55ecb639b1b0 2024-09-21T03:36:16.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.866+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/302550228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 msgr2=0x55ecb6782fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.866+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/302550228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 0x55ecb6782fd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe9c406ad30 tx=0x7fe9b4005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:16.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.866+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/302550228 shutdown_connections 2024-09-21T03:36:16.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.866+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/302550228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63a3390 0x55ecb676cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.866+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/302550228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 0x55ecb6782fd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.866+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/302550228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb63829f0 0x55ecb6778bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.867+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/302550228 >> 172.21.15.112:0/302550228 conn(0x55ecb63b8460 msgr2=0x55ecb676c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.867+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/302550228 shutdown_connections 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.867+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/302550228 wait complete. 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.867+0000 7fe9d1c2d740 1 -- start start 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.867+0000 7fe9d1c2d740 1 Processor -- start 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63829f0 0x55ecb6789f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 0x55ecb678a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9d1c2d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb63a3390 0x55ecb67f2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9d1c2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41aff70 con 0x55ecb63829f0 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9d1c2d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41b13e0 con 0x55ecb639b1b0 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9d1c2d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c41b0640 con 0x55ecb63a3390 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c9455640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63829f0 0x55ecb6789f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9d1c2d740 10 client.0 _ll_register_callbacks cb 0x7fe9c0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c9455640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63829f0 0x55ecb6789f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48142/0 (socket says 172.21.15.112:48142) 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c9455640 1 -- 172.21.15.112:0/1325458464 learned_addr learned my addr 172.21.15.112:0/1325458464 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c9c56640 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb63a3390 0x55ecb67f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 0x55ecb678a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c9455640 1 -- 172.21.15.112:0/1325458464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb63a3390 msgr2=0x55ecb67f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c9455640 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb63a3390 0x55ecb67f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c9455640 1 -- 172.21.15.112:0/1325458464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 msgr2=0x55ecb678a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c9455640 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 0x55ecb678a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c9455640 1 -- 172.21.15.112:0/1325458464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb678ad60 con 0x55ecb63829f0 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 0x55ecb678a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c9c56640 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb63a3390 0x55ecb67f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9acff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:36:16.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9c9455640 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63829f0 0x55ecb6789f50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00037e0 tx=0x7fe9c00ed4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb68b3d6c061cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.868+0000 7fe9acff9640 1 -- 172.21.15.112:0/1325458464 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0098070 con 0x55ecb63829f0 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9acff9640 1 -- 172.21.15.112:0/1325458464 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7fe9c00aca00 con 0x55ecb63829f0 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/1325458464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b4012270 con 0x55ecb63829f0 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9acff9640 1 -- 172.21.15.112:0/1325458464 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe9c00acf10 con 0x55ecb63829f0 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9d1c2d740 10 client.16722 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/1325458464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ecb678ad60 con 0x55ecb63829f0 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9acff9640 1 -- 172.21.15.112:0/1325458464 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b4012270 con 0x55ecb63829f0 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9acff9640 1 -- 172.21.15.112:0/1325458464 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe9c00ac470 con 0x55ecb63829f0 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9d1c2d740 10 client.16722 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9d1c2d740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9d1c2d740 20 client.16722 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ecb63c4070 0x55ecb67f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9d1c2d740 4 client.16722 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:36:16.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/1325458464 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ecb6769750 con 0x55ecb63c4070 2024-09-21T03:36:16.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.869+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ecb63c4070 0x55ecb67f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:16.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.870+0000 7fe9acff9640 10 client.16722 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:16.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.870+0000 7fe9c8c54640 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ecb63c4070 0x55ecb67f5be0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab5d7a7c5f007b server_cookie=22e8f1754ae551ed in_seq=0 out_seq=0 2024-09-21T03:36:16.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.870+0000 7fe9acff9640 1 -- 172.21.15.112:0/1325458464 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x55ecb6769750 con 0x55ecb63c4070 2024-09-21T03:36:16.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.870+0000 7fe9acff9640 10 client.16722 handle_command_reply: tid=0 2024-09-21T03:36:16.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.870+0000 7fe9d1c2d740 1 client.16722 shutdown 2024-09-21T03:36:16.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/1325458464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63829f0 msgr2=0x55ecb6789f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63829f0 0x55ecb6789f50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00037e0 tx=0x7fe9c00ed4d0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/1325458464 shutdown_connections 2024-09-21T03:36:16.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ecb63c4070 0x55ecb67f5be0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ecb63a3390 0x55ecb67f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecb639b1b0 0x55ecb678a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9acff9640 0 client.16722 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:16.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9d1c2d740 1 --2- 172.21.15.112:0/1325458464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ecb63829f0 0x55ecb6789f50 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/1325458464 >> 172.21.15.112:0/1325458464 conn(0x55ecb63b8460 msgr2=0x55ecb676cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/1325458464 shutdown_connections 2024-09-21T03:36:16.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9d1c2d740 1 -- 172.21.15.112:0/1325458464 wait complete. 2024-09-21T03:36:16.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.871+0000 7fe9d1c2d740 20 client.16722 trim_cache size 0 max 16384 2024-09-21T03:36:16.878 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:36:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:36:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:36:16.878 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.879 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.880 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:36:16.881 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.883 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:36:16.884 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:16.885 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:36:16.886 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:36:16.886 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:36:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/3682886135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b00b6680 msgr2=0x7fe9b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b00b6680 0x7fe9b00b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc006010 tx=0x7fe9bc009500 comp rx=0 tx=0).stop 2024-09-21T03:36:16.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/3682886135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c41049b0 msgr2=0x7fe9c419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:16.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c41049b0 0x7fe9c419a460 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000e9f0 tx=0x7fe9c000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/3682886135 shutdown_connections 2024-09-21T03:36:16.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe9b00b6680 0x7fe9b00b8b90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9c410de10 0x7fe9c419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c4105300 0x7fe9c419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 --2- 172.21.15.112:0/3682886135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe9c41049b0 0x7fe9c419a460 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:16.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/3682886135 >> 172.21.15.112:0/3682886135 conn(0x7fe9c4079730 msgr2=0x7fe9c40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:16.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/3682886135 shutdown_connections 2024-09-21T03:36:16.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:16.872+0000 7fe9cb6b9640 1 -- 172.21.15.112:0/3682886135 wait complete. 2024-09-21T03:36:17.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 760151 10.79 MB/sec execute 257 sec latency 13.357 ms 2024-09-21T03:36:17.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:36:17.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:36:18.013 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:18.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:17 smithi112 ceph-mon[31442]: pgmap v540: 97 pgs: 97 active+clean; 540 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-09-21T03:36:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:17 smithi203 ceph-mon[34723]: pgmap v540: 97 pgs: 97 active+clean; 540 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-09-21T03:36:18.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:17 smithi187 ceph-mon[34603]: pgmap v540: 97 pgs: 97 active+clean; 540 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-09-21T03:36:18.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 761642 10.77 MB/sec execute 258 sec latency 22.719 ms 2024-09-21T03:36:18.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.532+0000 7f20454ff640 1 -- 172.21.15.112:0/3735228375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20401083d0 msgr2=0x7f20401087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.532+0000 7f20454ff640 1 --2- 172.21.15.112:0/3735228375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20401083d0 0x7f20401087e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2034009900 tx=0x7f203402a410 comp rx=0 tx=0).stop 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.532+0000 7f20454ff640 1 -- 172.21.15.112:0/3735228375 shutdown_connections 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.532+0000 7f20454ff640 1 --2- 172.21.15.112:0/3735228375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2040108d20 0x7f204010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.532+0000 7f20454ff640 1 --2- 172.21.15.112:0/3735228375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20401083d0 0x7f20401087e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.532+0000 7f20454ff640 1 --2- 172.21.15.112:0/3735228375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2040102ae0 0x7f2040102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.532+0000 7f20454ff640 1 -- 172.21.15.112:0/3735228375 >> 172.21.15.112:0/3735228375 conn(0x7f2040079760 msgr2=0x7f2040079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.532+0000 7f20454ff640 1 -- 172.21.15.112:0/3735228375 shutdown_connections 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.532+0000 7f20454ff640 1 -- 172.21.15.112:0/3735228375 wait complete. 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.533+0000 7f20454ff640 1 Processor -- start 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.533+0000 7f20454ff640 1 -- start start 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.533+0000 7f20454ff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2040102ae0 0x7f2040199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.533+0000 7f20454ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20401083d0 0x7f20401996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.533+0000 7f20454ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2040108d20 0x7f2040199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.533+0000 7f20454ff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112990 con 0x7f20401083d0 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.533+0000 7f20454ff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112810 con 0x7f2040108d20 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.533+0000 7f20454ff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112b10 con 0x7f2040102ae0 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20401083d0 0x7f20401996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20401083d0 0x7f20401996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48162/0 (socket says 172.21.15.112:48162) 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2040102ae0 0x7f2040199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203e7fc640 1 -- 172.21.15.112:0/989533591 learned_addr learned my addr 172.21.15.112:0/989533591 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203f7fe640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2040108d20 0x7f2040199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203e7fc640 1 -- 172.21.15.112:0/989533591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2040102ae0 msgr2=0x7f2040199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203e7fc640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2040102ae0 0x7f2040199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203e7fc640 1 -- 172.21.15.112:0/989533591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2040108d20 msgr2=0x7f2040199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203e7fc640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2040108d20 0x7f2040199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203e7fc640 1 -- 172.21.15.112:0/989533591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20401a8820 con 0x7f20401083d0 2024-09-21T03:36:18.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203effd640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2040102ae0 0x7f2040199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f203e7fc640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20401083d0 0x7f20401996b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2034032fd0 tx=0x7f2034031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f2027fff640 1 -- 172.21.15.112:0/989533591 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203403f030 con 0x7f20401083d0 2024-09-21T03:36:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f20454ff640 1 -- 172.21.15.112:0/989533591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20401a8ab0 con 0x7f20401083d0 2024-09-21T03:36:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f2027fff640 1 -- 172.21.15.112:0/989533591 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2034004490 con 0x7f20401083d0 2024-09-21T03:36:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f2027fff640 1 -- 172.21.15.112:0/989533591 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034035070 con 0x7f20401083d0 2024-09-21T03:36:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.534+0000 7f20454ff640 1 -- 172.21.15.112:0/989533591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20401a8fe0 con 0x7f20401083d0 2024-09-21T03:36:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.536+0000 7f20454ff640 1 -- 172.21.15.112:0/989533591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2040102ef0 con 0x7f20401083d0 2024-09-21T03:36:18.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.537+0000 7f2027fff640 1 -- 172.21.15.112:0/989533591 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2034042030 con 0x7f20401083d0 2024-09-21T03:36:18.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.538+0000 7f2027fff640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f20280b6700 0x7f20280b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:18.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.538+0000 7f203effd640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f20280b6700 0x7f20280b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:18.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.538+0000 7f2027fff640 1 -- 172.21.15.112:0/989533591 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f2034107dd0 con 0x7f20401083d0 2024-09-21T03:36:18.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.539+0000 7f203effd640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f20280b6700 0x7f20280b8c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2040106030 tx=0x7f2030005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:18.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.542+0000 7f2027fff640 1 -- 172.21.15.112:0/989533591 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f20340d2730 con 0x7f20401083d0 2024-09-21T03:36:18.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.775+0000 7f20454ff640 1 -- 172.21.15.112:0/989533591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f204006bef0 con 0x7f20401083d0 2024-09-21T03:36:18.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.776+0000 7f2027fff640 1 -- 172.21.15.112:0/989533591 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f204006bef0 con 0x7f20401083d0 2024-09-21T03:36:18.778 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:36:18.778 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"btime":"2024-09-21T03:35:32:357323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:35:32.307199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:36:18.779 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-09-21T03:36:18.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.778+0000 7f20454ff640 1 -- 172.21.15.112:0/989533591 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f20280b6700 msgr2=0x7f20280b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.778+0000 7f20454ff640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f20280b6700 0x7f20280b8c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2040106030 tx=0x7f2030005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:18.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.778+0000 7f20454ff640 1 -- 172.21.15.112:0/989533591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20401083d0 msgr2=0x7f20401996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:18.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.778+0000 7f20454ff640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20401083d0 0x7f20401996b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2034032fd0 tx=0x7f2034031a20 comp rx=0 tx=0).stop 2024-09-21T03:36:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.778+0000 7f20454ff640 1 -- 172.21.15.112:0/989533591 shutdown_connections 2024-09-21T03:36:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.778+0000 7f20454ff640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f20280b6700 0x7f20280b8c10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.778+0000 7f20454ff640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2040108d20 0x7f2040199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.778+0000 7f20454ff640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20401083d0 0x7f20401996b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.778+0000 7f20454ff640 1 --2- 172.21.15.112:0/989533591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2040102ae0 0x7f2040199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.778+0000 7f20454ff640 1 -- 172.21.15.112:0/989533591 >> 172.21.15.112:0/989533591 conn(0x7f2040079760 msgr2=0x7f20401044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.779+0000 7f20454ff640 1 -- 172.21.15.112:0/989533591 shutdown_connections 2024-09-21T03:36:18.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:18.779+0000 7f20454ff640 1 -- 172.21.15.112:0/989533591 wait complete. 2024-09-21T03:36:19.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:18 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/989533591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:19.062 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:18 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/989533591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:19.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:18 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/989533591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:36:19.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 763772 10.79 MB/sec execute 259 sec latency 41.375 ms 2024-09-21T03:36:19.707 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:36:19.967 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:20.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:19 smithi112 ceph-mon[31442]: pgmap v541: 97 pgs: 97 active+clean; 543 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 290 op/s 2024-09-21T03:36:20.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:19 smithi203 ceph-mon[34723]: pgmap v541: 97 pgs: 97 active+clean; 543 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 290 op/s 2024-09-21T03:36:20.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:19 smithi187 ceph-mon[34603]: pgmap v541: 97 pgs: 97 active+clean; 543 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 290 op/s 2024-09-21T03:36:20.242 INFO:tasks.workunit.client.0.smithi112.stdout: 1 765389 10.77 MB/sec execute 260 sec latency 29.799 ms 2024-09-21T03:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.495+0000 7f7569fc4640 1 -- 172.21.15.112:0/1774465577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564108cf0 msgr2=0x7f756410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.495+0000 7f7569fc4640 1 --2- 172.21.15.112:0/1774465577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564108cf0 0x7f756410f520 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f755c009ca0 tx=0x7f755c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.495+0000 7f7569fc4640 1 -- 172.21.15.112:0/1774465577 shutdown_connections 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.495+0000 7f7569fc4640 1 --2- 172.21.15.112:0/1774465577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564108cf0 0x7f756410f520 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.495+0000 7f7569fc4640 1 --2- 172.21.15.112:0/1774465577 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75641083a0 0x7f75641087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.495+0000 7f7569fc4640 1 --2- 172.21.15.112:0/1774465577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7564102a60 0x7f7564102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.495+0000 7f7569fc4640 1 -- 172.21.15.112:0/1774465577 >> 172.21.15.112:0/1774465577 conn(0x7f7564079730 msgr2=0x7f7564079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.495+0000 7f7569fc4640 1 -- 172.21.15.112:0/1774465577 shutdown_connections 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.495+0000 7f7569fc4640 1 -- 172.21.15.112:0/1774465577 wait complete. 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.496+0000 7f7569fc4640 1 Processor -- start 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.496+0000 7f7569fc4640 1 -- start start 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.496+0000 7f7569fc4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f7564199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.496+0000 7f7569fc4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75641083a0 0x7f75641996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.496+0000 7f7569fc4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7564108cf0 0x7f7564199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.496+0000 7f7569fc4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7564112960 con 0x7f75641083a0 2024-09-21T03:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.496+0000 7f7569fc4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75641127e0 con 0x7f7564102a60 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.496+0000 7f7569fc4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7564112ae0 con 0x7f7564108cf0 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f75637fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f7564199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75641083a0 0x7f75641996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f7563fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7564108cf0 0x7f7564199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f75637fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f7564199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:54626/0 (socket says 172.21.15.112:54626) 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f75637fe640 1 -- 172.21.15.112:0/1314067120 learned_addr learned my addr 172.21.15.112:0/1314067120 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f75637fe640 1 -- 172.21.15.112:0/1314067120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7564108cf0 msgr2=0x7f7564199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f75637fe640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7564108cf0 0x7f7564199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f75637fe640 1 -- 172.21.15.112:0/1314067120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75641083a0 msgr2=0x7f75641996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f75637fe640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75641083a0 0x7f75641996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f75637fe640 1 -- 172.21.15.112:0/1314067120 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75641a87e0 con 0x7f7564102a60 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f7563fff640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7564108cf0 0x7f7564199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f75637fe640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f7564199160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f755400d770 tx=0x7f755400db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f7560ff9640 1 -- 172.21.15.112:0/1314067120 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554017070 con 0x7f7564102a60 2024-09-21T03:36:20.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f7560ff9640 1 -- 172.21.15.112:0/1314067120 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7554009820 con 0x7f7564102a60 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f7560ff9640 1 -- 172.21.15.112:0/1314067120 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554004bc0 con 0x7f7564102a60 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.497+0000 7f7569fc4640 1 -- 172.21.15.112:0/1314067120 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75641a8a70 con 0x7f7564102a60 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.498+0000 7f7569fc4640 1 -- 172.21.15.112:0/1314067120 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75641a8f60 con 0x7f7564102a60 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.500+0000 7f7570538740 1 Processor -- start 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.500+0000 7f7570538740 1 -- start start 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.500+0000 7f7560ff9640 1 -- 172.21.15.112:0/1314067120 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f75540099d0 con 0x7f7564102a60 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.500+0000 7f7570538740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7fb070 0x55bbdcba26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.500+0000 7f7570538740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcba2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.500+0000 7f7570538740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7da390 0x55bbdcba3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.500+0000 7f7570538740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcb18ea0 con 0x55bbdc7d21b0 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.500+0000 7f7570538740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcb082d0 con 0x55bbdc7fb070 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.500+0000 7f7570538740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcb0c0e0 con 0x55bbdc7da390 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.500+0000 7f75637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcba2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f75637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcba2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48188/0 (socket says 172.21.15.112:48188) 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f75637fe640 1 -- 172.21.15.112:0/2412906914 learned_addr learned my addr 172.21.15.112:0/2412906914 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f7562ffd640 1 --2- 172.21.15.112:0/2412906914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7da390 0x55bbdcba3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f7563fff640 1 --2- 172.21.15.112:0/2412906914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7fb070 0x55bbdcba26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f75637fe640 1 -- 172.21.15.112:0/2412906914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7da390 msgr2=0x55bbdcba3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f75637fe640 1 --2- 172.21.15.112:0/2412906914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7da390 0x55bbdcba3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f75637fe640 1 -- 172.21.15.112:0/2412906914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7fb070 msgr2=0x55bbdcba26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f75637fe640 1 --2- 172.21.15.112:0/2412906914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7fb070 0x55bbdcba26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f75637fe640 1 -- 172.21.15.112:0/2412906914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbdc866650 con 0x55bbdc7d21b0 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f7562ffd640 1 --2- 172.21.15.112:0/2412906914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7da390 0x55bbdcba3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f75637fe640 1 --2- 172.21.15.112:0/2412906914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcba2be0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f75540e4840 tx=0x7f75540ef910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afd68e219eca6b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f7560ff9640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 0x7f754c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f7549ffb640 1 -- 172.21.15.112:0/2412906914 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540fa070 con 0x55bbdc7d21b0 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f7563fff640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 0x7f754c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f7549ffb640 1 -- 172.21.15.112:0/2412906914 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75540acae0 con 0x55bbdc7d21b0 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.501+0000 7f7549ffb640 1 -- 172.21.15.112:0/2412906914 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755409e270 con 0x55bbdc7d21b0 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.502+0000 7f7560ff9640 1 -- 172.21.15.112:0/1314067120 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f75540e7150 con 0x7f7564102a60 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.502+0000 7f7570538740 1 -- 172.21.15.112:0/2412906914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 msgr2=0x55bbdcba2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.502+0000 7f7563fff640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 0x7f754c0b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f755c02f500 tx=0x7f755c02f310 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.502+0000 7f7570538740 1 --2- 172.21.15.112:0/2412906914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcba2be0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f75540e4840 tx=0x7f75540ef910 comp rx=0 tx=0).stop 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.502+0000 7f7570538740 1 -- 172.21.15.112:0/2412906914 shutdown_connections 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.502+0000 7f7570538740 1 --2- 172.21.15.112:0/2412906914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7da390 0x55bbdcba3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.502+0000 7f7570538740 1 --2- 172.21.15.112:0/2412906914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcba2be0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.502+0000 7f7570538740 1 --2- 172.21.15.112:0/2412906914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7fb070 0x55bbdcba26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.502+0000 7f7570538740 1 -- 172.21.15.112:0/2412906914 >> 172.21.15.112:0/2412906914 conn(0x55bbdc7ef460 msgr2=0x55bbdcba0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.502+0000 7f7570538740 1 -- 172.21.15.112:0/2412906914 shutdown_connections 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.502+0000 7f7570538740 1 -- 172.21.15.112:0/2412906914 wait complete. 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.503+0000 7f7570538740 1 -- start start 2024-09-21T03:36:20.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.503+0000 7f7570538740 1 Processor -- start 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.503+0000 7f7570538740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbadd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.503+0000 7f7570538740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7da390 0x55bbdcbae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.503+0000 7f7570538740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7fb070 0x55bbdcbba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.503+0000 7f7570538740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bbdc969700 con 0x55bbdc7da390 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.503+0000 7f7570538740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcaa3160 con 0x55bbdc7fb070 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.503+0000 7f7570538740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bbdcbb0e80 con 0x55bbdc7d21b0 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.503+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7da390 0x55bbdcbae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.503+0000 7f7570538740 10 client.0 _ll_register_callbacks cb 0x55bbdcabc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.503+0000 7f75637fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbadd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f75637fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbadd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:54376/0 (socket says 172.21.15.112:54376) 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f75637fe640 1 -- 172.21.15.112:0/3519657207 learned_addr learned my addr 172.21.15.112:0/3519657207 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7563fff640 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7fb070 0x55bbdcbba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7562ffd640 1 -- 172.21.15.112:0/3519657207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7d21b0 msgr2=0x55bbdcbadd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7562ffd640 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbadd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7562ffd640 1 -- 172.21.15.112:0/3519657207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7fb070 msgr2=0x55bbdcbba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7562ffd640 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7fb070 0x55bbdcbba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7562ffd640 1 -- 172.21.15.112:0/3519657207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbdcad0c90 con 0x55bbdc7da390 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f75637fe640 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbadd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7563fff640 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7fb070 0x55bbdcbba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7562ffd640 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7da390 0x55bbdcbae2d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f756406aaa0 tx=0x7f7558010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ba1f809e2d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7542ffd640 1 -- 172.21.15.112:0/3519657207 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755800fb90 con 0x55bbdc7da390 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7542ffd640 1 -- 172.21.15.112:0/3519657207 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f7558010e00 con 0x55bbdc7da390 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.504+0000 7f7570538740 1 -- 172.21.15.112:0/3519657207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75540acae0 con 0x55bbdc7da390 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f7542ffd640 1 -- 172.21.15.112:0/3519657207 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f755800fd30 con 0x55bbdc7da390 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f7570538740 10 client.16740 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:20.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f7570538740 1 -- 172.21.15.112:0/3519657207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bbdcbbcbf0 con 0x55bbdc7da390 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f7542ffd640 1 -- 172.21.15.112:0/3519657207 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75540acae0 con 0x55bbdc7da390 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f7542ffd640 1 -- 172.21.15.112:0/3519657207 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f755800e420 con 0x55bbdc7da390 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f7570538740 10 client.16740 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f7570538740 10 client.16740 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f7570538740 20 client.16740 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f7570538740 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55bbdc7b99f0 0x55bbdcbc0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f7570538740 4 client.16740 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f7570538740 1 -- 172.21.15.112:0/3519657207 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbdcb1adf0 con 0x55bbdc7b99f0 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.505+0000 7f75637fe640 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55bbdc7b99f0 0x55bbdcbc0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.506+0000 7f7542ffd640 10 client.16740 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.506+0000 7f75637fe640 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55bbdc7b99f0 0x55bbdcbc0180 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f406fb9e1cca621e server_cookie=6ce3f0311dce391a in_seq=0 out_seq=0 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7542ffd640 1 -- 172.21.15.112:0/3519657207 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bbdcb1adf0 con 0x55bbdc7b99f0 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7542ffd640 10 client.16740 handle_command_reply: tid=0 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 1 client.16740 shutdown 2024-09-21T03:36:20.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 1 -- 172.21.15.112:0/3519657207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7da390 msgr2=0x55bbdcbae2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7da390 0x55bbdcbae2d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f756406aaa0 tx=0x7f7558010c10 comp rx=0 tx=0).stop 2024-09-21T03:36:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 1 -- 172.21.15.112:0/3519657207 shutdown_connections 2024-09-21T03:36:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55bbdc7b99f0 0x55bbdcbc0180 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7fb070 0x55bbdcbba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7542ffd640 0 client.16740 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:36:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7da390 0x55bbdcbae2d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 1 --2- 172.21.15.112:0/3519657207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbadd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 1 -- 172.21.15.112:0/3519657207 >> 172.21.15.112:0/3519657207 conn(0x55bbdc7ef460 msgr2=0x55bbdcba5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 1 -- 172.21.15.112:0/3519657207 shutdown_connections 2024-09-21T03:36:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 1 -- 172.21.15.112:0/3519657207 wait complete. 2024-09-21T03:36:20.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.507+0000 7f7570538740 20 client.16740 trim_cache size 0 max 16384 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.522+0000 7f7570538740 1 Processor -- start 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7570538740 1 -- start start 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7570538740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7b99f0 0x55bbdcba3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7570538740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbaaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7570538740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7da390 0x55bbdcbab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7570538740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75641b4160 con 0x55bbdc7d21b0 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7570538740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75641b1a30 con 0x55bbdc7b99f0 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7570538740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75641af8a0 con 0x55bbdc7da390 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbaaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f75637fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7b99f0 0x55bbdcba3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbaaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48206/0 (socket says 172.21.15.112:48206) 2024-09-21T03:36:20.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7562ffd640 1 -- 172.21.15.112:0/3531514949 learned_addr learned my addr 172.21.15.112:0/3531514949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7563fff640 1 --2- 172.21.15.112:0/3531514949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7da390 0x55bbdcbab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f75637fe640 1 -- 172.21.15.112:0/3531514949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7da390 msgr2=0x55bbdcbab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f75637fe640 1 --2- 172.21.15.112:0/3531514949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7da390 0x55bbdcbab4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f75637fe640 1 -- 172.21.15.112:0/3531514949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 msgr2=0x55bbdcbaaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f75637fe640 1 --2- 172.21.15.112:0/3531514949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbaaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f75637fe640 1 -- 172.21.15.112:0/3531514949 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbdcba34c0 con 0x55bbdc7b99f0 2024-09-21T03:36:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.523+0000 7f7562ffd640 1 --2- 172.21.15.112:0/3531514949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbaaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.524+0000 7f75637fe640 1 --2- 172.21.15.112:0/3531514949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7b99f0 0x55bbdcba3de0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f75540accd0 tx=0x7f75540ead50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15016b344c4531e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.524+0000 7f75497fa640 1 -- 172.21.15.112:0/3531514949 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540a24c0 con 0x55bbdc7b99f0 2024-09-21T03:36:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.524+0000 7f75497fa640 1 -- 172.21.15.112:0/3531514949 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75540eaa70 con 0x55bbdc7b99f0 2024-09-21T03:36:20.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.524+0000 7f75497fa640 1 -- 172.21.15.112:0/3531514949 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540a2a40 con 0x55bbdc7b99f0 2024-09-21T03:36:20.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.525+0000 7f7570538740 1 -- 172.21.15.112:0/3531514949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7b99f0 msgr2=0x55bbdcba3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.525+0000 7f7570538740 1 --2- 172.21.15.112:0/3531514949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7b99f0 0x55bbdcba3de0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f75540accd0 tx=0x7f75540ead50 comp rx=0 tx=0).stop 2024-09-21T03:36:20.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.525+0000 7f7570538740 1 -- 172.21.15.112:0/3531514949 shutdown_connections 2024-09-21T03:36:20.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.525+0000 7f7570538740 1 --2- 172.21.15.112:0/3531514949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7da390 0x55bbdcbab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.525+0000 7f7570538740 1 --2- 172.21.15.112:0/3531514949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbaaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.525+0000 7f7570538740 1 --2- 172.21.15.112:0/3531514949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7b99f0 0x55bbdcba3de0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.525+0000 7f7570538740 1 -- 172.21.15.112:0/3531514949 >> 172.21.15.112:0/3531514949 conn(0x55bbdc7ef460 msgr2=0x55bbdcba36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.525+0000 7f7570538740 1 -- 172.21.15.112:0/3531514949 shutdown_connections 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.525+0000 7f7570538740 1 -- 172.21.15.112:0/3531514949 wait complete. 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.525+0000 7f7570538740 1 -- start start 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7570538740 1 Processor -- start 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7570538740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7b99f0 0x55bbdcbb6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7570538740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbb3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7570538740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7da390 0x55bbdcbb4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7570538740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75641afef0 con 0x55bbdc7d21b0 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7570538740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75641b1360 con 0x55bbdc7da390 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7570538740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75641b05c0 con 0x55bbdc7b99f0 2024-09-21T03:36:20.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f75637fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7b99f0 0x55bbdcbb6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbb3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbb3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48210/0 (socket says 172.21.15.112:48210) 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f75637fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7b99f0 0x55bbdcbb6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:54392/0 (socket says 172.21.15.112:54392) 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7562ffd640 1 -- 172.21.15.112:0/3310649919 learned_addr learned my addr 172.21.15.112:0/3310649919 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7570538740 10 client.0 _ll_register_callbacks cb 0x7f7558010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.526+0000 7f7563fff640 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7da390 0x55bbdcbb4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7562ffd640 1 -- 172.21.15.112:0/3310649919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7b99f0 msgr2=0x55bbdcbb6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7562ffd640 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7b99f0 0x55bbdcbb6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7562ffd640 1 -- 172.21.15.112:0/3310649919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7da390 msgr2=0x55bbdcbb4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7562ffd640 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7da390 0x55bbdcbb4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7562ffd640 1 -- 172.21.15.112:0/3310649919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbdcbb4d00 con 0x55bbdc7d21b0 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f75637fe640 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7b99f0 0x55bbdcbb6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7563fff640 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7da390 0x55bbdcbb4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:20.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7562ffd640 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbb3ef0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7558010010 tx=0x7f7558005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab15de07a527535 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7542ffd640 1 -- 172.21.15.112:0/3310649919 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755800ec90 con 0x55bbdc7d21b0 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7542ffd640 1 -- 172.21.15.112:0/3310649919 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f7558016040 con 0x55bbdc7d21b0 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7570538740 1 -- 172.21.15.112:0/3310649919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75540eaa70 con 0x55bbdc7d21b0 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7542ffd640 1 -- 172.21.15.112:0/3310649919 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f755800e3b0 con 0x55bbdc7d21b0 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7570538740 10 client.16746 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7570538740 1 -- 172.21.15.112:0/3310649919 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bbdcc2abd0 con 0x55bbdc7d21b0 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.527+0000 7f7542ffd640 1 -- 172.21.15.112:0/3310649919 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75540eaa70 con 0x55bbdc7d21b0 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.528+0000 7f7542ffd640 1 -- 172.21.15.112:0/3310649919 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7558034420 con 0x55bbdc7d21b0 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.528+0000 7f7570538740 10 client.16746 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.528+0000 7f7570538740 10 client.16746 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.528+0000 7f7570538740 20 client.16746 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:20.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.528+0000 7f7570538740 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55bbdc7fb070 0x55bbdcc2d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.528+0000 7f7570538740 4 client.16746 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.528+0000 7f7570538740 1 -- 172.21.15.112:0/3310649919 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbdcbae680 con 0x55bbdc7fb070 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.528+0000 7f75637fe640 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55bbdc7fb070 0x55bbdcc2d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.528+0000 7f7542ffd640 10 client.16746 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f75637fe640 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55bbdc7fb070 0x55bbdcc2d300 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=723e52d06a665f33 server_cookie=3ab026209a6ea80a in_seq=0 out_seq=0 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7542ffd640 1 -- 172.21.15.112:0/3310649919 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbdcbae680 con 0x55bbdc7fb070 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7542ffd640 10 client.16746 handle_command_reply: tid=0 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 1 client.16746 shutdown 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 1 -- 172.21.15.112:0/3310649919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 msgr2=0x55bbdcbb3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbb3ef0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7558010010 tx=0x7f7558005670 comp rx=0 tx=0).stop 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 1 -- 172.21.15.112:0/3310649919 shutdown_connections 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55bbdc7fb070 0x55bbdcc2d300 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbdc7da390 0x55bbdcbb4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bbdc7d21b0 0x55bbdcbb3ef0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7542ffd640 0 client.16746 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:36:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 1 --2- 172.21.15.112:0/3310649919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bbdc7b99f0 0x55bbdcbb6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 1 -- 172.21.15.112:0/3310649919 >> 172.21.15.112:0/3310649919 conn(0x55bbdc7ef460 msgr2=0x55bbdcbaf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 1 -- 172.21.15.112:0/3310649919 shutdown_connections 2024-09-21T03:36:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 1 -- 172.21.15.112:0/3310649919 wait complete. 2024-09-21T03:36:20.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.529+0000 7f7570538740 20 client.16746 trim_cache size 0 max 16384 2024-09-21T03:36:20.536 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:36:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 -- 172.21.15.112:0/1314067120 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 msgr2=0x7f754c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 0x7f754c0b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f755c02f500 tx=0x7f755c02f310 comp rx=0 tx=0).stop 2024-09-21T03:36:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 -- 172.21.15.112:0/1314067120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 msgr2=0x7f7564199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f7564199160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f755400d770 tx=0x7f755400db70 comp rx=0 tx=0).stop 2024-09-21T03:36:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 -- 172.21.15.112:0/1314067120 shutdown_connections 2024-09-21T03:36:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 0x7f754c0b8b90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7564108cf0 0x7f7564199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75641083a0 0x7f75641996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 --2- 172.21.15.112:0/1314067120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f7564199160 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 -- 172.21.15.112:0/1314067120 >> 172.21.15.112:0/1314067120 conn(0x7f7564079730 msgr2=0x7f7564104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:20.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 -- 172.21.15.112:0/1314067120 shutdown_connections 2024-09-21T03:36:20.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:20.530+0000 7f7569fc4640 1 -- 172.21.15.112:0/1314067120 wait complete. 2024-09-21T03:36:21.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 767496 10.78 MB/sec execute 261 sec latency 34.042 ms 2024-09-21T03:36:21.341 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:36:21.606 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:22.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:21 smithi112 ceph-mon[31442]: pgmap v542: 97 pgs: 97 active+clean; 544 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 259 op/s 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.104+0000 7f44bac10640 1 -- 172.21.15.112:0/888270936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b4108cf0 msgr2=0x7f44b410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.104+0000 7f44bac10640 1 --2- 172.21.15.112:0/888270936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b4108cf0 0x7f44b410f520 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009ca0 tx=0x7f44ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.104+0000 7f44bac10640 1 -- 172.21.15.112:0/888270936 shutdown_connections 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.104+0000 7f44bac10640 1 --2- 172.21.15.112:0/888270936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b4108cf0 0x7f44b410f520 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.104+0000 7f44bac10640 1 --2- 172.21.15.112:0/888270936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b41083a0 0x7f44b41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.104+0000 7f44bac10640 1 --2- 172.21.15.112:0/888270936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b4102a60 0x7f44b4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.104+0000 7f44bac10640 1 -- 172.21.15.112:0/888270936 >> 172.21.15.112:0/888270936 conn(0x7f44b4079730 msgr2=0x7f44b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.104+0000 7f44bac10640 1 -- 172.21.15.112:0/888270936 shutdown_connections 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.104+0000 7f44bac10640 1 -- 172.21.15.112:0/888270936 wait complete. 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44bac10640 1 Processor -- start 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44bac10640 1 -- start start 2024-09-21T03:36:22.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44bac10640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b4102a60 0x7f44b419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44bac10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b41083a0 0x7f44b419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44bac10640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b4108cf0 0x7f44b419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44bac10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4112960 con 0x7f44b41083a0 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44bac10640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41127e0 con 0x7f44b4108cf0 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44bac10640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4112ae0 con 0x7f44b4102a60 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b41083a0 0x7f44b419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44b89ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b4102a60 0x7f44b419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44b91ad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b4108cf0 0x7f44b419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44b89ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b4102a60 0x7f44b419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:54408/0 (socket says 172.21.15.112:54408) 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.105+0000 7f44b89ac640 1 -- 172.21.15.112:0/460182623 learned_addr learned my addr 172.21.15.112:0/460182623 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44b91ad640 1 -- 172.21.15.112:0/460182623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b4102a60 msgr2=0x7f44b419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44b91ad640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b4102a60 0x7f44b419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44b91ad640 1 -- 172.21.15.112:0/460182623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b41083a0 msgr2=0x7f44b419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44b91ad640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b41083a0 0x7f44b419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44b91ad640 1 -- 172.21.15.112:0/460182623 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b41a03b0 con 0x7f44b4108cf0 2024-09-21T03:36:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44b89ac640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b4102a60 0x7f44b419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44b3fff640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b41083a0 0x7f44b419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44b91ad640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b4108cf0 0x7f44b419fc80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f44ac02ade0 tx=0x7f44ac031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44b1ffb640 1 -- 172.21.15.112:0/460182623 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac04c070 con 0x7f44b4108cf0 2024-09-21T03:36:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44bac10640 1 -- 172.21.15.112:0/460182623 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b41a0640 con 0x7f44b4108cf0 2024-09-21T03:36:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44b1ffb640 1 -- 172.21.15.112:0/460182623 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44ac003e20 con 0x7f44b4108cf0 2024-09-21T03:36:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44b1ffb640 1 -- 172.21.15.112:0/460182623 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac0042a0 con 0x7f44b4108cf0 2024-09-21T03:36:22.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.106+0000 7f44bac10640 1 -- 172.21.15.112:0/460182623 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b40ff550 con 0x7f44b4108cf0 2024-09-21T03:36:22.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.109+0000 7f44c1184740 1 Processor -- start 2024-09-21T03:36:22.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b1ffb640 1 -- 172.21.15.112:0/460182623 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f44ac04f070 con 0x7f44b4108cf0 2024-09-21T03:36:22.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44c1184740 1 -- start start 2024-09-21T03:36:22.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44c1184740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835446070 0x5648357ed2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44c1184740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 0x5648357ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44c1184740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835425390 0x5648357edd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44c1184740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564835763ea0 con 0x564835446070 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44c1184740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5648357532d0 con 0x564835425390 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44c1184740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5648357570e0 con 0x56483541d1b0 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835446070 0x5648357ed2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b89ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 0x5648357ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835446070 0x5648357ed2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48230/0 (socket says 172.21.15.112:48230) 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b3fff640 1 -- 172.21.15.112:0/2902559539 learned_addr learned my addr 172.21.15.112:0/2902559539 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b89ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 0x5648357ed800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:54410/0 (socket says 172.21.15.112:54410) 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b3fff640 1 --2- 172.21.15.112:0/2902559539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835425390 0x5648357edd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b1ffb640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a40b6680 0x7f44a40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b89ac640 1 -- 172.21.15.112:0/2902559539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835425390 msgr2=0x5648357edd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b89ac640 1 --2- 172.21.15.112:0/2902559539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835425390 0x5648357edd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b89ac640 1 -- 172.21.15.112:0/2902559539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835446070 msgr2=0x5648357ed2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:22.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b89ac640 1 --2- 172.21.15.112:0/2902559539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835446070 0x5648357ed2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b89ac640 1 -- 172.21.15.112:0/2902559539 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648354b1650 con 0x56483541d1b0 2024-09-21T03:36:22.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.110+0000 7f44b3fff640 1 --2- 172.21.15.112:0/2902559539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835425390 0x5648357edd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:22.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.111+0000 7f44b91ad640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a40b6680 0x7f44a40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.111+0000 7f44b1ffb640 1 -- 172.21.15.112:0/460182623 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f44ac10bbb0 con 0x7f44b4108cf0 2024-09-21T03:36:22.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.111+0000 7f44b89ac640 1 --2- 172.21.15.112:0/2902559539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 0x5648357ed800 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f44a8011510 tx=0x7f44a80119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d189005cacccd7a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.111+0000 7f449effd640 1 -- 172.21.15.112:0/2902559539 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a801e070 con 0x56483541d1b0 2024-09-21T03:36:22.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.111+0000 7f449effd640 1 -- 172.21.15.112:0/2902559539 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a800fa70 con 0x56483541d1b0 2024-09-21T03:36:22.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.111+0000 7f449effd640 1 -- 172.21.15.112:0/2902559539 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8019b00 con 0x56483541d1b0 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.112+0000 7f44c1184740 1 -- 172.21.15.112:0/2902559539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 msgr2=0x5648357ed800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.112+0000 7f44b91ad640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a40b6680 0x7f44a40b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f44ac1093e0 tx=0x7f44ac0b9040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.112+0000 7f44c1184740 1 --2- 172.21.15.112:0/2902559539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 0x5648357ed800 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f44a8011510 tx=0x7f44a80119d0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.112+0000 7f44c1184740 1 -- 172.21.15.112:0/2902559539 shutdown_connections 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.112+0000 7f44c1184740 1 --2- 172.21.15.112:0/2902559539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835425390 0x5648357edd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.112+0000 7f44c1184740 1 --2- 172.21.15.112:0/2902559539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 0x5648357ed800 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.112+0000 7f44c1184740 1 --2- 172.21.15.112:0/2902559539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835446070 0x5648357ed2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.112+0000 7f44c1184740 1 -- 172.21.15.112:0/2902559539 >> 172.21.15.112:0/2902559539 conn(0x56483543a460 msgr2=0x5648357eb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.112+0000 7f44c1184740 1 -- 172.21.15.112:0/2902559539 shutdown_connections 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.112+0000 7f44c1184740 1 -- 172.21.15.112:0/2902559539 wait complete. 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.112+0000 7f44c1184740 1 -- start start 2024-09-21T03:36:22.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44c1184740 1 Processor -- start 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44c1184740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56483541d1b0 0x5648357f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44c1184740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44c1184740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564835446070 0x5648357f9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44c1184740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5648355b4700 con 0x564835425390 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44c1184740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5648356ee160 con 0x56483541d1b0 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44c1184740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5648357fbaa0 con 0x564835446070 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44c1184740 10 client.0 _ll_register_callbacks cb 0x5648357079e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44b89ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44b89ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357f8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48234/0 (socket says 172.21.15.112:48234) 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44b89ac640 1 -- 172.21.15.112:0/2887728937 learned_addr learned my addr 172.21.15.112:0/2887728937 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44b3fff640 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56483541d1b0 0x5648357f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.113+0000 7f44b3fff640 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564835446070 0x5648357f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f44b89ac640 1 -- 172.21.15.112:0/2887728937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564835446070 msgr2=0x5648357f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:22.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f44b89ac640 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564835446070 0x5648357f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f44b89ac640 1 -- 172.21.15.112:0/2887728937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56483541d1b0 msgr2=0x5648357f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f44b89ac640 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56483541d1b0 0x5648357f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f44b89ac640 1 -- 172.21.15.112:0/2887728937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56483571bc90 con 0x564835425390 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f44b3fff640 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56483541d1b0 0x5648357f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f44b89ac640 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357f8e20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f44a801c040 tx=0x7f44a8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6849f0ddfe1fab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f4493fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f4493fff640 1 -- 172.21.15.112:0/2887728937 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a801e040 con 0x564835425390 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f4493fff640 1 -- 172.21.15.112:0/2887728937 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f44a800f6c0 con 0x564835425390 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f44c1184740 1 -- 172.21.15.112:0/2887728937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a800fa70 con 0x564835425390 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f4493fff640 1 -- 172.21.15.112:0/2887728937 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44a8032050 con 0x564835425390 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f44c1184740 10 client.16758 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f44c1184740 1 -- 172.21.15.112:0/2887728937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564835807700 con 0x564835425390 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.114+0000 7f4493fff640 1 -- 172.21.15.112:0/2887728937 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a800fa70 con 0x564835425390 2024-09-21T03:36:22.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.115+0000 7f4493fff640 1 -- 172.21.15.112:0/2887728937 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f44a8021030 con 0x564835425390 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.115+0000 7f44c1184740 10 client.16758 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.115+0000 7f44c1184740 10 client.16758 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.115+0000 7f44c1184740 20 client.16758 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.115+0000 7f44c1184740 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5648354049f0 0x56483580ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.115+0000 7f44c1184740 4 client.16758 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.115+0000 7f44c1184740 1 -- 172.21.15.112:0/2887728937 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564835765df0 con 0x5648354049f0 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.115+0000 7f44b3fff640 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5648354049f0 0x56483580ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.115+0000 7f44b3fff640 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5648354049f0 0x56483580ab10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23ce5a0a71ba0e90 server_cookie=db4e0621d1b49d45 in_seq=0 out_seq=0 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.115+0000 7f4493fff640 10 client.16758 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f4493fff640 1 -- 172.21.15.112:0/2887728937 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564835765df0 con 0x5648354049f0 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f4493fff640 10 client.16758 handle_command_reply: tid=0 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f44c1184740 1 client.16758 shutdown 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f44c1184740 1 -- 172.21.15.112:0/2887728937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 msgr2=0x5648357f8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f44c1184740 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357f8e20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f44a801c040 tx=0x7f44a8019640 comp rx=0 tx=0).stop 2024-09-21T03:36:22.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f44c1184740 1 -- 172.21.15.112:0/2887728937 shutdown_connections 2024-09-21T03:36:22.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f44c1184740 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5648354049f0 0x56483580ab10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f44c1184740 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564835446070 0x5648357f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f44c1184740 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357f8e20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f44c1184740 1 --2- 172.21.15.112:0/2887728937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56483541d1b0 0x5648357f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f44c1184740 1 -- 172.21.15.112:0/2887728937 >> 172.21.15.112:0/2887728937 conn(0x56483543a460 msgr2=0x5648357f04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f4493fff640 0 client.16758 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:36:22.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f44c1184740 1 -- 172.21.15.112:0/2887728937 shutdown_connections 2024-09-21T03:36:22.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.116+0000 7f44c1184740 1 -- 172.21.15.112:0/2887728937 wait complete. 2024-09-21T03:36:22.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.117+0000 7f44c1184740 20 client.16758 trim_cache size 0 max 16384 2024-09-21T03:36:22.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.129+0000 7f44c1184740 1 Processor -- start 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44c1184740 1 -- start start 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44c1184740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56483541d1b0 0x5648357fa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44c1184740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835425390 0x5648357f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44c1184740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564835446070 0x5648357f6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44c1184740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41afcc0 con 0x56483541d1b0 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44c1184740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41ad590 con 0x564835425390 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44c1184740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41ab400 con 0x564835446070 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56483541d1b0 0x5648357fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44b3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56483541d1b0 0x5648357fa510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48242/0 (socket says 172.21.15.112:48242) 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44b3fff640 1 -- 172.21.15.112:0/1116103039 learned_addr learned my addr 172.21.15.112:0/1116103039 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44b3fff640 1 --2- 172.21.15.112:0/1116103039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564835446070 0x5648357f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44b89ac640 1 --2- 172.21.15.112:0/1116103039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835425390 0x5648357f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44b3fff640 1 -- 172.21.15.112:0/1116103039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564835446070 msgr2=0x5648357f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:22.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44b3fff640 1 --2- 172.21.15.112:0/1116103039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564835446070 0x5648357f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44b3fff640 1 -- 172.21.15.112:0/1116103039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835425390 msgr2=0x5648357f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:22.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44b3fff640 1 --2- 172.21.15.112:0/1116103039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835425390 0x5648357f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.130+0000 7f44b3fff640 1 -- 172.21.15.112:0/1116103039 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648357fb410 con 0x56483541d1b0 2024-09-21T03:36:22.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.131+0000 7f44b3fff640 1 --2- 172.21.15.112:0/1116103039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56483541d1b0 0x5648357fa510 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f44a0000c00 tx=0x7f44a0009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1c78f1e6229255 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.131+0000 7f449e7fc640 1 -- 172.21.15.112:0/1116103039 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a001c040 con 0x56483541d1b0 2024-09-21T03:36:22.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.131+0000 7f449e7fc640 1 -- 172.21.15.112:0/1116103039 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a0010800 con 0x56483541d1b0 2024-09-21T03:36:22.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.131+0000 7f449e7fc640 1 -- 172.21.15.112:0/1116103039 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a00044e0 con 0x56483541d1b0 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.132+0000 7f44c1184740 1 -- 172.21.15.112:0/1116103039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56483541d1b0 msgr2=0x5648357fa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.132+0000 7f44c1184740 1 --2- 172.21.15.112:0/1116103039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56483541d1b0 0x5648357fa510 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f44a0000c00 tx=0x7f44a0009ce0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.132+0000 7f44c1184740 1 -- 172.21.15.112:0/1116103039 shutdown_connections 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.132+0000 7f44c1184740 1 --2- 172.21.15.112:0/1116103039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564835446070 0x5648357f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.132+0000 7f44c1184740 1 --2- 172.21.15.112:0/1116103039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835425390 0x5648357f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.132+0000 7f44c1184740 1 --2- 172.21.15.112:0/1116103039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56483541d1b0 0x5648357fa510 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.132+0000 7f44c1184740 1 -- 172.21.15.112:0/1116103039 >> 172.21.15.112:0/1116103039 conn(0x56483543a460 msgr2=0x5648357f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.132+0000 7f44c1184740 1 -- 172.21.15.112:0/1116103039 shutdown_connections 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.132+0000 7f44c1184740 1 -- 172.21.15.112:0/1116103039 wait complete. 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.132+0000 7f44c1184740 1 -- start start 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.132+0000 7f44c1184740 1 Processor -- start 2024-09-21T03:36:22.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44c1184740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 0x5648357fe560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44c1184740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357feb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44c1184740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835446070 0x5648358733c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44c1184740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41aba50 con 0x564835425390 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44c1184740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41acec0 con 0x564835446070 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44c1184740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41ac120 con 0x56483541d1b0 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44c1184740 10 client.0 _ll_register_callbacks cb 0x7f44a801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44b89ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357feb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44b3fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 0x5648357fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44b89ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357feb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48248/0 (socket says 172.21.15.112:48248) 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44b89ac640 1 -- 172.21.15.112:0/2356248021 learned_addr learned my addr 172.21.15.112:0/2356248021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44b3fff640 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835446070 0x5648358733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44b89ac640 1 -- 172.21.15.112:0/2356248021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 msgr2=0x5648357fe560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44b89ac640 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 0x5648357fe560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44b89ac640 1 -- 172.21.15.112:0/2356248021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835446070 msgr2=0x5648358733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44b89ac640 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835446070 0x5648358733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44b89ac640 1 -- 172.21.15.112:0/2356248021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564835804c90 con 0x564835425390 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.133+0000 7f44b3fff640 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835446070 0x5648358733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f44b89ac640 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357feb40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f44a8005590 tx=0x7f44a8011270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b5cb0d2d75a6dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f4493fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f4493fff640 1 -- 172.21.15.112:0/2356248021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a801e040 con 0x564835425390 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f4493fff640 1 -- 172.21.15.112:0/2356248021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f44a800f7d0 con 0x564835425390 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f44c1184740 1 -- 172.21.15.112:0/2356248021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a0010800 con 0x564835425390 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f4493fff640 1 -- 172.21.15.112:0/2356248021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44a8009cf0 con 0x564835425390 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f44c1184740 10 client.16764 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f44c1184740 1 -- 172.21.15.112:0/2356248021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564835874ae0 con 0x564835425390 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f4493fff640 1 -- 172.21.15.112:0/2356248021 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a0010800 con 0x564835425390 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f4493fff640 1 -- 172.21.15.112:0/2356248021 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f44a8021070 con 0x564835425390 2024-09-21T03:36:22.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f44c1184740 10 client.16764 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f44c1184740 10 client.16764 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f44c1184740 20 client.16764 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f44c1184740 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5648353d9da0 0x564835877210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f44c1184740 4 client.16764 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.134+0000 7f44c1184740 1 -- 172.21.15.112:0/2356248021 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648357618a0 con 0x5648353d9da0 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.135+0000 7f44b3fff640 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5648353d9da0 0x564835877210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.135+0000 7f4493fff640 10 client.16764 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.135+0000 7f44b3fff640 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5648353d9da0 0x564835877210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1313f1dcb8c1027 server_cookie=7cdba15b7b103f44 in_seq=0 out_seq=0 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.135+0000 7f4493fff640 1 -- 172.21.15.112:0/2356248021 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648357618a0 con 0x5648353d9da0 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.135+0000 7f4493fff640 10 client.16764 handle_command_reply: tid=0 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.135+0000 7f44c1184740 1 client.16764 shutdown 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.135+0000 7f44c1184740 1 -- 172.21.15.112:0/2356248021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 msgr2=0x5648357feb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.135+0000 7f44c1184740 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357feb40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f44a8005590 tx=0x7f44a8011270 comp rx=0 tx=0).stop 2024-09-21T03:36:22.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.135+0000 7f44c1184740 1 -- 172.21.15.112:0/2356248021 shutdown_connections 2024-09-21T03:36:22.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.135+0000 7f44c1184740 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5648353d9da0 0x564835877210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44c1184740 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564835446070 0x5648358733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f4493fff640 0 client.16764 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:36:22.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44c1184740 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564835425390 0x5648357feb40 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44c1184740 1 --2- 172.21.15.112:0/2356248021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56483541d1b0 0x5648357fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44c1184740 1 -- 172.21.15.112:0/2356248021 >> 172.21.15.112:0/2356248021 conn(0x56483543a460 msgr2=0x5648357ee2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44c1184740 1 -- 172.21.15.112:0/2356248021 shutdown_connections 2024-09-21T03:36:22.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44c1184740 1 -- 172.21.15.112:0/2356248021 wait complete. 2024-09-21T03:36:22.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44c1184740 20 client.16764 trim_cache size 0 max 16384 2024-09-21T03:36:22.142 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:36:22.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44bac10640 1 -- 172.21.15.112:0/460182623 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a40b6680 msgr2=0x7f44a40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44bac10640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a40b6680 0x7f44a40b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f44ac1093e0 tx=0x7f44ac0b9040 comp rx=0 tx=0).stop 2024-09-21T03:36:22.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44bac10640 1 -- 172.21.15.112:0/460182623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b4108cf0 msgr2=0x7f44b419fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:22.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44bac10640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b4108cf0 0x7f44b419fc80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f44ac02ade0 tx=0x7f44ac031660 comp rx=0 tx=0).stop 2024-09-21T03:36:22.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44bac10640 1 -- 172.21.15.112:0/460182623 shutdown_connections 2024-09-21T03:36:22.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44bac10640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44a40b6680 0x7f44a40b8b90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44bac10640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44b4108cf0 0x7f44b419fc80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44bac10640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44b41083a0 0x7f44b419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44bac10640 1 --2- 172.21.15.112:0/460182623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44b4102a60 0x7f44b419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.136+0000 7f44bac10640 1 -- 172.21.15.112:0/460182623 >> 172.21.15.112:0/460182623 conn(0x7f44b4079730 msgr2=0x7f44b406b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.137+0000 7f44bac10640 1 -- 172.21.15.112:0/460182623 shutdown_connections 2024-09-21T03:36:22.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:22.137+0000 7f44bac10640 1 -- 172.21.15.112:0/460182623 wait complete. 2024-09-21T03:36:22.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:21 smithi203 ceph-mon[34723]: pgmap v542: 97 pgs: 97 active+clean; 544 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 259 op/s 2024-09-21T03:36:22.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:21 smithi187 ceph-mon[34603]: pgmap v542: 97 pgs: 97 active+clean; 544 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 259 op/s 2024-09-21T03:36:22.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 769831 10.78 MB/sec execute 262 sec latency 29.981 ms 2024-09-21T03:36:23.083 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:36:23.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 771326 10.77 MB/sec execute 263 sec latency 21.877 ms 2024-09-21T03:36:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.909+0000 7f14775e4640 1 -- 172.21.15.112:0/2251496982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701094a0 msgr2=0x7f14701098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.909+0000 7f14775e4640 1 --2- 172.21.15.112:0/2251496982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701094a0 0x7f14701098b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1460009900 tx=0x7f146002a410 comp rx=0 tx=0).stop 2024-09-21T03:36:23.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.909+0000 7f14775e4640 1 -- 172.21.15.112:0/2251496982 shutdown_connections 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.909+0000 7f14775e4640 1 --2- 172.21.15.112:0/2251496982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1470109df0 0x7f1470110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.909+0000 7f14775e4640 1 --2- 172.21.15.112:0/2251496982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701094a0 0x7f14701098b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.909+0000 7f14775e4640 1 --2- 172.21.15.112:0/2251496982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1470103b60 0x7f1470103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.909+0000 7f14775e4640 1 -- 172.21.15.112:0/2251496982 >> 172.21.15.112:0/2251496982 conn(0x7f147007b850 msgr2=0x7f147007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.909+0000 7f14775e4640 1 -- 172.21.15.112:0/2251496982 shutdown_connections 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.909+0000 7f14775e4640 1 -- 172.21.15.112:0/2251496982 wait complete. 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.910+0000 7f14775e4640 1 Processor -- start 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.910+0000 7f14775e4640 1 -- start start 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.910+0000 7f14775e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1470103b60 0x7f147019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.910+0000 7f14775e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701094a0 0x7f147019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.910+0000 7f14775e4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1470109df0 0x7f147019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.910+0000 7f14775e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1470113ab0 con 0x7f14701094a0 2024-09-21T03:36:23.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.910+0000 7f14775e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1470113930 con 0x7f1470103b60 2024-09-21T03:36:23.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.910+0000 7f14775e4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1470113c30 con 0x7f1470109df0 2024-09-21T03:36:23.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1474b7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701094a0 0x7f147019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1474b7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701094a0 0x7f147019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48270/0 (socket says 172.21.15.112:48270) 2024-09-21T03:36:23.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1474b7f640 1 -- 172.21.15.112:0/323127428 learned_addr learned my addr 172.21.15.112:0/323127428 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:23.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1475b81640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1470109df0 0x7f147019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1475380640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1470103b60 0x7f147019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1474b7f640 1 -- 172.21.15.112:0/323127428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1470109df0 msgr2=0x7f147019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1474b7f640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1470109df0 0x7f147019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1474b7f640 1 -- 172.21.15.112:0/323127428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1470103b60 msgr2=0x7f147019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1474b7f640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1470103b60 0x7f147019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1474b7f640 1 -- 172.21.15.112:0/323127428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14701a04d0 con 0x7f14701094a0 2024-09-21T03:36:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1475b81640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1470109df0 0x7f147019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f1474b7f640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701094a0 0x7f147019ab00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f146002a930 tx=0x7f1460031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f14667fc640 1 -- 172.21.15.112:0/323127428 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146003f030 con 0x7f14701094a0 2024-09-21T03:36:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f14667fc640 1 -- 172.21.15.112:0/323127428 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1460004490 con 0x7f14701094a0 2024-09-21T03:36:23.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f14775e4640 1 -- 172.21.15.112:0/323127428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14701a0760 con 0x7f14701094a0 2024-09-21T03:36:23.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.911+0000 7f14667fc640 1 -- 172.21.15.112:0/323127428 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460035070 con 0x7f14701094a0 2024-09-21T03:36:23.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.912+0000 7f14775e4640 1 -- 172.21.15.112:0/323127428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14701a6f20 con 0x7f14701094a0 2024-09-21T03:36:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.914+0000 7f147db58740 1 Processor -- start 2024-09-21T03:36:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.914+0000 7f14667fc640 1 -- 172.21.15.112:0/323127428 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f1460042030 con 0x7f14701094a0 2024-09-21T03:36:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.914+0000 7f147db58740 1 -- start start 2024-09-21T03:36:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.915+0000 7f147db58740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ef070 0x5566bd79a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.915+0000 7f147db58740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd79ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.915+0000 7f147db58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 0x5566bd79b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.915+0000 7f147db58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5566bd70cea0 con 0x5566bd3ce390 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.915+0000 7f147db58740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5566bd6fc2d0 con 0x5566bd3c61b0 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.915+0000 7f147db58740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5566bd7000e0 con 0x5566bd3ef070 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.915+0000 7f1474b7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 0x5566bd79b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.915+0000 7f1475b81640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ef070 0x5566bd79a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.915+0000 7f1474b7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 0x5566bd79b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48280/0 (socket says 172.21.15.112:48280) 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.915+0000 7f1474b7f640 1 -- 172.21.15.112:0/1716645015 learned_addr learned my addr 172.21.15.112:0/1716645015 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.915+0000 7f1475380640 1 --2- 172.21.15.112:0/1716645015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd79ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f1475b81640 1 -- 172.21.15.112:0/1716645015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 msgr2=0x5566bd79ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f1475b81640 1 --2- 172.21.15.112:0/1716645015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd79ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f1475b81640 1 -- 172.21.15.112:0/1716645015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 msgr2=0x5566bd79b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:23.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f1475b81640 1 --2- 172.21.15.112:0/1716645015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 0x5566bd79b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f1475b81640 1 -- 172.21.15.112:0/1716645015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bd45a650 con 0x5566bd3ef070 2024-09-21T03:36:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f1475380640 1 --2- 172.21.15.112:0/1716645015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd79ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f14667fc640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f145c0b6680 0x7f145c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f1475b81640 1 --2- 172.21.15.112:0/1716645015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ef070 0x5566bd79a6d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1468005fd0 tx=0x7f1468011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fe6aa4581cfcf15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f1475380640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f145c0b6680 0x7f145c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f14667fc640 1 -- 172.21.15.112:0/323127428 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f14601079b0 con 0x7f14701094a0 2024-09-21T03:36:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f145b7fe640 1 -- 172.21.15.112:0/1716645015 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468004d20 con 0x5566bd3ef070 2024-09-21T03:36:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f145b7fe640 1 -- 172.21.15.112:0/1716645015 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f146800f9a0 con 0x5566bd3ef070 2024-09-21T03:36:23.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.916+0000 7f145b7fe640 1 -- 172.21.15.112:0/1716645015 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468004470 con 0x5566bd3ef070 2024-09-21T03:36:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.917+0000 7f147db58740 1 -- 172.21.15.112:0/1716645015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ef070 msgr2=0x5566bd79a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.917+0000 7f1475380640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f145c0b6680 0x7f145c0b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5566bd799350 tx=0x7f146c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.917+0000 7f147db58740 1 --2- 172.21.15.112:0/1716645015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ef070 0x5566bd79a6d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1468005fd0 tx=0x7f1468011bf0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.917+0000 7f147db58740 1 -- 172.21.15.112:0/1716645015 shutdown_connections 2024-09-21T03:36:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.917+0000 7f147db58740 1 --2- 172.21.15.112:0/1716645015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 0x5566bd79b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.917+0000 7f147db58740 1 --2- 172.21.15.112:0/1716645015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd79ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.917+0000 7f147db58740 1 --2- 172.21.15.112:0/1716645015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ef070 0x5566bd79a6d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.917+0000 7f147db58740 1 -- 172.21.15.112:0/1716645015 >> 172.21.15.112:0/1716645015 conn(0x5566bd3e3460 msgr2=0x5566bd7985c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.917+0000 7f147db58740 1 -- 172.21.15.112:0/1716645015 shutdown_connections 2024-09-21T03:36:23.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.917+0000 7f147db58740 1 -- 172.21.15.112:0/1716645015 wait complete. 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f147db58740 1 -- start start 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f147db58740 1 Processor -- start 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f147db58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3c61b0 0x5566bd7a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f147db58740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ce390 0x5566bd7a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f147db58740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3ef070 0x5566bd7b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f147db58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5566bd55d700 con 0x5566bd3c61b0 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f147db58740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5566bd697160 con 0x5566bd3ef070 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f147db58740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5566bd7a8eb0 con 0x5566bd3ce390 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f147db58740 10 client.0 _ll_register_callbacks cb 0x5566bd6b09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f1475380640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3c61b0 0x5566bd7a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f1475380640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3c61b0 0x5566bd7a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48284/0 (socket says 172.21.15.112:48284) 2024-09-21T03:36:23.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f1475380640 1 -- 172.21.15.112:0/3865858130 learned_addr learned my addr 172.21.15.112:0/3865858130 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.918+0000 7f1474b7f640 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ce390 0x5566bd7a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1475b81640 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3ef070 0x5566bd7b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1475380640 1 -- 172.21.15.112:0/3865858130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ce390 msgr2=0x5566bd7a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1475380640 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ce390 0x5566bd7a62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1475380640 1 -- 172.21.15.112:0/3865858130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3ef070 msgr2=0x5566bd7b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1475380640 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3ef070 0x5566bd7b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1475380640 1 -- 172.21.15.112:0/3865858130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bd6c4c90 con 0x5566bd3c61b0 2024-09-21T03:36:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1474b7f640 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ce390 0x5566bd7a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:23.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1475380640 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3c61b0 0x5566bd7a5d70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f146c005bc0 tx=0x7f146c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05dceb7dbe8593a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1458ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1458ff9640 1 -- 172.21.15.112:0/3865858130 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c02e060 con 0x5566bd3c61b0 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1458ff9640 1 -- 172.21.15.112:0/3865858130 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f146c018030 con 0x5566bd3c61b0 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f147db58740 1 -- 172.21.15.112:0/3865858130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f146800f9a0 con 0x5566bd3c61b0 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.919+0000 7f1458ff9640 1 -- 172.21.15.112:0/3865858130 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f146c033480 con 0x5566bd3c61b0 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.920+0000 7f147db58740 10 client.16776 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.920+0000 7f147db58740 1 -- 172.21.15.112:0/3865858130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566bd7b4c30 con 0x5566bd3c61b0 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.920+0000 7f1458ff9640 1 -- 172.21.15.112:0/3865858130 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f146800f9a0 con 0x5566bd3c61b0 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.920+0000 7f1458ff9640 1 -- 172.21.15.112:0/3865858130 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f146c01c4a0 con 0x5566bd3c61b0 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.920+0000 7f147db58740 10 client.16776 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.920+0000 7f147db58740 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:36:23.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.920+0000 7f147db58740 20 client.16776 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.920+0000 7f147db58740 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5566bd3ad9f0 0x5566bd7b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.920+0000 7f147db58740 4 client.16776 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.920+0000 7f147db58740 1 -- 172.21.15.112:0/3865858130 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566bd70edf0 con 0x5566bd3ad9f0 2024-09-21T03:36:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.920+0000 7f1474b7f640 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5566bd3ad9f0 0x5566bd7b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.921+0000 7f1474b7f640 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5566bd3ad9f0 0x5566bd7b85b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=619666d82c1beec8 server_cookie=8f6ac24796491dd2 in_seq=0 out_seq=0 2024-09-21T03:36:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.921+0000 7f1458ff9640 10 client.16776 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.921+0000 7f1458ff9640 1 -- 172.21.15.112:0/3865858130 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5566bd70edf0 con 0x5566bd3ad9f0 2024-09-21T03:36:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.921+0000 7f1458ff9640 10 client.16776 handle_command_reply: tid=0 2024-09-21T03:36:23.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.921+0000 7f147db58740 1 client.16776 shutdown 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.921+0000 7f147db58740 1 -- 172.21.15.112:0/3865858130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3c61b0 msgr2=0x5566bd7a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.921+0000 7f147db58740 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3c61b0 0x5566bd7a5d70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f146c005bc0 tx=0x7f146c004730 comp rx=0 tx=0).stop 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.922+0000 7f147db58740 1 -- 172.21.15.112:0/3865858130 shutdown_connections 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.922+0000 7f147db58740 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5566bd3ad9f0 0x5566bd7b85b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.922+0000 7f147db58740 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3ef070 0x5566bd7b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.922+0000 7f147db58740 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ce390 0x5566bd7a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.922+0000 7f147db58740 1 --2- 172.21.15.112:0/3865858130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3c61b0 0x5566bd7a5d70 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.922+0000 7f147db58740 1 -- 172.21.15.112:0/3865858130 >> 172.21.15.112:0/3865858130 conn(0x5566bd3e3460 msgr2=0x5566bd79d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.922+0000 7f1458ff9640 0 client.16776 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.922+0000 7f147db58740 1 -- 172.21.15.112:0/3865858130 shutdown_connections 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.922+0000 7f147db58740 1 -- 172.21.15.112:0/3865858130 wait complete. 2024-09-21T03:36:23.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.922+0000 7f147db58740 20 client.16776 trim_cache size 0 max 16384 2024-09-21T03:36:23.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.936+0000 7f147db58740 1 Processor -- start 2024-09-21T03:36:23.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.936+0000 7f147db58740 1 -- start start 2024-09-21T03:36:23.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f147db58740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd7a6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f147db58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 0x5566bd7b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f147db58740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ef070 0x5566bd7a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f147db58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14701afe10 con 0x5566bd3ce390 2024-09-21T03:36:23.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f147db58740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14701ad6e0 con 0x5566bd3c61b0 2024-09-21T03:36:23.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f147db58740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14701ab550 con 0x5566bd3ef070 2024-09-21T03:36:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1474b7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 0x5566bd7b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1474b7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 0x5566bd7b20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48296/0 (socket says 172.21.15.112:48296) 2024-09-21T03:36:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1474b7f640 1 -- 172.21.15.112:0/1218224751 learned_addr learned my addr 172.21.15.112:0/1218224751 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1475380640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd7a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1475b81640 1 --2- 172.21.15.112:0/1218224751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ef070 0x5566bd7a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1474b7f640 1 -- 172.21.15.112:0/1218224751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ef070 msgr2=0x5566bd7a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1474b7f640 1 --2- 172.21.15.112:0/1218224751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ef070 0x5566bd7a2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1474b7f640 1 -- 172.21.15.112:0/1218224751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 msgr2=0x5566bd7a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1474b7f640 1 --2- 172.21.15.112:0/1218224751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd7a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1474b7f640 1 -- 172.21.15.112:0/1218224751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bd78e460 con 0x5566bd3ce390 2024-09-21T03:36:23.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1475380640 1 --2- 172.21.15.112:0/1218224751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd7a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.937+0000 7f1474b7f640 1 --2- 172.21.15.112:0/1218224751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 0x5566bd7b20d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f14600cab50 tx=0x7f146010b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78be71245b25e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.938+0000 7f145affd640 1 -- 172.21.15.112:0/1218224751 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460107e60 con 0x5566bd3ce390 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.938+0000 7f145affd640 1 -- 172.21.15.112:0/1218224751 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14600cd3f0 con 0x5566bd3ce390 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.938+0000 7f145affd640 1 -- 172.21.15.112:0/1218224751 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460049040 con 0x5566bd3ce390 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.939+0000 7f147db58740 1 -- 172.21.15.112:0/1218224751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 msgr2=0x5566bd7b20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.939+0000 7f147db58740 1 --2- 172.21.15.112:0/1218224751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 0x5566bd7b20d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f14600cab50 tx=0x7f146010b500 comp rx=0 tx=0).stop 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.939+0000 7f147db58740 1 -- 172.21.15.112:0/1218224751 shutdown_connections 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.939+0000 7f147db58740 1 --2- 172.21.15.112:0/1218224751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ef070 0x5566bd7a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.939+0000 7f147db58740 1 --2- 172.21.15.112:0/1218224751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ce390 0x5566bd7b20d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.939+0000 7f147db58740 1 --2- 172.21.15.112:0/1218224751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd7a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.939+0000 7f147db58740 1 -- 172.21.15.112:0/1218224751 >> 172.21.15.112:0/1218224751 conn(0x5566bd3e3460 msgr2=0x5566bd7a5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:23.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.939+0000 7f147db58740 1 -- 172.21.15.112:0/1218224751 shutdown_connections 2024-09-21T03:36:23.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.939+0000 7f147db58740 1 -- 172.21.15.112:0/1218224751 wait complete. 2024-09-21T03:36:23.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.939+0000 7f147db58740 1 -- start start 2024-09-21T03:36:23.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.939+0000 7f147db58740 1 Processor -- start 2024-09-21T03:36:23.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f147db58740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd7ae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f147db58740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ce390 0x5566bd7a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f147db58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ef070 0x5566bd7aa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f147db58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14701abba0 con 0x5566bd3ef070 2024-09-21T03:36:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f147db58740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14701ad010 con 0x5566bd3c61b0 2024-09-21T03:36:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f147db58740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14701ac270 con 0x5566bd3ce390 2024-09-21T03:36:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f147db58740 10 client.0 _ll_register_callbacks cb 0x7f146c01e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f1475b81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ef070 0x5566bd7aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f1474b7f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ce390 0x5566bd7a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f1475b81640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ef070 0x5566bd7aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48312/0 (socket says 172.21.15.112:48312) 2024-09-21T03:36:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f1475b81640 1 -- 172.21.15.112:0/2568566274 learned_addr learned my addr 172.21.15.112:0/2568566274 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.940+0000 7f1475380640 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd7ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.941+0000 7f1475b81640 1 -- 172.21.15.112:0/2568566274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ce390 msgr2=0x5566bd7a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:23.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.941+0000 7f1475b81640 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ce390 0x5566bd7a9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.941+0000 7f1475b81640 1 -- 172.21.15.112:0/2568566274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 msgr2=0x5566bd7ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.941+0000 7f1475b81640 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd7ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.941+0000 7f1475b81640 1 -- 172.21.15.112:0/2568566274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bd79cf10 con 0x5566bd3ef070 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.941+0000 7f1475380640 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd7ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.941+0000 7f1475b81640 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ef070 0x5566bd7aa300 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1468000f80 tx=0x7f1468003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1769d5ffed460f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.941+0000 7f1458ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.941+0000 7f1458ff9640 1 -- 172.21.15.112:0/2568566274 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146800fbb0 con 0x5566bd3ef070 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.941+0000 7f1458ff9640 1 -- 172.21.15.112:0/2568566274 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f146800fd50 con 0x5566bd3ef070 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.941+0000 7f147db58740 1 -- 172.21.15.112:0/2568566274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14600cd3f0 con 0x5566bd3ef070 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.942+0000 7f1458ff9640 1 -- 172.21.15.112:0/2568566274 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1468004fd0 con 0x5566bd3ef070 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.942+0000 7f147db58740 10 client.16782 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:23.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.942+0000 7f147db58740 1 -- 172.21.15.112:0/2568566274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566bd793be0 con 0x5566bd3ef070 2024-09-21T03:36:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.942+0000 7f1458ff9640 1 -- 172.21.15.112:0/2568566274 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14600cd3f0 con 0x5566bd3ef070 2024-09-21T03:36:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.942+0000 7f1458ff9640 1 -- 172.21.15.112:0/2568566274 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1468018070 con 0x5566bd3ef070 2024-09-21T03:36:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.942+0000 7f147db58740 10 client.16782 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.942+0000 7f147db58740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:36:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.942+0000 7f147db58740 20 client.16782 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.942+0000 7f147db58740 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5566bd382da0 0x5566bd796160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.942+0000 7f147db58740 4 client.16782 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:36:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.942+0000 7f147db58740 1 -- 172.21.15.112:0/2568566274 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566bd70a8a0 con 0x5566bd382da0 2024-09-21T03:36:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.943+0000 7f1475380640 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5566bd382da0 0x5566bd796160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.943+0000 7f1458ff9640 10 client.16782 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:23.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.943+0000 7f1475380640 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5566bd382da0 0x5566bd796160 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8f4bc0d3320797 server_cookie=bc03f392decb6f8a in_seq=0 out_seq=0 2024-09-21T03:36:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.946+0000 7f1458ff9640 1 -- 172.21.15.112:0/2568566274 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566bd70a8a0 con 0x5566bd382da0 2024-09-21T03:36:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.946+0000 7f1458ff9640 10 client.16782 handle_command_reply: tid=0 2024-09-21T03:36:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.946+0000 7f147db58740 1 client.16782 shutdown 2024-09-21T03:36:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f147db58740 1 -- 172.21.15.112:0/2568566274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ef070 msgr2=0x5566bd7aa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f147db58740 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ef070 0x5566bd7aa300 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1468000f80 tx=0x7f1468003da0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f147db58740 1 -- 172.21.15.112:0/2568566274 shutdown_connections 2024-09-21T03:36:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f147db58740 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5566bd382da0 0x5566bd796160 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f147db58740 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566bd3ef070 0x5566bd7aa300 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f147db58740 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566bd3ce390 0x5566bd7a9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f1458ff9640 0 client.16782 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f147db58740 1 --2- 172.21.15.112:0/2568566274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566bd3c61b0 0x5566bd7ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f147db58740 1 -- 172.21.15.112:0/2568566274 >> 172.21.15.112:0/2568566274 conn(0x5566bd3e3460 msgr2=0x5566bd79b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f147db58740 1 -- 172.21.15.112:0/2568566274 shutdown_connections 2024-09-21T03:36:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f147db58740 1 -- 172.21.15.112:0/2568566274 wait complete. 2024-09-21T03:36:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.947+0000 7f147db58740 20 client.16782 trim_cache size 0 max 16384 2024-09-21T03:36:23.953 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:36:23.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 -- 172.21.15.112:0/323127428 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f145c0b6680 msgr2=0x7f145c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:23.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f145c0b6680 0x7f145c0b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5566bd799350 tx=0x7f146c005f90 comp rx=0 tx=0).stop 2024-09-21T03:36:23.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 -- 172.21.15.112:0/323127428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701094a0 msgr2=0x7f147019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:23.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701094a0 0x7f147019ab00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f146002a930 tx=0x7f1460031a20 comp rx=0 tx=0).stop 2024-09-21T03:36:23.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 -- 172.21.15.112:0/323127428 shutdown_connections 2024-09-21T03:36:23.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f145c0b6680 0x7f145c0b8b90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1470109df0 0x7f147019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701094a0 0x7f147019ab00 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 --2- 172.21.15.112:0/323127428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1470103b60 0x7f147019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:23.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 -- 172.21.15.112:0/323127428 >> 172.21.15.112:0/323127428 conn(0x7f147007b850 msgr2=0x7f14701075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:23.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 -- 172.21.15.112:0/323127428 shutdown_connections 2024-09-21T03:36:23.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:23.948+0000 7f14775e4640 1 -- 172.21.15.112:0/323127428 wait complete. 2024-09-21T03:36:24.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:23 smithi203 ceph-mon[34723]: pgmap v543: 97 pgs: 97 active+clean; 548 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 257 op/s 2024-09-21T03:36:24.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:23 smithi187 ceph-mon[34603]: pgmap v543: 97 pgs: 97 active+clean; 548 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 257 op/s 2024-09-21T03:36:24.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 774023 10.79 MB/sec execute 264 sec latency 19.452 ms 2024-09-21T03:36:24.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:23 smithi112 ceph-mon[31442]: pgmap v543: 97 pgs: 97 active+clean; 548 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 257 op/s 2024-09-21T03:36:25.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 774992 10.76 MB/sec execute 265 sec latency 69.436 ms 2024-09-21T03:36:26.038 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:36:26.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:36:26.061 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:25 smithi112 ceph-mon[31442]: pgmap v544: 97 pgs: 97 active+clean; 548 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-09-21T03:36:26.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:25 smithi203 ceph-mon[34723]: pgmap v544: 97 pgs: 97 active+clean; 548 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-09-21T03:36:26.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:25 smithi187 ceph-mon[34603]: pgmap v544: 97 pgs: 97 active+clean; 548 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-09-21T03:36:26.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 777681 10.78 MB/sec execute 266 sec latency 17.056 ms 2024-09-21T03:36:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:26.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.788+0000 7fb204b0b640 1 -- 172.21.15.112:0/3451713889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb200108420 msgr2=0x7fb200108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.788+0000 7fb204b0b640 1 --2- 172.21.15.112:0/3451713889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb200108420 0x7fb200108830 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009900 tx=0x7fb1f402a410 comp rx=0 tx=0).stop 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.788+0000 7fb204b0b640 1 -- 172.21.15.112:0/3451713889 shutdown_connections 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.788+0000 7fb204b0b640 1 --2- 172.21.15.112:0/3451713889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb200108d70 0x7fb20010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.788+0000 7fb204b0b640 1 --2- 172.21.15.112:0/3451713889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb200108420 0x7fb200108830 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.788+0000 7fb204b0b640 1 --2- 172.21.15.112:0/3451713889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb200102ae0 0x7fb200102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.788+0000 7fb204b0b640 1 -- 172.21.15.112:0/3451713889 >> 172.21.15.112:0/3451713889 conn(0x7fb200079770 msgr2=0x7fb200079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.788+0000 7fb204b0b640 1 -- 172.21.15.112:0/3451713889 shutdown_connections 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.788+0000 7fb204b0b640 1 -- 172.21.15.112:0/3451713889 wait complete. 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.789+0000 7fb204b0b640 1 Processor -- start 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.789+0000 7fb204b0b640 1 -- start start 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb204b0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb200102ae0 0x7fb20019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb204b0b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb200108420 0x7fb20019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb204b0b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb200108d70 0x7fb20019fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb204b0b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb200112a30 con 0x7fb200102ae0 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb204b0b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001128b0 con 0x7fb200108420 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb204b0b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb200112bb0 con 0x7fb200108d70 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb200102ae0 0x7fb20019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb200102ae0 0x7fb20019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38316/0 (socket says 172.21.15.112:38316) 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fe59c640 1 -- 172.21.15.112:0/3289068225 learned_addr learned my addr 172.21.15.112:0/3289068225 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fed9d640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb200108d70 0x7fb20019fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb200108420 0x7fb20019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fe59c640 1 -- 172.21.15.112:0/3289068225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb200108d70 msgr2=0x7fb20019fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fe59c640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb200108d70 0x7fb20019fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fe59c640 1 -- 172.21.15.112:0/3289068225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb200108420 msgr2=0x7fb20019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fe59c640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb200108420 0x7fb20019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fe59c640 1 -- 172.21.15.112:0/3289068225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2001a0560 con 0x7fb200102ae0 2024-09-21T03:36:26.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb200108420 0x7fb20019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:36:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1fe59c640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb200102ae0 0x7fb20019a5f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000e9f0 tx=0x7fb1f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1ef7fe640 1 -- 172.21.15.112:0/3289068225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f000cd80 con 0x7fb200102ae0 2024-09-21T03:36:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb204b0b640 1 -- 172.21.15.112:0/3289068225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2001a0850 con 0x7fb200102ae0 2024-09-21T03:36:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1ef7fe640 1 -- 172.21.15.112:0/3289068225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f00054d0 con 0x7fb200102ae0 2024-09-21T03:36:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.790+0000 7fb1ef7fe640 1 -- 172.21.15.112:0/3289068225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0010690 con 0x7fb200102ae0 2024-09-21T03:36:26.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.791+0000 7fb204b0b640 1 -- 172.21.15.112:0/3289068225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2001a6fb0 con 0x7fb200102ae0 2024-09-21T03:36:26.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.793+0000 7fb1ef7fe640 1 -- 172.21.15.112:0/3289068225 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb1f0004e90 con 0x7fb200102ae0 2024-09-21T03:36:26.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.793+0000 7fb20b07f740 1 Processor -- start 2024-09-21T03:36:26.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb20b07f740 1 -- start start 2024-09-21T03:36:26.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb222350 0x558cfb5c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5c9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb201670 0x558cfb5c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb20b07f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558cfb53fe60 con 0x558cfb1f9490 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb20b07f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558cfb52f290 con 0x558cfb201670 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb20b07f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558cfb5330a0 con 0x558cfb222350 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1fe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1fe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38324/0 (socket says 172.21.15.112:38324) 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1fe59c640 1 -- 172.21.15.112:0/3990493515 learned_addr learned my addr 172.21.15.112:0/3990493515 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1fed9d640 1 --2- 172.21.15.112:0/3990493515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb222350 0x558cfb5c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/3990493515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb201670 0x558cfb5c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1ef7fe640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1e80b6680 0x7fb1e80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1fed9d640 1 -- 172.21.15.112:0/3990493515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb201670 msgr2=0x558cfb5c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1fed9d640 1 --2- 172.21.15.112:0/3990493515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb201670 0x558cfb5c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1fed9d640 1 -- 172.21.15.112:0/3990493515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 msgr2=0x558cfb5c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1fed9d640 1 --2- 172.21.15.112:0/3990493515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1fed9d640 1 -- 172.21.15.112:0/3990493515 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfb28d930 con 0x558cfb222350 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.794+0000 7fb1fe59c640 1 --2- 172.21.15.112:0/3990493515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.795+0000 7fb1ef7fe640 1 -- 172.21.15.112:0/3289068225 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7fb1f00e7b70 con 0x7fb200102ae0 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.795+0000 7fb1fed9d640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1e80b6680 0x7fb1e80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.795+0000 7fb1fed9d640 1 --2- 172.21.15.112:0/3990493515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb222350 0x558cfb5c9430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009f00 tx=0x7fb1f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a79af4a9be002e9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.795+0000 7fb1ecff9640 1 -- 172.21.15.112:0/3990493515 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8004250 con 0x558cfb222350 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.795+0000 7fb1ecff9640 1 -- 172.21.15.112:0/3990493515 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f8003e70 con 0x558cfb222350 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.795+0000 7fb1ecff9640 1 -- 172.21.15.112:0/3990493515 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8019d00 con 0x558cfb222350 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.796+0000 7fb20b07f740 1 -- 172.21.15.112:0/3990493515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb222350 msgr2=0x558cfb5c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.796+0000 7fb1fed9d640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1e80b6680 0x7fb1e80b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8020000 tx=0x7fb1f8021000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.796+0000 7fb20b07f740 1 --2- 172.21.15.112:0/3990493515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb222350 0x558cfb5c9430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009f00 tx=0x7fb1f8011810 comp rx=0 tx=0).stop 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.796+0000 7fb20b07f740 1 -- 172.21.15.112:0/3990493515 shutdown_connections 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.796+0000 7fb20b07f740 1 --2- 172.21.15.112:0/3990493515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb201670 0x558cfb5c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.796+0000 7fb20b07f740 1 --2- 172.21.15.112:0/3990493515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.796+0000 7fb20b07f740 1 --2- 172.21.15.112:0/3990493515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb222350 0x558cfb5c9430 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.796+0000 7fb20b07f740 1 -- 172.21.15.112:0/3990493515 >> 172.21.15.112:0/3990493515 conn(0x558cfb216740 msgr2=0x558cfb5c7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.796+0000 7fb20b07f740 1 -- 172.21.15.112:0/3990493515 shutdown_connections 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.796+0000 7fb20b07f740 1 -- 172.21.15.112:0/3990493515 wait complete. 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.796+0000 7fb20b07f740 1 -- start start 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb20b07f740 1 Processor -- start 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5d4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb201670 0x558cfb5d5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb222350 0x558cfb5e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb20b07f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558cfb390890 con 0x558cfb1f9490 2024-09-21T03:36:26.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb20b07f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558cfb4c9d40 con 0x558cfb201670 2024-09-21T03:36:26.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb20b07f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558cfb5d7c10 con 0x558cfb222350 2024-09-21T03:36:26.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb20b07f740 10 client.0 _ll_register_callbacks cb 0x558cfb52dc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:26.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb1fe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5d4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb1fe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5d4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38336/0 (socket says 172.21.15.112:38336) 2024-09-21T03:36:26.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb1fdd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb201670 0x558cfb5d5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb1fe59c640 1 -- 172.21.15.112:0/110599266 learned_addr learned my addr 172.21.15.112:0/110599266 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:26.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb1fed9d640 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb222350 0x558cfb5e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb1fe59c640 1 -- 172.21.15.112:0/110599266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb222350 msgr2=0x558cfb5e0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:26.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb1fe59c640 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb222350 0x558cfb5e0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb1fe59c640 1 -- 172.21.15.112:0/110599266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb201670 msgr2=0x558cfb5d5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb1fe59c640 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb201670 0x558cfb5d5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb1fe59c640 1 -- 172.21.15.112:0/110599266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfb5e58b0 con 0x558cfb1f9490 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.797+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb201670 0x558cfb5d5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb1fe59c640 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5d4ff0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00aac60 tx=0x7fb1f0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b4c96b73558ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb1e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/110599266 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00a9890 con 0x558cfb1f9490 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/110599266 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7fb1f00acc50 con 0x558cfb1f9490 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb20b07f740 1 -- 172.21.15.112:0/110599266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f8003e70 con 0x558cfb1f9490 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/110599266 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1f0005180 con 0x558cfb1f9490 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb20b07f740 10 client.16794 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb20b07f740 1 -- 172.21.15.112:0/110599266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558cfb5e8230 con 0x558cfb1f9490 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/110599266 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f8003e70 con 0x558cfb1f9490 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/110599266 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb1f00b20d0 con 0x558cfb1f9490 2024-09-21T03:36:26.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb20b07f740 10 client.16794 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb20b07f740 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb20b07f740 20 client.16794 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb20b07f740 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558cfb1e0cd0 0x558cfb5eb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb20b07f740 4 client.16794 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.798+0000 7fb20b07f740 1 -- 172.21.15.112:0/110599266 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cfb541db0 con 0x558cfb1e0cd0 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.799+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558cfb1e0cd0 0x558cfb5eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.799+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558cfb1e0cd0 0x558cfb5eb740 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=625b8d3847bc2752 server_cookie=5604b138c40cc44f in_seq=0 out_seq=0 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.799+0000 7fb1e5ffb640 10 client.16794 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/110599266 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558cfb541db0 con 0x558cfb1e0cd0 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb1e5ffb640 10 client.16794 handle_command_reply: tid=0 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 1 client.16794 shutdown 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 1 -- 172.21.15.112:0/110599266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 msgr2=0x558cfb5d4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5d4ff0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00aac60 tx=0x7fb1f0009310 comp rx=0 tx=0).stop 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 1 -- 172.21.15.112:0/110599266 shutdown_connections 2024-09-21T03:36:26.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558cfb1e0cd0 0x558cfb5eb740 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb222350 0x558cfb5e0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb201670 0x558cfb5d5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb1e5ffb640 0 client.16794 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:26.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 1 --2- 172.21.15.112:0/110599266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb1f9490 0x558cfb5d4ff0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 1 -- 172.21.15.112:0/110599266 >> 172.21.15.112:0/110599266 conn(0x558cfb216740 msgr2=0x558cfb5cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 1 -- 172.21.15.112:0/110599266 shutdown_connections 2024-09-21T03:36:26.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 1 -- 172.21.15.112:0/110599266 wait complete. 2024-09-21T03:36:26.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.800+0000 7fb20b07f740 20 client.16794 trim_cache size 0 max 16384 2024-09-21T03:36:26.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.818+0000 7fb20b07f740 1 Processor -- start 2024-09-21T03:36:26.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.818+0000 7fb20b07f740 1 -- start start 2024-09-21T03:36:26.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.818+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb1f9490 0x558cfb5e0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.818+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.818+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb222350 0x558cfb5d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.818+0000 7fb20b07f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001afec0 con 0x558cfb201670 2024-09-21T03:36:26.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.818+0000 7fb20b07f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001ad790 con 0x558cfb222350 2024-09-21T03:36:26.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.818+0000 7fb20b07f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001ab600 con 0x558cfb1f9490 2024-09-21T03:36:26.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.818+0000 7fb1fdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.818+0000 7fb1fdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38340/0 (socket says 172.21.15.112:38340) 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.818+0000 7fb1fdd9b640 1 -- 172.21.15.112:0/3144798599 learned_addr learned my addr 172.21.15.112:0/3144798599 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.819+0000 7fb1fed9d640 1 --2- 172.21.15.112:0/3144798599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb222350 0x558cfb5d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.819+0000 7fb1fe59c640 1 --2- 172.21.15.112:0/3144798599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb1f9490 0x558cfb5e0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.819+0000 7fb1fdd9b640 1 -- 172.21.15.112:0/3144798599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb1f9490 msgr2=0x558cfb5e0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.819+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/3144798599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb1f9490 0x558cfb5e0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.819+0000 7fb1fdd9b640 1 -- 172.21.15.112:0/3144798599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb222350 msgr2=0x558cfb5d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.819+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/3144798599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb222350 0x558cfb5d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.819+0000 7fb1fdd9b640 1 -- 172.21.15.112:0/3144798599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfb5d60b0 con 0x558cfb201670 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.819+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/3144798599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5cf550 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4005ab0 tx=0x7fb1f4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b271fa73eb90ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.819+0000 7fb1e7fff640 1 -- 172.21.15.112:0/3144798599 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4035040 con 0x558cfb201670 2024-09-21T03:36:26.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.819+0000 7fb1e7fff640 1 -- 172.21.15.112:0/3144798599 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f4049070 con 0x558cfb201670 2024-09-21T03:36:26.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.819+0000 7fb1e7fff640 1 -- 172.21.15.112:0/3144798599 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40049b0 con 0x558cfb201670 2024-09-21T03:36:26.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.820+0000 7fb20b07f740 1 -- 172.21.15.112:0/3144798599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 msgr2=0x558cfb5cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.820+0000 7fb20b07f740 1 --2- 172.21.15.112:0/3144798599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5cf550 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4005ab0 tx=0x7fb1f4005a20 comp rx=0 tx=0).stop 2024-09-21T03:36:26.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.820+0000 7fb20b07f740 1 -- 172.21.15.112:0/3144798599 shutdown_connections 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.820+0000 7fb20b07f740 1 --2- 172.21.15.112:0/3144798599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb222350 0x558cfb5d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.820+0000 7fb20b07f740 1 --2- 172.21.15.112:0/3144798599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5cf550 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.820+0000 7fb20b07f740 1 --2- 172.21.15.112:0/3144798599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb1f9490 0x558cfb5e0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.820+0000 7fb20b07f740 1 -- 172.21.15.112:0/3144798599 >> 172.21.15.112:0/3144798599 conn(0x558cfb216740 msgr2=0x558cfb5d3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.820+0000 7fb20b07f740 1 -- 172.21.15.112:0/3144798599 shutdown_connections 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.820+0000 7fb20b07f740 1 -- 172.21.15.112:0/3144798599 wait complete. 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.820+0000 7fb20b07f740 1 -- start start 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb20b07f740 1 Processor -- start 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb1f9490 0x558cfb5d9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5d9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb20b07f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb222350 0x558cfb655150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb20b07f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001abc50 con 0x558cfb201670 2024-09-21T03:36:26.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb20b07f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001ad0c0 con 0x558cfb222350 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb20b07f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001ac320 con 0x558cfb1f9490 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb20b07f740 10 client.0 _ll_register_callbacks cb 0x7fb1f0009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1fdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5d9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1fe59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb1f9490 0x558cfb5d9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1fdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5d9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38350/0 (socket says 172.21.15.112:38350) 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1fdd9b640 1 -- 172.21.15.112:0/2367068500 learned_addr learned my addr 172.21.15.112:0/2367068500 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1fed9d640 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb222350 0x558cfb655150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1fdd9b640 1 -- 172.21.15.112:0/2367068500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb1f9490 msgr2=0x558cfb5d9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb1f9490 0x558cfb5d9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1fdd9b640 1 -- 172.21.15.112:0/2367068500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb222350 msgr2=0x558cfb655150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb222350 0x558cfb655150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1fdd9b640 1 -- 172.21.15.112:0/2367068500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfb655690 con 0x558cfb201670 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1fdd9b640 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5d9a60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009230 tx=0x7fb1f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b83bc10bd6bc73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.821+0000 7fb1e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/2367068500 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4051070 con 0x558cfb201670 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/2367068500 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7fb1f40035f0 con 0x558cfb201670 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb20b07f740 1 -- 172.21.15.112:0/2367068500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f4049070 con 0x558cfb201670 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/2367068500 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1f4035070 con 0x558cfb201670 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb20b07f740 10 client.16800 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb20b07f740 1 -- 172.21.15.112:0/2367068500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558cfb6568c0 con 0x558cfb201670 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/2367068500 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f4049070 con 0x558cfb201670 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/2367068500 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb1f4032400 con 0x558cfb201670 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb20b07f740 10 client.16800 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb20b07f740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb20b07f740 20 client.16800 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb20b07f740 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558cfb1b6080 0x558cfb659000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb20b07f740 4 client.16800 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:36:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb20b07f740 1 -- 172.21.15.112:0/2367068500 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558cfb53d860 con 0x558cfb1b6080 2024-09-21T03:36:26.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.822+0000 7fb1fe59c640 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558cfb1b6080 0x558cfb659000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:26.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.823+0000 7fb1e5ffb640 10 client.16800 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:26.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.823+0000 7fb1fe59c640 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558cfb1b6080 0x558cfb659000 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d0f1e89c33b1e9 server_cookie=e339d2c64595fa72 in_seq=0 out_seq=0 2024-09-21T03:36:26.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb1e5ffb640 1 -- 172.21.15.112:0/2367068500 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558cfb53d860 con 0x558cfb1b6080 2024-09-21T03:36:26.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb1e5ffb640 10 client.16800 handle_command_reply: tid=0 2024-09-21T03:36:26.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 1 client.16800 shutdown 2024-09-21T03:36:26.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 1 -- 172.21.15.112:0/2367068500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 msgr2=0x558cfb5d9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5d9a60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009230 tx=0x7fb1f4000bc0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 1 -- 172.21.15.112:0/2367068500 shutdown_connections 2024-09-21T03:36:26.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x558cfb1b6080 0x558cfb659000 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cfb222350 0x558cfb655150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558cfb201670 0x558cfb5d9a60 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 1 --2- 172.21.15.112:0/2367068500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558cfb1f9490 0x558cfb5d9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb1e5ffb640 0 client.16800 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:26.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 1 -- 172.21.15.112:0/2367068500 >> 172.21.15.112:0/2367068500 conn(0x558cfb216740 msgr2=0x558cfb5d6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 1 -- 172.21.15.112:0/2367068500 shutdown_connections 2024-09-21T03:36:26.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 1 -- 172.21.15.112:0/2367068500 wait complete. 2024-09-21T03:36:26.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.825+0000 7fb20b07f740 20 client.16800 trim_cache size 0 max 16384 2024-09-21T03:36:26.831 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:36:26.831 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:36:26.832 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "df8c91a7-299b-410b-babe-4f4ca1487b26", 2024-09-21T03:36:26.832 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:36:26.832 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:36:26.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 -- 172.21.15.112:0/3289068225 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1e80b6680 msgr2=0x7fb1e80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1e80b6680 0x7fb1e80b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8020000 tx=0x7fb1f8021000 comp rx=0 tx=0).stop 2024-09-21T03:36:26.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 -- 172.21.15.112:0/3289068225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb200102ae0 msgr2=0x7fb20019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:26.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb200102ae0 0x7fb20019a5f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000e9f0 tx=0x7fb1f000edf0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 -- 172.21.15.112:0/3289068225 shutdown_connections 2024-09-21T03:36:26.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb1e80b6680 0x7fb1e80b8b90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb200108d70 0x7fb20019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb200108420 0x7fb20019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 --2- 172.21.15.112:0/3289068225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb200102ae0 0x7fb20019a5f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:26.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 -- 172.21.15.112:0/3289068225 >> 172.21.15.112:0/3289068225 conn(0x7fb200079770 msgr2=0x7fb200101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:26.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 -- 172.21.15.112:0/3289068225 shutdown_connections 2024-09-21T03:36:26.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:26.826+0000 7fb204b0b640 1 -- 172.21.15.112:0/3289068225 wait complete. 2024-09-21T03:36:27.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 778569 10.75 MB/sec execute 267 sec latency 101.972 ms 2024-09-21T03:36:27.671 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:36:27.967 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:28.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:27 smithi112 ceph-mon[31442]: pgmap v545: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 296 op/s 2024-09-21T03:36:28.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:27 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:36:28.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:27 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:27 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:36:28.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:27 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:36:28.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:27 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:36:28.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:27 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:27 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:27 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:27 smithi203 ceph-mon[34723]: pgmap v545: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 296 op/s 2024-09-21T03:36:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:27 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:36:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:27 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:27 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:36:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:27 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:36:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:27 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:36:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:27 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:27 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:27 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:27 smithi187 ceph-mon[34603]: pgmap v545: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 296 op/s 2024-09-21T03:36:28.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:27 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:36:28.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:27 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:27 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:36:28.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:27 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:36:28.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:27 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:36:28.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:27 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:27 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:27 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:36:28.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 781214 10.78 MB/sec execute 268 sec latency 23.899 ms 2024-09-21T03:36:28.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.474+0000 7f96ab9b9640 1 -- 172.21.15.112:0/147170995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96a4108d40 msgr2=0x7f96a410f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:28.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.474+0000 7f96ab9b9640 1 --2- 172.21.15.112:0/147170995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96a4108d40 0x7f96a410f570 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f969c009ca0 tx=0x7f969c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.474+0000 7f96ab9b9640 1 -- 172.21.15.112:0/147170995 shutdown_connections 2024-09-21T03:36:28.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.474+0000 7f96ab9b9640 1 --2- 172.21.15.112:0/147170995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96a4108d40 0x7f96a410f570 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.474+0000 7f96ab9b9640 1 --2- 172.21.15.112:0/147170995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a41083f0 0x7f96a4108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.474+0000 7f96ab9b9640 1 --2- 172.21.15.112:0/147170995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96a4102ab0 0x7f96a4102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.474+0000 7f96ab9b9640 1 -- 172.21.15.112:0/147170995 >> 172.21.15.112:0/147170995 conn(0x7f96a40796b0 msgr2=0x7f96a4079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:28.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.474+0000 7f96ab9b9640 1 -- 172.21.15.112:0/147170995 shutdown_connections 2024-09-21T03:36:28.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.474+0000 7f96ab9b9640 1 -- 172.21.15.112:0/147170995 wait complete. 2024-09-21T03:36:28.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96ab9b9640 1 Processor -- start 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96ab9b9640 1 -- start start 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96ab9b9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96a4102ab0 0x7f96a40ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96ab9b9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a41083f0 0x7f96a40ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96ab9b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96a4108d40 0x7f96a4100120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96ab9b9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41129b0 con 0x7f96a4108d40 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96ab9b9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4112830 con 0x7f96a4102ab0 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96ab9b9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4112b30 con 0x7f96a41083f0 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96a9755640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96a4102ab0 0x7f96a40ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96a8f54640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a41083f0 0x7f96a40ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96a9f56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96a4108d40 0x7f96a4100120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96a9755640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96a4102ab0 0x7f96a40ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:37812/0 (socket says 172.21.15.112:37812) 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96a8f54640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a41083f0 0x7f96a40ffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:58146/0 (socket says 172.21.15.112:58146) 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96a9f56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96a4108d40 0x7f96a4100120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38358/0 (socket says 172.21.15.112:38358) 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.475+0000 7f96a9755640 1 -- 172.21.15.112:0/2655893413 learned_addr learned my addr 172.21.15.112:0/2655893413 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:28.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.476+0000 7f96a9755640 1 -- 172.21.15.112:0/2655893413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a41083f0 msgr2=0x7f96a40ffbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:28.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.476+0000 7f96a9755640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a41083f0 0x7f96a40ffbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.476+0000 7f96a9755640 1 -- 172.21.15.112:0/2655893413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96a4108d40 msgr2=0x7f96a4100120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:28.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.476+0000 7f96a9755640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96a4108d40 0x7f96a4100120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.476+0000 7f96a9755640 1 -- 172.21.15.112:0/2655893413 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a41a8770 con 0x7f96a4102ab0 2024-09-21T03:36:28.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.476+0000 7f96a9755640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96a4102ab0 0x7f96a40ff6a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f96a000ea60 tx=0x7f96a000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:28.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.476+0000 7f969a7fc640 1 -- 172.21.15.112:0/2655893413 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a000cde0 con 0x7f96a4102ab0 2024-09-21T03:36:28.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.476+0000 7f96ab9b9640 1 -- 172.21.15.112:0/2655893413 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a41a8a60 con 0x7f96a4102ab0 2024-09-21T03:36:28.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.476+0000 7f969a7fc640 1 -- 172.21.15.112:0/2655893413 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96a00035f0 con 0x7f96a4102ab0 2024-09-21T03:36:28.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.476+0000 7f969a7fc640 1 -- 172.21.15.112:0/2655893413 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0010690 con 0x7f96a4102ab0 2024-09-21T03:36:28.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.476+0000 7f96ab9b9640 1 -- 172.21.15.112:0/2655893413 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a41a8fa0 con 0x7f96a4102ab0 2024-09-21T03:36:28.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.479+0000 7f96b1f2d740 1 Processor -- start 2024-09-21T03:36:28.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.479+0000 7f96b1f2d740 1 -- start start 2024-09-21T03:36:28.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.479+0000 7f969a7fc640 1 -- 172.21.15.112:0/2655893413 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f96a00037a0 con 0x7f96a4102ab0 2024-09-21T03:36:28.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.479+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafcb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.479+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafcb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.479+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafcbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.479+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5555eaf41e50 con 0x5555eac03390 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.479+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5555eaf31280 con 0x5555eac24070 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.479+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5555eaf35090 con 0x5555eabfb1b0 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f96a8f54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafcbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f96a8f54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafcbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38370/0 (socket says 172.21.15.112:38370) 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f96a8f54640 1 -- 172.21.15.112:0/2877863922 learned_addr learned my addr 172.21.15.112:0/2877863922 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f96a9755640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafcb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f96a9f56640 1 --2- 172.21.15.112:0/2877863922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafcb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f96a8f54640 1 -- 172.21.15.112:0/2877863922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 msgr2=0x5555eafcb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f96a8f54640 1 --2- 172.21.15.112:0/2877863922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafcb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f96a8f54640 1 -- 172.21.15.112:0/2877863922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 msgr2=0x5555eafcb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f96a8f54640 1 --2- 172.21.15.112:0/2877863922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafcb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f96a8f54640 1 -- 172.21.15.112:0/2877863922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555eac8f650 con 0x5555eac03390 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f96a9755640 1 --2- 172.21.15.112:0/2877863922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafcb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.480+0000 7f969a7fc640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96900b6680 0x7f96900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.481+0000 7f96a8f54640 1 --2- 172.21.15.112:0/2877863922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafcbcf0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9694009c00 tx=0x7f9694011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0878356df6d382 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.481+0000 7f96a9f56640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96900b6680 0x7f96900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.481+0000 7f969a7fc640 1 -- 172.21.15.112:0/2655893413 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f96a00e79d0 con 0x7f96a4102ab0 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.481+0000 7f968f7fe640 1 -- 172.21.15.112:0/2877863922 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694004230 con 0x5555eac03390 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.481+0000 7f968f7fe640 1 -- 172.21.15.112:0/2877863922 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96940043d0 con 0x5555eac03390 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.481+0000 7f968f7fe640 1 -- 172.21.15.112:0/2877863922 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694019660 con 0x5555eac03390 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.482+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2877863922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 msgr2=0x5555eafcbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.482+0000 7f96a9f56640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96900b6680 0x7f96900b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f969c02f500 tx=0x7f969c02f310 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.482+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/2877863922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafcbcf0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9694009c00 tx=0x7f9694011810 comp rx=0 tx=0).stop 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.482+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2877863922 shutdown_connections 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.482+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/2877863922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafcbcf0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.482+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/2877863922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafcb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.482+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/2877863922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafcb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.482+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2877863922 >> 172.21.15.112:0/2877863922 conn(0x5555eac18460 msgr2=0x5555eafc9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.482+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2877863922 shutdown_connections 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.483+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2877863922 wait complete. 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.483+0000 7f96b1f2d740 1 -- start start 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.483+0000 7f96b1f2d740 1 Processor -- start 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafd68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafd6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafe2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5555ead92700 con 0x5555eac03390 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5555eaecc110 con 0x5555eac24070 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5555eafd9a50 con 0x5555eabfb1b0 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96b1f2d740 10 client.0 _ll_register_callbacks cb 0x5555eaee5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96a8f54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafd6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96a9755640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafd68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96a8f54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafd6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38382/0 (socket says 172.21.15.112:38382) 2024-09-21T03:36:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96a8f54640 1 -- 172.21.15.112:0/351050112 learned_addr learned my addr 172.21.15.112:0/351050112 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96a9f56640 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafe2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96a9755640 1 -- 172.21.15.112:0/351050112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 msgr2=0x5555eafe2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96a9755640 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafe2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96a9755640 1 -- 172.21.15.112:0/351050112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 msgr2=0x5555eafd6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96a9755640 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafd6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96a9755640 1 -- 172.21.15.112:0/351050112 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555eaef9c40 con 0x5555eabfb1b0 2024-09-21T03:36:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f96a9755640 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafd68d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f96a00e5150 tx=0x7f96a00e5650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45424d83f2d641cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.484+0000 7f968cff9640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:36:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f968cff9640 1 -- 172.21.15.112:0/351050112 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a00a9ea0 con 0x5555eabfb1b0 2024-09-21T03:36:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f968cff9640 1 -- 172.21.15.112:0/351050112 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f96a0004200 con 0x5555eabfb1b0 2024-09-21T03:36:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f96b1f2d740 1 -- 172.21.15.112:0/351050112 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96940043d0 con 0x5555eabfb1b0 2024-09-21T03:36:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f968cff9640 1 -- 172.21.15.112:0/351050112 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f96a00acfc0 con 0x5555eabfb1b0 2024-09-21T03:36:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f96b1f2d740 10 client.26371 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f96b1f2d740 1 -- 172.21.15.112:0/351050112 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555eafe5760 con 0x5555eabfb1b0 2024-09-21T03:36:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f968cff9640 1 -- 172.21.15.112:0/351050112 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96940043d0 con 0x5555eabfb1b0 2024-09-21T03:36:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f968cff9640 1 -- 172.21.15.112:0/351050112 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f96a00ac520 con 0x5555eabfb1b0 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f96b1f2d740 10 client.26371 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f96b1f2d740 10 client.26371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f96b1f2d740 20 client.26371 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555eabe29f0 0x5555eafe8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f96b1f2d740 4 client.26371 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.485+0000 7f96b1f2d740 1 -- 172.21.15.112:0/351050112 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555eaf43da0 con 0x5555eabe29f0 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.486+0000 7f96a8f54640 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555eabe29f0 0x5555eafe8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.486+0000 7f96a8f54640 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555eabe29f0 0x5555eafe8ce0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68fb2a8edf6683c server_cookie=60ce561891b51820 in_seq=0 out_seq=0 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.486+0000 7f968cff9640 10 client.26371 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f968cff9640 1 -- 172.21.15.112:0/351050112 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5555eaf43da0 con 0x5555eabe29f0 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f968cff9640 10 client.26371 handle_command_reply: tid=0 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 1 client.26371 shutdown 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 1 -- 172.21.15.112:0/351050112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 msgr2=0x5555eafd68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafd68d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f96a00e5150 tx=0x7f96a00e5650 comp rx=0 tx=0).stop 2024-09-21T03:36:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 1 -- 172.21.15.112:0/351050112 shutdown_connections 2024-09-21T03:36:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555eabe29f0 0x5555eafe8ce0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafe2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafd6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/351050112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafd68d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f968cff9640 0 client.26371 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 1 -- 172.21.15.112:0/351050112 >> 172.21.15.112:0/351050112 conn(0x5555eac18460 msgr2=0x5555eafce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 1 -- 172.21.15.112:0/351050112 shutdown_connections 2024-09-21T03:36:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 1 -- 172.21.15.112:0/351050112 wait complete. 2024-09-21T03:36:28.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.487+0000 7f96b1f2d740 20 client.26371 trim_cache size 0 max 16384 2024-09-21T03:36:28.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.505+0000 7f96b1f2d740 1 Processor -- start 2024-09-21T03:36:28.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.505+0000 7f96b1f2d740 1 -- start start 2024-09-21T03:36:28.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.505+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafcc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.505+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafd3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafd4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b40c0 con 0x5555eac03390 2024-09-21T03:36:28.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b1990 con 0x5555eac24070 2024-09-21T03:36:28.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41af800 con 0x5555eabfb1b0 2024-09-21T03:36:28.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96a8f54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafd3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96a8f54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafd3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38384/0 (socket says 172.21.15.112:38384) 2024-09-21T03:36:28.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96a8f54640 1 -- 172.21.15.112:0/346804248 learned_addr learned my addr 172.21.15.112:0/346804248 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96a9755640 1 --2- 172.21.15.112:0/346804248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafcc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96a9f56640 1 --2- 172.21.15.112:0/346804248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafd4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96a8f54640 1 -- 172.21.15.112:0/346804248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 msgr2=0x5555eafcc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96a8f54640 1 --2- 172.21.15.112:0/346804248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafcc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96a8f54640 1 -- 172.21.15.112:0/346804248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 msgr2=0x5555eafd4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96a8f54640 1 --2- 172.21.15.112:0/346804248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafd4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96a8f54640 1 -- 172.21.15.112:0/346804248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555eafcd830 con 0x5555eac03390 2024-09-21T03:36:28.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f96a8f54640 1 --2- 172.21.15.112:0/346804248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafd3ac0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9694009c00 tx=0x7f9694009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d0b47302abc0d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:28.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.506+0000 7f968effd640 1 -- 172.21.15.112:0/346804248 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694033070 con 0x5555eac03390 2024-09-21T03:36:28.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.507+0000 7f968effd640 1 -- 172.21.15.112:0/346804248 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9694009e70 con 0x5555eac03390 2024-09-21T03:36:28.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.507+0000 7f968effd640 1 -- 172.21.15.112:0/346804248 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969401f040 con 0x5555eac03390 2024-09-21T03:36:28.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.507+0000 7f96b1f2d740 1 -- 172.21.15.112:0/346804248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 msgr2=0x5555eafd3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:28.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.507+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/346804248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafd3ac0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9694009c00 tx=0x7f9694009620 comp rx=0 tx=0).stop 2024-09-21T03:36:28.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.507+0000 7f96b1f2d740 1 -- 172.21.15.112:0/346804248 shutdown_connections 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.507+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/346804248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafd4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.507+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/346804248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafd3ac0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.507+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/346804248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafcc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.508+0000 7f96b1f2d740 1 -- 172.21.15.112:0/346804248 >> 172.21.15.112:0/346804248 conn(0x5555eac18460 msgr2=0x5555eafcc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.508+0000 7f96b1f2d740 1 -- 172.21.15.112:0/346804248 shutdown_connections 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.508+0000 7f96b1f2d740 1 -- 172.21.15.112:0/346804248 wait complete. 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.508+0000 7f96b1f2d740 1 -- start start 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.508+0000 7f96b1f2d740 1 Processor -- start 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafdf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafda880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96b1f2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafdadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41afe50 con 0x5555eac03390 2024-09-21T03:36:28.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b12c0 con 0x5555eac24070 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96b1f2d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b0520 con 0x5555eabfb1b0 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96b1f2d740 10 client.0 _ll_register_callbacks cb 0x7f96a00eb1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96a9755640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafdf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96a8f54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafda880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96a9755640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafdf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:58184/0 (socket says 172.21.15.112:58184) 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96a9755640 1 -- 172.21.15.112:0/2227875910 learned_addr learned my addr 172.21.15.112:0/2227875910 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96a9f56640 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafdadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96a9755640 1 -- 172.21.15.112:0/2227875910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 msgr2=0x5555eafdadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96a9755640 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafdadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96a9755640 1 -- 172.21.15.112:0/2227875910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 msgr2=0x5555eafda880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96a9755640 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafda880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96a9755640 1 -- 172.21.15.112:0/2227875910 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555eafdb4b0 con 0x5555eabfb1b0 2024-09-21T03:36:28.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.509+0000 7f96a8f54640 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafda880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:36:28.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.510+0000 7f96a9755640 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafdf3d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f96a00e5750 tx=0x7f96a00b2260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d405974e2782ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.510+0000 7f968cff9640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.510+0000 7f968cff9640 1 -- 172.21.15.112:0/2227875910 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a00ad1a0 con 0x5555eabfb1b0 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.510+0000 7f968cff9640 1 -- 172.21.15.112:0/2227875910 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f96a0004000 con 0x5555eabfb1b0 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.510+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2227875910 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9694009e70 con 0x5555eabfb1b0 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.510+0000 7f968cff9640 1 -- 172.21.15.112:0/2227875910 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f96a00ad620 con 0x5555eabfb1b0 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.510+0000 7f96b1f2d740 10 client.26377 fetch_fsmap learned FSMap version 30 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.510+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2227875910 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555eb053540 con 0x5555eabfb1b0 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.510+0000 7f968cff9640 1 -- 172.21.15.112:0/2227875910 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9694009e70 con 0x5555eabfb1b0 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.511+0000 7f968cff9640 1 -- 172.21.15.112:0/2227875910 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f96a00b34c0 con 0x5555eabfb1b0 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.511+0000 7f96b1f2d740 10 client.26377 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.511+0000 7f96b1f2d740 10 client.26377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.511+0000 7f96b1f2d740 20 client.26377 populate_metadata read hostname 'smithi112' 2024-09-21T03:36:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.511+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555eabb7da0 0x5555eb055bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.511+0000 7f96b1f2d740 4 client.26377 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.511+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2227875910 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5555eaf3f850 con 0x5555eabb7da0 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.511+0000 7f96a8f54640 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555eabb7da0 0x5555eb055bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.511+0000 7f968cff9640 10 client.26377 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.511+0000 7f96a8f54640 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555eabb7da0 0x5555eb055bc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c1882b2a9f7bf37 server_cookie=6733cb2d55f2741c in_seq=0 out_seq=0 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f968cff9640 1 -- 172.21.15.112:0/2227875910 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x5555eaf3f850 con 0x5555eabb7da0 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f968cff9640 10 client.26377 handle_command_reply: tid=0 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 1 client.26377 shutdown 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2227875910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 msgr2=0x5555eafdf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafdf3d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f96a00e5750 tx=0x7f96a00b2260 comp rx=0 tx=0).stop 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2227875910 shutdown_connections 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555eabb7da0 0x5555eb055bc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f968cff9640 0 client.26377 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:36:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555eac24070 0x5555eafdadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555eac03390 0x5555eafda880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 1 --2- 172.21.15.112:0/2227875910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555eabfb1b0 0x5555eafdf3d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2227875910 >> 172.21.15.112:0/2227875910 conn(0x5555eac18460 msgr2=0x5555eafd86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2227875910 shutdown_connections 2024-09-21T03:36:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 1 -- 172.21.15.112:0/2227875910 wait complete. 2024-09-21T03:36:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.512+0000 7f96b1f2d740 20 client.26377 trim_cache size 0 max 16384 2024-09-21T03:36:28.518 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:36:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:36:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:36:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:36:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "df8c91a7-299b-410b-babe-4f4ca1487b26", 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:36:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:36:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:36:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:36:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:36:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:36:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:36:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:36:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:36:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:36:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:36:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:36:28.526 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 -- 172.21.15.112:0/2655893413 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96900b6680 msgr2=0x7f96900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96900b6680 0x7f96900b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f969c02f500 tx=0x7f969c02f310 comp rx=0 tx=0).stop 2024-09-21T03:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 -- 172.21.15.112:0/2655893413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96a4102ab0 msgr2=0x7f96a40ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96a4102ab0 0x7f96a40ff6a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f96a000ea60 tx=0x7f96a000ef20 comp rx=0 tx=0).stop 2024-09-21T03:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 -- 172.21.15.112:0/2655893413 shutdown_connections 2024-09-21T03:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f96900b6680 0x7f96900b8b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96a4108d40 0x7f96a4100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a41083f0 0x7f96a40ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 --2- 172.21.15.112:0/2655893413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96a4102ab0 0x7f96a40ff6a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:36:28.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 -- 172.21.15.112:0/2655893413 >> 172.21.15.112:0/2655893413 conn(0x7f96a40796b0 msgr2=0x7f96a406bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:36:28.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 -- 172.21.15.112:0/2655893413 shutdown_connections 2024-09-21T03:36:28.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:36:28.513+0000 7f96ab9b9640 1 -- 172.21.15.112:0/2655893413 wait complete. 2024-09-21T03:36:29.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 782357 10.75 MB/sec execute 269 sec latency 41.733 ms 2024-09-21T03:36:29.252 INFO:tasks.cephfs.filesystem:scrub status for tag:df8c91a7-299b-410b-babe-4f4ca1487b26 - {'path': '/', 'tag': 'df8c91a7-299b-410b-babe-4f4ca1487b26', 'options': 'recursive,force'} 2024-09-21T03:36:30.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:29 smithi203 ceph-mon[34723]: pgmap v546: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-09-21T03:36:30.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:29 smithi187 ceph-mon[34603]: pgmap v546: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-09-21T03:36:30.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 784754 10.77 MB/sec execute 270 sec latency 52.702 ms 2024-09-21T03:36:30.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:29 smithi112 ceph-mon[31442]: pgmap v546: 97 pgs: 97 active+clean; 559 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-09-21T03:36:31.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 786106 10.75 MB/sec execute 271 sec latency 19.811 ms 2024-09-21T03:36:32.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:31 smithi203 ceph-mon[34723]: pgmap v547: 97 pgs: 97 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 306 op/s 2024-09-21T03:36:32.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:31 smithi187 ceph-mon[34603]: pgmap v547: 97 pgs: 97 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 306 op/s 2024-09-21T03:36:32.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 788471 10.76 MB/sec execute 272 sec latency 14.026 ms 2024-09-21T03:36:32.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:31 smithi112 ceph-mon[31442]: pgmap v547: 97 pgs: 97 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 306 op/s 2024-09-21T03:36:33.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 789897 10.74 MB/sec execute 273 sec latency 16.667 ms 2024-09-21T03:36:34.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:33 smithi203 ceph-mon[34723]: pgmap v548: 97 pgs: 97 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-09-21T03:36:34.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:33 smithi187 ceph-mon[34603]: pgmap v548: 97 pgs: 97 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-09-21T03:36:34.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 792100 10.75 MB/sec execute 274 sec latency 38.145 ms 2024-09-21T03:36:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:33 smithi112 ceph-mon[31442]: pgmap v548: 97 pgs: 97 active+clean; 568 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-09-21T03:36:35.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:34 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:36:35.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:34 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:36:35.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:34 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:36:35.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:34 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:36:35.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:34 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:36:35.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:34 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:36:35.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 793464 10.73 MB/sec execute 275 sec latency 26.675 ms 2024-09-21T03:36:35.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:34 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:36:35.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:34 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:36:35.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:34 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:36:36.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:35 smithi203 ceph-mon[34723]: pgmap v549: 97 pgs: 97 active+clean; 569 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-09-21T03:36:36.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:35 smithi187 ceph-mon[34603]: pgmap v549: 97 pgs: 97 active+clean; 569 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-09-21T03:36:36.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 795692 10.75 MB/sec execute 276 sec latency 14.152 ms 2024-09-21T03:36:36.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:35 smithi112 ceph-mon[31442]: pgmap v549: 97 pgs: 97 active+clean; 569 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-09-21T03:36:37.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 797237 10.73 MB/sec execute 277 sec latency 20.013 ms 2024-09-21T03:36:38.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:37 smithi203 ceph-mon[34723]: pgmap v550: 97 pgs: 97 active+clean; 576 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 319 op/s 2024-09-21T03:36:38.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:37 smithi187 ceph-mon[34603]: pgmap v550: 97 pgs: 97 active+clean; 576 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 319 op/s 2024-09-21T03:36:38.243 INFO:tasks.workunit.client.0.smithi112.stdout: 1 799273 10.74 MB/sec execute 278 sec latency 14.313 ms 2024-09-21T03:36:38.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:37 smithi112 ceph-mon[31442]: pgmap v550: 97 pgs: 97 active+clean; 576 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 319 op/s 2024-09-21T03:36:39.247 INFO:tasks.workunit.client.0.smithi112.stdout: 1 801197 10.72 MB/sec execute 279 sec latency 20.638 ms 2024-09-21T03:36:40.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:39 smithi112 ceph-mon[31442]: pgmap v551: 97 pgs: 97 active+clean; 576 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-09-21T03:36:40.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:39 smithi203 ceph-mon[34723]: pgmap v551: 97 pgs: 97 active+clean; 576 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-09-21T03:36:40.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:39 smithi187 ceph-mon[34603]: pgmap v551: 97 pgs: 97 active+clean; 576 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-09-21T03:36:40.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 802811 10.73 MB/sec execute 280 sec latency 35.775 ms 2024-09-21T03:36:41.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 804828 10.72 MB/sec execute 281 sec latency 19.935 ms 2024-09-21T03:36:42.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:41 smithi203 ceph-mon[34723]: pgmap v552: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 296 op/s 2024-09-21T03:36:42.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:41 smithi187 ceph-mon[34603]: pgmap v552: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 296 op/s 2024-09-21T03:36:42.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 806566 10.72 MB/sec execute 282 sec latency 16.933 ms 2024-09-21T03:36:42.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:41 smithi112 ceph-mon[31442]: pgmap v552: 97 pgs: 97 active+clean; 582 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 296 op/s 2024-09-21T03:36:43.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 808975 10.73 MB/sec execute 283 sec latency 14.767 ms 2024-09-21T03:36:44.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:43 smithi203 ceph-mon[34723]: pgmap v553: 97 pgs: 97 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-09-21T03:36:44.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:43 smithi187 ceph-mon[34603]: pgmap v553: 97 pgs: 97 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-09-21T03:36:44.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 810481 10.72 MB/sec execute 284 sec latency 20.906 ms 2024-09-21T03:36:44.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:43 smithi112 ceph-mon[31442]: pgmap v553: 97 pgs: 97 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-09-21T03:36:45.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 813046 10.74 MB/sec execute 285 sec latency 25.547 ms 2024-09-21T03:36:46.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:45 smithi203 ceph-mon[34723]: pgmap v554: 97 pgs: 97 active+clean; 586 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 232 op/s 2024-09-21T03:36:46.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:45 smithi187 ceph-mon[34603]: pgmap v554: 97 pgs: 97 active+clean; 586 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 232 op/s 2024-09-21T03:36:46.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 814469 10.72 MB/sec execute 286 sec latency 17.172 ms 2024-09-21T03:36:46.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:45 smithi112 ceph-mon[31442]: pgmap v554: 97 pgs: 97 active+clean; 586 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 232 op/s 2024-09-21T03:36:47.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 816838 10.73 MB/sec execute 287 sec latency 14.835 ms 2024-09-21T03:36:48.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:47 smithi203 ceph-mon[34723]: pgmap v555: 97 pgs: 97 active+clean; 593 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 304 op/s 2024-09-21T03:36:48.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:47 smithi187 ceph-mon[34603]: pgmap v555: 97 pgs: 97 active+clean; 593 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 304 op/s 2024-09-21T03:36:48.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 818252 10.71 MB/sec execute 288 sec latency 19.305 ms 2024-09-21T03:36:48.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:47 smithi112 ceph-mon[31442]: pgmap v555: 97 pgs: 97 active+clean; 593 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 304 op/s 2024-09-21T03:36:49.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 820527 10.73 MB/sec execute 289 sec latency 13.963 ms 2024-09-21T03:36:50.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:49 smithi203 ceph-mon[34723]: pgmap v556: 97 pgs: 97 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2024-09-21T03:36:50.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:49 smithi187 ceph-mon[34603]: pgmap v556: 97 pgs: 97 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2024-09-21T03:36:50.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 822029 10.71 MB/sec execute 290 sec latency 47.883 ms 2024-09-21T03:36:50.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:49 smithi112 ceph-mon[31442]: pgmap v556: 97 pgs: 97 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2024-09-21T03:36:51.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 824146 10.72 MB/sec execute 291 sec latency 31.576 ms 2024-09-21T03:36:52.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:51 smithi203 ceph-mon[34723]: pgmap v557: 97 pgs: 97 active+clean; 599 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 294 op/s 2024-09-21T03:36:52.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:51 smithi187 ceph-mon[34603]: pgmap v557: 97 pgs: 97 active+clean; 599 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 294 op/s 2024-09-21T03:36:52.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 826583 10.72 MB/sec execute 292 sec latency 17.681 ms 2024-09-21T03:36:52.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:51 smithi112 ceph-mon[31442]: pgmap v557: 97 pgs: 97 active+clean; 599 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 294 op/s 2024-09-21T03:36:53.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 828206 10.71 MB/sec execute 293 sec latency 12.956 ms 2024-09-21T03:36:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:53 smithi203 ceph-mon[34723]: pgmap v558: 97 pgs: 97 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 255 op/s 2024-09-21T03:36:54.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:53 smithi187 ceph-mon[34603]: pgmap v558: 97 pgs: 97 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 255 op/s 2024-09-21T03:36:54.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 830971 10.74 MB/sec execute 294 sec latency 19.542 ms 2024-09-21T03:36:54.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:53 smithi112 ceph-mon[31442]: pgmap v558: 97 pgs: 97 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 255 op/s 2024-09-21T03:36:55.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 832281 10.72 MB/sec execute 295 sec latency 59.801 ms 2024-09-21T03:36:56.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:55 smithi203 ceph-mon[34723]: pgmap v559: 97 pgs: 97 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-21T03:36:56.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:55 smithi187 ceph-mon[34603]: pgmap v559: 97 pgs: 97 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-21T03:36:56.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 834606 10.73 MB/sec execute 296 sec latency 41.941 ms 2024-09-21T03:36:56.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:55 smithi112 ceph-mon[31442]: pgmap v559: 97 pgs: 97 active+clean; 607 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 254 op/s 2024-09-21T03:36:57.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 836325 10.71 MB/sec execute 297 sec latency 24.460 ms 2024-09-21T03:36:58.103 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:57 smithi112 ceph-mon[31442]: pgmap v560: 97 pgs: 97 active+clean; 613 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 315 op/s 2024-09-21T03:36:58.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:57 smithi203 ceph-mon[34723]: pgmap v560: 97 pgs: 97 active+clean; 613 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 315 op/s 2024-09-21T03:36:58.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:57 smithi187 ceph-mon[34603]: pgmap v560: 97 pgs: 97 active+clean; 613 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 315 op/s 2024-09-21T03:36:58.246 INFO:tasks.workunit.client.0.smithi112.stdout: 1 838230 10.72 MB/sec execute 298 sec latency 148.877 ms 2024-09-21T03:36:59.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 839887 10.70 MB/sec execute 299 sec latency 24.277 ms 2024-09-21T03:36:59.253 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:36:59.526 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:36:59.922 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:36:59 smithi112 ceph-mon[31442]: pgmap v561: 97 pgs: 97 active+clean; 617 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-09-21T03:37:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.125+0000 7f58fd7cf640 1 -- 172.21.15.112:0/2298765915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f8108d30 msgr2=0x7f58f810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.125+0000 7f58fd7cf640 1 --2- 172.21.15.112:0/2298765915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f8108d30 0x7f58f810f560 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009ca0 tx=0x7f58f002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.125+0000 7f58fd7cf640 1 -- 172.21.15.112:0/2298765915 shutdown_connections 2024-09-21T03:37:00.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.125+0000 7f58fd7cf640 1 --2- 172.21.15.112:0/2298765915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f8108d30 0x7f58f810f560 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.125+0000 7f58fd7cf640 1 --2- 172.21.15.112:0/2298765915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f81083e0 0x7f58f81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.125+0000 7f58fd7cf640 1 --2- 172.21.15.112:0/2298765915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f8102aa0 0x7f58f8102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.125+0000 7f58fd7cf640 1 -- 172.21.15.112:0/2298765915 >> 172.21.15.112:0/2298765915 conn(0x7f58f80797e0 msgr2=0x7f58f8079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.126+0000 7f58fd7cf640 1 -- 172.21.15.112:0/2298765915 shutdown_connections 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.126+0000 7f58fd7cf640 1 -- 172.21.15.112:0/2298765915 wait complete. 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.126+0000 7f58fd7cf640 1 Processor -- start 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.126+0000 7f58fd7cf640 1 -- start start 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58fd7cf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f8102aa0 0x7f58f8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58fd7cf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f81083e0 0x7f58f81998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58fd7cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f8108d30 0x7f58f819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58fd7cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81129a0 con 0x7f58f8108d30 2024-09-21T03:37:00.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58fd7cf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58f8112820 con 0x7f58f81083e0 2024-09-21T03:37:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58fd7cf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58f8112b20 con 0x7f58f8102aa0 2024-09-21T03:37:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58ee7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f81083e0 0x7f58f81998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58f6ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f8102aa0 0x7f58f8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f8108d30 0x7f58f819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58ee7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f81083e0 0x7f58f81998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:48868/0 (socket says 172.21.15.112:48868) 2024-09-21T03:37:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58ee7fc640 1 -- 172.21.15.112:0/3313302753 learned_addr learned my addr 172.21.15.112:0/3313302753 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58ee7fc640 1 -- 172.21.15.112:0/3313302753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f8102aa0 msgr2=0x7f58f8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f8102aa0 0x7f58f8199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58ee7fc640 1 -- 172.21.15.112:0/3313302753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f8108d30 msgr2=0x7f58f819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f8108d30 0x7f58f819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58ee7fc640 1 -- 172.21.15.112:0/3313302753 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f819f2c0 con 0x7f58f81083e0 2024-09-21T03:37:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58f77fe640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f8108d30 0x7f58f819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.127+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f81083e0 0x7f58f81998f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58e000e9c0 tx=0x7f58e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.128+0000 7f58f4ff9640 1 -- 172.21.15.112:0/3313302753 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e000cd80 con 0x7f58f81083e0 2024-09-21T03:37:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.128+0000 7f58f4ff9640 1 -- 172.21.15.112:0/3313302753 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e00054d0 con 0x7f58f81083e0 2024-09-21T03:37:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.128+0000 7f58fd7cf640 1 -- 172.21.15.112:0/3313302753 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f819f5b0 con 0x7f58f81083e0 2024-09-21T03:37:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.128+0000 7f58f4ff9640 1 -- 172.21.15.112:0/3313302753 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0010430 con 0x7f58f81083e0 2024-09-21T03:37:00.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.128+0000 7f58fd7cf640 1 -- 172.21.15.112:0/3313302753 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f81a5d10 con 0x7f58f81083e0 2024-09-21T03:37:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.131+0000 7f5903d43740 1 Processor -- start 2024-09-21T03:37:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.131+0000 7f5903d43740 1 -- start start 2024-09-21T03:37:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.131+0000 7f58f4ff9640 1 -- 172.21.15.112:0/3313302753 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f58e0004fd0 con 0x7f58f81083e0 2024-09-21T03:37:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.131+0000 7f5903d43740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370fc070 0x55c3374a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f5903d43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370d31b0 0x55c3374a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f5903d43740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370db390 0x55c3374a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f5903d43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c337419e50 con 0x55c3370d31b0 2024-09-21T03:37:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f5903d43740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c337409280 con 0x55c3370fc070 2024-09-21T03:37:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f5903d43740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c33740d090 con 0x55c3370db390 2024-09-21T03:37:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58f6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370d31b0 0x55c3374a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58f6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370d31b0 0x55c3374a37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44438/0 (socket says 172.21.15.112:44438) 2024-09-21T03:37:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58f6ffd640 1 -- 172.21.15.112:0/3663726588 learned_addr learned my addr 172.21.15.112:0/3663726588 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58f77fe640 1 --2- 172.21.15.112:0/3663726588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370fc070 0x55c3374a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/3663726588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370db390 0x55c3374a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58f6ffd640 1 -- 172.21.15.112:0/3663726588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370db390 msgr2=0x55c3374a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58f6ffd640 1 --2- 172.21.15.112:0/3663726588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370db390 0x55c3374a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58f6ffd640 1 -- 172.21.15.112:0/3663726588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370fc070 msgr2=0x55c3374a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58f6ffd640 1 --2- 172.21.15.112:0/3663726588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370fc070 0x55c3374a3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58f6ffd640 1 -- 172.21.15.112:0/3663726588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c337167650 con 0x55c3370d31b0 2024-09-21T03:37:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58f77fe640 1 --2- 172.21.15.112:0/3663726588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370fc070 0x55c3374a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.132+0000 7f58f6ffd640 1 --2- 172.21.15.112:0/3663726588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370d31b0 0x55c3374a37b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009d20 tx=0x7f58e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635d14245a37ec92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.133+0000 7f58ed7fa640 1 -- 172.21.15.112:0/3663726588 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e801c070 con 0x55c3370d31b0 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.133+0000 7f58ed7fa640 1 -- 172.21.15.112:0/3663726588 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e8018b30 con 0x55c3370d31b0 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.133+0000 7f58ed7fa640 1 -- 172.21.15.112:0/3663726588 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8017a30 con 0x55c3370d31b0 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.133+0000 7f58f4ff9640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58e40b6680 0x7f58e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.133+0000 7f58f77fe640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58e40b6680 0x7f58e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.133+0000 7f58f4ff9640 1 -- 172.21.15.112:0/3313302753 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f58e00e7b00 con 0x7f58f81083e0 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f5903d43740 1 -- 172.21.15.112:0/3663726588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370d31b0 msgr2=0x55c3374a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f58f77fe640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58e40b6680 0x7f58e40b8b90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f58f002f500 tx=0x7f58f002f350 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f5903d43740 1 --2- 172.21.15.112:0/3663726588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370d31b0 0x55c3374a37b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009d20 tx=0x7f58e8011810 comp rx=0 tx=0).stop 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f5903d43740 1 -- 172.21.15.112:0/3663726588 shutdown_connections 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f5903d43740 1 --2- 172.21.15.112:0/3663726588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370db390 0x55c3374a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f5903d43740 1 --2- 172.21.15.112:0/3663726588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370d31b0 0x55c3374a37b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f5903d43740 1 --2- 172.21.15.112:0/3663726588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370fc070 0x55c3374a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f5903d43740 1 -- 172.21.15.112:0/3663726588 >> 172.21.15.112:0/3663726588 conn(0x55c3370f0460 msgr2=0x55c3374a1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f5903d43740 1 -- 172.21.15.112:0/3663726588 shutdown_connections 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f5903d43740 1 -- 172.21.15.112:0/3663726588 wait complete. 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f5903d43740 1 -- start start 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.134+0000 7f5903d43740 1 Processor -- start 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f5903d43740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370d31b0 0x55c3374ae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f5903d43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f5903d43740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370fc070 0x55c3374af310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f5903d43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c33726a700 con 0x55c3370db390 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f5903d43740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c3373a4110 con 0x55c3370fc070 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f5903d43740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c3374b1a50 con 0x55c3370d31b0 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f5903d43740 10 client.0 _ll_register_callbacks cb 0x55c3373bd990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58ee7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58f6ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370d31b0 0x55c3374ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58ee7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44452/0 (socket says 172.21.15.112:44452) 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58ee7fc640 1 -- 172.21.15.112:0/132785531 learned_addr learned my addr 172.21.15.112:0/132785531 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58f77fe640 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370fc070 0x55c3374af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58ee7fc640 1 -- 172.21.15.112:0/132785531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370d31b0 msgr2=0x55c3374ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370d31b0 0x55c3374ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58ee7fc640 1 -- 172.21.15.112:0/132785531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370fc070 msgr2=0x55c3374af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370fc070 0x55c3374af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58ee7fc640 1 -- 172.21.15.112:0/132785531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3373d1c40 con 0x55c3370db390 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374aedd0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f58e00ad680 tx=0x7f58e00a9ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23714232339d601b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.135+0000 7f58de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f58de7fc640 1 -- 172.21.15.112:0/132785531 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e00ad2d0 con 0x55c3370db390 2024-09-21T03:37:00.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f58de7fc640 1 -- 172.21.15.112:0/132785531 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f58e00b21b0 con 0x55c3370db390 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f5903d43740 1 -- 172.21.15.112:0/132785531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e8018b30 con 0x55c3370db390 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f58de7fc640 1 -- 172.21.15.112:0/132785531 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58e00a90f0 con 0x55c3370db390 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f5903d43740 10 client.16830 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f5903d43740 1 -- 172.21.15.112:0/132785531 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c3374bd660 con 0x55c3370db390 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f58de7fc640 1 -- 172.21.15.112:0/132785531 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e8018b30 con 0x55c3370db390 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f58de7fc640 1 -- 172.21.15.112:0/132785531 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f58e00ac650 con 0x55c3370db390 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f5903d43740 10 client.16830 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f5903d43740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f5903d43740 20 client.16830 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f5903d43740 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c3370ba9f0 0x55c3374c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f5903d43740 4 client.16830 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.136+0000 7f5903d43740 1 -- 172.21.15.112:0/132785531 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c33741bda0 con 0x55c3370ba9f0 2024-09-21T03:37:00.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.137+0000 7f58f6ffd640 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c3370ba9f0 0x55c3374c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.137+0000 7f58de7fc640 10 client.16830 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.137+0000 7f58f6ffd640 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c3370ba9f0 0x55c3374c0af0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=288935adee223b2d server_cookie=245a950fdc76ad91 in_seq=0 out_seq=0 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f58de7fc640 1 -- 172.21.15.112:0/132785531 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c33741bda0 con 0x55c3370ba9f0 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f58de7fc640 10 client.16830 handle_command_reply: tid=0 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 1 client.16830 shutdown 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 1 -- 172.21.15.112:0/132785531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 msgr2=0x55c3374aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374aedd0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f58e00ad680 tx=0x7f58e00a9ea0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 1 -- 172.21.15.112:0/132785531 shutdown_connections 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c3370ba9f0 0x55c3374c0af0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370fc070 0x55c3374af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374aedd0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f58de7fc640 0 client.16830 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:00.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 1 --2- 172.21.15.112:0/132785531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370d31b0 0x55c3374ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 1 -- 172.21.15.112:0/132785531 >> 172.21.15.112:0/132785531 conn(0x55c3370f0460 msgr2=0x55c3374a6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 1 -- 172.21.15.112:0/132785531 shutdown_connections 2024-09-21T03:37:00.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 1 -- 172.21.15.112:0/132785531 wait complete. 2024-09-21T03:37:00.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.138+0000 7f5903d43740 20 client.16830 trim_cache size 0 max 16384 2024-09-21T03:37:00.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.155+0000 7f5903d43740 1 Processor -- start 2024-09-21T03:37:00.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.155+0000 7f5903d43740 1 -- start start 2024-09-21T03:37:00.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f5903d43740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 0x55c3374b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f5903d43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f5903d43740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370fc070 0x55c3374a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f5903d43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81afc20 con 0x55c3370db390 2024-09-21T03:37:00.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f5903d43740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81ad4f0 con 0x55c3370d31b0 2024-09-21T03:37:00.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f5903d43740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81ab360 con 0x55c3370fc070 2024-09-21T03:37:00.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f58ee7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f58ee7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374a42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44466/0 (socket says 172.21.15.112:44466) 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f58ee7fc640 1 -- 172.21.15.112:0/2959319132 learned_addr learned my addr 172.21.15.112:0/2959319132 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f58f6ffd640 1 --2- 172.21.15.112:0/2959319132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 0x55c3374b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f58f77fe640 1 --2- 172.21.15.112:0/2959319132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370fc070 0x55c3374a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f58ee7fc640 1 -- 172.21.15.112:0/2959319132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370fc070 msgr2=0x55c3374a6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/2959319132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370fc070 0x55c3374a6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f58ee7fc640 1 -- 172.21.15.112:0/2959319132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 msgr2=0x55c3374b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/2959319132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 0x55c3374b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.156+0000 7f58ee7fc640 1 -- 172.21.15.112:0/2959319132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c33749c580 con 0x55c3370db390 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.157+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/2959319132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374a42a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f58e00a9f50 tx=0x7f58e0005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfd23642a4b360 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.157+0000 7f58ecff9640 1 -- 172.21.15.112:0/2959319132 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e00ad0f0 con 0x55c3370db390 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.157+0000 7f58ecff9640 1 -- 172.21.15.112:0/2959319132 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e00046c0 con 0x55c3370db390 2024-09-21T03:37:00.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.157+0000 7f58ecff9640 1 -- 172.21.15.112:0/2959319132 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e00b21b0 con 0x55c3370db390 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 -- 172.21.15.112:0/2959319132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 msgr2=0x55c3374a42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 --2- 172.21.15.112:0/2959319132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374a42a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f58e00a9f50 tx=0x7f58e0005680 comp rx=0 tx=0).stop 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 -- 172.21.15.112:0/2959319132 shutdown_connections 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 --2- 172.21.15.112:0/2959319132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370fc070 0x55c3374a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 --2- 172.21.15.112:0/2959319132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370db390 0x55c3374a42a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 --2- 172.21.15.112:0/2959319132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 0x55c3374b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 -- 172.21.15.112:0/2959319132 >> 172.21.15.112:0/2959319132 conn(0x55c3370f0460 msgr2=0x55c3374a6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 -- 172.21.15.112:0/2959319132 shutdown_connections 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 -- 172.21.15.112:0/2959319132 wait complete. 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 -- start start 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 Processor -- start 2024-09-21T03:37:00.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 0x55c3374b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370db390 0x55c3374b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.158+0000 7f5903d43740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370fc070 0x55c33752a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f5903d43740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81ab9b0 con 0x55c3370fc070 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f5903d43740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81ace20 con 0x55c3370d31b0 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f5903d43740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81ac080 con 0x55c3370db390 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f5903d43740 10 client.0 _ll_register_callbacks cb 0x7f58e00a9040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58f6ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 0x55c3374b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370fc070 0x55c33752a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58f6ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 0x55c3374b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:48902/0 (socket says 172.21.15.112:48902) 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58f6ffd640 1 -- 172.21.15.112:0/3509657343 learned_addr learned my addr 172.21.15.112:0/3509657343 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370fc070 0x55c33752a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44468/0 (socket says 172.21.15.112:44468) 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370db390 0x55c3374b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58f6ffd640 1 -- 172.21.15.112:0/3509657343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370db390 msgr2=0x55c3374b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58f6ffd640 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370db390 0x55c3374b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58f6ffd640 1 -- 172.21.15.112:0/3509657343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370fc070 msgr2=0x55c33752a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58f6ffd640 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370fc070 0x55c33752a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58f6ffd640 1 -- 172.21.15.112:0/3509657343 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3374b3e60 con 0x55c3370d31b0 2024-09-21T03:37:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58f6ffd640 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 0x55c3374b3050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f58e8011d10 tx=0x7f58e8009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6911c3b6764aaa46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:37:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58de7fc640 1 -- 172.21.15.112:0/3509657343 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e801c040 con 0x55c3370d31b0 2024-09-21T03:37:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f5903d43740 1 -- 172.21.15.112:0/3509657343 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c337417850 con 0x55c3370d31b0 2024-09-21T03:37:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.159+0000 7f58de7fc640 1 -- 172.21.15.112:0/3509657343 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f58e8025c70 con 0x55c3370d31b0 2024-09-21T03:37:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f58de7fc640 1 -- 172.21.15.112:0/3509657343 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58e803d050 con 0x55c3370d31b0 2024-09-21T03:37:00.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f5903d43740 10 client.26393 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f5903d43740 1 -- 172.21.15.112:0/3509657343 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c33752ba80 con 0x55c3370d31b0 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f58de7fc640 1 -- 172.21.15.112:0/3509657343 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c337417850 con 0x55c3370d31b0 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f58de7fc640 1 -- 172.21.15.112:0/3509657343 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f58e8017a80 con 0x55c3370d31b0 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f5903d43740 10 client.26393 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f5903d43740 10 client.26393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f5903d43740 20 client.26393 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f5903d43740 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c33708fda0 0x55c33752dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f5903d43740 4 client.26393 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f5903d43740 1 -- 172.21.15.112:0/3509657343 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c337414c80 con 0x55c33708fda0 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.160+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c33708fda0 0x55c33752dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.161+0000 7f58de7fc640 10 client.26393 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:00.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.161+0000 7f58ee7fc640 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c33708fda0 0x55c33752dfc0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=193bbb7d5c35c650 server_cookie=c7330b5b3394361f in_seq=0 out_seq=0 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.161+0000 7f58de7fc640 1 -- 172.21.15.112:0/3509657343 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x55c337414c80 con 0x55c33708fda0 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.161+0000 7f58de7fc640 10 client.26393 handle_command_reply: tid=0 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.161+0000 7f5903d43740 1 client.26393 shutdown 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.161+0000 7f5903d43740 1 -- 172.21.15.112:0/3509657343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 msgr2=0x55c3374b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.161+0000 7f5903d43740 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 0x55c3374b3050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f58e8011d10 tx=0x7f58e8009ed0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.161+0000 7f5903d43740 1 -- 172.21.15.112:0/3509657343 shutdown_connections 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f5903d43740 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c33708fda0 0x55c33752dfc0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f5903d43740 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3370fc070 0x55c33752a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f58de7fc640 0 client.26393 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f5903d43740 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3370db390 0x55c3374b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f5903d43740 1 --2- 172.21.15.112:0/3509657343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3370d31b0 0x55c3374b3050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f5903d43740 1 -- 172.21.15.112:0/3509657343 >> 172.21.15.112:0/3509657343 conn(0x55c3370f0460 msgr2=0x55c3374a6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f5903d43740 1 -- 172.21.15.112:0/3509657343 shutdown_connections 2024-09-21T03:37:00.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f5903d43740 1 -- 172.21.15.112:0/3509657343 wait complete. 2024-09-21T03:37:00.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f5903d43740 20 client.26393 trim_cache size 0 max 16384 2024-09-21T03:37:00.172 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:37:00.172 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:37:00.172 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:37:00.172 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:37:00.172 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:37:00.172 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:37:00.175 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:37:00.176 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.176 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:37:00.176 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:37:00.176 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.177 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.177 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.177 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.177 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:37:00.177 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:37:00.177 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.177 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.177 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.177 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.177 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.178 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:37:00.178 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:37:00.178 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.178 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:37:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:37:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:37:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:37:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:00.180 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:37:00.181 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:37:00.181 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:00.181 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:00.181 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:00.181 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:00.181 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:00.181 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:37:00.181 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:37:00.181 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:37:00.183 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:36:59 smithi187 ceph-mon[34603]: pgmap v561: 97 pgs: 97 active+clean; 617 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-09-21T03:37:00.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:36:59 smithi203 ceph-mon[34723]: pgmap v561: 97 pgs: 97 active+clean; 617 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 256 op/s 2024-09-21T03:37:00.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f58fd7cf640 1 -- 172.21.15.112:0/3313302753 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58e40b6680 msgr2=0x7f58e40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f58fd7cf640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58e40b6680 0x7f58e40b8b90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f58f002f500 tx=0x7f58f002f350 comp rx=0 tx=0).stop 2024-09-21T03:37:00.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f58fd7cf640 1 -- 172.21.15.112:0/3313302753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f81083e0 msgr2=0x7f58f81998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:00.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.162+0000 7f58fd7cf640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f81083e0 0x7f58f81998f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58e000e9c0 tx=0x7f58e000ee80 comp rx=0 tx=0).stop 2024-09-21T03:37:00.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.163+0000 7f58fd7cf640 1 -- 172.21.15.112:0/3313302753 shutdown_connections 2024-09-21T03:37:00.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.163+0000 7f58fd7cf640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f58e40b6680 0x7f58e40b8b90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.163+0000 7f58fd7cf640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58f8108d30 0x7f58f819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.163+0000 7f58fd7cf640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58f81083e0 0x7f58f81998f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.163+0000 7f58fd7cf640 1 --2- 172.21.15.112:0/3313302753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f8102aa0 0x7f58f8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:00.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.163+0000 7f58fd7cf640 1 -- 172.21.15.112:0/3313302753 >> 172.21.15.112:0/3313302753 conn(0x7f58f80797e0 msgr2=0x7f58f81043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:00.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.163+0000 7f58fd7cf640 1 -- 172.21.15.112:0/3313302753 shutdown_connections 2024-09-21T03:37:00.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:00.163+0000 7f58fd7cf640 1 -- 172.21.15.112:0/3313302753 wait complete. 2024-09-21T03:37:00.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 841836 10.71 MB/sec execute 300 sec latency 41.700 ms 2024-09-21T03:37:01.186 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:37:01.186 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:37:01.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 842521 10.69 MB/sec execute 301 sec latency 60.226 ms 2024-09-21T03:37:01.442 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:01.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.946+0000 7f980c236640 1 -- 172.21.15.112:0/3341388041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9804102a60 msgr2=0x7f9804102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.946+0000 7f980c236640 1 --2- 172.21.15.112:0/3341388041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9804102a60 0x7f9804102e70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9800009900 tx=0x7f980002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.946+0000 7f980c236640 1 -- 172.21.15.112:0/3341388041 shutdown_connections 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.946+0000 7f980c236640 1 --2- 172.21.15.112:0/3341388041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9804108cf0 0x7f980410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.946+0000 7f980c236640 1 --2- 172.21.15.112:0/3341388041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98041083a0 0x7f98041087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.946+0000 7f980c236640 1 --2- 172.21.15.112:0/3341388041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9804102a60 0x7f9804102e70 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.946+0000 7f980c236640 1 -- 172.21.15.112:0/3341388041 >> 172.21.15.112:0/3341388041 conn(0x7f9804079730 msgr2=0x7f9804079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.946+0000 7f980c236640 1 -- 172.21.15.112:0/3341388041 shutdown_connections 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.946+0000 7f980c236640 1 -- 172.21.15.112:0/3341388041 wait complete. 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.947+0000 7f980c236640 1 Processor -- start 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.947+0000 7f980c236640 1 -- start start 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.947+0000 7f980c236640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9804102a60 0x7f9804199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.947+0000 7f980c236640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98041083a0 0x7f9804199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:01.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.947+0000 7f980c236640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9804108cf0 0x7f980419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.947+0000 7f980c236640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9804112960 con 0x7f9804102a60 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.947+0000 7f980c236640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98041127e0 con 0x7f9804108cf0 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.947+0000 7f980c236640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9804112ae0 con 0x7f98041083a0 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.947+0000 7f9809fd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9804102a60 0x7f9804199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f9809fd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9804102a60 0x7f9804199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44488/0 (socket says 172.21.15.112:44488) 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f9809fd2640 1 -- 172.21.15.112:0/592736858 learned_addr learned my addr 172.21.15.112:0/592736858 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f98097d1640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98041083a0 0x7f9804199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f980a7d3640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9804108cf0 0x7f980419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f9809fd2640 1 -- 172.21.15.112:0/592736858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98041083a0 msgr2=0x7f9804199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f9809fd2640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98041083a0 0x7f9804199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f9809fd2640 1 -- 172.21.15.112:0/592736858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9804108cf0 msgr2=0x7f980419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f9809fd2640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9804108cf0 0x7f980419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f9809fd2640 1 -- 172.21.15.112:0/592736858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980419f2e0 con 0x7f9804102a60 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f9809fd2640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9804102a60 0x7f9804199370 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9800005940 tx=0x7f98000040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:01.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f97faffd640 1 -- 172.21.15.112:0/592736858 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980003f030 con 0x7f9804102a60 2024-09-21T03:37:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f97faffd640 1 -- 172.21.15.112:0/592736858 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98000044c0 con 0x7f9804102a60 2024-09-21T03:37:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f97faffd640 1 -- 172.21.15.112:0/592736858 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800035070 con 0x7f9804102a60 2024-09-21T03:37:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f980c236640 1 -- 172.21.15.112:0/592736858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980419f570 con 0x7f9804102a60 2024-09-21T03:37:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.948+0000 7f980c236640 1 -- 172.21.15.112:0/592736858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98041a5d30 con 0x7f9804102a60 2024-09-21T03:37:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.950+0000 7f980c236640 1 -- 172.21.15.112:0/592736858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9804102e70 con 0x7f9804102a60 2024-09-21T03:37:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.951+0000 7f97faffd640 1 -- 172.21.15.112:0/592736858 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9800042030 con 0x7f9804102a60 2024-09-21T03:37:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.952+0000 7f97faffd640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97f00b6700 0x7f97f00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.952+0000 7f98097d1640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97f00b6700 0x7f97f00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.952+0000 7f97faffd640 1 -- 172.21.15.112:0/592736858 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7f98001079f0 con 0x7f9804102a60 2024-09-21T03:37:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.952+0000 7f98097d1640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97f00b6700 0x7f97f00b8c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9804100d20 tx=0x7f97f4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:01.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:01.956+0000 7f97faffd640 1 -- 172.21.15.112:0/592736858 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98000d2350 con 0x7f9804102a60 2024-09-21T03:37:02.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:01 smithi203 ceph-mon[34723]: pgmap v562: 97 pgs: 97 active+clean; 620 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 290 op/s 2024-09-21T03:37:02.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:01 smithi187 ceph-mon[34603]: pgmap v562: 97 pgs: 97 active+clean; 620 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 290 op/s 2024-09-21T03:37:02.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.193+0000 7f980c236640 1 -- 172.21.15.112:0/592736858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f980410aac0 con 0x7f9804102a60 2024-09-21T03:37:02.195 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:01 smithi112 ceph-mon[31442]: pgmap v562: 97 pgs: 97 active+clean; 620 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 290 op/s 2024-09-21T03:37:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.194+0000 7f97faffd640 1 -- 172.21.15.112:0/592736858 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f98000d7290 con 0x7f9804102a60 2024-09-21T03:37:02.196 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:37:02.196 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"btime":"2024-09-21T03:35:32:357323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:35:32.307199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:37:02.196 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-09-21T03:37:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.196+0000 7f980c236640 1 -- 172.21.15.112:0/592736858 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97f00b6700 msgr2=0x7f97f00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.196+0000 7f980c236640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97f00b6700 0x7f97f00b8c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9804100d20 tx=0x7f97f4005f90 comp rx=0 tx=0).stop 2024-09-21T03:37:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.196+0000 7f980c236640 1 -- 172.21.15.112:0/592736858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9804102a60 msgr2=0x7f9804199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.196+0000 7f980c236640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9804102a60 0x7f9804199370 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9800005940 tx=0x7f98000040c0 comp rx=0 tx=0).stop 2024-09-21T03:37:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.196+0000 7f980c236640 1 -- 172.21.15.112:0/592736858 shutdown_connections 2024-09-21T03:37:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.196+0000 7f980c236640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97f00b6700 0x7f97f00b8c10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:02.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.196+0000 7f980c236640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9804108cf0 0x7f980419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.196+0000 7f980c236640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98041083a0 0x7f9804199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.196+0000 7f980c236640 1 --2- 172.21.15.112:0/592736858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9804102a60 0x7f9804199370 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.197+0000 7f980c236640 1 -- 172.21.15.112:0/592736858 >> 172.21.15.112:0/592736858 conn(0x7f9804079730 msgr2=0x7f9804101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.197+0000 7f980c236640 1 -- 172.21.15.112:0/592736858 shutdown_connections 2024-09-21T03:37:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:02.197+0000 7f980c236640 1 -- 172.21.15.112:0/592736858 wait complete. 2024-09-21T03:37:02.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 845007 10.70 MB/sec execute 302 sec latency 52.146 ms 2024-09-21T03:37:03.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:37:03.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:02 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/592736858' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:03.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:02 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/592736858' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:03.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 846276 10.68 MB/sec execute 303 sec latency 13.963 ms 2024-09-21T03:37:03.265 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:03.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:02 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/592736858' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.752+0000 7fb50c368640 1 -- 172.21.15.112:0/3316248019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb504102a60 msgr2=0x7fb504102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.752+0000 7fb50c368640 1 --2- 172.21.15.112:0/3316248019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb504102a60 0x7fb504102e70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb500009900 tx=0x7fb50002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.752+0000 7fb50c368640 1 -- 172.21.15.112:0/3316248019 shutdown_connections 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.752+0000 7fb50c368640 1 --2- 172.21.15.112:0/3316248019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb504108cf0 0x7fb50410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.752+0000 7fb50c368640 1 --2- 172.21.15.112:0/3316248019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5041083a0 0x7fb5041087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.752+0000 7fb50c368640 1 --2- 172.21.15.112:0/3316248019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb504102a60 0x7fb504102e70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.752+0000 7fb50c368640 1 -- 172.21.15.112:0/3316248019 >> 172.21.15.112:0/3316248019 conn(0x7fb504079730 msgr2=0x7fb504079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.752+0000 7fb50c368640 1 -- 172.21.15.112:0/3316248019 shutdown_connections 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.752+0000 7fb50c368640 1 -- 172.21.15.112:0/3316248019 wait complete. 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.753+0000 7fb50c368640 1 Processor -- start 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.753+0000 7fb50c368640 1 -- start start 2024-09-21T03:37:03.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.753+0000 7fb50c368640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb504102a60 0x7fb50419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.753+0000 7fb50c368640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5041083a0 0x7fb50419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.753+0000 7fb50c368640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb504108cf0 0x7fb50419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.753+0000 7fb50c368640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb504113a80 con 0x7fb504102a60 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.753+0000 7fb50c368640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb504113900 con 0x7fb504108cf0 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.753+0000 7fb50c368640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb504113c00 con 0x7fb5041083a0 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50a104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb504102a60 0x7fb50419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50a104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb504102a60 0x7fb50419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44500/0 (socket says 172.21.15.112:44500) 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50a104640 1 -- 172.21.15.112:0/2853010394 learned_addr learned my addr 172.21.15.112:0/2853010394 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb509903640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5041083a0 0x7fb50419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50a905640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb504108cf0 0x7fb50419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50a104640 1 -- 172.21.15.112:0/2853010394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5041083a0 msgr2=0x7fb50419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50a104640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5041083a0 0x7fb50419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50a104640 1 -- 172.21.15.112:0/2853010394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb504108cf0 msgr2=0x7fb50419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50a104640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb504108cf0 0x7fb50419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50a104640 1 -- 172.21.15.112:0/2853010394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5041a03a0 con 0x7fb504102a60 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50a104640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb504102a60 0x7fb50419a430 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb500005940 tx=0x7fb5000040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb4fb7fe640 1 -- 172.21.15.112:0/2853010394 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50003f030 con 0x7fb504102a60 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb4fb7fe640 1 -- 172.21.15.112:0/2853010394 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5000044c0 con 0x7fb504102a60 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50c368640 1 -- 172.21.15.112:0/2853010394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041a0630 con 0x7fb504102a60 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb4fb7fe640 1 -- 172.21.15.112:0/2853010394 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500035070 con 0x7fb504102a60 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.754+0000 7fb50c368640 1 -- 172.21.15.112:0/2853010394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041a6df0 con 0x7fb504102a60 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.757+0000 7fb4fb7fe640 1 -- 172.21.15.112:0/2853010394 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb500042030 con 0x7fb504102a60 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.757+0000 7fb5128dc740 1 Processor -- start 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.757+0000 7fb5128dc740 1 -- start start 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.757+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 0x556e77f9d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.757+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 0x556e77f9dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.757+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77f9e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb5128dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556e77f13ea0 con 0x556e77bcd1b0 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb5128dc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556e77f032d0 con 0x556e77bf6070 2024-09-21T03:37:03.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb5128dc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556e77f070e0 con 0x556e77bd5390 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb50a104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 0x556e77f9dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb50a905640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 0x556e77f9d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb509903640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77f9e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb50a104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 0x556e77f9dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44516/0 (socket says 172.21.15.112:44516) 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb50a104640 1 -- 172.21.15.112:0/2922103506 learned_addr learned my addr 172.21.15.112:0/2922103506 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb50a905640 1 --2- 172.21.15.112:0/2922103506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 0x556e77f9d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:48954/0 (socket says 172.21.15.112:48954) 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb509903640 1 -- 172.21.15.112:0/2922103506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 msgr2=0x556e77f9d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb509903640 1 --2- 172.21.15.112:0/2922103506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 0x556e77f9d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb509903640 1 -- 172.21.15.112:0/2922103506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 msgr2=0x556e77f9dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb509903640 1 --2- 172.21.15.112:0/2922103506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 0x556e77f9dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb509903640 1 -- 172.21.15.112:0/2922103506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e77c61650 con 0x556e77bd5390 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb50a905640 1 --2- 172.21.15.112:0/2922103506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 0x556e77f9d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb4fb7fe640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb4f00b6680 0x7fb4f00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb50a905640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb4f00b6680 0x7fb4f00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb509903640 1 --2- 172.21.15.112:0/2922103506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77f9e120 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb50406aaa0 tx=0x7fb4f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd420d1e21d51ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb4fb7fe640 1 -- 172.21.15.112:0/2853010394 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7fb500107a20 con 0x7fb504102a60 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.758+0000 7fb4f8d9a640 1 -- 172.21.15.112:0/2922103506 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4004050 con 0x556e77bd5390 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.759+0000 7fb4f8d9a640 1 -- 172.21.15.112:0/2922103506 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f40054d0 con 0x556e77bd5390 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.759+0000 7fb4f8d9a640 1 -- 172.21.15.112:0/2922103506 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4004c20 con 0x556e77bd5390 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.759+0000 7fb5128dc740 1 -- 172.21.15.112:0/2922103506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 msgr2=0x556e77f9e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.759+0000 7fb50a905640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb4f00b6680 0x7fb4f00b8b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009fd0 tx=0x7fb4fc009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.759+0000 7fb5128dc740 1 --2- 172.21.15.112:0/2922103506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77f9e120 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb50406aaa0 tx=0x7fb4f4011810 comp rx=0 tx=0).stop 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 -- 172.21.15.112:0/2922103506 shutdown_connections 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 --2- 172.21.15.112:0/2922103506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77f9e120 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 --2- 172.21.15.112:0/2922103506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 0x556e77f9dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 --2- 172.21.15.112:0/2922103506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 0x556e77f9d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 -- 172.21.15.112:0/2922103506 >> 172.21.15.112:0/2922103506 conn(0x556e77bea460 msgr2=0x556e77f9b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 -- 172.21.15.112:0/2922103506 shutdown_connections 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 -- 172.21.15.112:0/2922103506 wait complete. 2024-09-21T03:37:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 -- start start 2024-09-21T03:37:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 Processor -- start 2024-09-21T03:37:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 0x556e77fa8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bd5390 0x556e77fa9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bf6070 0x556e77fa9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556e77d64700 con 0x556e77bd5390 2024-09-21T03:37:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.760+0000 7fb5128dc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556e77e9e160 con 0x556e77bcd1b0 2024-09-21T03:37:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb5128dc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556e77fabe80 con 0x556e77bf6070 2024-09-21T03:37:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb509903640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bd5390 0x556e77fa9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb5128dc740 10 client.0 _ll_register_callbacks cb 0x556e77eb79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb509903640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bd5390 0x556e77fa9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44522/0 (socket says 172.21.15.112:44522) 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb509903640 1 -- 172.21.15.112:0/1775341194 learned_addr learned my addr 172.21.15.112:0/1775341194 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb50a104640 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 0x556e77fa8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb50a905640 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bf6070 0x556e77fa9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb509903640 1 -- 172.21.15.112:0/1775341194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bf6070 msgr2=0x556e77fa9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb509903640 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bf6070 0x556e77fa9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb509903640 1 -- 172.21.15.112:0/1775341194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 msgr2=0x556e77fa8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb509903640 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 0x556e77fa8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb509903640 1 -- 172.21.15.112:0/1775341194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e77ecbc90 con 0x556e77bd5390 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb50a104640 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 0x556e77fa8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb509903640 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bd5390 0x556e77fa9230 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb4f401b040 tx=0x7fb4f4005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aebd7d837a0edd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:03.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb4edffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.761+0000 7fb4edffb640 1 -- 172.21.15.112:0/1775341194 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4029070 con 0x556e77bd5390 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb4edffb640 1 -- 172.21.15.112:0/1775341194 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7fb4f4003a00 con 0x556e77bd5390 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb5128dc740 1 -- 172.21.15.112:0/1775341194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f40054d0 con 0x556e77bd5390 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb4edffb640 1 -- 172.21.15.112:0/1775341194 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb4f4035050 con 0x556e77bd5390 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb5128dc740 10 client.16854 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb5128dc740 1 -- 172.21.15.112:0/1775341194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e77fb7b30 con 0x556e77bd5390 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb4edffb640 1 -- 172.21.15.112:0/1775341194 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f40054d0 con 0x556e77bd5390 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb4edffb640 1 -- 172.21.15.112:0/1775341194 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb4f4005030 con 0x556e77bd5390 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb5128dc740 10 client.16854 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb5128dc740 10 client.16854 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb5128dc740 20 client.16854 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb5128dc740 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556e77bb49f0 0x556e77fbae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb5128dc740 4 client.16854 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb5128dc740 1 -- 172.21.15.112:0/1775341194 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e77f15df0 con 0x556e77bb49f0 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.762+0000 7fb50a104640 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556e77bb49f0 0x556e77fbae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.763+0000 7fb50a104640 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556e77bb49f0 0x556e77fbae90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4347fc5f48f4dc12 server_cookie=c7957e7c30f3610e in_seq=0 out_seq=0 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.763+0000 7fb4edffb640 10 client.16854 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.763+0000 7fb4edffb640 1 -- 172.21.15.112:0/1775341194 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e77f15df0 con 0x556e77bb49f0 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.763+0000 7fb4edffb640 10 client.16854 handle_command_reply: tid=0 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 1 client.16854 shutdown 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 1 -- 172.21.15.112:0/1775341194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bd5390 msgr2=0x556e77fa9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bd5390 0x556e77fa9230 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb4f401b040 tx=0x7fb4f4005680 comp rx=0 tx=0).stop 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 1 -- 172.21.15.112:0/1775341194 shutdown_connections 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556e77bb49f0 0x556e77fbae90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bf6070 0x556e77fa9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bd5390 0x556e77fa9230 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 1 --2- 172.21.15.112:0/1775341194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 0x556e77fa8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 1 -- 172.21.15.112:0/1775341194 >> 172.21.15.112:0/1775341194 conn(0x556e77bea460 msgr2=0x556e77fa0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb4edffb640 0 client.16854 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:37:03.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 1 -- 172.21.15.112:0/1775341194 shutdown_connections 2024-09-21T03:37:03.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 1 -- 172.21.15.112:0/1775341194 wait complete. 2024-09-21T03:37:03.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.764+0000 7fb5128dc740 20 client.16854 trim_cache size 0 max 16384 2024-09-21T03:37:03.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb5128dc740 1 Processor -- start 2024-09-21T03:37:03.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb5128dc740 1 -- start start 2024-09-21T03:37:03.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 0x556e77faabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77f9e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 0x556e77fb47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb5128dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041b1ee0 con 0x556e77bcd1b0 2024-09-21T03:37:03.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb5128dc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041af7b0 con 0x556e77bf6070 2024-09-21T03:37:03.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb5128dc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041ad620 con 0x556e77bd5390 2024-09-21T03:37:03.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb50a104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 0x556e77faabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb50a104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 0x556e77faabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44524/0 (socket says 172.21.15.112:44524) 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb50a104640 1 -- 172.21.15.112:0/3503982550 learned_addr learned my addr 172.21.15.112:0/3503982550 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.778+0000 7fb509903640 1 --2- 172.21.15.112:0/3503982550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77f9e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.779+0000 7fb50a905640 1 --2- 172.21.15.112:0/3503982550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 0x556e77fb47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.779+0000 7fb50a104640 1 -- 172.21.15.112:0/3503982550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 msgr2=0x556e77f9e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.779+0000 7fb50a104640 1 --2- 172.21.15.112:0/3503982550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77f9e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.779+0000 7fb50a104640 1 -- 172.21.15.112:0/3503982550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 msgr2=0x556e77fb47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.779+0000 7fb50a104640 1 --2- 172.21.15.112:0/3503982550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 0x556e77fb47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.779+0000 7fb50a104640 1 -- 172.21.15.112:0/3503982550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e77bb4c20 con 0x556e77bcd1b0 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.779+0000 7fb50a905640 1 --2- 172.21.15.112:0/3503982550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 0x556e77fb47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.779+0000 7fb50a104640 1 --2- 172.21.15.112:0/3503982550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 0x556e77faabd0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb5000f10d0 tx=0x7fb50010b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbd92d30ef1cf92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.779+0000 7fb4effff640 1 -- 172.21.15.112:0/3503982550 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50010e070 con 0x556e77bcd1b0 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.779+0000 7fb4effff640 1 -- 172.21.15.112:0/3503982550 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb500110040 con 0x556e77bcd1b0 2024-09-21T03:37:03.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.779+0000 7fb4effff640 1 -- 172.21.15.112:0/3503982550 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500049040 con 0x556e77bcd1b0 2024-09-21T03:37:03.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.780+0000 7fb5128dc740 1 -- 172.21.15.112:0/3503982550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 msgr2=0x556e77faabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:03.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.780+0000 7fb5128dc740 1 --2- 172.21.15.112:0/3503982550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 0x556e77faabd0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb5000f10d0 tx=0x7fb50010b4c0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.780+0000 7fb5128dc740 1 -- 172.21.15.112:0/3503982550 shutdown_connections 2024-09-21T03:37:03.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.780+0000 7fb5128dc740 1 --2- 172.21.15.112:0/3503982550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bf6070 0x556e77fb47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.780+0000 7fb5128dc740 1 --2- 172.21.15.112:0/3503982550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77f9e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.780+0000 7fb5128dc740 1 --2- 172.21.15.112:0/3503982550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bcd1b0 0x556e77faabd0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.780+0000 7fb5128dc740 1 -- 172.21.15.112:0/3503982550 >> 172.21.15.112:0/3503982550 conn(0x556e77bea460 msgr2=0x556e77fb40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.780+0000 7fb5128dc740 1 -- 172.21.15.112:0/3503982550 shutdown_connections 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.780+0000 7fb5128dc740 1 -- 172.21.15.112:0/3503982550 wait complete. 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.780+0000 7fb5128dc740 1 -- start start 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.780+0000 7fb5128dc740 1 Processor -- start 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 0x556e77facb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77fad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb5128dc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bf6070 0x556e77fb1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb5128dc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041adc70 con 0x556e77bf6070 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb5128dc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041af0e0 con 0x556e77bcd1b0 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb5128dc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041ae340 con 0x556e77bd5390 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb5128dc740 10 client.0 _ll_register_callbacks cb 0x7fb50010c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb50a905640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bf6070 0x556e77fb1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb50a905640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bf6070 0x556e77fb1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44538/0 (socket says 172.21.15.112:44538) 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb50a905640 1 -- 172.21.15.112:0/3513504100 learned_addr learned my addr 172.21.15.112:0/3513504100 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb50a104640 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 0x556e77facb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb509903640 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77fad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb50a905640 1 -- 172.21.15.112:0/3513504100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 msgr2=0x556e77fad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb50a905640 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77fad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb50a905640 1 -- 172.21.15.112:0/3513504100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 msgr2=0x556e77facb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb50a905640 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 0x556e77facb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb50a905640 1 -- 172.21.15.112:0/3513504100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e77fb16e0 con 0x556e77bf6070 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb50a104640 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 0x556e77facb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb50a905640 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bf6070 0x556e77fb1070 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc011ff0 tx=0x7fb4fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0099515aa25cab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.781+0000 7fb4edffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:03.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb4edffb640 1 -- 172.21.15.112:0/3513504100 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc003970 con 0x556e77bf6070 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb4edffb640 1 -- 172.21.15.112:0/3513504100 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8470+0+0 (secure 0 0 0) 0x7fb4fc003b10 con 0x556e77bf6070 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb5128dc740 1 -- 172.21.15.112:0/3513504100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb500110040 con 0x556e77bf6070 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb4edffb640 1 -- 172.21.15.112:0/3513504100 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb4fc003d50 con 0x556e77bf6070 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb5128dc740 10 client.16860 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb5128dc740 1 -- 172.21.15.112:0/3513504100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e780252d0 con 0x556e77bf6070 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb4edffb640 1 -- 172.21.15.112:0/3513504100 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb500110040 con 0x556e77bf6070 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb4edffb640 1 -- 172.21.15.112:0/3513504100 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb4fc004240 con 0x556e77bf6070 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb5128dc740 10 client.16860 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb5128dc740 10 client.16860 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb5128dc740 20 client.16860 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb5128dc740 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556e77b89da0 0x556e78027b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb5128dc740 4 client.16860 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.782+0000 7fb5128dc740 1 -- 172.21.15.112:0/3513504100 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e77f118a0 con 0x556e77b89da0 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb50a104640 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556e77b89da0 0x556e78027b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:03.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb50a104640 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556e77b89da0 0x556e78027b40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9b340ce67fe3975 server_cookie=c3bc6b727f75ffdd in_seq=0 out_seq=0 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb4edffb640 10 client.16860 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb4edffb640 1 -- 172.21.15.112:0/3513504100 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e77f118a0 con 0x556e77b89da0 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb4edffb640 10 client.16860 handle_command_reply: tid=0 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb5128dc740 1 client.16860 shutdown 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb5128dc740 1 -- 172.21.15.112:0/3513504100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bf6070 msgr2=0x556e77fb1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb5128dc740 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bf6070 0x556e77fb1070 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc011ff0 tx=0x7fb4fc004a40 comp rx=0 tx=0).stop 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb5128dc740 1 -- 172.21.15.112:0/3513504100 shutdown_connections 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb5128dc740 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556e77b89da0 0x556e78027b40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb5128dc740 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556e77bf6070 0x556e77fb1070 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb5128dc740 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556e77bd5390 0x556e77fad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb5128dc740 1 --2- 172.21.15.112:0/3513504100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e77bcd1b0 0x556e77facb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb5128dc740 1 -- 172.21.15.112:0/3513504100 >> 172.21.15.112:0/3513504100 conn(0x556e77bea460 msgr2=0x556e77fa9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb4edffb640 0 client.16860 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb5128dc740 1 -- 172.21.15.112:0/3513504100 shutdown_connections 2024-09-21T03:37:03.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.783+0000 7fb5128dc740 1 -- 172.21.15.112:0/3513504100 wait complete. 2024-09-21T03:37:03.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb5128dc740 20 client.16860 trim_cache size 0 max 16384 2024-09-21T03:37:03.791 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 -- 172.21.15.112:0/2853010394 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb4f00b6680 msgr2=0x7fb4f00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb4f00b6680 0x7fb4f00b8b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009fd0 tx=0x7fb4fc009500 comp rx=0 tx=0).stop 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 -- 172.21.15.112:0/2853010394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb504102a60 msgr2=0x7fb50419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb504102a60 0x7fb50419a430 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb500005940 tx=0x7fb5000040c0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 -- 172.21.15.112:0/2853010394 shutdown_connections 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb4f00b6680 0x7fb4f00b8b90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb504108cf0 0x7fb50419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5041083a0 0x7fb50419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 --2- 172.21.15.112:0/2853010394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb504102a60 0x7fb50419a430 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 -- 172.21.15.112:0/2853010394 >> 172.21.15.112:0/2853010394 conn(0x7fb504079730 msgr2=0x7fb5041013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 -- 172.21.15.112:0/2853010394 shutdown_connections 2024-09-21T03:37:03.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:03.784+0000 7fb50c368640 1 -- 172.21.15.112:0/2853010394 wait complete. 2024-09-21T03:37:04.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:03 smithi112 ceph-mon[31442]: pgmap v563: 97 pgs: 97 active+clean; 621 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 233 op/s 2024-09-21T03:37:04.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:03 smithi203 ceph-mon[34723]: pgmap v563: 97 pgs: 97 active+clean; 621 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 233 op/s 2024-09-21T03:37:04.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:03 smithi187 ceph-mon[34603]: pgmap v563: 97 pgs: 97 active+clean; 621 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 233 op/s 2024-09-21T03:37:04.244 INFO:tasks.workunit.client.0.smithi112.stdout: 1 848766 10.70 MB/sec execute 304 sec latency 19.819 ms 2024-09-21T03:37:04.629 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:37:04.906 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:05.263 INFO:tasks.workunit.client.0.smithi112.stdout: 1 850056 10.68 MB/sec execute 305 sec latency 26.302 ms 2024-09-21T03:37:05.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.517+0000 7ff0ed360640 1 -- 172.21.15.112:0/971705278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e8109df0 msgr2=0x7ff0e8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.517+0000 7ff0ed360640 1 --2- 172.21.15.112:0/971705278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e8109df0 0x7ff0e8110620 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009ca0 tx=0x7ff0e002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.517+0000 7ff0ed360640 1 -- 172.21.15.112:0/971705278 shutdown_connections 2024-09-21T03:37:05.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.517+0000 7ff0ed360640 1 --2- 172.21.15.112:0/971705278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e8109df0 0x7ff0e8110620 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.517+0000 7ff0ed360640 1 --2- 172.21.15.112:0/971705278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0e81094a0 0x7ff0e81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.517+0000 7ff0ed360640 1 --2- 172.21.15.112:0/971705278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e8103b60 0x7ff0e8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.517+0000 7ff0ed360640 1 -- 172.21.15.112:0/971705278 >> 172.21.15.112:0/971705278 conn(0x7ff0e807b850 msgr2=0x7ff0e807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:05.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.517+0000 7ff0ed360640 1 -- 172.21.15.112:0/971705278 shutdown_connections 2024-09-21T03:37:05.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.517+0000 7ff0ed360640 1 -- 172.21.15.112:0/971705278 wait complete. 2024-09-21T03:37:05.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.518+0000 7ff0ed360640 1 Processor -- start 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.518+0000 7ff0ed360640 1 -- start start 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.518+0000 7ff0ed360640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0e8103b60 0x7ff0e819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.518+0000 7ff0ed360640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e81094a0 0x7ff0e819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.518+0000 7ff0ed360640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e8109df0 0x7ff0e819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.518+0000 7ff0ed360640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e8113ab0 con 0x7ff0e81094a0 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.518+0000 7ff0ed360640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e8113930 con 0x7ff0e8109df0 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.518+0000 7ff0ed360640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e8113c30 con 0x7ff0e8103b60 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0de7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e81094a0 0x7ff0e819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0de7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e81094a0 0x7ff0e819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49216/0 (socket says 172.21.15.112:49216) 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0de7fc640 1 -- 172.21.15.112:0/1175479588 learned_addr learned my addr 172.21.15.112:0/1175479588 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0e8103b60 0x7ff0e819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0e77fe640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e8109df0 0x7ff0e819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0de7fc640 1 -- 172.21.15.112:0/1175479588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0e8103b60 msgr2=0x7ff0e819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0de7fc640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0e8103b60 0x7ff0e819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0de7fc640 1 -- 172.21.15.112:0/1175479588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e8109df0 msgr2=0x7ff0e819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0de7fc640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e8109df0 0x7ff0e819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0de7fc640 1 -- 172.21.15.112:0/1175479588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e81a03f0 con 0x7ff0e81094a0 2024-09-21T03:37:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0e8103b60 0x7ff0e819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0e77fe640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e8109df0 0x7ff0e819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0de7fc640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e81094a0 0x7ff0e819aa20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000d750 tx=0x7ff0d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0e4ff9640 1 -- 172.21.15.112:0/1175479588 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0017070 con 0x7ff0e81094a0 2024-09-21T03:37:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0ed360640 1 -- 172.21.15.112:0/1175479588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e81a06e0 con 0x7ff0e81094a0 2024-09-21T03:37:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0e4ff9640 1 -- 172.21.15.112:0/1175479588 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d00054d0 con 0x7ff0e81094a0 2024-09-21T03:37:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0e4ff9640 1 -- 172.21.15.112:0/1175479588 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0004c30 con 0x7ff0e81094a0 2024-09-21T03:37:05.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.519+0000 7ff0ed360640 1 -- 172.21.15.112:0/1175479588 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e81a6e40 con 0x7ff0e81094a0 2024-09-21T03:37:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.522+0000 7ff0f38d4740 1 Processor -- start 2024-09-21T03:37:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.522+0000 7ff0f38d4740 1 -- start start 2024-09-21T03:37:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.522+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 0x5560b53e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.522+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b50101b0 0x5560b53e0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.522+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5018390 0x5560b53e1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.522+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5560b5356ea0 con 0x5560b5039070 2024-09-21T03:37:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5560b53462d0 con 0x5560b5018390 2024-09-21T03:37:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5560b534a0e0 con 0x5560b50101b0 2024-09-21T03:37:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0e4ff9640 1 -- 172.21.15.112:0/1175479588 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff0d0010460 con 0x7ff0e81094a0 2024-09-21T03:37:05.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 0x5560b53e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 0x5560b53e06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49224/0 (socket says 172.21.15.112:49224) 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0e77fe640 1 -- 172.21.15.112:0/1607869870 learned_addr learned my addr 172.21.15.112:0/1607869870 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/1607869870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b50101b0 0x5560b53e0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0de7fc640 1 --2- 172.21.15.112:0/1607869870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5018390 0x5560b53e1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0e6ffd640 1 -- 172.21.15.112:0/1607869870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5018390 msgr2=0x5560b53e1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/1607869870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5018390 0x5560b53e1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0e6ffd640 1 -- 172.21.15.112:0/1607869870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 msgr2=0x5560b53e06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/1607869870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 0x5560b53e06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0e6ffd640 1 -- 172.21.15.112:0/1607869870 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b50a4650 con 0x5560b50101b0 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/1607869870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b50101b0 0x5560b53e0be0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009fd0 tx=0x7ff0d80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f81a858eca0698f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0dd7fa640 1 -- 172.21.15.112:0/1607869870 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8004030 con 0x5560b50101b0 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.523+0000 7ff0dd7fa640 1 -- 172.21.15.112:0/1607869870 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d80054d0 con 0x5560b50101b0 2024-09-21T03:37:05.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.524+0000 7ff0dd7fa640 1 -- 172.21.15.112:0/1607869870 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8004c40 con 0x5560b50101b0 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.524+0000 7ff0e4ff9640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0d40b6680 0x7ff0d40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.524+0000 7ff0e77fe640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0d40b6680 0x7ff0d40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.524+0000 7ff0e4ff9640 1 -- 172.21.15.112:0/1175479588 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7ff0d00e7090 con 0x7ff0e81094a0 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0e77fe640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0d40b6680 0x7ff0d40b8b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0035000 tx=0x7ff0e0032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0f38d4740 1 -- 172.21.15.112:0/1607869870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b50101b0 msgr2=0x5560b53e0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/1607869870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b50101b0 0x5560b53e0be0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009fd0 tx=0x7ff0d80118d0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0f38d4740 1 -- 172.21.15.112:0/1607869870 shutdown_connections 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/1607869870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5018390 0x5560b53e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/1607869870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b50101b0 0x5560b53e0be0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/1607869870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 0x5560b53e06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0f38d4740 1 -- 172.21.15.112:0/1607869870 >> 172.21.15.112:0/1607869870 conn(0x5560b502d460 msgr2=0x5560b53de590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0f38d4740 1 -- 172.21.15.112:0/1607869870 shutdown_connections 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0f38d4740 1 -- 172.21.15.112:0/1607869870 wait complete. 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0f38d4740 1 -- start start 2024-09-21T03:37:05.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.525+0000 7ff0f38d4740 1 Processor -- start 2024-09-21T03:37:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b50101b0 0x5560b53ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5018390 0x5560b53ec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 0x5560b53ec710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5560b51a7700 con 0x5560b5039070 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5560b52e1160 con 0x5560b50101b0 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5560b53eee80 con 0x5560b5018390 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0f38d4740 10 client.0 _ll_register_callbacks cb 0x5560b52fa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0de7fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5018390 0x5560b53ec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 0x5560b53ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0e6ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b50101b0 0x5560b53ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0de7fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5018390 0x5560b53ec1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:57888/0 (socket says 172.21.15.112:57888) 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0de7fc640 1 -- 172.21.15.112:0/3463854553 learned_addr learned my addr 172.21.15.112:0/3463854553 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0e77fe640 1 -- 172.21.15.112:0/3463854553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5018390 msgr2=0x5560b53ec1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0e77fe640 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5018390 0x5560b53ec1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0e77fe640 1 -- 172.21.15.112:0/3463854553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b50101b0 msgr2=0x5560b53ebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0e77fe640 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b50101b0 0x5560b53ebc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0e77fe640 1 -- 172.21.15.112:0/3463854553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b530ec90 con 0x5560b5039070 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b50101b0 0x5560b53ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0e77fe640 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 0x5560b53ec710 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00094e0 tx=0x7ff0e002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b56ea2820f32960 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.526+0000 7ff0ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.527+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/3463854553 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e005b060 con 0x5560b5039070 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.527+0000 7ff0f38d4740 1 -- 172.21.15.112:0/3463854553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d80054d0 con 0x5560b5039070 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.527+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/3463854553 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7ff0e0043030 con 0x5560b5039070 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.527+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/3463854553 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff0e004b7a0 con 0x5560b5039070 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.527+0000 7ff0f38d4740 10 client.16872 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.527+0000 7ff0f38d4740 1 -- 172.21.15.112:0/3463854553 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5560b53faae0 con 0x5560b5039070 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.527+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/3463854553 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d80054d0 con 0x5560b5039070 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.527+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/3463854553 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff0e002f660 con 0x5560b5039070 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.528+0000 7ff0f38d4740 10 client.16872 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.528+0000 7ff0f38d4740 10 client.16872 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.528+0000 7ff0f38d4740 20 client.16872 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:05.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.528+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5560b4ff79f0 0x5560b53fe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.528+0000 7ff0f38d4740 4 client.16872 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.528+0000 7ff0f38d4740 1 -- 172.21.15.112:0/3463854553 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560b5358df0 con 0x5560b4ff79f0 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.528+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5560b4ff79f0 0x5560b53fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.529+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5560b4ff79f0 0x5560b53fe000 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e247066240cfb891 server_cookie=9f20d44bcc22b0bd in_seq=0 out_seq=0 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.529+0000 7ff0ce7fc640 10 client.16872 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.529+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/3463854553 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5560b5358df0 con 0x5560b4ff79f0 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.529+0000 7ff0ce7fc640 10 client.16872 handle_command_reply: tid=0 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.529+0000 7ff0f38d4740 1 client.16872 shutdown 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.529+0000 7ff0f38d4740 1 -- 172.21.15.112:0/3463854553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 msgr2=0x5560b53ec710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.529+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 0x5560b53ec710 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00094e0 tx=0x7ff0e002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.529+0000 7ff0f38d4740 1 -- 172.21.15.112:0/3463854553 shutdown_connections 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.530+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5560b4ff79f0 0x5560b53fe000 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.530+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b5039070 0x5560b53ec710 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.530+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5018390 0x5560b53ec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.530+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/3463854553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b50101b0 0x5560b53ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.530+0000 7ff0f38d4740 1 -- 172.21.15.112:0/3463854553 >> 172.21.15.112:0/3463854553 conn(0x5560b502d460 msgr2=0x5560b53e3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.530+0000 7ff0ce7fc640 0 client.16872 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:37:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.530+0000 7ff0f38d4740 1 -- 172.21.15.112:0/3463854553 shutdown_connections 2024-09-21T03:37:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.530+0000 7ff0f38d4740 1 -- 172.21.15.112:0/3463854553 wait complete. 2024-09-21T03:37:05.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.530+0000 7ff0f38d4740 20 client.16872 trim_cache size 0 max 16384 2024-09-21T03:37:05.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.543+0000 7ff0f38d4740 1 Processor -- start 2024-09-21T03:37:05.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.543+0000 7ff0f38d4740 1 -- start start 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.543+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.543+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5018390 0x5560b53e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.543+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5039070 0x5560b53ddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.543+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81afdd0 con 0x5560b50101b0 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.543+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81ad6a0 con 0x5560b5039070 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.543+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81ab530 con 0x5560b5018390 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53edb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49244/0 (socket says 172.21.15.112:49244) 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0de7fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5018390 0x5560b53e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0e6ffd640 1 -- 172.21.15.112:0/2720991801 learned_addr learned my addr 172.21.15.112:0/2720991801 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0e77fe640 1 --2- 172.21.15.112:0/2720991801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5039070 0x5560b53ddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0e6ffd640 1 -- 172.21.15.112:0/2720991801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5018390 msgr2=0x5560b53e1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/2720991801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5018390 0x5560b53e1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0e6ffd640 1 -- 172.21.15.112:0/2720991801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5039070 msgr2=0x5560b53ddbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/2720991801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5039070 0x5560b53ddbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0e6ffd640 1 -- 172.21.15.112:0/2720991801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b53ecf50 con 0x5560b50101b0 2024-09-21T03:37:05.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/2720991801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53edb70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009380 tx=0x7ff0d8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace9cb549ea3c27b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:05.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0dcff9640 1 -- 172.21.15.112:0/2720991801 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800fb20 con 0x5560b50101b0 2024-09-21T03:37:05.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0dcff9640 1 -- 172.21.15.112:0/2720991801 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d800fcc0 con 0x5560b50101b0 2024-09-21T03:37:05.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.544+0000 7ff0dcff9640 1 -- 172.21.15.112:0/2720991801 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8024670 con 0x5560b50101b0 2024-09-21T03:37:05.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.545+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2720991801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 msgr2=0x5560b53edb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.545+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/2720991801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53edb70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009380 tx=0x7ff0d8005f90 comp rx=0 tx=0).stop 2024-09-21T03:37:05.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.545+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2720991801 shutdown_connections 2024-09-21T03:37:05.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.545+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/2720991801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5039070 0x5560b53ddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.545+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/2720991801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5018390 0x5560b53e1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.545+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/2720991801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53edb70 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.545+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2720991801 >> 172.21.15.112:0/2720991801 conn(0x5560b502d460 msgr2=0x5560b53e16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:05.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.545+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2720991801 shutdown_connections 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.545+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2720991801 wait complete. 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0f38d4740 1 -- start start 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0f38d4740 1 Processor -- start 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53f0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5018390 0x5560b53f0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0f38d4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5039070 0x5560b53f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81abb80 con 0x5560b50101b0 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81acfd0 con 0x5560b5018390 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0f38d4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81ac230 con 0x5560b5039070 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0f38d4740 10 client.0 _ll_register_callbacks cb 0x7ff0e0009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53f0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53f0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49250/0 (socket says 172.21.15.112:49250) 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0e6ffd640 1 -- 172.21.15.112:0/2042293035 learned_addr learned my addr 172.21.15.112:0/2042293035 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:05.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0de7fc640 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5018390 0x5560b53f0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.546+0000 7ff0e77fe640 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5039070 0x5560b53f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0e6ffd640 1 -- 172.21.15.112:0/2042293035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5039070 msgr2=0x5560b53f0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5039070 0x5560b53f0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0e6ffd640 1 -- 172.21.15.112:0/2042293035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5018390 msgr2=0x5560b53f0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5018390 0x5560b53f0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0e6ffd640 1 -- 172.21.15.112:0/2042293035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b54679c0 con 0x5560b50101b0 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0de7fc640 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5018390 0x5560b53f0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0e6ffd640 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53f0020 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8005bf0 tx=0x7ff0d8019f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deaadaadcb4db530 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/2042293035 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800fb20 con 0x5560b50101b0 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/2042293035 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7ff0d8005320 con 0x5560b50101b0 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2042293035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d800fcc0 con 0x5560b50101b0 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/2042293035 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff0d8019450 con 0x5560b50101b0 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0f38d4740 10 client.16878 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:05.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.547+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2042293035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5560b5468b80 con 0x5560b50101b0 2024-09-21T03:37:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.548+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/2042293035 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d800fcc0 con 0x5560b50101b0 2024-09-21T03:37:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.548+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/2042293035 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff0d8003a80 con 0x5560b50101b0 2024-09-21T03:37:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.548+0000 7ff0f38d4740 10 client.16878 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.548+0000 7ff0f38d4740 10 client.16878 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:37:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.548+0000 7ff0f38d4740 20 client.16878 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.548+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5560b4fccda0 0x5560b546b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.548+0000 7ff0f38d4740 4 client.16878 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:37:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.548+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2042293035 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560b53548a0 con 0x5560b4fccda0 2024-09-21T03:37:05.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.548+0000 7ff0de7fc640 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5560b4fccda0 0x5560b546b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.549+0000 7ff0ce7fc640 10 client.16878 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:37:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.549+0000 7ff0de7fc640 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5560b4fccda0 0x5560b546b110 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ad3322373b557fd server_cookie=3e8ad86fa0e4a3d4 in_seq=0 out_seq=0 2024-09-21T03:37:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.549+0000 7ff0ce7fc640 1 -- 172.21.15.112:0/2042293035 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560b53548a0 con 0x5560b4fccda0 2024-09-21T03:37:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.549+0000 7ff0ce7fc640 10 client.16878 handle_command_reply: tid=0 2024-09-21T03:37:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 1 client.16878 shutdown 2024-09-21T03:37:05.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2042293035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 msgr2=0x5560b53f0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53f0020 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8005bf0 tx=0x7ff0d8019f20 comp rx=0 tx=0).stop 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2042293035 shutdown_connections 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5560b4fccda0 0x5560b546b110 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5560b5039070 0x5560b53f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5560b5018390 0x5560b53f0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 1 --2- 172.21.15.112:0/2042293035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5560b50101b0 0x5560b53f0020 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0ce7fc640 0 client.16878 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2042293035 >> 172.21.15.112:0/2042293035 conn(0x5560b502d460 msgr2=0x5560b53ea110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2042293035 shutdown_connections 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 1 -- 172.21.15.112:0/2042293035 wait complete. 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0f38d4740 20 client.16878 trim_cache size 0 max 16384 2024-09-21T03:37:05.555 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:37:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0ed360640 1 -- 172.21.15.112:0/1175479588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0d40b6680 msgr2=0x7ff0d40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0ed360640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0d40b6680 0x7ff0d40b8b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0035000 tx=0x7ff0e0032ed0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0ed360640 1 -- 172.21.15.112:0/1175479588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e81094a0 msgr2=0x7ff0e819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.550+0000 7ff0ed360640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e81094a0 0x7ff0e819aa20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000d750 tx=0x7ff0d000dc10 comp rx=0 tx=0).stop 2024-09-21T03:37:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.551+0000 7ff0ed360640 1 -- 172.21.15.112:0/1175479588 shutdown_connections 2024-09-21T03:37:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.551+0000 7ff0ed360640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff0d40b6680 0x7ff0d40b8b90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.551+0000 7ff0ed360640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0e8109df0 0x7ff0e819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.551+0000 7ff0ed360640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e81094a0 0x7ff0e819aa20 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.551+0000 7ff0ed360640 1 --2- 172.21.15.112:0/1175479588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0e8103b60 0x7ff0e819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:05.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.551+0000 7ff0ed360640 1 -- 172.21.15.112:0/1175479588 >> 172.21.15.112:0/1175479588 conn(0x7ff0e807b850 msgr2=0x7ff0e8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:05.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.551+0000 7ff0ed360640 1 -- 172.21.15.112:0/1175479588 shutdown_connections 2024-09-21T03:37:05.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:05.551+0000 7ff0ed360640 1 -- 172.21.15.112:0/1175479588 wait complete. 2024-09-21T03:37:06.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:05 smithi203 ceph-mon[34723]: pgmap v564: 97 pgs: 97 active+clean; 625 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-09-21T03:37:06.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:05 smithi203 ceph-mon[34723]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:37:06.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:05 smithi187 ceph-mon[34603]: pgmap v564: 97 pgs: 97 active+clean; 625 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-09-21T03:37:06.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:05 smithi187 ceph-mon[34603]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:37:06.263 INFO:tasks.workunit.client.0.smithi112.stdout: 1 852258 10.69 MB/sec execute 306 sec latency 40.563 ms 2024-09-21T03:37:06.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:05 smithi112 ceph-mon[31442]: pgmap v564: 97 pgs: 97 active+clean; 625 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-09-21T03:37:06.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:05 smithi112 ceph-mon[31442]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:37:06.356 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:37:06.613 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.124+0000 7f1528fbd640 1 -- 172.21.15.112:0/2192730147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15241083e0 msgr2=0x7f15241087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.124+0000 7f1528fbd640 1 --2- 172.21.15.112:0/2192730147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15241083e0 0x7f15241087f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f150c009900 tx=0x7f150c02a410 comp rx=0 tx=0).stop 2024-09-21T03:37:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.124+0000 7f1528fbd640 1 -- 172.21.15.112:0/2192730147 shutdown_connections 2024-09-21T03:37:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.124+0000 7f1528fbd640 1 --2- 172.21.15.112:0/2192730147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1524108d30 0x7f152410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.124+0000 7f1528fbd640 1 --2- 172.21.15.112:0/2192730147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15241083e0 0x7f15241087f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.124+0000 7f1528fbd640 1 --2- 172.21.15.112:0/2192730147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1524102aa0 0x7f1524102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.124+0000 7f1528fbd640 1 -- 172.21.15.112:0/2192730147 >> 172.21.15.112:0/2192730147 conn(0x7f1524079750 msgr2=0x7f1524079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.124+0000 7f1528fbd640 1 -- 172.21.15.112:0/2192730147 shutdown_connections 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.124+0000 7f1528fbd640 1 -- 172.21.15.112:0/2192730147 wait complete. 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.125+0000 7f1528fbd640 1 Processor -- start 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.125+0000 7f1528fbd640 1 -- start start 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f1528fbd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1524102aa0 0x7f152419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f1528fbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15241083e0 0x7f152419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f1528fbd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1524108d30 0x7f152419bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f1528fbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15241129f0 con 0x7f15241083e0 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f1528fbd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1524112870 con 0x7f1524108d30 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f1528fbd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1524112b70 con 0x7f1524102aa0 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f151bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15241083e0 0x7f152419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f152259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1524102aa0 0x7f152419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f151bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15241083e0 0x7f152419b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49272/0 (socket says 172.21.15.112:49272) 2024-09-21T03:37:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f151bfff640 1 -- 172.21.15.112:0/1978820973 learned_addr learned my addr 172.21.15.112:0/1978820973 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f1522d9d640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1524108d30 0x7f152419bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f151bfff640 1 -- 172.21.15.112:0/1978820973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1524102aa0 msgr2=0x7f152419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f151bfff640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1524102aa0 0x7f152419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f151bfff640 1 -- 172.21.15.112:0/1978820973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1524108d30 msgr2=0x7f152419bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f151bfff640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1524108d30 0x7f152419bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f151bfff640 1 -- 172.21.15.112:0/1978820973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15241a65f0 con 0x7f15241083e0 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.126+0000 7f151bfff640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15241083e0 0x7f152419b8a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f150c02a930 tx=0x7f150c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.127+0000 7f151b7fe640 1 -- 172.21.15.112:0/1978820973 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c03f030 con 0x7f15241083e0 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.127+0000 7f1528fbd640 1 -- 172.21.15.112:0/1978820973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15241a6880 con 0x7f15241083e0 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.127+0000 7f151b7fe640 1 -- 172.21.15.112:0/1978820973 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f150c004490 con 0x7f15241083e0 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.127+0000 7f151b7fe640 1 -- 172.21.15.112:0/1978820973 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c035070 con 0x7f15241083e0 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.127+0000 7f1528fbd640 1 -- 172.21.15.112:0/1978820973 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15241a6db0 con 0x7f15241083e0 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.129+0000 7f152f531740 1 Processor -- start 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.129+0000 7f151b7fe640 1 -- 172.21.15.112:0/1978820973 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f150c042030 con 0x7f15241083e0 2024-09-21T03:37:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f152f531740 1 -- start start 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f152f531740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4ea070 0x55ebce8912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f152f531740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 0x55ebce891800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f152f531740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4c9390 0x55ebce891d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f152f531740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ebce807ea0 con 0x55ebce4c11b0 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f152f531740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ebce7f72d0 con 0x55ebce4ea070 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f152f531740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ebce7fb0e0 con 0x55ebce4c9390 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f1522d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4ea070 0x55ebce8912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f152259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 0x55ebce891800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f1522d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4ea070 0x55ebce8912c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43700/0 (socket says 172.21.15.112:43700) 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f1522d9d640 1 -- 172.21.15.112:0/3976538106 learned_addr learned my addr 172.21.15.112:0/3976538106 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.130+0000 7f151bfff640 1 --2- 172.21.15.112:0/3976538106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4c9390 0x55ebce891d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f1522d9d640 1 -- 172.21.15.112:0/3976538106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4c9390 msgr2=0x55ebce891d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f1522d9d640 1 --2- 172.21.15.112:0/3976538106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4c9390 0x55ebce891d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f1522d9d640 1 -- 172.21.15.112:0/3976538106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 msgr2=0x55ebce891800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f1522d9d640 1 --2- 172.21.15.112:0/3976538106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 0x55ebce891800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f1522d9d640 1 -- 172.21.15.112:0/3976538106 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebce555650 con 0x55ebce4ea070 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f151bfff640 1 --2- 172.21.15.112:0/3976538106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4c9390 0x55ebce891d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f151b7fe640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15100b66b0 0x7f15100b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f1522d9d640 1 --2- 172.21.15.112:0/3976538106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4ea070 0x55ebce8912c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f151c009f50 tx=0x7f151c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebecb072d0b99e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f152259c640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15100b66b0 0x7f15100b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f151b7fe640 1 -- 172.21.15.112:0/1978820973 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7f150c1079f0 con 0x7f15241083e0 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f1518ff9640 1 -- 172.21.15.112:0/3976538106 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c004d20 con 0x55ebce4ea070 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f1518ff9640 1 -- 172.21.15.112:0/3976538106 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c004050 con 0x55ebce4ea070 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.131+0000 7f1518ff9640 1 -- 172.21.15.112:0/3976538106 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c005280 con 0x55ebce4ea070 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.132+0000 7f152f531740 1 -- 172.21.15.112:0/3976538106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4ea070 msgr2=0x55ebce8912c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.132+0000 7f152259c640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15100b66b0 0x7f15100b8bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55ebce88ff40 tx=0x7f1514015040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.132+0000 7f152f531740 1 --2- 172.21.15.112:0/3976538106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4ea070 0x55ebce8912c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f151c009f50 tx=0x7f151c0118d0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.132+0000 7f152f531740 1 -- 172.21.15.112:0/3976538106 shutdown_connections 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.132+0000 7f152f531740 1 --2- 172.21.15.112:0/3976538106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4c9390 0x55ebce891d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.132+0000 7f152f531740 1 --2- 172.21.15.112:0/3976538106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 0x55ebce891800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.132+0000 7f152f531740 1 --2- 172.21.15.112:0/3976538106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4ea070 0x55ebce8912c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.132+0000 7f152f531740 1 -- 172.21.15.112:0/3976538106 >> 172.21.15.112:0/3976538106 conn(0x55ebce4de460 msgr2=0x55ebce88f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.132+0000 7f152f531740 1 -- 172.21.15.112:0/3976538106 shutdown_connections 2024-09-21T03:37:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.132+0000 7f152f531740 1 -- 172.21.15.112:0/3976538106 wait complete. 2024-09-21T03:37:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f152f531740 1 -- start start 2024-09-21T03:37:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f152f531740 1 Processor -- start 2024-09-21T03:37:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f152f531740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4c11b0 0x55ebce89ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f152f531740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c9390 0x55ebce89cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f152f531740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4ea070 0x55ebce8a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f152f531740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ebce658700 con 0x55ebce4ea070 2024-09-21T03:37:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f152f531740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ebce792160 con 0x55ebce4c9390 2024-09-21T03:37:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f152f531740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ebce89faa0 con 0x55ebce4c11b0 2024-09-21T03:37:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f152f531740 10 client.0 _ll_register_callbacks cb 0x55ebce7ab9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f1522d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4ea070 0x55ebce8a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f152259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4c11b0 0x55ebce89ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f1522d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4ea070 0x55ebce8a8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49296/0 (socket says 172.21.15.112:49296) 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f1522d9d640 1 -- 172.21.15.112:0/1059350784 learned_addr learned my addr 172.21.15.112:0/1059350784 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.133+0000 7f151bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c9390 0x55ebce89cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f151bfff640 1 -- 172.21.15.112:0/1059350784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4c11b0 msgr2=0x55ebce89ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f151bfff640 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4c11b0 0x55ebce89ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f151bfff640 1 -- 172.21.15.112:0/1059350784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4ea070 msgr2=0x55ebce8a8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f151bfff640 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4ea070 0x55ebce8a8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f151bfff640 1 -- 172.21.15.112:0/1059350784 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebce7bfc90 con 0x55ebce4c9390 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f1522d9d640 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4ea070 0x55ebce8a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f151bfff640 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c9390 0x55ebce89cf50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f150c105140 tx=0x7f150c003f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9865cf73269879c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f1509ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:37:07.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f1509ffb640 1 -- 172.21.15.112:0/1059350784 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c049070 con 0x55ebce4c9390 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f152f531740 1 -- 172.21.15.112:0/1059350784 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151c004050 con 0x55ebce4c9390 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f1509ffb640 1 -- 172.21.15.112:0/1059350784 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7f150c03bc30 con 0x55ebce4c9390 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f1509ffb640 1 -- 172.21.15.112:0/1059350784 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f150c0cd6c0 con 0x55ebce4c9390 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f152f531740 10 client.26447 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f152f531740 1 -- 172.21.15.112:0/1059350784 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ebce8ab7a0 con 0x55ebce4c9390 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.134+0000 7f1509ffb640 1 -- 172.21.15.112:0/1059350784 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c004050 con 0x55ebce4c9390 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.135+0000 7f1509ffb640 1 -- 172.21.15.112:0/1059350784 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f150c0d6dd0 con 0x55ebce4c9390 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.135+0000 7f152f531740 10 client.26447 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.135+0000 7f152f531740 10 client.26447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.135+0000 7f152f531740 20 client.26447 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.135+0000 7f152f531740 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ebce4a89f0 0x55ebce8aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.135+0000 7f152f531740 4 client.26447 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.135+0000 7f152f531740 1 -- 172.21.15.112:0/1059350784 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebce809df0 con 0x55ebce4a89f0 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.135+0000 7f152259c640 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ebce4a89f0 0x55ebce8aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.135+0000 7f1509ffb640 10 client.26447 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.135+0000 7f152259c640 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ebce4a89f0 0x55ebce8aec60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21c5d9ec01ed4f9 server_cookie=259fe3d2b1cd3c3d in_seq=0 out_seq=0 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f1509ffb640 1 -- 172.21.15.112:0/1059350784 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ebce809df0 con 0x55ebce4a89f0 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f1509ffb640 10 client.26447 handle_command_reply: tid=0 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 1 client.26447 shutdown 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 1 -- 172.21.15.112:0/1059350784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c9390 msgr2=0x55ebce89cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c9390 0x55ebce89cf50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f150c105140 tx=0x7f150c003f60 comp rx=0 tx=0).stop 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 1 -- 172.21.15.112:0/1059350784 shutdown_connections 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ebce4a89f0 0x55ebce8aec60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4ea070 0x55ebce8a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c9390 0x55ebce89cf50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f1509ffb640 0 client.26447 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 1 --2- 172.21.15.112:0/1059350784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4c11b0 0x55ebce89ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 1 -- 172.21.15.112:0/1059350784 >> 172.21.15.112:0/1059350784 conn(0x55ebce4de460 msgr2=0x55ebce894450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:07.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 1 -- 172.21.15.112:0/1059350784 shutdown_connections 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 1 -- 172.21.15.112:0/1059350784 wait complete. 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.136+0000 7f152f531740 20 client.26447 trim_cache size 0 max 16384 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f152f531740 1 Processor -- start 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f152f531740 1 -- start start 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f152f531740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 0x55ebce88f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f152f531740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c11b0 0x55ebce8a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f152f531740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c9390 0x55ebce8943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f152f531740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15241afe20 con 0x55ebce4c9390 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f152f531740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15241ad6f0 con 0x55ebce4c11b0 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f152f531740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f15241ab560 con 0x55ebce4a89f0 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f1522d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c9390 0x55ebce8943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f1522d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c9390 0x55ebce8943c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49306/0 (socket says 172.21.15.112:49306) 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f1522d9d640 1 -- 172.21.15.112:0/3616049878 learned_addr learned my addr 172.21.15.112:0/3616049878 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:07.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f152259c640 1 --2- 172.21.15.112:0/3616049878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 0x55ebce88f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.150+0000 7f151bfff640 1 --2- 172.21.15.112:0/3616049878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c11b0 0x55ebce8a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.151+0000 7f1522d9d640 1 -- 172.21.15.112:0/3616049878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 msgr2=0x55ebce88f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.151+0000 7f1522d9d640 1 --2- 172.21.15.112:0/3616049878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 0x55ebce88f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.151+0000 7f1522d9d640 1 -- 172.21.15.112:0/3616049878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c11b0 msgr2=0x55ebce8a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.151+0000 7f1522d9d640 1 --2- 172.21.15.112:0/3616049878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c11b0 0x55ebce8a8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.151+0000 7f1522d9d640 1 -- 172.21.15.112:0/3616049878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebce8935b0 con 0x55ebce4c9390 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.151+0000 7f152259c640 1 --2- 172.21.15.112:0/3616049878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 0x55ebce88f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.151+0000 7f151bfff640 1 --2- 172.21.15.112:0/3616049878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c11b0 0x55ebce8a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.151+0000 7f1522d9d640 1 --2- 172.21.15.112:0/3616049878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c9390 0x55ebce8943c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f151c009f50 tx=0x7f151c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fd100a685e62cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.151+0000 7f150bfff640 1 -- 172.21.15.112:0/3616049878 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c004c80 con 0x55ebce4c9390 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.151+0000 7f150bfff640 1 -- 172.21.15.112:0/3616049878 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c004e20 con 0x55ebce4c9390 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.151+0000 7f150bfff640 1 -- 172.21.15.112:0/3616049878 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c0056a0 con 0x55ebce4c9390 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.152+0000 7f152f531740 1 -- 172.21.15.112:0/3616049878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c9390 msgr2=0x55ebce8943c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.152+0000 7f152f531740 1 --2- 172.21.15.112:0/3616049878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c9390 0x55ebce8943c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f151c009f50 tx=0x7f151c009be0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.152+0000 7f152f531740 1 -- 172.21.15.112:0/3616049878 shutdown_connections 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.152+0000 7f152f531740 1 --2- 172.21.15.112:0/3616049878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c9390 0x55ebce8943c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.152+0000 7f152f531740 1 --2- 172.21.15.112:0/3616049878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c11b0 0x55ebce8a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.152+0000 7f152f531740 1 --2- 172.21.15.112:0/3616049878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 0x55ebce88f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.152+0000 7f152f531740 1 -- 172.21.15.112:0/3616049878 >> 172.21.15.112:0/3616049878 conn(0x55ebce4de460 msgr2=0x55ebce88eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.152+0000 7f152f531740 1 -- 172.21.15.112:0/3616049878 shutdown_connections 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.152+0000 7f152f531740 1 -- 172.21.15.112:0/3616049878 wait complete. 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.152+0000 7f152f531740 1 -- start start 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f152f531740 1 Processor -- start 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f152f531740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 0x55ebce8a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f152f531740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 0x55ebce8a1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f152f531740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c9390 0x55ebce8a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f152f531740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15241abbb0 con 0x55ebce4c11b0 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f152f531740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15241ad020 con 0x55ebce4c9390 2024-09-21T03:37:07.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f152f531740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f15241ac280 con 0x55ebce4a89f0 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f151bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 0x55ebce8a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f152f531740 10 client.0 _ll_register_callbacks cb 0x7f150c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f151bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 0x55ebce8a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49322/0 (socket says 172.21.15.112:49322) 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f151bfff640 1 -- 172.21.15.112:0/1301093046 learned_addr learned my addr 172.21.15.112:0/1301093046 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f152259c640 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 0x55ebce8a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f1522d9d640 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c9390 0x55ebce8a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f151bfff640 1 -- 172.21.15.112:0/1301093046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 msgr2=0x55ebce8a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f151bfff640 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 0x55ebce8a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f151bfff640 1 -- 172.21.15.112:0/1301093046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c9390 msgr2=0x55ebce8a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f151bfff640 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c9390 0x55ebce8a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f151bfff640 1 -- 172.21.15.112:0/1301093046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebce9185e0 con 0x55ebce4c11b0 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.153+0000 7f152259c640 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 0x55ebce8a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f151bfff640 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 0x55ebce8a1220 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f150c0cab60 tx=0x7f150c03bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e321f2ea0721b0c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f1509ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:07.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f1509ffb640 1 -- 172.21.15.112:0/1301093046 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c049070 con 0x55ebce4c11b0 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f1509ffb640 1 -- 172.21.15.112:0/1301093046 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7f150c0ccb50 con 0x55ebce4c11b0 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f152f531740 1 -- 172.21.15.112:0/1301093046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151c004e20 con 0x55ebce4c11b0 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f1509ffb640 1 -- 172.21.15.112:0/1301093046 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f150c0cd060 con 0x55ebce4c11b0 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f152f531740 10 client.16896 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f152f531740 1 -- 172.21.15.112:0/1301093046 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ebce919840 con 0x55ebce4c11b0 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f1509ffb640 1 -- 172.21.15.112:0/1301093046 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c004e20 con 0x55ebce4c11b0 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f1509ffb640 1 -- 172.21.15.112:0/1301093046 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f150c0cc5c0 con 0x55ebce4c11b0 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f152f531740 10 client.16896 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f152f531740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f152f531740 20 client.16896 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f152f531740 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ebce4ea070 0x55ebce91bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f152f531740 4 client.16896 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.154+0000 7f152f531740 1 -- 172.21.15.112:0/1301093046 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ebce8058a0 con 0x55ebce4ea070 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.155+0000 7f152259c640 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ebce4ea070 0x55ebce91bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:07.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.155+0000 7f152259c640 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ebce4ea070 0x55ebce91bec0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc03dc3fd2d5388c server_cookie=5a38a08a0e866908 in_seq=0 out_seq=0 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f1509ffb640 10 client.16896 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f1509ffb640 1 -- 172.21.15.112:0/1301093046 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebce8058a0 con 0x55ebce4ea070 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f1509ffb640 10 client.16896 handle_command_reply: tid=0 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 1 client.16896 shutdown 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 1 -- 172.21.15.112:0/1301093046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 msgr2=0x55ebce8a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 0x55ebce8a1220 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f150c0cab60 tx=0x7f150c03bc60 comp rx=0 tx=0).stop 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 1 -- 172.21.15.112:0/1301093046 shutdown_connections 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ebce4ea070 0x55ebce91bec0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ebce4c9390 0x55ebce8a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ebce4c11b0 0x55ebce8a1220 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 1 --2- 172.21.15.112:0/1301093046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebce4a89f0 0x55ebce8a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 1 -- 172.21.15.112:0/1301093046 >> 172.21.15.112:0/1301093046 conn(0x55ebce4de460 msgr2=0x55ebce8943c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f1509ffb640 0 client.16896 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 1 -- 172.21.15.112:0/1301093046 shutdown_connections 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 1 -- 172.21.15.112:0/1301093046 wait complete. 2024-09-21T03:37:07.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.156+0000 7f152f531740 20 client.16896 trim_cache size 0 max 16384 2024-09-21T03:37:07.173 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:37:07.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.157+0000 7f1528fbd640 1 -- 172.21.15.112:0/1978820973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15100b66b0 msgr2=0x7f15100b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.157+0000 7f1528fbd640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15100b66b0 0x7f15100b8bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55ebce88ff40 tx=0x7f1514015040 comp rx=0 tx=0).stop 2024-09-21T03:37:07.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.157+0000 7f1528fbd640 1 -- 172.21.15.112:0/1978820973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15241083e0 msgr2=0x7f152419b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:07.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.157+0000 7f1528fbd640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15241083e0 0x7f152419b8a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f150c02a930 tx=0x7f150c031a20 comp rx=0 tx=0).stop 2024-09-21T03:37:07.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.157+0000 7f1528fbd640 1 -- 172.21.15.112:0/1978820973 shutdown_connections 2024-09-21T03:37:07.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.157+0000 7f1528fbd640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f15100b66b0 0x7f15100b8bc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.157+0000 7f1528fbd640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1524108d30 0x7f152419bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.157+0000 7f1528fbd640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15241083e0 0x7f152419b8a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.157+0000 7f1528fbd640 1 --2- 172.21.15.112:0/1978820973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1524102aa0 0x7f152419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:07.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.157+0000 7f1528fbd640 1 -- 172.21.15.112:0/1978820973 >> 172.21.15.112:0/1978820973 conn(0x7f1524079750 msgr2=0x7f15241064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:07.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.158+0000 7f1528fbd640 1 -- 172.21.15.112:0/1978820973 shutdown_connections 2024-09-21T03:37:07.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:07.158+0000 7f1528fbd640 1 -- 172.21.15.112:0/1978820973 wait complete. 2024-09-21T03:37:07.263 INFO:tasks.workunit.client.0.smithi112.stdout: 1 853604 10.67 MB/sec execute 307 sec latency 19.909 ms 2024-09-21T03:37:08.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:07 smithi112 ceph-mon[31442]: pgmap v566: 97 pgs: 97 active+clean; 633 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 304 op/s 2024-09-21T03:37:08.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:07 smithi203 ceph-mon[34723]: pgmap v566: 97 pgs: 97 active+clean; 633 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 304 op/s 2024-09-21T03:37:08.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:07 smithi187 ceph-mon[34603]: pgmap v566: 97 pgs: 97 active+clean; 633 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 304 op/s 2024-09-21T03:37:08.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 856002 10.68 MB/sec execute 308 sec latency 13.961 ms 2024-09-21T03:37:08.868 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:37:08.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:37:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:09.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 857838 10.67 MB/sec execute 309 sec latency 15.769 ms 2024-09-21T03:37:09.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.648+0000 7fc89fb63640 1 -- 172.21.15.112:0/2191004782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc898108d20 msgr2=0x7fc89810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.648+0000 7fc89fb63640 1 --2- 172.21.15.112:0/2191004782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc898108d20 0x7fc89810f550 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc890009ca0 tx=0x7fc89002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.648+0000 7fc89fb63640 1 -- 172.21.15.112:0/2191004782 shutdown_connections 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.648+0000 7fc89fb63640 1 --2- 172.21.15.112:0/2191004782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc898108d20 0x7fc89810f550 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.648+0000 7fc89fb63640 1 --2- 172.21.15.112:0/2191004782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8981083d0 0x7fc8981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.648+0000 7fc89fb63640 1 --2- 172.21.15.112:0/2191004782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898102ae0 0x7fc898102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.648+0000 7fc89fb63640 1 -- 172.21.15.112:0/2191004782 >> 172.21.15.112:0/2191004782 conn(0x7fc898079790 msgr2=0x7fc898079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.648+0000 7fc89fb63640 1 -- 172.21.15.112:0/2191004782 shutdown_connections 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.648+0000 7fc89fb63640 1 -- 172.21.15.112:0/2191004782 wait complete. 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89fb63640 1 Processor -- start 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89fb63640 1 -- start start 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89fb63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc898102ae0 0x7fc898199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89fb63640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8981083d0 0x7fc8981998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89fb63640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898108d20 0x7fc89819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89fb63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981129e0 con 0x7fc898102ae0 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89fb63640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc898112860 con 0x7fc898108d20 2024-09-21T03:37:09.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89fb63640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc898112b60 con 0x7fc8981083d0 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89d8ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc898102ae0 0x7fc898199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89d8ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc898102ae0 0x7fc898199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49352/0 (socket says 172.21.15.112:49352) 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89d0fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8981083d0 0x7fc8981998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.649+0000 7fc89d8ff640 1 -- 172.21.15.112:0/868121867 learned_addr learned my addr 172.21.15.112:0/868121867 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc89e100640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898108d20 0x7fc89819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc89d0fe640 1 -- 172.21.15.112:0/868121867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898108d20 msgr2=0x7fc89819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc89d0fe640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898108d20 0x7fc89819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc89d0fe640 1 -- 172.21.15.112:0/868121867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc898102ae0 msgr2=0x7fc898199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc89d0fe640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc898102ae0 0x7fc898199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc89d0fe640 1 -- 172.21.15.112:0/868121867 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89819f280 con 0x7fc8981083d0 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc89d8ff640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc898102ae0 0x7fc898199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc89e100640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898108d20 0x7fc89819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc89d0fe640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8981083d0 0x7fc8981998b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc89806ab20 tx=0x7fc88800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:09.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc88effd640 1 -- 172.21.15.112:0/868121867 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88800cc40 con 0x7fc8981083d0 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc89fb63640 1 -- 172.21.15.112:0/868121867 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89819f570 con 0x7fc8981083d0 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc88effd640 1 -- 172.21.15.112:0/868121867 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88800cde0 con 0x7fc8981083d0 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc89fb63640 1 -- 172.21.15.112:0/868121867 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8981a5cd0 con 0x7fc8981083d0 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.650+0000 7fc88effd640 1 -- 172.21.15.112:0/868121867 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888004a00 con 0x7fc8981083d0 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.653+0000 7fc88effd640 1 -- 172.21.15.112:0/868121867 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc888004ba0 con 0x7fc8981083d0 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.653+0000 7fc8a60d7740 1 Processor -- start 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc8a60d7740 1 -- start start 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 0x55df9bd4b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b97b490 0x55df9bd4b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd4beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55df9bcc1e60 con 0x55df9b9a4350 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55df9bcb1290 con 0x55df9b983670 2024-09-21T03:37:09.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55df9bcb50a0 con 0x55df9b97b490 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc89e100640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 0x55df9bd4b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc89e100640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 0x55df9bd4b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49362/0 (socket says 172.21.15.112:49362) 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc89e100640 1 -- 172.21.15.112:0/1262373688 learned_addr learned my addr 172.21.15.112:0/1262373688 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc89d8ff640 1 --2- 172.21.15.112:0/1262373688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b97b490 0x55df9bd4b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.654+0000 7fc89d0fe640 1 --2- 172.21.15.112:0/1262373688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd4beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.655+0000 7fc88effd640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc8840b6680 0x7fc8840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.655+0000 7fc89e100640 1 -- 172.21.15.112:0/1262373688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b97b490 msgr2=0x55df9bd4b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.655+0000 7fc89e100640 1 --2- 172.21.15.112:0/1262373688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b97b490 0x55df9bd4b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.655+0000 7fc89e100640 1 -- 172.21.15.112:0/1262373688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 msgr2=0x55df9bd4beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.655+0000 7fc89e100640 1 --2- 172.21.15.112:0/1262373688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd4beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.655+0000 7fc89e100640 1 -- 172.21.15.112:0/1262373688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df9ba0f930 con 0x55df9b9a4350 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.655+0000 7fc89d8ff640 1 --2- 172.21.15.112:0/1262373688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b97b490 0x55df9bd4b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:09.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.655+0000 7fc88effd640 1 -- 172.21.15.112:0/868121867 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7fc8880e76f0 con 0x7fc8981083d0 2024-09-21T03:37:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.656+0000 7fc89e100640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc8840b6680 0x7fc8840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.656+0000 7fc89e100640 1 --2- 172.21.15.112:0/1262373688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 0x55df9bd4b430 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc890009470 tx=0x7fc8900094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbbd90ed8612424 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.656+0000 7fc883fff640 1 -- 172.21.15.112:0/1262373688 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890003b70 con 0x55df9b9a4350 2024-09-21T03:37:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.656+0000 7fc883fff640 1 -- 172.21.15.112:0/1262373688 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc890036040 con 0x55df9b9a4350 2024-09-21T03:37:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.656+0000 7fc883fff640 1 -- 172.21.15.112:0/1262373688 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89002f860 con 0x55df9b9a4350 2024-09-21T03:37:09.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.657+0000 7fc8a60d7740 1 -- 172.21.15.112:0/1262373688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 msgr2=0x55df9bd4b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.657+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/1262373688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 0x55df9bd4b430 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc890009470 tx=0x7fc8900094a0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.657+0000 7fc89e100640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc8840b6680 0x7fc8840b8b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc89004c000 tx=0x7fc89004d000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.657+0000 7fc8a60d7740 1 -- 172.21.15.112:0/1262373688 shutdown_connections 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.657+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/1262373688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd4beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.657+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/1262373688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b97b490 0x55df9bd4b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.657+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/1262373688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 0x55df9bd4b430 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.657+0000 7fc8a60d7740 1 -- 172.21.15.112:0/1262373688 >> 172.21.15.112:0/1262373688 conn(0x55df9b998740 msgr2=0x55df9bd49320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.657+0000 7fc8a60d7740 1 -- 172.21.15.112:0/1262373688 shutdown_connections 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.657+0000 7fc8a60d7740 1 -- 172.21.15.112:0/1262373688 wait complete. 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.658+0000 7fc8a60d7740 1 -- start start 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.658+0000 7fc8a60d7740 1 Processor -- start 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.658+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.658+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd59700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b9a4350 0x55df9bd650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55df9bb12890 con 0x55df9b97b490 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55df9bc4bd40 con 0x55df9b983670 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55df9bd5bd70 con 0x55df9b9a4350 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc8a60d7740 10 client.0 _ll_register_callbacks cb 0x55df9bcafc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d0fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd59700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d8ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d0fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd59700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43760/0 (socket says 172.21.15.112:43760) 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89e100640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b9a4350 0x55df9bd650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d0fe640 1 -- 172.21.15.112:0/4166362680 learned_addr learned my addr 172.21.15.112:0/4166362680 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d8ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49376/0 (socket says 172.21.15.112:49376) 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d0fe640 1 -- 172.21.15.112:0/4166362680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b9a4350 msgr2=0x55df9bd650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d0fe640 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b9a4350 0x55df9bd650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d0fe640 1 -- 172.21.15.112:0/4166362680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 msgr2=0x55df9bd591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d0fe640 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d0fe640 1 -- 172.21.15.112:0/4166362680 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df9bd65610 con 0x55df9b983670 2024-09-21T03:37:09.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d8ff640 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc89d0fe640 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd59700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc88800c6a0 tx=0x7fc8880a9a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a9dff4dab5123ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.659+0000 7fc8817fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8817fa640 1 -- 172.21.15.112:0/4166362680 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88801c070 con 0x55df9b983670 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8817fa640 1 -- 172.21.15.112:0/4166362680 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7fc8880e7ba0 con 0x55df9b983670 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8a60d7740 1 -- 172.21.15.112:0/4166362680 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc890036040 con 0x55df9b983670 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8817fa640 1 -- 172.21.15.112:0/4166362680 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc8880ace90 con 0x55df9b983670 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8a60d7740 10 client.26465 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8a60d7740 1 -- 172.21.15.112:0/4166362680 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df9bd51760 con 0x55df9b983670 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8817fa640 1 -- 172.21.15.112:0/4166362680 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc890036040 con 0x55df9b983670 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8817fa640 1 -- 172.21.15.112:0/4166362680 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc8880a8e90 con 0x55df9b983670 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8a60d7740 10 client.26465 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8a60d7740 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8a60d7740 20 client.26465 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:09.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df9b962cd0 0x55df9bd52c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8a60d7740 4 client.26465 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc8a60d7740 1 -- 172.21.15.112:0/4166362680 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df9bcc3db0 con 0x55df9b962cd0 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.660+0000 7fc89d8ff640 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df9b962cd0 0x55df9bd52c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.661+0000 7fc8817fa640 10 client.26465 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.661+0000 7fc89d8ff640 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df9b962cd0 0x55df9bd52c20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=298479e44aab95ca server_cookie=e1e15df79e4013d in_seq=0 out_seq=0 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.662+0000 7fc8817fa640 1 -- 172.21.15.112:0/4166362680 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df9bcc3db0 con 0x55df9b962cd0 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.662+0000 7fc8817fa640 10 client.26465 handle_command_reply: tid=0 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.662+0000 7fc8a60d7740 1 client.26465 shutdown 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.662+0000 7fc8a60d7740 1 -- 172.21.15.112:0/4166362680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 msgr2=0x55df9bd59700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.662+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd59700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc88800c6a0 tx=0x7fc8880a9a30 comp rx=0 tx=0).stop 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.662+0000 7fc8a60d7740 1 -- 172.21.15.112:0/4166362680 shutdown_connections 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.662+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df9b962cd0 0x55df9bd52c20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.662+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b9a4350 0x55df9bd650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.662+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd59700 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.662+0000 7fc8817fa640 0 client.26465 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:09.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.663+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/4166362680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.663+0000 7fc8a60d7740 1 -- 172.21.15.112:0/4166362680 >> 172.21.15.112:0/4166362680 conn(0x55df9b998740 msgr2=0x55df9bd47850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:09.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.663+0000 7fc8a60d7740 1 -- 172.21.15.112:0/4166362680 shutdown_connections 2024-09-21T03:37:09.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.663+0000 7fc8a60d7740 1 -- 172.21.15.112:0/4166362680 wait complete. 2024-09-21T03:37:09.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.663+0000 7fc8a60d7740 20 client.26465 trim_cache size 0 max 16384 2024-09-21T03:37:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.682+0000 7fc8a60d7740 1 Processor -- start 2024-09-21T03:37:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc8a60d7740 1 -- start start 2024-09-21T03:37:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd547a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd65010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b9a4350 0x55df9bd58850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981afd10 con 0x55df9b97b490 2024-09-21T03:37:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981ad5e0 con 0x55df9b983670 2024-09-21T03:37:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981ab450 con 0x55df9b9a4350 2024-09-21T03:37:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc89d8ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc89d8ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49390/0 (socket says 172.21.15.112:49390) 2024-09-21T03:37:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc89d8ff640 1 -- 172.21.15.112:0/339129638 learned_addr learned my addr 172.21.15.112:0/339129638 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc89e100640 1 --2- 172.21.15.112:0/339129638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b9a4350 0x55df9bd58850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc89d0fe640 1 --2- 172.21.15.112:0/339129638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd65010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc89d8ff640 1 -- 172.21.15.112:0/339129638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b9a4350 msgr2=0x55df9bd58850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc89d8ff640 1 --2- 172.21.15.112:0/339129638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b9a4350 0x55df9bd58850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc89d8ff640 1 -- 172.21.15.112:0/339129638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 msgr2=0x55df9bd65010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc89d8ff640 1 --2- 172.21.15.112:0/339129638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd65010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.683+0000 7fc89d8ff640 1 -- 172.21.15.112:0/339129638 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df9bd53c00 con 0x55df9b97b490 2024-09-21T03:37:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.684+0000 7fc89d8ff640 1 --2- 172.21.15.112:0/339129638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd547a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55df9bd53310 tx=0x7fc894005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc170bcca9c01eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.684+0000 7fc8837fe640 1 -- 172.21.15.112:0/339129638 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89400fca0 con 0x55df9b97b490 2024-09-21T03:37:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.684+0000 7fc8837fe640 1 -- 172.21.15.112:0/339129638 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89400fe40 con 0x55df9b97b490 2024-09-21T03:37:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.684+0000 7fc8837fe640 1 -- 172.21.15.112:0/339129638 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894004a00 con 0x55df9b97b490 2024-09-21T03:37:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.685+0000 7fc8a60d7740 1 -- 172.21.15.112:0/339129638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 msgr2=0x55df9bd547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.685+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/339129638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd547a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55df9bd53310 tx=0x7fc894005f90 comp rx=0 tx=0).stop 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.685+0000 7fc8a60d7740 1 -- 172.21.15.112:0/339129638 shutdown_connections 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.685+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/339129638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b9a4350 0x55df9bd58850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.685+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/339129638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b983670 0x55df9bd65010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.685+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/339129638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b97b490 0x55df9bd547a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.685+0000 7fc8a60d7740 1 -- 172.21.15.112:0/339129638 >> 172.21.15.112:0/339129638 conn(0x55df9b998740 msgr2=0x55df9bd5a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.685+0000 7fc8a60d7740 1 -- 172.21.15.112:0/339129638 shutdown_connections 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.685+0000 7fc8a60d7740 1 -- 172.21.15.112:0/339129638 wait complete. 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.685+0000 7fc8a60d7740 1 -- start start 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.685+0000 7fc8a60d7740 1 Processor -- start 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b97b490 0x55df9bd6d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b983670 0x55df9bd6a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc8a60d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 0x55df9bd6adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981abaa0 con 0x55df9b9a4350 2024-09-21T03:37:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981acf10 con 0x55df9b97b490 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc8a60d7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc8981ac170 con 0x55df9b983670 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc8a60d7740 10 client.0 _ll_register_callbacks cb 0x7fc8880046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89d0fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b983670 0x55df9bd6a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89e100640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 0x55df9bd6adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89d8ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b97b490 0x55df9bd6d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89d8ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b97b490 0x55df9bd6d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43774/0 (socket says 172.21.15.112:43774) 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89d0fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b983670 0x55df9bd6a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:58000/0 (socket says 172.21.15.112:58000) 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89e100640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 0x55df9bd6adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49400/0 (socket says 172.21.15.112:49400) 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89d0fe640 1 -- 172.21.15.112:0/3454975585 learned_addr learned my addr 172.21.15.112:0/3454975585 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89d0fe640 1 -- 172.21.15.112:0/3454975585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b97b490 msgr2=0x55df9bd6d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89d0fe640 1 --2- 172.21.15.112:0/3454975585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b97b490 0x55df9bd6d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89d0fe640 1 -- 172.21.15.112:0/3454975585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 msgr2=0x55df9bd6adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89d0fe640 1 --2- 172.21.15.112:0/3454975585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 0x55df9bd6adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89d0fe640 1 -- 172.21.15.112:0/3454975585 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df9bd6b2f0 con 0x55df9b983670 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc89d0fe640 1 --2- 172.21.15.112:0/3454975585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b983670 0x55df9bd6a870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc8880a9250 tx=0x7fc888005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce49784d39cf9158 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.686+0000 7fc8817fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8817fa640 1 -- 172.21.15.112:0/3454975585 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88801c040 con 0x55df9b983670 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8817fa640 1 -- 172.21.15.112:0/3454975585 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7fc8880e7d50 con 0x55df9b983670 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8a60d7740 1 -- 172.21.15.112:0/3454975585 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc89400fe40 con 0x55df9b983670 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8817fa640 1 -- 172.21.15.112:0/3454975585 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc8880accb0 con 0x55df9b983670 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8a60d7740 10 client.26473 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8a60d7740 1 -- 172.21.15.112:0/3454975585 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df9bdd3e60 con 0x55df9b983670 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8817fa640 1 -- 172.21.15.112:0/3454975585 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89400fe40 con 0x55df9b983670 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8817fa640 1 -- 172.21.15.112:0/3454975585 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc88806ba90 con 0x55df9b983670 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8a60d7740 10 client.26473 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8a60d7740 10 client.26473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8a60d7740 20 client.26473 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/3454975585 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df9b938080 0x55df9bdd65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8a60d7740 4 client.26473 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:37:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc8a60d7740 1 -- 172.21.15.112:0/3454975585 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55df9bcbf860 con 0x55df9b938080 2024-09-21T03:37:09.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.687+0000 7fc89d8ff640 1 --2- 172.21.15.112:0/3454975585 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df9b938080 0x55df9bdd65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.688+0000 7fc8817fa640 10 client.26473 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.688+0000 7fc89d8ff640 1 --2- 172.21.15.112:0/3454975585 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df9b938080 0x55df9bdd65d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c00abfc4834051c7 server_cookie=9bfddd56ffa7ebb3 in_seq=0 out_seq=0 2024-09-21T03:37:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8817fa640 1 -- 172.21.15.112:0/3454975585 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55df9bcbf860 con 0x55df9b938080 2024-09-21T03:37:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8817fa640 10 client.26473 handle_command_reply: tid=0 2024-09-21T03:37:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 1 client.26473 shutdown 2024-09-21T03:37:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 1 -- 172.21.15.112:0/3454975585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b983670 msgr2=0x55df9bd6a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/3454975585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b983670 0x55df9bd6a870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc8880a9250 tx=0x7fc888005b20 comp rx=0 tx=0).stop 2024-09-21T03:37:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 1 -- 172.21.15.112:0/3454975585 shutdown_connections 2024-09-21T03:37:09.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/3454975585 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df9b938080 0x55df9bdd65d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/3454975585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df9b9a4350 0x55df9bd6adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/3454975585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df9b983670 0x55df9bd6a870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8817fa640 0 client.26473 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 1 --2- 172.21.15.112:0/3454975585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df9b97b490 0x55df9bd6d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 1 -- 172.21.15.112:0/3454975585 >> 172.21.15.112:0/3454975585 conn(0x55df9b998740 msgr2=0x55df9bd4cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:09.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 1 -- 172.21.15.112:0/3454975585 shutdown_connections 2024-09-21T03:37:09.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 1 -- 172.21.15.112:0/3454975585 wait complete. 2024-09-21T03:37:09.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.692+0000 7fc8a60d7740 20 client.26473 trim_cache size 0 max 16384 2024-09-21T03:37:09.699 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:37:09.699 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:37:09.699 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "575b64ed-beba-489e-b2ec-f4aac3fe0d1c", 2024-09-21T03:37:09.699 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:37:09.699 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:37:09.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 -- 172.21.15.112:0/868121867 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc8840b6680 msgr2=0x7fc8840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc8840b6680 0x7fc8840b8b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc89004c000 tx=0x7fc89004d000 comp rx=0 tx=0).stop 2024-09-21T03:37:09.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 -- 172.21.15.112:0/868121867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8981083d0 msgr2=0x7fc8981998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:09.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8981083d0 0x7fc8981998b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc89806ab20 tx=0x7fc88800ee50 comp rx=0 tx=0).stop 2024-09-21T03:37:09.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 -- 172.21.15.112:0/868121867 shutdown_connections 2024-09-21T03:37:09.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc8840b6680 0x7fc8840b8b90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc898108d20 0x7fc89819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8981083d0 0x7fc8981998b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 --2- 172.21.15.112:0/868121867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc898102ae0 0x7fc898199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:09.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 -- 172.21.15.112:0/868121867 >> 172.21.15.112:0/868121867 conn(0x7fc898079790 msgr2=0x7fc8981063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:09.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 -- 172.21.15.112:0/868121867 shutdown_connections 2024-09-21T03:37:09.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:09.693+0000 7fc89fb63640 1 -- 172.21.15.112:0/868121867 wait complete. 2024-09-21T03:37:10.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:09 smithi203 ceph-mon[34723]: pgmap v567: 97 pgs: 97 active+clean; 634 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-09-21T03:37:10.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:09 smithi187 ceph-mon[34603]: pgmap v567: 97 pgs: 97 active+clean; 634 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-09-21T03:37:10.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:09 smithi112 ceph-mon[31442]: pgmap v567: 97 pgs: 97 active+clean; 634 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-09-21T03:37:10.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 859648 10.68 MB/sec execute 310 sec latency 39.804 ms 2024-09-21T03:37:10.561 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:37:10.824 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:11.139 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:10 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:37:11.139 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:10 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.139 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:10 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:37:11.139 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:10 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:37:11.139 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:10 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:37:11.139 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:10 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.139 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:10 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.139 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:10 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:10 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:37:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:10 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:10 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:37:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:10 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:37:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:10 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:37:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:10 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:10 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:10 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:10 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:37:11.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:10 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:10 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:37:11.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:10 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:37:11.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:10 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:37:11.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:10 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:10 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:10 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:11.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 861587 10.67 MB/sec execute 311 sec latency 59.204 ms 2024-09-21T03:37:11.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.363+0000 7fcb5bf77640 1 -- 172.21.15.112:0/990436312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb541083d0 msgr2=0x7fcb541087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.363+0000 7fcb5bf77640 1 --2- 172.21.15.112:0/990436312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb541083d0 0x7fcb541087e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009900 tx=0x7fcb4402a410 comp rx=0 tx=0).stop 2024-09-21T03:37:11.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.364+0000 7fcb5bf77640 1 -- 172.21.15.112:0/990436312 shutdown_connections 2024-09-21T03:37:11.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.364+0000 7fcb5bf77640 1 --2- 172.21.15.112:0/990436312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb54108d20 0x7fcb5410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.364+0000 7fcb5bf77640 1 --2- 172.21.15.112:0/990436312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb541083d0 0x7fcb541087e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.364+0000 7fcb5bf77640 1 --2- 172.21.15.112:0/990436312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb54102ae0 0x7fcb54102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.364+0000 7fcb5bf77640 1 -- 172.21.15.112:0/990436312 >> 172.21.15.112:0/990436312 conn(0x7fcb54079760 msgr2=0x7fcb54079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:11.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.364+0000 7fcb5bf77640 1 -- 172.21.15.112:0/990436312 shutdown_connections 2024-09-21T03:37:11.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.364+0000 7fcb5bf77640 1 -- 172.21.15.112:0/990436312 wait complete. 2024-09-21T03:37:11.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.364+0000 7fcb5bf77640 1 Processor -- start 2024-09-21T03:37:11.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.364+0000 7fcb5bf77640 1 -- start start 2024-09-21T03:37:11.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb5bf77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb54102ae0 0x7fcb5419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb5bf77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb541083d0 0x7fcb5419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb5bf77640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb54108d20 0x7fcb5419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb5bf77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb54112990 con 0x7fcb541083d0 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb5bf77640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcb54112810 con 0x7fcb54102ae0 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb5bf77640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcb54112b10 con 0x7fcb54108d20 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb59512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb541083d0 0x7fcb5419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb59d13640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb54102ae0 0x7fcb5419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb59512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb541083d0 0x7fcb5419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49430/0 (socket says 172.21.15.112:49430) 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb59d13640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb54102ae0 0x7fcb5419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43788/0 (socket says 172.21.15.112:43788) 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb59512640 1 -- 172.21.15.112:0/2332955877 learned_addr learned my addr 172.21.15.112:0/2332955877 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb5a514640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb54108d20 0x7fcb5419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb59512640 1 -- 172.21.15.112:0/2332955877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb54108d20 msgr2=0x7fcb5419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb59512640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb54108d20 0x7fcb5419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb59512640 1 -- 172.21.15.112:0/2332955877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb54102ae0 msgr2=0x7fcb5419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb59512640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb54102ae0 0x7fcb5419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.365+0000 7fcb59512640 1 -- 172.21.15.112:0/2332955877 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb541a0400 con 0x7fcb541083d0 2024-09-21T03:37:11.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.366+0000 7fcb59512640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb541083d0 0x7fcb5419aa30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcb44032fd0 tx=0x7fcb44031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:11.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.366+0000 7fcb4affd640 1 -- 172.21.15.112:0/2332955877 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4403f030 con 0x7fcb541083d0 2024-09-21T03:37:11.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.366+0000 7fcb4affd640 1 -- 172.21.15.112:0/2332955877 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb44004490 con 0x7fcb541083d0 2024-09-21T03:37:11.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.366+0000 7fcb4affd640 1 -- 172.21.15.112:0/2332955877 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44035070 con 0x7fcb541083d0 2024-09-21T03:37:11.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.366+0000 7fcb5bf77640 1 -- 172.21.15.112:0/2332955877 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb541a0690 con 0x7fcb541083d0 2024-09-21T03:37:11.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.366+0000 7fcb5bf77640 1 -- 172.21.15.112:0/2332955877 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb541a6e50 con 0x7fcb541083d0 2024-09-21T03:37:11.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.368+0000 7fcb624eb740 1 Processor -- start 2024-09-21T03:37:11.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb624eb740 1 -- start start 2024-09-21T03:37:11.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb4affd640 1 -- 172.21.15.112:0/2332955877 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fcb44042030 con 0x7fcb541083d0 2024-09-21T03:37:11.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfff070 0x55b31c3a6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 0x55b31c3a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 0x55b31c3a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb624eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b31c31ce50 con 0x55b31bfde390 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb624eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b31c30c280 con 0x55b31bfff070 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb624eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b31c310090 con 0x55b31bfd61b0 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb5a514640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfff070 0x55b31c3a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb5a514640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfff070 0x55b31c3a6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43802/0 (socket says 172.21.15.112:43802) 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb5a514640 1 -- 172.21.15.112:0/712731928 learned_addr learned my addr 172.21.15.112:0/712731928 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb59d13640 1 --2- 172.21.15.112:0/712731928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 0x55b31c3a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb5a514640 1 -- 172.21.15.112:0/712731928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 msgr2=0x55b31c3a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb5a514640 1 --2- 172.21.15.112:0/712731928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 0x55b31c3a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.369+0000 7fcb59512640 1 --2- 172.21.15.112:0/712731928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 0x55b31c3a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.370+0000 7fcb5a514640 1 -- 172.21.15.112:0/712731928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 msgr2=0x55b31c3a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.370+0000 7fcb5a514640 1 --2- 172.21.15.112:0/712731928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 0x55b31c3a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.370+0000 7fcb5a514640 1 -- 172.21.15.112:0/712731928 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31c06a650 con 0x55b31bfff070 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.370+0000 7fcb59512640 1 --2- 172.21.15.112:0/712731928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 0x55b31c3a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:11.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.370+0000 7fcb4affd640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcb400b6700 0x7fcb400b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.370+0000 7fcb59d13640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcb400b6700 0x7fcb400b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.370+0000 7fcb5a514640 1 --2- 172.21.15.112:0/712731928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfff070 0x55b31c3a6270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009fd0 tx=0x7fcb4c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58780da201af382c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.370+0000 7fcb4affd640 1 -- 172.21.15.112:0/2332955877 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7fcb441079f0 con 0x7fcb541083d0 2024-09-21T03:37:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.370+0000 7fcb3ffff640 1 -- 172.21.15.112:0/712731928 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c004d20 con 0x55b31bfff070 2024-09-21T03:37:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.370+0000 7fcb3ffff640 1 -- 172.21.15.112:0/712731928 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb4c004050 con 0x55b31bfff070 2024-09-21T03:37:11.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.370+0000 7fcb3ffff640 1 -- 172.21.15.112:0/712731928 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c005280 con 0x55b31bfff070 2024-09-21T03:37:11.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.371+0000 7fcb624eb740 1 -- 172.21.15.112:0/712731928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfff070 msgr2=0x55b31c3a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.371+0000 7fcb59d13640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcb400b6700 0x7fcb400b8c10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55b31c3a4ef0 tx=0x7fcb50005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:11.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.371+0000 7fcb624eb740 1 --2- 172.21.15.112:0/712731928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfff070 0x55b31c3a6270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009fd0 tx=0x7fcb4c0118d0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.371+0000 7fcb624eb740 1 -- 172.21.15.112:0/712731928 shutdown_connections 2024-09-21T03:37:11.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.371+0000 7fcb624eb740 1 --2- 172.21.15.112:0/712731928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 0x55b31c3a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.371+0000 7fcb624eb740 1 --2- 172.21.15.112:0/712731928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 0x55b31c3a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.371+0000 7fcb624eb740 1 --2- 172.21.15.112:0/712731928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfff070 0x55b31c3a6270 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.371+0000 7fcb624eb740 1 -- 172.21.15.112:0/712731928 >> 172.21.15.112:0/712731928 conn(0x55b31bff3460 msgr2=0x55b31c3a4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.371+0000 7fcb624eb740 1 -- 172.21.15.112:0/712731928 shutdown_connections 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.371+0000 7fcb624eb740 1 -- 172.21.15.112:0/712731928 wait complete. 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.372+0000 7fcb624eb740 1 -- start start 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.372+0000 7fcb624eb740 1 Processor -- start 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.372+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfd61b0 0x55b31c3b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.372+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 0x55b31c3b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.372+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 0x55b31c3c2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.372+0000 7fcb624eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b31c16d700 con 0x55b31bfd61b0 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.372+0000 7fcb624eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b31c2a7110 con 0x55b31bfde390 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.372+0000 7fcb624eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b31c3b8e60 con 0x55b31bfff070 2024-09-21T03:37:11.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb59d13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfd61b0 0x55b31c3b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb59d13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfd61b0 0x55b31c3b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49438/0 (socket says 172.21.15.112:49438) 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb59d13640 1 -- 172.21.15.112:0/3945502514 learned_addr learned my addr 172.21.15.112:0/3945502514 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb624eb740 10 client.0 _ll_register_callbacks cb 0x55b31c2c0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb5a514640 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 0x55b31c3c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb59512640 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 0x55b31c3b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb5a514640 1 -- 172.21.15.112:0/3945502514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 msgr2=0x55b31c3b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb5a514640 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 0x55b31c3b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb5a514640 1 -- 172.21.15.112:0/3945502514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfd61b0 msgr2=0x55b31c3b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb5a514640 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfd61b0 0x55b31c3b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb5a514640 1 -- 172.21.15.112:0/3945502514 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31c2d4c40 con 0x55b31bfff070 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb59d13640 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfd61b0 0x55b31c3b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb5a514640 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 0x55b31c3c2180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c011dd0 tx=0x7fcb4c004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae9f4fedfff6a51b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.373+0000 7fcb3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:37:11.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/3945502514 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c028020 con 0x55b31bfff070 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/3945502514 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7fcb4c004a40 con 0x55b31bfff070 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb624eb740 1 -- 172.21.15.112:0/3945502514 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb4c004050 con 0x55b31bfff070 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/3945502514 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb4c033050 con 0x55b31bfff070 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb624eb740 10 client.26485 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb624eb740 1 -- 172.21.15.112:0/3945502514 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b31c3c4b40 con 0x55b31bfff070 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/3945502514 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb4c004050 con 0x55b31bfff070 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/3945502514 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcb4c00f4e0 con 0x55b31bfff070 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb624eb740 10 client.26485 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb624eb740 10 client.26485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb624eb740 20 client.26485 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb624eb740 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b31bfbd9f0 0x55b31c3c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb624eb740 4 client.26485 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:11.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.374+0000 7fcb624eb740 1 -- 172.21.15.112:0/3945502514 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b31c31eda0 con 0x55b31bfbd9f0 2024-09-21T03:37:11.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.375+0000 7fcb59d13640 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b31bfbd9f0 0x55b31c3c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.375+0000 7fcb3d7fa640 10 client.26485 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:11.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.375+0000 7fcb59d13640 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b31bfbd9f0 0x55b31c3c7fb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed45203cf5884e7 server_cookie=ac792d4ddd38c0a7 in_seq=0 out_seq=0 2024-09-21T03:37:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.376+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/3945502514 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b31c31eda0 con 0x55b31bfbd9f0 2024-09-21T03:37:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.376+0000 7fcb3d7fa640 10 client.26485 handle_command_reply: tid=0 2024-09-21T03:37:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.376+0000 7fcb624eb740 1 client.26485 shutdown 2024-09-21T03:37:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.376+0000 7fcb624eb740 1 -- 172.21.15.112:0/3945502514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 msgr2=0x55b31c3c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.376+0000 7fcb624eb740 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 0x55b31c3c2180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c011dd0 tx=0x7fcb4c004200 comp rx=0 tx=0).stop 2024-09-21T03:37:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.377+0000 7fcb624eb740 1 -- 172.21.15.112:0/3945502514 shutdown_connections 2024-09-21T03:37:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.377+0000 7fcb624eb740 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b31bfbd9f0 0x55b31c3c7fb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.377+0000 7fcb624eb740 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 0x55b31c3c2180 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.377+0000 7fcb624eb740 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 0x55b31c3b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.377+0000 7fcb624eb740 1 --2- 172.21.15.112:0/3945502514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfd61b0 0x55b31c3b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.377+0000 7fcb3d7fa640 0 client.26485 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.377+0000 7fcb624eb740 1 -- 172.21.15.112:0/3945502514 >> 172.21.15.112:0/3945502514 conn(0x55b31bff3460 msgr2=0x55b31c3a9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.377+0000 7fcb624eb740 1 -- 172.21.15.112:0/3945502514 shutdown_connections 2024-09-21T03:37:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.377+0000 7fcb624eb740 1 -- 172.21.15.112:0/3945502514 wait complete. 2024-09-21T03:37:11.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.377+0000 7fcb624eb740 20 client.26485 trim_cache size 0 max 16384 2024-09-21T03:37:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.396+0000 7fcb624eb740 1 Processor -- start 2024-09-21T03:37:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.396+0000 7fcb624eb740 1 -- start start 2024-09-21T03:37:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfd61b0 0x55b31c3b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 0x55b31c3a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 0x55b31c3c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb624eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb541afd70 con 0x55b31bfde390 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb624eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcb541ad640 con 0x55b31bfd61b0 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb624eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcb541ab4b0 con 0x55b31bfff070 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb59512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 0x55b31c3a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb59512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 0x55b31c3a79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49452/0 (socket says 172.21.15.112:49452) 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb59512640 1 -- 172.21.15.112:0/2148825943 learned_addr learned my addr 172.21.15.112:0/2148825943 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb5a514640 1 --2- 172.21.15.112:0/2148825943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 0x55b31c3c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb59d13640 1 --2- 172.21.15.112:0/2148825943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfd61b0 0x55b31c3b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb59512640 1 -- 172.21.15.112:0/2148825943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 msgr2=0x55b31c3c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb59512640 1 --2- 172.21.15.112:0/2148825943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 0x55b31c3c2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb59512640 1 -- 172.21.15.112:0/2148825943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfd61b0 msgr2=0x55b31c3b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb59512640 1 --2- 172.21.15.112:0/2148825943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfd61b0 0x55b31c3b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb59512640 1 -- 172.21.15.112:0/2148825943 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31c3a90d0 con 0x55b31bfde390 2024-09-21T03:37:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb5a514640 1 --2- 172.21.15.112:0/2148825943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 0x55b31c3c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.397+0000 7fcb59512640 1 --2- 172.21.15.112:0/2148825943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 0x55b31c3a79b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcb44105740 tx=0x7fcb4402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c311cbb1fe645009 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.398+0000 7fcb3f7fe640 1 -- 172.21.15.112:0/2148825943 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44049040 con 0x55b31bfde390 2024-09-21T03:37:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.398+0000 7fcb3f7fe640 1 -- 172.21.15.112:0/2148825943 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb440cd400 con 0x55b31bfde390 2024-09-21T03:37:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.398+0000 7fcb3f7fe640 1 -- 172.21.15.112:0/2148825943 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb440d7030 con 0x55b31bfde390 2024-09-21T03:37:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.398+0000 7fcb624eb740 1 -- 172.21.15.112:0/2148825943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 msgr2=0x55b31c3a79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.399+0000 7fcb624eb740 1 --2- 172.21.15.112:0/2148825943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 0x55b31c3a79b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcb44105740 tx=0x7fcb4402a410 comp rx=0 tx=0).stop 2024-09-21T03:37:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.399+0000 7fcb624eb740 1 -- 172.21.15.112:0/2148825943 shutdown_connections 2024-09-21T03:37:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.399+0000 7fcb624eb740 1 --2- 172.21.15.112:0/2148825943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfff070 0x55b31c3c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.399+0000 7fcb624eb740 1 --2- 172.21.15.112:0/2148825943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfde390 0x55b31c3a79b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.399+0000 7fcb624eb740 1 --2- 172.21.15.112:0/2148825943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfd61b0 0x55b31c3b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.399+0000 7fcb624eb740 1 -- 172.21.15.112:0/2148825943 >> 172.21.15.112:0/2148825943 conn(0x55b31bff3460 msgr2=0x55b31c3a72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.399+0000 7fcb624eb740 1 -- 172.21.15.112:0/2148825943 shutdown_connections 2024-09-21T03:37:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.399+0000 7fcb624eb740 1 -- 172.21.15.112:0/2148825943 wait complete. 2024-09-21T03:37:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.399+0000 7fcb624eb740 1 -- start start 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.399+0000 7fcb624eb740 1 Processor -- start 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 0x55b31c3ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 0x55b31c3ba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb624eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfff070 0x55b31c3babc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb624eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb541abb00 con 0x55b31bfff070 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb624eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcb541acf70 con 0x55b31bfde390 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb624eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcb541ac1d0 con 0x55b31bfd61b0 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb5a514640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfff070 0x55b31c3babc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59512640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 0x55b31c3ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59d13640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 0x55b31c3ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59512640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 0x55b31c3ba680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43830/0 (socket says 172.21.15.112:43830) 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59d13640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 0x55b31c3ba140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:58042/0 (socket says 172.21.15.112:58042) 2024-09-21T03:37:11.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59512640 1 -- 172.21.15.112:0/1529703831 learned_addr learned my addr 172.21.15.112:0/1529703831 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb624eb740 10 client.0 _ll_register_callbacks cb 0x7fcb500097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59d13640 1 -- 172.21.15.112:0/1529703831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 msgr2=0x55b31c3ba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59d13640 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 0x55b31c3ba680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59d13640 1 -- 172.21.15.112:0/1529703831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfff070 msgr2=0x55b31c3babc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59d13640 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfff070 0x55b31c3babc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59d13640 1 -- 172.21.15.112:0/1529703831 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31c431830 con 0x55b31bfd61b0 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59512640 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 0x55b31c3ba680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb59d13640 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 0x55b31c3ba140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55b31c16d850 tx=0x7fcb500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=200d67b01fefad0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.400+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/1529703831 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50003e60 con 0x55b31bfd61b0 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/1529703831 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8514+0+0 (secure 0 0 0) 0x7fcb50018030 con 0x55b31bfd61b0 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb624eb740 1 -- 172.21.15.112:0/1529703831 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb440cd400 con 0x55b31bfd61b0 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/1529703831 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb50005020 con 0x55b31bfd61b0 2024-09-21T03:37:11.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb624eb740 10 client.26491 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb624eb740 1 -- 172.21.15.112:0/1529703831 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b31c432b20 con 0x55b31bfd61b0 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/1529703831 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb440cd400 con 0x55b31bfd61b0 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/1529703831 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcb50031450 con 0x55b31bfd61b0 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb624eb740 10 client.26491 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb624eb740 10 client.26491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb624eb740 20 client.26491 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb624eb740 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b31bf92da0 0x55b31c4352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb624eb740 4 client.26491 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb624eb740 1 -- 172.21.15.112:0/1529703831 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b31c31a850 con 0x55b31bf92da0 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.401+0000 7fcb59512640 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b31bf92da0 0x55b31c4352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.402+0000 7fcb3d7fa640 10 client.26491 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:11.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.402+0000 7fcb59512640 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b31bf92da0 0x55b31c4352e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26153c87e02158d server_cookie=c51ae2a03bc0bceb in_seq=0 out_seq=0 2024-09-21T03:37:11.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb3d7fa640 1 -- 172.21.15.112:0/1529703831 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x55b31c31a850 con 0x55b31bf92da0 2024-09-21T03:37:11.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb3d7fa640 10 client.26491 handle_command_reply: tid=0 2024-09-21T03:37:11.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 1 client.26491 shutdown 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 1 -- 172.21.15.112:0/1529703831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 msgr2=0x55b31c3ba140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 0x55b31c3ba140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55b31c16d850 tx=0x7fcb500046c0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 1 -- 172.21.15.112:0/1529703831 shutdown_connections 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b31bf92da0 0x55b31c4352e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b31bfff070 0x55b31c3babc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b31bfde390 0x55b31c3ba680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 1 --2- 172.21.15.112:0/1529703831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b31bfd61b0 0x55b31c3ba140 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb3d7fa640 0 client.26491 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 1 -- 172.21.15.112:0/1529703831 >> 172.21.15.112:0/1529703831 conn(0x55b31bff3460 msgr2=0x55b31c3b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 1 -- 172.21.15.112:0/1529703831 shutdown_connections 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 1 -- 172.21.15.112:0/1529703831 wait complete. 2024-09-21T03:37:11.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.404+0000 7fcb624eb740 20 client.26491 trim_cache size 0 max 16384 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "575b64ed-beba-489e-b2ec-f4aac3fe0d1c", 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:37:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:37:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:37:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:37:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:37:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:37:11.420 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:37:11.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 -- 172.21.15.112:0/2332955877 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcb400b6700 msgr2=0x7fcb400b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcb400b6700 0x7fcb400b8c10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55b31c3a4ef0 tx=0x7fcb50005f90 comp rx=0 tx=0).stop 2024-09-21T03:37:11.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 -- 172.21.15.112:0/2332955877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb541083d0 msgr2=0x7fcb5419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:11.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb541083d0 0x7fcb5419aa30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcb44032fd0 tx=0x7fcb44031a20 comp rx=0 tx=0).stop 2024-09-21T03:37:11.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 -- 172.21.15.112:0/2332955877 shutdown_connections 2024-09-21T03:37:11.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcb400b6700 0x7fcb400b8c10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb54108d20 0x7fcb5419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb541083d0 0x7fcb5419aa30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 --2- 172.21.15.112:0/2332955877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb54102ae0 0x7fcb5419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:11.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 -- 172.21.15.112:0/2332955877 >> 172.21.15.112:0/2332955877 conn(0x7fcb54079760 msgr2=0x7fcb5406be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:11.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 -- 172.21.15.112:0/2332955877 shutdown_connections 2024-09-21T03:37:11.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:11.405+0000 7fcb5bf77640 1 -- 172.21.15.112:0/2332955877 wait complete. 2024-09-21T03:37:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:11 smithi203 ceph-mon[34723]: pgmap v568: 97 pgs: 97 active+clean; 636 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-21T03:37:12.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:11 smithi187 ceph-mon[34603]: pgmap v568: 97 pgs: 97 active+clean; 636 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-21T03:37:12.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:11 smithi112 ceph-mon[31442]: pgmap v568: 97 pgs: 97 active+clean; 636 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-21T03:37:12.220 INFO:tasks.cephfs.filesystem:scrub status for tag:575b64ed-beba-489e-b2ec-f4aac3fe0d1c - {'path': '/', 'tag': '575b64ed-beba-489e-b2ec-f4aac3fe0d1c', 'options': 'recursive,force'} 2024-09-21T03:37:12.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 863337 10.67 MB/sec execute 312 sec latency 17.494 ms 2024-09-21T03:37:13.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 865754 10.68 MB/sec execute 313 sec latency 17.679 ms 2024-09-21T03:37:14.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:13 smithi203 ceph-mon[34723]: pgmap v569: 97 pgs: 97 active+clean; 641 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 334 op/s 2024-09-21T03:37:14.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:14.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:14.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:14.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:37:14.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:13 smithi187 ceph-mon[34603]: pgmap v569: 97 pgs: 97 active+clean; 641 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 334 op/s 2024-09-21T03:37:14.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:14.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:14.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:14.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:37:14.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 867285 10.67 MB/sec execute 314 sec latency 12.519 ms 2024-09-21T03:37:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:13 smithi112 ceph-mon[31442]: pgmap v569: 97 pgs: 97 active+clean; 641 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 334 op/s 2024-09-21T03:37:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:37:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:37:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:37:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:37:15.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 869986 10.69 MB/sec execute 315 sec latency 16.220 ms 2024-09-21T03:37:16.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:15 smithi203 ceph-mon[34723]: pgmap v570: 97 pgs: 97 active+clean; 644 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 310 op/s 2024-09-21T03:37:16.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:15 smithi203 ceph-mon[34723]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:37:16.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:15 smithi187 ceph-mon[34603]: pgmap v570: 97 pgs: 97 active+clean; 644 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 310 op/s 2024-09-21T03:37:16.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:15 smithi187 ceph-mon[34603]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:37:16.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 871288 10.67 MB/sec execute 316 sec latency 54.620 ms 2024-09-21T03:37:16.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:15 smithi112 ceph-mon[31442]: pgmap v570: 97 pgs: 97 active+clean; 644 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 310 op/s 2024-09-21T03:37:16.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:15 smithi112 ceph-mon[31442]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:37:17.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 873665 10.68 MB/sec execute 317 sec latency 17.747 ms 2024-09-21T03:37:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:17 smithi203 ceph-mon[34723]: pgmap v572: 97 pgs: 97 active+clean; 648 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-09-21T03:37:18.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:17 smithi187 ceph-mon[34603]: pgmap v572: 97 pgs: 97 active+clean; 648 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-09-21T03:37:18.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 875088 10.67 MB/sec execute 318 sec latency 20.533 ms 2024-09-21T03:37:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:17 smithi112 ceph-mon[31442]: pgmap v572: 97 pgs: 97 active+clean; 648 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-09-21T03:37:19.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 877370 10.68 MB/sec execute 319 sec latency 12.205 ms 2024-09-21T03:37:20.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:19 smithi203 ceph-mon[34723]: pgmap v573: 97 pgs: 97 active+clean; 648 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 253 op/s 2024-09-21T03:37:20.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:19 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:37:20.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:19 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:37:20.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:19 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:37:20.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:19 smithi187 ceph-mon[34603]: pgmap v573: 97 pgs: 97 active+clean; 648 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 253 op/s 2024-09-21T03:37:20.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:19 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:37:20.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:19 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:37:20.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:19 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:37:20.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 879405 10.67 MB/sec execute 320 sec latency 16.205 ms 2024-09-21T03:37:20.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:19 smithi112 ceph-mon[31442]: pgmap v573: 97 pgs: 97 active+clean; 648 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 253 op/s 2024-09-21T03:37:20.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:19 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:37:20.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:19 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:37:20.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:19 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:37:21.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 881005 10.67 MB/sec execute 321 sec latency 45.691 ms 2024-09-21T03:37:22.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:21 smithi203 ceph-mon[34723]: pgmap v574: 97 pgs: 97 active+clean; 653 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 246 op/s 2024-09-21T03:37:22.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:21 smithi187 ceph-mon[34603]: pgmap v574: 97 pgs: 97 active+clean; 653 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 246 op/s 2024-09-21T03:37:22.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 883058 10.66 MB/sec execute 322 sec latency 21.301 ms 2024-09-21T03:37:22.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:21 smithi112 ceph-mon[31442]: pgmap v574: 97 pgs: 97 active+clean; 653 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 246 op/s 2024-09-21T03:37:23.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 884637 10.66 MB/sec execute 323 sec latency 28.760 ms 2024-09-21T03:37:24.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:23 smithi203 ceph-mon[34723]: pgmap v575: 97 pgs: 97 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 263 op/s 2024-09-21T03:37:24.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:23 smithi187 ceph-mon[34603]: pgmap v575: 97 pgs: 97 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 263 op/s 2024-09-21T03:37:24.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 887185 10.67 MB/sec execute 324 sec latency 16.302 ms 2024-09-21T03:37:24.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:23 smithi112 ceph-mon[31442]: pgmap v575: 97 pgs: 97 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 263 op/s 2024-09-21T03:37:25.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 888659 10.66 MB/sec execute 325 sec latency 13.443 ms 2024-09-21T03:37:26.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:25 smithi203 ceph-mon[34723]: pgmap v576: 97 pgs: 97 active+clean; 662 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 262 op/s 2024-09-21T03:37:26.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:25 smithi187 ceph-mon[34603]: pgmap v576: 97 pgs: 97 active+clean; 662 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 262 op/s 2024-09-21T03:37:26.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 891217 10.68 MB/sec execute 326 sec latency 33.781 ms 2024-09-21T03:37:26.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:25 smithi112 ceph-mon[31442]: pgmap v576: 97 pgs: 97 active+clean; 662 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 262 op/s 2024-09-21T03:37:27.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 892562 10.66 MB/sec execute 327 sec latency 19.674 ms 2024-09-21T03:37:28.089 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:27 smithi112 ceph-mon[31442]: pgmap v577: 97 pgs: 97 active+clean; 668 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 309 op/s 2024-09-21T03:37:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:27 smithi203 ceph-mon[34723]: pgmap v577: 97 pgs: 97 active+clean; 668 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 309 op/s 2024-09-21T03:37:28.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:27 smithi187 ceph-mon[34603]: pgmap v577: 97 pgs: 97 active+clean; 668 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 309 op/s 2024-09-21T03:37:28.264 INFO:tasks.workunit.client.0.smithi112.stdout: 1 894882 10.67 MB/sec execute 328 sec latency 16.284 ms 2024-09-21T03:37:29.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 896176 10.66 MB/sec execute 329 sec latency 24.523 ms 2024-09-21T03:37:30.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:29 smithi203 ceph-mon[34723]: pgmap v578: 97 pgs: 97 active+clean; 668 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 234 op/s 2024-09-21T03:37:30.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:29 smithi187 ceph-mon[34603]: pgmap v578: 97 pgs: 97 active+clean; 668 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 234 op/s 2024-09-21T03:37:30.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 898495 10.67 MB/sec execute 330 sec latency 15.496 ms 2024-09-21T03:37:30.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:29 smithi112 ceph-mon[31442]: pgmap v578: 97 pgs: 97 active+clean; 668 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 234 op/s 2024-09-21T03:37:31.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 899905 10.65 MB/sec execute 331 sec latency 26.428 ms 2024-09-21T03:37:32.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:31 smithi203 ceph-mon[34723]: pgmap v579: 97 pgs: 97 active+clean; 675 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 307 op/s 2024-09-21T03:37:32.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:31 smithi187 ceph-mon[34603]: pgmap v579: 97 pgs: 97 active+clean; 675 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 307 op/s 2024-09-21T03:37:32.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 902136 10.66 MB/sec execute 332 sec latency 17.508 ms 2024-09-21T03:37:32.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:31 smithi112 ceph-mon[31442]: pgmap v579: 97 pgs: 97 active+clean; 675 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 307 op/s 2024-09-21T03:37:33.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 904196 10.65 MB/sec execute 333 sec latency 15.377 ms 2024-09-21T03:37:34.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:33 smithi203 ceph-mon[34723]: pgmap v580: 97 pgs: 97 active+clean; 678 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 289 op/s 2024-09-21T03:37:34.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:33 smithi187 ceph-mon[34603]: pgmap v580: 97 pgs: 97 active+clean; 678 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 289 op/s 2024-09-21T03:37:34.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 905848 10.65 MB/sec execute 334 sec latency 15.302 ms 2024-09-21T03:37:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:33 smithi112 ceph-mon[31442]: pgmap v580: 97 pgs: 97 active+clean; 678 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 289 op/s 2024-09-21T03:37:35.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 908059 10.65 MB/sec execute 335 sec latency 17.628 ms 2024-09-21T03:37:36.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:35 smithi203 ceph-mon[34723]: pgmap v581: 97 pgs: 97 active+clean; 678 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-09-21T03:37:36.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:35 smithi187 ceph-mon[34603]: pgmap v581: 97 pgs: 97 active+clean; 678 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-09-21T03:37:36.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 909395 10.65 MB/sec execute 336 sec latency 56.767 ms 2024-09-21T03:37:36.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:35 smithi112 ceph-mon[31442]: pgmap v581: 97 pgs: 97 active+clean; 678 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-09-21T03:37:37.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 911612 10.65 MB/sec execute 337 sec latency 20.349 ms 2024-09-21T03:37:38.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:37 smithi203 ceph-mon[34723]: pgmap v582: 97 pgs: 97 active+clean; 685 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 305 op/s 2024-09-21T03:37:38.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:37 smithi187 ceph-mon[34603]: pgmap v582: 97 pgs: 97 active+clean; 685 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 305 op/s 2024-09-21T03:37:38.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 913172 10.64 MB/sec execute 338 sec latency 48.898 ms 2024-09-21T03:37:38.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:37 smithi112 ceph-mon[31442]: pgmap v582: 97 pgs: 97 active+clean; 685 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 305 op/s 2024-09-21T03:37:39.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 915999 10.66 MB/sec execute 339 sec latency 16.072 ms 2024-09-21T03:37:40.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:39 smithi203 ceph-mon[34723]: pgmap v583: 97 pgs: 97 active+clean; 685 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 224 op/s 2024-09-21T03:37:40.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:39 smithi187 ceph-mon[34603]: pgmap v583: 97 pgs: 97 active+clean; 685 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 224 op/s 2024-09-21T03:37:40.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 916895 10.66 MB/sec execute 340 sec latency 66.105 ms 2024-09-21T03:37:40.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:39 smithi112 ceph-mon[31442]: pgmap v583: 97 pgs: 97 active+clean; 685 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 224 op/s 2024-09-21T03:37:41.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 918368 10.68 MB/sec execute 341 sec latency 41.447 ms 2024-09-21T03:37:42.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:41 smithi203 ceph-mon[34723]: pgmap v584: 97 pgs: 97 active+clean; 693 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 310 op/s 2024-09-21T03:37:42.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:41 smithi187 ceph-mon[34603]: pgmap v584: 97 pgs: 97 active+clean; 693 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 310 op/s 2024-09-21T03:37:42.220 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:37:42.244 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:41 smithi112 ceph-mon[31442]: pgmap v584: 97 pgs: 97 active+clean; 693 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 310 op/s 2024-09-21T03:37:42.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 920523 10.69 MB/sec execute 342 sec latency 27.051 ms 2024-09-21T03:37:42.479 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:43.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.083+0000 7fe767390640 1 -- 172.21.15.112:0/2566828875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7601094a0 msgr2=0x7fe7601098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.083+0000 7fe767390640 1 --2- 172.21.15.112:0/2566828875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7601094a0 0x7fe7601098b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe750009900 tx=0x7fe75002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:43.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.083+0000 7fe767390640 1 -- 172.21.15.112:0/2566828875 shutdown_connections 2024-09-21T03:37:43.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.083+0000 7fe767390640 1 --2- 172.21.15.112:0/2566828875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe760109df0 0x7fe760110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.083+0000 7fe767390640 1 --2- 172.21.15.112:0/2566828875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7601094a0 0x7fe7601098b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.083+0000 7fe767390640 1 --2- 172.21.15.112:0/2566828875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe760103b60 0x7fe760103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.083+0000 7fe767390640 1 -- 172.21.15.112:0/2566828875 >> 172.21.15.112:0/2566828875 conn(0x7fe76007b850 msgr2=0x7fe76007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.083+0000 7fe767390640 1 -- 172.21.15.112:0/2566828875 shutdown_connections 2024-09-21T03:37:43.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.083+0000 7fe767390640 1 -- 172.21.15.112:0/2566828875 wait complete. 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.083+0000 7fe767390640 1 Processor -- start 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe767390640 1 -- start start 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe767390640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe760103b60 0x7fe76019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe767390640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7601094a0 0x7fe76019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe767390640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe760109df0 0x7fe76019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe767390640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe760113ab0 con 0x7fe760109df0 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe767390640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe760113930 con 0x7fe7601094a0 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe767390640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe760113c30 con 0x7fe760103b60 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe76592d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe760109df0 0x7fe76019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe76592d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe760109df0 0x7fe76019fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37202/0 (socket says 172.21.15.112:37202) 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe76592d640 1 -- 172.21.15.112:0/396844056 learned_addr learned my addr 172.21.15.112:0/396844056 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe76512c640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe760103b60 0x7fe76019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe76492b640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7601094a0 0x7fe76019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe76592d640 1 -- 172.21.15.112:0/396844056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe760103b60 msgr2=0x7fe76019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe76592d640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe760103b60 0x7fe76019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe76592d640 1 -- 172.21.15.112:0/396844056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7601094a0 msgr2=0x7fe76019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe76592d640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7601094a0 0x7fe76019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe76592d640 1 -- 172.21.15.112:0/396844056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7601a04f0 con 0x7fe760109df0 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.084+0000 7fe76592d640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe760109df0 0x7fe76019fdc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe75800eb10 tx=0x7fe75800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.085+0000 7fe7567fc640 1 -- 172.21.15.112:0/396844056 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758004b20 con 0x7fe760109df0 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.085+0000 7fe7567fc640 1 -- 172.21.15.112:0/396844056 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe758004cc0 con 0x7fe760109df0 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.085+0000 7fe767390640 1 -- 172.21.15.112:0/396844056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7601a07e0 con 0x7fe760109df0 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.085+0000 7fe7567fc640 1 -- 172.21.15.112:0/396844056 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758010670 con 0x7fe760109df0 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.085+0000 7fe767390640 1 -- 172.21.15.112:0/396844056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7601a6f40 con 0x7fe760109df0 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.087+0000 7fe7567fc640 1 -- 172.21.15.112:0/396844056 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe758005020 con 0x7fe760109df0 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.087+0000 7fe76d904740 1 Processor -- start 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.087+0000 7fe76d904740 1 -- start start 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76d904740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbf2070 0x5577abf99650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76d904740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 0x5577abf99b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76d904740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbd1390 0x5577abf9a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76d904740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5577abf0fe50 con 0x5577abbd1390 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76d904740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5577abeff280 con 0x5577abbc91b0 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76d904740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5577abf03090 con 0x5577abbf2070 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76492b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbf2070 0x5577abf99650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76492b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbd1390 0x5577abf9a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76512c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 0x5577abf99b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76512c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 0x5577abf99b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49486/0 (socket says 172.21.15.112:49486) 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76512c640 1 -- 172.21.15.112:0/3045930040 learned_addr learned my addr 172.21.15.112:0/3045930040 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76492b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbd1390 0x5577abf9a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37216/0 (socket says 172.21.15.112:37216) 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76512c640 1 -- 172.21.15.112:0/3045930040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbf2070 msgr2=0x5577abf99650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76512c640 1 --2- 172.21.15.112:0/3045930040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbf2070 0x5577abf99650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76512c640 1 -- 172.21.15.112:0/3045930040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbd1390 msgr2=0x5577abf9a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76512c640 1 --2- 172.21.15.112:0/3045930040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbd1390 0x5577abf9a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe7567fc640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe74c0b6700 0x7fe74c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76512c640 1 -- 172.21.15.112:0/3045930040 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577abc5d650 con 0x5577abbc91b0 2024-09-21T03:37:43.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.088+0000 7fe76492b640 1 --2- 172.21.15.112:0/3045930040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbd1390 0x5577abf9a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.089+0000 7fe76592d640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe74c0b6700 0x7fe74c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.089+0000 7fe76512c640 1 --2- 172.21.15.112:0/3045930040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 0x5577abf99b90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009d20 tx=0x7fe75c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e75606e5e47d2ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.089+0000 7fe7567fc640 1 -- 172.21.15.112:0/396844056 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7fe7580e7ca0 con 0x7fe760109df0 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.089+0000 7fe74b7fe640 1 -- 172.21.15.112:0/3045930040 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c01e070 con 0x5577abbc91b0 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.089+0000 7fe74b7fe640 1 -- 172.21.15.112:0/3045930040 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75c019ae0 con 0x5577abbc91b0 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.089+0000 7fe74b7fe640 1 -- 172.21.15.112:0/3045930040 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00e9f0 con 0x5577abbc91b0 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76d904740 1 -- 172.21.15.112:0/3045930040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 msgr2=0x5577abf99b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76592d640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe74c0b6700 0x7fe74c0b8c10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe7580e54d0 tx=0x7fe758079040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76d904740 1 --2- 172.21.15.112:0/3045930040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 0x5577abf99b90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009d20 tx=0x7fe75c011810 comp rx=0 tx=0).stop 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76d904740 1 -- 172.21.15.112:0/3045930040 shutdown_connections 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76d904740 1 --2- 172.21.15.112:0/3045930040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbd1390 0x5577abf9a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76d904740 1 --2- 172.21.15.112:0/3045930040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 0x5577abf99b90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76d904740 1 --2- 172.21.15.112:0/3045930040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbf2070 0x5577abf99650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76d904740 1 -- 172.21.15.112:0/3045930040 >> 172.21.15.112:0/3045930040 conn(0x5577abbe6460 msgr2=0x5577abf97540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76d904740 1 -- 172.21.15.112:0/3045930040 shutdown_connections 2024-09-21T03:37:43.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76d904740 1 -- 172.21.15.112:0/3045930040 wait complete. 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76d904740 1 -- start start 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.090+0000 7fe76d904740 1 Processor -- start 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76d904740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbc91b0 0x5577abfa4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76d904740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 0x5577abfa5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76d904740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbf2070 0x5577abfb1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76d904740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5577abd60700 con 0x5577abbf2070 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76d904740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5577abe9a110 con 0x5577abbd1390 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76d904740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5577abfa7e30 con 0x5577abbc91b0 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76d904740 10 client.0 _ll_register_callbacks cb 0x5577abeb3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76492b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbf2070 0x5577abfb1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76512c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 0x5577abfa5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76492b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbf2070 0x5577abfb1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37224/0 (socket says 172.21.15.112:37224) 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76492b640 1 -- 172.21.15.112:0/457173229 learned_addr learned my addr 172.21.15.112:0/457173229 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76492b640 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbc91b0 0x5577abfa4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76492b640 1 -- 172.21.15.112:0/457173229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbc91b0 msgr2=0x5577abfa4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76492b640 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbc91b0 0x5577abfa4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76492b640 1 -- 172.21.15.112:0/457173229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 msgr2=0x5577abfa5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76492b640 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 0x5577abfa5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76492b640 1 -- 172.21.15.112:0/457173229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577abec7c40 con 0x5577abbf2070 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.091+0000 7fe76512c640 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 0x5577abfa5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe76492b640 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbf2070 0x5577abfb1150 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe750009340 tx=0x7fe750005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459c7923313ac51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe748ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe748ff9640 1 -- 172.21.15.112:0/457173229 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750035040 con 0x5577abbf2070 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe748ff9640 1 -- 172.21.15.112:0/457173229 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7fe750004260 con 0x5577abbf2070 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe76d904740 1 -- 172.21.15.112:0/457173229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75c019ae0 con 0x5577abbf2070 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe748ff9640 1 -- 172.21.15.112:0/457173229 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe750051050 con 0x5577abbf2070 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe76d904740 10 client.16941 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe76d904740 1 -- 172.21.15.112:0/457173229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5577abfb3b30 con 0x5577abbf2070 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe748ff9640 1 -- 172.21.15.112:0/457173229 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75c019ae0 con 0x5577abbf2070 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe748ff9640 1 -- 172.21.15.112:0/457173229 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe75003d480 con 0x5577abbf2070 2024-09-21T03:37:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe76d904740 10 client.16941 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe76d904740 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe76d904740 20 client.16941 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe76d904740 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5577abbb09f0 0x5577abfb7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe76d904740 4 client.16941 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.092+0000 7fe76d904740 1 -- 172.21.15.112:0/457173229 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577abf11da0 con 0x5577abbb09f0 2024-09-21T03:37:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.093+0000 7fe76512c640 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5577abbb09f0 0x5577abfb7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.093+0000 7fe748ff9640 10 client.16941 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.093+0000 7fe76512c640 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5577abbb09f0 0x5577abfb7000 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689c0646145155c8 server_cookie=1cf2fa733465df3c in_seq=0 out_seq=0 2024-09-21T03:37:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.094+0000 7fe748ff9640 1 -- 172.21.15.112:0/457173229 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577abf11da0 con 0x5577abbb09f0 2024-09-21T03:37:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.094+0000 7fe748ff9640 10 client.16941 handle_command_reply: tid=0 2024-09-21T03:37:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.094+0000 7fe76d904740 1 client.16941 shutdown 2024-09-21T03:37:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe76d904740 1 -- 172.21.15.112:0/457173229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbf2070 msgr2=0x5577abfb1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe76d904740 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbf2070 0x5577abfb1150 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe750009340 tx=0x7fe750005e50 comp rx=0 tx=0).stop 2024-09-21T03:37:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe76d904740 1 -- 172.21.15.112:0/457173229 shutdown_connections 2024-09-21T03:37:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe76d904740 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5577abbb09f0 0x5577abfb7000 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe76d904740 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbf2070 0x5577abfb1150 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe76d904740 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 0x5577abfa5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe76d904740 1 --2- 172.21.15.112:0/457173229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbc91b0 0x5577abfa4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe748ff9640 0 client.16941 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:43.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe76d904740 1 -- 172.21.15.112:0/457173229 >> 172.21.15.112:0/457173229 conn(0x5577abbe6460 msgr2=0x5577abf9c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe76d904740 1 -- 172.21.15.112:0/457173229 shutdown_connections 2024-09-21T03:37:43.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe76d904740 1 -- 172.21.15.112:0/457173229 wait complete. 2024-09-21T03:37:43.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.095+0000 7fe76d904740 20 client.16941 trim_cache size 0 max 16384 2024-09-21T03:37:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.112+0000 7fe76d904740 1 Processor -- start 2024-09-21T03:37:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.112+0000 7fe76d904740 1 -- start start 2024-09-21T03:37:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76d904740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbb09f0 0x5577abf9ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76d904740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 0x5577abfad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76d904740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbd1390 0x5577abfad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76d904740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601b2070 con 0x5577abbb09f0 2024-09-21T03:37:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76d904740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601af940 con 0x5577abbc91b0 2024-09-21T03:37:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76d904740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601ad7b0 con 0x5577abbd1390 2024-09-21T03:37:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76492b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbb09f0 0x5577abf9ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76492b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbb09f0 0x5577abf9ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37236/0 (socket says 172.21.15.112:37236) 2024-09-21T03:37:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76492b640 1 -- 172.21.15.112:0/3047309798 learned_addr learned my addr 172.21.15.112:0/3047309798 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76512c640 1 --2- 172.21.15.112:0/3047309798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 0x5577abfad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76492b640 1 --2- 172.21.15.112:0/3047309798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbd1390 0x5577abfad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76492b640 1 -- 172.21.15.112:0/3047309798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbd1390 msgr2=0x5577abfad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76492b640 1 --2- 172.21.15.112:0/3047309798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbd1390 0x5577abfad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76492b640 1 -- 172.21.15.112:0/3047309798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 msgr2=0x5577abfad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76492b640 1 --2- 172.21.15.112:0/3047309798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 0x5577abfad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.113+0000 7fe76492b640 1 -- 172.21.15.112:0/3047309798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577abf9bd80 con 0x5577abbb09f0 2024-09-21T03:37:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.114+0000 7fe76492b640 1 --2- 172.21.15.112:0/3047309798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbb09f0 0x5577abf9ad90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe750000c00 tx=0x7fe750040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e798ac06122eda5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.114+0000 7fe74affd640 1 -- 172.21.15.112:0/3047309798 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750035070 con 0x5577abbb09f0 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.114+0000 7fe74affd640 1 -- 172.21.15.112:0/3047309798 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe750031e20 con 0x5577abbb09f0 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.114+0000 7fe74affd640 1 -- 172.21.15.112:0/3047309798 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750003850 con 0x5577abbb09f0 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.115+0000 7fe76d904740 1 -- 172.21.15.112:0/3047309798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbb09f0 msgr2=0x5577abf9ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.115+0000 7fe76d904740 1 --2- 172.21.15.112:0/3047309798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbb09f0 0x5577abf9ad90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe750000c00 tx=0x7fe750040040 comp rx=0 tx=0).stop 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.115+0000 7fe76d904740 1 -- 172.21.15.112:0/3047309798 shutdown_connections 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.115+0000 7fe76d904740 1 --2- 172.21.15.112:0/3047309798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbd1390 0x5577abfad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.115+0000 7fe76d904740 1 --2- 172.21.15.112:0/3047309798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbc91b0 0x5577abfad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.115+0000 7fe76d904740 1 --2- 172.21.15.112:0/3047309798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbb09f0 0x5577abf9ad90 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.115+0000 7fe76d904740 1 -- 172.21.15.112:0/3047309798 >> 172.21.15.112:0/3047309798 conn(0x5577abbe6460 msgr2=0x5577abf9a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.115+0000 7fe76d904740 1 -- 172.21.15.112:0/3047309798 shutdown_connections 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.115+0000 7fe76d904740 1 -- 172.21.15.112:0/3047309798 wait complete. 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.115+0000 7fe76d904740 1 -- start start 2024-09-21T03:37:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76d904740 1 Processor -- start 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76d904740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbb09f0 0x5577ac019910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76d904740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbc91b0 0x5577ac019e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76d904740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 0x5577ac024990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76d904740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601ade00 con 0x5577abbc91b0 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76d904740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601af270 con 0x5577abbd1390 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76d904740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601ae4d0 con 0x5577abbb09f0 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76d904740 10 client.0 _ll_register_callbacks cb 0x7fe750009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76512c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbc91b0 0x5577ac019e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76512c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbc91b0 0x5577ac019e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37240/0 (socket says 172.21.15.112:37240) 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76492b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbb09f0 0x5577ac019910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.116+0000 7fe76512c640 1 -- 172.21.15.112:0/2506966209 learned_addr learned my addr 172.21.15.112:0/2506966209 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe76492b640 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 0x5577ac024990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe76512c640 1 -- 172.21.15.112:0/2506966209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbb09f0 msgr2=0x5577ac019910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe76512c640 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbb09f0 0x5577ac019910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe76512c640 1 -- 172.21.15.112:0/2506966209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 msgr2=0x5577ac024990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe76512c640 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 0x5577ac024990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe76512c640 1 -- 172.21.15.112:0/2506966209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577ac024ed0 con 0x5577abbc91b0 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe76492b640 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 0x5577ac024990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe76512c640 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbc91b0 0x5577ac019e90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0058c0 tx=0x7fe75c009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ede07858a08200d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe748ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe748ff9640 1 -- 172.21.15.112:0/2506966209 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c032020 con 0x5577abbc91b0 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe748ff9640 1 -- 172.21.15.112:0/2506966209 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7fe75c005540 con 0x5577abbc91b0 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe76d904740 1 -- 172.21.15.112:0/2506966209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe750031e20 con 0x5577abbc91b0 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe748ff9640 1 -- 172.21.15.112:0/2506966209 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe75c01e040 con 0x5577abbc91b0 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe76d904740 10 client.16947 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.117+0000 7fe76d904740 1 -- 172.21.15.112:0/2506966209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5577ac026030 con 0x5577abbc91b0 2024-09-21T03:37:43.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.118+0000 7fe748ff9640 1 -- 172.21.15.112:0/2506966209 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe750031e20 con 0x5577abbc91b0 2024-09-21T03:37:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.118+0000 7fe748ff9640 1 -- 172.21.15.112:0/2506966209 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe75c0037d0 con 0x5577abbc91b0 2024-09-21T03:37:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.118+0000 7fe76d904740 10 client.16947 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.118+0000 7fe76d904740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.118+0000 7fe76d904740 20 client.16947 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.118+0000 7fe76d904740 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5577abbf2070 0x5577ac028680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.118+0000 7fe76d904740 4 client.16947 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:37:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.118+0000 7fe76d904740 1 -- 172.21.15.112:0/2506966209 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5577abf0d850 con 0x5577abbf2070 2024-09-21T03:37:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.118+0000 7fe76492b640 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5577abbf2070 0x5577ac028680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.118+0000 7fe748ff9640 10 client.16947 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:43.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.118+0000 7fe76492b640 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5577abbf2070 0x5577ac028680 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f104465addc082 server_cookie=e908029d75ab1b32 in_seq=0 out_seq=0 2024-09-21T03:37:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe748ff9640 1 -- 172.21.15.112:0/2506966209 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x5577abf0d850 con 0x5577abbf2070 2024-09-21T03:37:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe748ff9640 10 client.16947 handle_command_reply: tid=0 2024-09-21T03:37:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 1 client.16947 shutdown 2024-09-21T03:37:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 1 -- 172.21.15.112:0/2506966209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbc91b0 msgr2=0x5577ac019e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbc91b0 0x5577ac019e90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0058c0 tx=0x7fe75c009960 comp rx=0 tx=0).stop 2024-09-21T03:37:43.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 1 -- 172.21.15.112:0/2506966209 shutdown_connections 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5577abbf2070 0x5577ac028680 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577abbd1390 0x5577ac024990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5577abbc91b0 0x5577ac019e90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 1 --2- 172.21.15.112:0/2506966209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5577abbb09f0 0x5577ac019910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 1 -- 172.21.15.112:0/2506966209 >> 172.21.15.112:0/2506966209 conn(0x5577abbe6460 msgr2=0x5577abf9a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe748ff9640 0 client.16947 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 1 -- 172.21.15.112:0/2506966209 shutdown_connections 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 1 -- 172.21.15.112:0/2506966209 wait complete. 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.120+0000 7fe76d904740 20 client.16947 trim_cache size 0 max 16384 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:37:43.125 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.126 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.127 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:37:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:37:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:37:43.130 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:43.132 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:37:43.133 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:37:43.133 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:43.133 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:43.133 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:43.133 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:43.133 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:43.133 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:37:43.133 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:37:43.133 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 -- 172.21.15.112:0/396844056 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe74c0b6700 msgr2=0x7fe74c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe74c0b6700 0x7fe74c0b8c10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe7580e54d0 tx=0x7fe758079040 comp rx=0 tx=0).stop 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 -- 172.21.15.112:0/396844056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe760109df0 msgr2=0x7fe76019fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe760109df0 0x7fe76019fdc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe75800eb10 tx=0x7fe75800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 -- 172.21.15.112:0/396844056 shutdown_connections 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe74c0b6700 0x7fe74c0b8c10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe760109df0 0x7fe76019fdc0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7601094a0 0x7fe76019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 --2- 172.21.15.112:0/396844056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe760103b60 0x7fe76019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 -- 172.21.15.112:0/396844056 >> 172.21.15.112:0/396844056 conn(0x7fe76007b850 msgr2=0x7fe760105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 -- 172.21.15.112:0/396844056 shutdown_connections 2024-09-21T03:37:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:43.121+0000 7fe767390640 1 -- 172.21.15.112:0/396844056 wait complete. 2024-09-21T03:37:43.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 921805 10.67 MB/sec execute 343 sec latency 40.301 ms 2024-09-21T03:37:44.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:43 smithi112 ceph-mon[31442]: pgmap v585: 97 pgs: 97 active+clean; 696 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-09-21T03:37:44.105 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:37:44.105 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:37:44.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:43 smithi203 ceph-mon[34723]: pgmap v585: 97 pgs: 97 active+clean; 696 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-09-21T03:37:44.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:43 smithi187 ceph-mon[34603]: pgmap v585: 97 pgs: 97 active+clean; 696 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-09-21T03:37:44.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 924068 10.68 MB/sec execute 344 sec latency 15.866 ms 2024-09-21T03:37:44.420 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.922+0000 7fad7b1d3640 1 -- 172.21.15.112:0/1664929496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad74102ad0 msgr2=0x7fad74102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.922+0000 7fad7b1d3640 1 --2- 172.21.15.112:0/1664929496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad74102ad0 0x7fad74102ee0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fad70009900 tx=0x7fad7002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.922+0000 7fad7b1d3640 1 -- 172.21.15.112:0/1664929496 shutdown_connections 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.922+0000 7fad7b1d3640 1 --2- 172.21.15.112:0/1664929496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad74108d10 0x7fad7410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.922+0000 7fad7b1d3640 1 --2- 172.21.15.112:0/1664929496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad741083c0 0x7fad741087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.922+0000 7fad7b1d3640 1 --2- 172.21.15.112:0/1664929496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad74102ad0 0x7fad74102ee0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.922+0000 7fad7b1d3640 1 -- 172.21.15.112:0/1664929496 >> 172.21.15.112:0/1664929496 conn(0x7fad740796d0 msgr2=0x7fad74079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.922+0000 7fad7b1d3640 1 -- 172.21.15.112:0/1664929496 shutdown_connections 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.922+0000 7fad7b1d3640 1 -- 172.21.15.112:0/1664929496 wait complete. 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad7b1d3640 1 Processor -- start 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad7b1d3640 1 -- start start 2024-09-21T03:37:44.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad7b1d3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad74102ad0 0x7fad7419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad7b1d3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad741083c0 0x7fad7419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad7b1d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad74108d10 0x7fad7419bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad7b1d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad741129d0 con 0x7fad74108d10 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad7b1d3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fad74112850 con 0x7fad741083c0 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad7b1d3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fad74112b50 con 0x7fad74102ad0 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad79770640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad74108d10 0x7fad7419bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad79770640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad74108d10 0x7fad7419bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37250/0 (socket says 172.21.15.112:37250) 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad79770640 1 -- 172.21.15.112:0/2030307803 learned_addr learned my addr 172.21.15.112:0/2030307803 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.924+0000 7fad6bfff640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad741083c0 0x7fad7419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.923+0000 7fad78f6f640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad74102ad0 0x7fad7419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.924+0000 7fad79770640 1 -- 172.21.15.112:0/2030307803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad74102ad0 msgr2=0x7fad7419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.924+0000 7fad79770640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad74102ad0 0x7fad7419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.924+0000 7fad79770640 1 -- 172.21.15.112:0/2030307803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad741083c0 msgr2=0x7fad7419b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.924+0000 7fad79770640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad741083c0 0x7fad7419b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:44.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.924+0000 7fad79770640 1 -- 172.21.15.112:0/2030307803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad741a65f0 con 0x7fad74108d10 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.924+0000 7fad79770640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad74108d10 0x7fad7419bde0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00eb10 tx=0x7fad6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.924+0000 7fad69ffb640 1 -- 172.21.15.112:0/2030307803 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004b20 con 0x7fad74108d10 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.924+0000 7fad69ffb640 1 -- 172.21.15.112:0/2030307803 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad6c004cc0 con 0x7fad74108d10 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.924+0000 7fad7b1d3640 1 -- 172.21.15.112:0/2030307803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad741a68e0 con 0x7fad74108d10 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.924+0000 7fad69ffb640 1 -- 172.21.15.112:0/2030307803 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c010690 con 0x7fad74108d10 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.925+0000 7fad7b1d3640 1 -- 172.21.15.112:0/2030307803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad741a6e20 con 0x7fad74108d10 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.926+0000 7fad7b1d3640 1 -- 172.21.15.112:0/2030307803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad74103d00 con 0x7fad74108d10 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.927+0000 7fad69ffb640 1 -- 172.21.15.112:0/2030307803 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fad6c005000 con 0x7fad74108d10 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.928+0000 7fad69ffb640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fad5c0b66b0 0x7fad5c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.929+0000 7fad78f6f640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fad5c0b66b0 0x7fad5c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.929+0000 7fad69ffb640 1 -- 172.21.15.112:0/2030307803 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7fad6c0e7dc0 con 0x7fad74108d10 2024-09-21T03:37:44.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.929+0000 7fad78f6f640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fad5c0b66b0 0x7fad5c0b8bc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fad70000c00 tx=0x7fad70031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:44.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:44.932+0000 7fad69ffb640 1 -- 172.21.15.112:0/2030307803 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad6c056710 con 0x7fad74108d10 2024-09-21T03:37:45.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.173+0000 7fad7b1d3640 1 -- 172.21.15.112:0/2030307803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad74102ee0 con 0x7fad74108d10 2024-09-21T03:37:45.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.174+0000 7fad69ffb640 1 -- 172.21.15.112:0/2030307803 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7fad74102ee0 con 0x7fad74108d10 2024-09-21T03:37:45.177 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:37:45.177 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"btime":"2024-09-21T03:35:32:357323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:35:32.307199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:37:45.177 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-09-21T03:37:45.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.176+0000 7fad7b1d3640 1 -- 172.21.15.112:0/2030307803 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fad5c0b66b0 msgr2=0x7fad5c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:45.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.176+0000 7fad7b1d3640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fad5c0b66b0 0x7fad5c0b8bc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fad70000c00 tx=0x7fad70031400 comp rx=0 tx=0).stop 2024-09-21T03:37:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.176+0000 7fad7b1d3640 1 -- 172.21.15.112:0/2030307803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad74108d10 msgr2=0x7fad7419bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.176+0000 7fad7b1d3640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad74108d10 0x7fad7419bde0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00eb10 tx=0x7fad6c00c2d0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.176+0000 7fad7b1d3640 1 -- 172.21.15.112:0/2030307803 shutdown_connections 2024-09-21T03:37:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.176+0000 7fad7b1d3640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fad5c0b66b0 0x7fad5c0b8bc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.176+0000 7fad7b1d3640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad74108d10 0x7fad7419bde0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.176+0000 7fad7b1d3640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad741083c0 0x7fad7419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.176+0000 7fad7b1d3640 1 --2- 172.21.15.112:0/2030307803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad74102ad0 0x7fad7419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.176+0000 7fad7b1d3640 1 -- 172.21.15.112:0/2030307803 >> 172.21.15.112:0/2030307803 conn(0x7fad740796d0 msgr2=0x7fad74106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.177+0000 7fad7b1d3640 1 -- 172.21.15.112:0/2030307803 shutdown_connections 2024-09-21T03:37:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:45.177+0000 7fad7b1d3640 1 -- 172.21.15.112:0/2030307803 wait complete. 2024-09-21T03:37:45.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 925320 10.67 MB/sec execute 345 sec latency 40.514 ms 2024-09-21T03:37:45.957 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:37:46.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:45 smithi203 ceph-mon[34723]: pgmap v586: 97 pgs: 97 active+clean; 696 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-09-21T03:37:46.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:45 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2030307803' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:46.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:45 smithi187 ceph-mon[34603]: pgmap v586: 97 pgs: 97 active+clean; 696 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-09-21T03:37:46.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:45 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2030307803' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:46.243 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:46.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 927621 10.68 MB/sec execute 346 sec latency 28.997 ms 2024-09-21T03:37:46.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:45 smithi112 ceph-mon[31442]: pgmap v586: 97 pgs: 97 active+clean; 696 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-09-21T03:37:46.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:45 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2030307803' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:37:46.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.759+0000 7f3e094ba640 1 -- 172.21.15.112:0/1972290408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04108cf0 msgr2=0x7f3e0410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.760+0000 7f3e094ba640 1 --2- 172.21.15.112:0/1972290408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04108cf0 0x7f3e0410f520 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009ca0 tx=0x7f3dfc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.760+0000 7f3e094ba640 1 -- 172.21.15.112:0/1972290408 shutdown_connections 2024-09-21T03:37:46.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.760+0000 7f3e094ba640 1 --2- 172.21.15.112:0/1972290408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04108cf0 0x7f3e0410f520 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.760+0000 7f3e094ba640 1 --2- 172.21.15.112:0/1972290408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e041083a0 0x7f3e041087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.760+0000 7f3e094ba640 1 --2- 172.21.15.112:0/1972290408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e04102a60 0x7f3e04102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.760+0000 7f3e094ba640 1 -- 172.21.15.112:0/1972290408 >> 172.21.15.112:0/1972290408 conn(0x7f3e04079730 msgr2=0x7f3e04079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:46.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.760+0000 7f3e094ba640 1 -- 172.21.15.112:0/1972290408 shutdown_connections 2024-09-21T03:37:46.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.760+0000 7f3e094ba640 1 -- 172.21.15.112:0/1972290408 wait complete. 2024-09-21T03:37:46.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.760+0000 7f3e094ba640 1 Processor -- start 2024-09-21T03:37:46.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.760+0000 7f3e094ba640 1 -- start start 2024-09-21T03:37:46.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3e094ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e04102a60 0x7f3e0419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3e094ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e041083a0 0x7f3e0419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3e094ba640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e04108cf0 0x7f3e0419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3e094ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041129b0 con 0x7f3e041083a0 2024-09-21T03:37:46.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3e094ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04112830 con 0x7f3e04102a60 2024-09-21T03:37:46.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3e094ba640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04112b30 con 0x7f3e04108cf0 2024-09-21T03:37:46.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3dfbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e041083a0 0x7f3e0419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3dfbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e041083a0 0x7f3e0419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45092/0 (socket says 172.21.15.112:45092) 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3dfbfff640 1 -- 172.21.15.112:0/1905580347 learned_addr learned my addr 172.21.15.112:0/1905580347 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3e037fe640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e04108cf0 0x7f3e0419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3e02ffd640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e04102a60 0x7f3e0419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3dfbfff640 1 -- 172.21.15.112:0/1905580347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e04108cf0 msgr2=0x7f3e0419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3dfbfff640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e04108cf0 0x7f3e0419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3dfbfff640 1 -- 172.21.15.112:0/1905580347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e04102a60 msgr2=0x7f3e0419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3dfbfff640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e04102a60 0x7f3e0419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3dfbfff640 1 -- 172.21.15.112:0/1905580347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e041a0380 con 0x7f3e041083a0 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.761+0000 7f3dfbfff640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e041083a0 0x7f3e0419a9b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00e9c0 tx=0x7f3dec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.762+0000 7f3e00ff9640 1 -- 172.21.15.112:0/1905580347 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec00cd80 con 0x7f3e041083a0 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.762+0000 7f3e00ff9640 1 -- 172.21.15.112:0/1905580347 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dec0054d0 con 0x7f3e041083a0 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.762+0000 7f3e094ba640 1 -- 172.21.15.112:0/1905580347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e041a0670 con 0x7f3e041083a0 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.762+0000 7f3e00ff9640 1 -- 172.21.15.112:0/1905580347 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec010430 con 0x7f3e041083a0 2024-09-21T03:37:46.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.762+0000 7f3e094ba640 1 -- 172.21.15.112:0/1905580347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e041a6dd0 con 0x7f3e041083a0 2024-09-21T03:37:46.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.765+0000 7f3e00ff9640 1 -- 172.21.15.112:0/1905580347 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3dec004fd0 con 0x7f3e041083a0 2024-09-21T03:37:46.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.765+0000 7f3e0fa2e740 1 Processor -- start 2024-09-21T03:37:46.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.765+0000 7f3e0fa2e740 1 -- start start 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.765+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f39a05070 0x556f39dac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.765+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399dc1b0 0x556f39dac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.765+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399e4390 0x556f39dacd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556f39d22ea0 con 0x556f399e4390 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556f39d122d0 con 0x556f39a05070 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556f39d160e0 con 0x556f399dc1b0 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e02ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399dc1b0 0x556f39dac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e02ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399dc1b0 0x556f39dac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46748/0 (socket says 172.21.15.112:46748) 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3dfbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399e4390 0x556f39dacd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e02ffd640 1 -- 172.21.15.112:0/2691033358 learned_addr learned my addr 172.21.15.112:0/2691033358 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e037fe640 1 --2- 172.21.15.112:0/2691033358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f39a05070 0x556f39dac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e02ffd640 1 -- 172.21.15.112:0/2691033358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f39a05070 msgr2=0x556f39dac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:46.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e02ffd640 1 --2- 172.21.15.112:0/2691033358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f39a05070 0x556f39dac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e02ffd640 1 -- 172.21.15.112:0/2691033358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399e4390 msgr2=0x556f39dacd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:46.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e02ffd640 1 --2- 172.21.15.112:0/2691033358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399e4390 0x556f39dacd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e02ffd640 1 -- 172.21.15.112:0/2691033358 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f39a70650 con 0x556f399dc1b0 2024-09-21T03:37:46.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e00ff9640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3df00b6680 0x7f3df00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e02ffd640 1 --2- 172.21.15.112:0/2691033358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399dc1b0 0x556f39dac800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009cf0 tx=0x7f3df4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c97127c6a21371ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:46.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3e037fe640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3df00b6680 0x7f3df00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.766+0000 7f3df97fa640 1 -- 172.21.15.112:0/2691033358 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4004230 con 0x556f399dc1b0 2024-09-21T03:37:46.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.767+0000 7f3e00ff9640 1 -- 172.21.15.112:0/1905580347 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7f3dec0e7b10 con 0x7f3e041083a0 2024-09-21T03:37:46.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.767+0000 7f3df97fa640 1 -- 172.21.15.112:0/2691033358 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df40043d0 con 0x556f399dc1b0 2024-09-21T03:37:46.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.767+0000 7f3df97fa640 1 -- 172.21.15.112:0/2691033358 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4019820 con 0x556f399dc1b0 2024-09-21T03:37:46.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.767+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2691033358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399dc1b0 msgr2=0x556f39dac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.767+0000 7f3e037fe640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3df00b6680 0x7f3df00b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc006010 tx=0x7f3dfc005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:46.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.767+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/2691033358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399dc1b0 0x556f39dac800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009cf0 tx=0x7f3df4011810 comp rx=0 tx=0).stop 2024-09-21T03:37:46.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2691033358 shutdown_connections 2024-09-21T03:37:46.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/2691033358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399e4390 0x556f39dacd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/2691033358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399dc1b0 0x556f39dac800 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/2691033358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f39a05070 0x556f39dac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2691033358 >> 172.21.15.112:0/2691033358 conn(0x556f399f9460 msgr2=0x556f39daa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2691033358 shutdown_connections 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2691033358 wait complete. 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 -- start start 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 Processor -- start 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399dc1b0 0x556f39db7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399e4390 0x556f39db7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.768+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f39a05070 0x556f39dc3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556f39b73700 con 0x556f399dc1b0 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556f39cad160 con 0x556f39a05070 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556f39dbaaa0 con 0x556f399e4390 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e02ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399dc1b0 0x556f39db7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e02ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399dc1b0 0x556f39db7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45098/0 (socket says 172.21.15.112:45098) 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e02ffd640 1 -- 172.21.15.112:0/2667939698 learned_addr learned my addr 172.21.15.112:0/2667939698 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e0fa2e740 10 client.0 _ll_register_callbacks cb 0x556f39cc69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3dfbfff640 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399e4390 0x556f39db7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e037fe640 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f39a05070 0x556f39dc3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e02ffd640 1 -- 172.21.15.112:0/2667939698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399e4390 msgr2=0x556f39db7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e02ffd640 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399e4390 0x556f39db7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e02ffd640 1 -- 172.21.15.112:0/2667939698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f39a05070 msgr2=0x556f39dc3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e02ffd640 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f39a05070 0x556f39dc3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e02ffd640 1 -- 172.21.15.112:0/2667939698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f39cdac90 con 0x556f399dc1b0 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e02ffd640 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399dc1b0 0x556f39db7950 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3df4019460 tx=0x7f3df40096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192aec4d3d54525f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3dea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3dea7fc640 1 -- 172.21.15.112:0/2667939698 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4027070 con 0x556f399dc1b0 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3dea7fc640 1 -- 172.21.15.112:0/2667939698 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7f3df4005040 con 0x556f399dc1b0 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.769+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2667939698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3df40043d0 con 0x556f399dc1b0 2024-09-21T03:37:46.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.770+0000 7f3dea7fc640 1 -- 172.21.15.112:0/2667939698 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3df4033050 con 0x556f399dc1b0 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.770+0000 7f3e0fa2e740 10 client.16965 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.770+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2667939698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f39dc6790 con 0x556f399dc1b0 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.770+0000 7f3dea7fc640 1 -- 172.21.15.112:0/2667939698 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df40043d0 con 0x556f399dc1b0 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.770+0000 7f3dea7fc640 1 -- 172.21.15.112:0/2667939698 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3df40049f0 con 0x556f399dc1b0 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.770+0000 7f3e0fa2e740 10 client.16965 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.770+0000 7f3e0fa2e740 10 client.16965 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.770+0000 7f3e0fa2e740 20 client.16965 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.770+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556f399c39f0 0x556f39dc9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.771+0000 7f3e0fa2e740 4 client.16965 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.771+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2667939698 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f39d24df0 con 0x556f399c39f0 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.771+0000 7f3dfbfff640 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556f399c39f0 0x556f39dc9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.771+0000 7f3dfbfff640 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556f399c39f0 0x556f39dc9b50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37282c51804b3fef server_cookie=b93e12934f75a457 in_seq=0 out_seq=0 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.771+0000 7f3dea7fc640 10 client.16965 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3dea7fc640 1 -- 172.21.15.112:0/2667939698 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f39d24df0 con 0x556f399c39f0 2024-09-21T03:37:46.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3dea7fc640 10 client.16965 handle_command_reply: tid=0 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3e0fa2e740 1 client.16965 shutdown 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2667939698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399dc1b0 msgr2=0x556f39db7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399dc1b0 0x556f39db7950 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3df4019460 tx=0x7f3df40096f0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2667939698 shutdown_connections 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556f399c39f0 0x556f39dc9b50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f39a05070 0x556f39dc3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f399e4390 0x556f39db7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/2667939698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f399dc1b0 0x556f39db7950 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2667939698 >> 172.21.15.112:0/2667939698 conn(0x556f399f9460 msgr2=0x556f39daf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3dea7fc640 0 client.16965 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2667939698 shutdown_connections 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.772+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/2667939698 wait complete. 2024-09-21T03:37:46.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.773+0000 7f3e0fa2e740 20 client.16965 trim_cache size 0 max 16384 2024-09-21T03:37:46.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.787+0000 7f3e0fa2e740 1 Processor -- start 2024-09-21T03:37:46.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e0fa2e740 1 -- start start 2024-09-21T03:37:46.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f399e4390 0x556f39db3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f39a05070 0x556f39db68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39998da0 0x556f39dc3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041afda0 con 0x556f39998da0 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041ad670 con 0x556f399e4390 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041ab4e0 con 0x556f39a05070 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39998da0 0x556f39dc3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39998da0 0x556f39dc3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45112/0 (socket says 172.21.15.112:45112) 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e037fe640 1 -- 172.21.15.112:0/1701990534 learned_addr learned my addr 172.21.15.112:0/1701990534 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3dfbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f39a05070 0x556f39db68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e02ffd640 1 --2- 172.21.15.112:0/1701990534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f399e4390 0x556f39db3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e037fe640 1 -- 172.21.15.112:0/1701990534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f39a05070 msgr2=0x556f39db68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e037fe640 1 --2- 172.21.15.112:0/1701990534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f39a05070 0x556f39db68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e037fe640 1 -- 172.21.15.112:0/1701990534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f399e4390 msgr2=0x556f39db3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e037fe640 1 --2- 172.21.15.112:0/1701990534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f399e4390 0x556f39db3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.788+0000 7f3e037fe640 1 -- 172.21.15.112:0/1701990534 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f39db5e50 con 0x556f39998da0 2024-09-21T03:37:46.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.789+0000 7f3e037fe640 1 --2- 172.21.15.112:0/1701990534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39998da0 0x556f39dc3cc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009ca0 tx=0x7f3dfc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d661c4c97ff96b23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.789+0000 7f3df8ff9640 1 -- 172.21.15.112:0/1701990534 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc003970 con 0x556f39998da0 2024-09-21T03:37:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.789+0000 7f3df8ff9640 1 -- 172.21.15.112:0/1701990534 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dfc036050 con 0x556f39998da0 2024-09-21T03:37:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.789+0000 7f3df8ff9640 1 -- 172.21.15.112:0/1701990534 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0043c0 con 0x556f39998da0 2024-09-21T03:37:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.790+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/1701990534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39998da0 msgr2=0x556f39dc3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.790+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/1701990534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39998da0 0x556f39dc3cc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009ca0 tx=0x7f3dfc041000 comp rx=0 tx=0).stop 2024-09-21T03:37:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.790+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/1701990534 shutdown_connections 2024-09-21T03:37:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.790+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/1701990534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39998da0 0x556f39dc3cc0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.790+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/1701990534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f39a05070 0x556f39db68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.790+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/1701990534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f399e4390 0x556f39db3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.790+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/1701990534 >> 172.21.15.112:0/1701990534 conn(0x556f399f9460 msgr2=0x556f39db61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:46.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.790+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/1701990534 shutdown_connections 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.790+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/1701990534 wait complete. 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.790+0000 7f3e0fa2e740 1 -- start start 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.790+0000 7f3e0fa2e740 1 Processor -- start 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f39998da0 0x556f39dc0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f399e4390 0x556f39dbb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e0fa2e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39a05070 0x556f39dbbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041abb30 con 0x556f39a05070 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041acfa0 con 0x556f399e4390 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e0fa2e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041ac200 con 0x556f39998da0 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e0fa2e740 10 client.0 _ll_register_callbacks cb 0x7f3dec0eb530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39a05070 0x556f39dbbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e02ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f39998da0 0x556f39dc0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3dfbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f399e4390 0x556f39dbb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3dfbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f399e4390 0x556f39dbb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:38220/0 (socket says 172.21.15.112:38220) 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39a05070 0x556f39dbbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45118/0 (socket says 172.21.15.112:45118) 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e02ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f39998da0 0x556f39dc0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46770/0 (socket says 172.21.15.112:46770) 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3dfbfff640 1 -- 172.21.15.112:0/81475628 learned_addr learned my addr 172.21.15.112:0/81475628 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e037fe640 1 -- 172.21.15.112:0/81475628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f39998da0 msgr2=0x556f39dc0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e037fe640 1 --2- 172.21.15.112:0/81475628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f39998da0 0x556f39dc0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e037fe640 1 -- 172.21.15.112:0/81475628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f399e4390 msgr2=0x556f39dbb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e037fe640 1 --2- 172.21.15.112:0/81475628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f399e4390 0x556f39dbb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e037fe640 1 -- 172.21.15.112:0/81475628 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f39dbc140 con 0x556f39a05070 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3e037fe640 1 --2- 172.21.15.112:0/81475628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39a05070 0x556f39dbbc00 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc02aeb0 tx=0x7f3dfc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622e534216008408 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.791+0000 7f3dea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3dea7fc640 1 -- 172.21.15.112:0/81475628 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc064060 con 0x556f39a05070 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3dea7fc640 1 -- 172.21.15.112:0/81475628 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7f3dfc003b80 con 0x556f39a05070 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/81475628 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dfc036050 con 0x556f39a05070 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3dea7fc640 1 -- 172.21.15.112:0/81475628 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3dfc003d80 con 0x556f39a05070 2024-09-21T03:37:46.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3e0fa2e740 10 client.16971 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/81475628 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f39e34080 con 0x556f39a05070 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3dea7fc640 1 -- 172.21.15.112:0/81475628 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dfc036050 con 0x556f39a05070 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3dea7fc640 1 -- 172.21.15.112:0/81475628 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3dfc055070 con 0x556f39a05070 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3e0fa2e740 10 client.16971 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3e0fa2e740 10 client.16971 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3e0fa2e740 20 client.16971 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/81475628 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556f39e347a0 0x556f39e36bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3e0fa2e740 4 client.16971 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/81475628 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f39d208a0 con 0x556f39e347a0 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.792+0000 7f3e02ffd640 1 --2- 172.21.15.112:0/81475628 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556f39e347a0 0x556f39e36bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3dea7fc640 10 client.16971 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3e02ffd640 1 --2- 172.21.15.112:0/81475628 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556f39e347a0 0x556f39e36bc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1568f2dae49eead server_cookie=60606b3f32c1e850 in_seq=0 out_seq=0 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3dea7fc640 1 -- 172.21.15.112:0/81475628 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f39d208a0 con 0x556f39e347a0 2024-09-21T03:37:46.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3dea7fc640 10 client.16971 handle_command_reply: tid=0 2024-09-21T03:37:46.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3e0fa2e740 1 client.16971 shutdown 2024-09-21T03:37:46.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/81475628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39a05070 msgr2=0x556f39dbbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/81475628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39a05070 0x556f39dbbc00 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc02aeb0 tx=0x7f3dfc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/81475628 shutdown_connections 2024-09-21T03:37:46.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/81475628 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x556f39e347a0 0x556f39e36bc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/81475628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f39a05070 0x556f39dbbc00 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/81475628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f399e4390 0x556f39dbb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3e0fa2e740 1 --2- 172.21.15.112:0/81475628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f39998da0 0x556f39dc0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.793+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/81475628 >> 172.21.15.112:0/81475628 conn(0x556f399f9460 msgr2=0x556f39dad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:46.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.794+0000 7f3dea7fc640 0 client.16971 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:37:46.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.794+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/81475628 shutdown_connections 2024-09-21T03:37:46.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.794+0000 7f3e0fa2e740 1 -- 172.21.15.112:0/81475628 wait complete. 2024-09-21T03:37:46.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.794+0000 7f3e0fa2e740 20 client.16971 trim_cache size 0 max 16384 2024-09-21T03:37:46.801 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:37:46.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.794+0000 7f3e094ba640 1 -- 172.21.15.112:0/1905580347 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3df00b6680 msgr2=0x7f3df00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.794+0000 7f3e094ba640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3df00b6680 0x7f3df00b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc006010 tx=0x7f3dfc005d80 comp rx=0 tx=0).stop 2024-09-21T03:37:46.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.794+0000 7f3e094ba640 1 -- 172.21.15.112:0/1905580347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e041083a0 msgr2=0x7f3e0419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:46.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.794+0000 7f3e094ba640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e041083a0 0x7f3e0419a9b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00e9c0 tx=0x7f3dec00ee80 comp rx=0 tx=0).stop 2024-09-21T03:37:46.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.795+0000 7f3e094ba640 1 -- 172.21.15.112:0/1905580347 shutdown_connections 2024-09-21T03:37:46.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.795+0000 7f3e094ba640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3df00b6680 0x7f3df00b8b90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.795+0000 7f3e094ba640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e04108cf0 0x7f3e0419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.795+0000 7f3e094ba640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e041083a0 0x7f3e0419a9b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.795+0000 7f3e094ba640 1 --2- 172.21.15.112:0/1905580347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e04102a60 0x7f3e0419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:46.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.795+0000 7f3e094ba640 1 -- 172.21.15.112:0/1905580347 >> 172.21.15.112:0/1905580347 conn(0x7f3e04079730 msgr2=0x7f3e0406b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:46.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.795+0000 7f3e094ba640 1 -- 172.21.15.112:0/1905580347 shutdown_connections 2024-09-21T03:37:46.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:46.795+0000 7f3e094ba640 1 -- 172.21.15.112:0/1905580347 wait complete. 2024-09-21T03:37:47.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 928896 10.66 MB/sec execute 347 sec latency 46.458 ms 2024-09-21T03:37:47.787 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:37:48.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:47 smithi112 ceph-mon[31442]: pgmap v587: 97 pgs: 97 active+clean; 703 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-09-21T03:37:48.058 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:48.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:47 smithi203 ceph-mon[34723]: pgmap v587: 97 pgs: 97 active+clean; 703 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-09-21T03:37:48.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:47 smithi187 ceph-mon[34603]: pgmap v587: 97 pgs: 97 active+clean; 703 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-09-21T03:37:48.265 INFO:tasks.workunit.client.0.smithi112.stdout: 1 931139 10.67 MB/sec execute 348 sec latency 30.743 ms 2024-09-21T03:37:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.581+0000 7fc0668a7640 1 -- 172.21.15.112:0/2651800188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060103c30 msgr2=0x7fc060104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.581+0000 7fc0668a7640 1 --2- 172.21.15.112:0/2651800188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060103c30 0x7fc060104040 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc050009900 tx=0x7fc05002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.581+0000 7fc0668a7640 1 -- 172.21.15.112:0/2651800188 shutdown_connections 2024-09-21T03:37:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.581+0000 7fc0668a7640 1 --2- 172.21.15.112:0/2651800188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc060109ec0 0x7fc0601106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.581+0000 7fc0668a7640 1 --2- 172.21.15.112:0/2651800188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc060109570 0x7fc060109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.581+0000 7fc0668a7640 1 --2- 172.21.15.112:0/2651800188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060103c30 0x7fc060104040 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.581+0000 7fc0668a7640 1 -- 172.21.15.112:0/2651800188 >> 172.21.15.112:0/2651800188 conn(0x7fc06007b8d0 msgr2=0x7fc06007bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.581+0000 7fc0668a7640 1 -- 172.21.15.112:0/2651800188 shutdown_connections 2024-09-21T03:37:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.581+0000 7fc0668a7640 1 -- 172.21.15.112:0/2651800188 wait complete. 2024-09-21T03:37:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc0668a7640 1 Processor -- start 2024-09-21T03:37:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc0668a7640 1 -- start start 2024-09-21T03:37:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc0668a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc060103c30 0x7fc06019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc0668a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060109570 0x7fc06019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc0668a7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc060109ec0 0x7fc06019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc0668a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc060113ae0 con 0x7fc060109570 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc0668a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc060113960 con 0x7fc060103c30 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc0668a7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc060113c60 con 0x7fc060109ec0 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc057fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060109570 0x7fc06019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc057fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060109570 0x7fc06019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45132/0 (socket says 172.21.15.112:45132) 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc057fff640 1 -- 172.21.15.112:0/394651067 learned_addr learned my addr 172.21.15.112:0/394651067 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc064e44640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc060109ec0 0x7fc06019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.582+0000 7fc05ffff640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc060103c30 0x7fc06019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc057fff640 1 -- 172.21.15.112:0/394651067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc060109ec0 msgr2=0x7fc06019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc057fff640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc060109ec0 0x7fc06019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc057fff640 1 -- 172.21.15.112:0/394651067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc060103c30 msgr2=0x7fc06019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc057fff640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc060103c30 0x7fc06019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc057fff640 1 -- 172.21.15.112:0/394651067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0601a0540 con 0x7fc060109570 2024-09-21T03:37:48.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc064e44640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc060109ec0 0x7fc06019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:48.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc057fff640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060109570 0x7fc06019ab70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e9f0 tx=0x7fc04800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:48.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc05dffb640 1 -- 172.21.15.112:0/394651067 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04800ccc0 con 0x7fc060109570 2024-09-21T03:37:48.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc05dffb640 1 -- 172.21.15.112:0/394651067 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0480108e0 con 0x7fc060109570 2024-09-21T03:37:48.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc05dffb640 1 -- 172.21.15.112:0/394651067 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048018860 con 0x7fc060109570 2024-09-21T03:37:48.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc0668a7640 1 -- 172.21.15.112:0/394651067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0601a0830 con 0x7fc060109570 2024-09-21T03:37:48.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.583+0000 7fc0668a7640 1 -- 172.21.15.112:0/394651067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0601a6f90 con 0x7fc060109570 2024-09-21T03:37:48.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.587+0000 7fc05dffb640 1 -- 172.21.15.112:0/394651067 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc048010430 con 0x7fc060109570 2024-09-21T03:37:48.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.587+0000 7fc06ce1b740 1 Processor -- start 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.587+0000 7fc06ce1b740 1 -- start start 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.587+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042d6070 0x55810467d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.587+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042ad1b0 0x55810467d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.587+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x55810467dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.587+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581045f3ea0 con 0x5581042ad1b0 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.587+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5581045e32d0 con 0x5581042b5390 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.587+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5581045e70e0 con 0x5581042d6070 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042ad1b0 0x55810467d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc064e44640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042d6070 0x55810467d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042ad1b0 0x55810467d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45136/0 (socket says 172.21.15.112:45136) 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc064e44640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042d6070 0x55810467d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46798/0 (socket says 172.21.15.112:46798) 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc05ffff640 1 -- 172.21.15.112:0/3667115729 learned_addr learned my addr 172.21.15.112:0/3667115729 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc057fff640 1 --2- 172.21.15.112:0/3667115729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x55810467dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc064e44640 1 -- 172.21.15.112:0/3667115729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 msgr2=0x55810467dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc064e44640 1 --2- 172.21.15.112:0/3667115729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x55810467dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc064e44640 1 -- 172.21.15.112:0/3667115729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042ad1b0 msgr2=0x55810467d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc064e44640 1 --2- 172.21.15.112:0/3667115729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042ad1b0 0x55810467d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc064e44640 1 -- 172.21.15.112:0/3667115729 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558104341650 con 0x5581042d6070 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc05ffff640 1 --2- 172.21.15.112:0/3667115729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042ad1b0 0x55810467d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc05dffb640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc04c0b6680 0x7fc04c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc064e44640 1 --2- 172.21.15.112:0/3667115729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042d6070 0x55810467d2c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc058005fd0 tx=0x7fc058011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58d68bc2778a4f72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc05ffff640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc04c0b6680 0x7fc04c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc0567fc640 1 -- 172.21.15.112:0/3667115729 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058004d20 con 0x5581042d6070 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.588+0000 7fc05dffb640 1 -- 172.21.15.112:0/394651067 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7fc0480e7930 con 0x7fc060109570 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.589+0000 7fc0567fc640 1 -- 172.21.15.112:0/3667115729 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05800f9a0 con 0x5581042d6070 2024-09-21T03:37:48.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.589+0000 7fc0567fc640 1 -- 172.21.15.112:0/3667115729 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058004450 con 0x5581042d6070 2024-09-21T03:37:48.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.589+0000 7fc06ce1b740 1 -- 172.21.15.112:0/3667115729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042d6070 msgr2=0x55810467d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.589+0000 7fc05ffff640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc04c0b6680 0x7fc04c0b8b90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc0500095d0 tx=0x7fc050009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:48.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.589+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/3667115729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042d6070 0x55810467d2c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc058005fd0 tx=0x7fc058011bf0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.589+0000 7fc06ce1b740 1 -- 172.21.15.112:0/3667115729 shutdown_connections 2024-09-21T03:37:48.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.589+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/3667115729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x55810467dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.589+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/3667115729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042ad1b0 0x55810467d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.589+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/3667115729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042d6070 0x55810467d2c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.590+0000 7fc06ce1b740 1 -- 172.21.15.112:0/3667115729 >> 172.21.15.112:0/3667115729 conn(0x5581042ca460 msgr2=0x55810467b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.590+0000 7fc06ce1b740 1 -- 172.21.15.112:0/3667115729 shutdown_connections 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.590+0000 7fc06ce1b740 1 -- 172.21.15.112:0/3667115729 wait complete. 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.590+0000 7fc06ce1b740 1 -- start start 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.590+0000 7fc06ce1b740 1 Processor -- start 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104688950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x558104688e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 0x558104689410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558104444700 con 0x5581042d6070 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55810457e160 con 0x5581042b5390 2024-09-21T03:37:48.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55810468baa0 con 0x5581042ad1b0 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc06ce1b740 10 client.0 _ll_register_callbacks cb 0x5581045979e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104688950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc064e44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 0x558104689410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104688950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46812/0 (socket says 172.21.15.112:46812) 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc05ffff640 1 -- 172.21.15.112:0/1868859007 learned_addr learned my addr 172.21.15.112:0/1868859007 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc057fff640 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x558104688e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc05ffff640 1 -- 172.21.15.112:0/1868859007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 msgr2=0x558104688e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc05ffff640 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x558104688e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc05ffff640 1 -- 172.21.15.112:0/1868859007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 msgr2=0x558104689410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc05ffff640 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 0x558104689410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc05ffff640 1 -- 172.21.15.112:0/1868859007 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581045abc90 con 0x5581042ad1b0 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc064e44640 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 0x558104689410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc05ffff640 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104688950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc050006010 tx=0x7fc050009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a56ac9e9763f1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:48.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc03f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.591+0000 7fc03f7fe640 1 -- 172.21.15.112:0/1868859007 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05004d070 con 0x5581042ad1b0 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc03f7fe640 1 -- 172.21.15.112:0/1868859007 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7fc050004450 con 0x5581042ad1b0 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc06ce1b740 1 -- 172.21.15.112:0/1868859007 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc05800f9a0 con 0x5581042ad1b0 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc03f7fe640 1 -- 172.21.15.112:0/1868859007 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc050059050 con 0x5581042ad1b0 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc06ce1b740 10 client.26545 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc06ce1b740 1 -- 172.21.15.112:0/1868859007 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558104697770 con 0x5581042ad1b0 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc03f7fe640 1 -- 172.21.15.112:0/1868859007 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05800f9a0 con 0x5581042ad1b0 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc03f7fe640 1 -- 172.21.15.112:0/1868859007 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc0500035f0 con 0x5581042ad1b0 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc06ce1b740 10 client.26545 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc06ce1b740 10 client.26545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc06ce1b740 20 client.26545 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5581042949f0 0x55810469ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc06ce1b740 4 client.26545 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:48.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc06ce1b740 1 -- 172.21.15.112:0/1868859007 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581045f5df0 con 0x5581042949f0 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.592+0000 7fc057fff640 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5581042949f0 0x55810469ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.593+0000 7fc03f7fe640 10 client.26545 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.593+0000 7fc057fff640 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5581042949f0 0x55810469ad30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99122e1a8cdb4bb7 server_cookie=9526450f32694984 in_seq=0 out_seq=0 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc03f7fe640 1 -- 172.21.15.112:0/1868859007 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5581045f5df0 con 0x5581042949f0 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc03f7fe640 10 client.26545 handle_command_reply: tid=0 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 1 client.26545 shutdown 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 1 -- 172.21.15.112:0/1868859007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 msgr2=0x558104688950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104688950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc050006010 tx=0x7fc050009260 comp rx=0 tx=0).stop 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 1 -- 172.21.15.112:0/1868859007 shutdown_connections 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5581042949f0 0x55810469ad30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 0x558104689410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc03f7fe640 0 client.26545 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x558104688e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/1868859007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104688950 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 1 -- 172.21.15.112:0/1868859007 >> 172.21.15.112:0/1868859007 conn(0x5581042ca460 msgr2=0x558104680450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:48.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 1 -- 172.21.15.112:0/1868859007 shutdown_connections 2024-09-21T03:37:48.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 1 -- 172.21.15.112:0/1868859007 wait complete. 2024-09-21T03:37:48.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.594+0000 7fc06ce1b740 20 client.26545 trim_cache size 0 max 16384 2024-09-21T03:37:48.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.609+0000 7fc06ce1b740 1 Processor -- start 2024-09-21T03:37:48.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.609+0000 7fc06ce1b740 1 -- start start 2024-09-21T03:37:48.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104688e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x55810467a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 0x55810467aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0601b1fc0 con 0x5581042d6070 2024-09-21T03:37:48.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc0601af890 con 0x5581042b5390 2024-09-21T03:37:48.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc0601ad700 con 0x5581042ad1b0 2024-09-21T03:37:48.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc064e44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 0x55810467aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104688e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc064e44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 0x55810467aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45160/0 (socket says 172.21.15.112:45160) 2024-09-21T03:37:48.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc064e44640 1 -- 172.21.15.112:0/3361395255 learned_addr learned my addr 172.21.15.112:0/3361395255 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:48.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc057fff640 1 --2- 172.21.15.112:0/3361395255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x55810467a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc064e44640 1 -- 172.21.15.112:0/3361395255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 msgr2=0x558104688e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:48.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc064e44640 1 --2- 172.21.15.112:0/3361395255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104688e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc064e44640 1 -- 172.21.15.112:0/3361395255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 msgr2=0x55810467a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:48.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc064e44640 1 --2- 172.21.15.112:0/3361395255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x55810467a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc064e44640 1 -- 172.21.15.112:0/3361395255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558104680120 con 0x5581042d6070 2024-09-21T03:37:48.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.610+0000 7fc064e44640 1 --2- 172.21.15.112:0/3361395255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 0x55810467aca0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc0580040c0 tx=0x7fc058004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d478f49550bc3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:48.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.611+0000 7fc055ffb640 1 -- 172.21.15.112:0/3361395255 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058029020 con 0x5581042d6070 2024-09-21T03:37:48.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.611+0000 7fc055ffb640 1 -- 172.21.15.112:0/3361395255 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05801f040 con 0x5581042d6070 2024-09-21T03:37:48.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.611+0000 7fc055ffb640 1 -- 172.21.15.112:0/3361395255 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05801e040 con 0x5581042d6070 2024-09-21T03:37:48.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.611+0000 7fc06ce1b740 1 -- 172.21.15.112:0/3361395255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 msgr2=0x55810467aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.611+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/3361395255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 0x55810467aca0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc0580040c0 tx=0x7fc058004e20 comp rx=0 tx=0).stop 2024-09-21T03:37:48.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.612+0000 7fc06ce1b740 1 -- 172.21.15.112:0/3361395255 shutdown_connections 2024-09-21T03:37:48.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.612+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/3361395255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042d6070 0x55810467aca0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.612+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/3361395255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042b5390 0x55810467a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.612+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/3361395255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104688e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.612+0000 7fc06ce1b740 1 -- 172.21.15.112:0/3361395255 >> 172.21.15.112:0/3361395255 conn(0x5581042ca460 msgr2=0x558104688700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.612+0000 7fc06ce1b740 1 -- 172.21.15.112:0/3361395255 shutdown_connections 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.612+0000 7fc06ce1b740 1 -- 172.21.15.112:0/3361395255 wait complete. 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.612+0000 7fc06ce1b740 1 -- start start 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.612+0000 7fc06ce1b740 1 Processor -- start 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.612+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104691070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042b5390 0x558104691630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc06ce1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042d6070 0x5581046fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0601add50 con 0x5581042b5390 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc0601af1c0 con 0x5581042d6070 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc06ce1b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc0601ae420 con 0x5581042ad1b0 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc06ce1b740 10 client.0 _ll_register_callbacks cb 0x7fc050009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104691070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc057fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042b5390 0x558104691630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104691070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46828/0 (socket says 172.21.15.112:46828) 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc05ffff640 1 -- 172.21.15.112:0/2888475123 learned_addr learned my addr 172.21.15.112:0/2888475123 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc064e44640 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042d6070 0x5581046fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc057fff640 1 -- 172.21.15.112:0/2888475123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 msgr2=0x558104691070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc057fff640 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104691070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc057fff640 1 -- 172.21.15.112:0/2888475123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042d6070 msgr2=0x5581046fe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc057fff640 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042d6070 0x5581046fe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc057fff640 1 -- 172.21.15.112:0/2888475123 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581046fef00 con 0x5581042b5390 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc064e44640 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042d6070 0x5581046fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc05ffff640 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104691070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc057fff640 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042b5390 0x558104691630 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc0480e50b0 tx=0x7fc0480096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8068ed364fb982de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc03f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc03f7fe640 1 -- 172.21.15.112:0/2888475123 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0480b2290 con 0x5581042b5390 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc03f7fe640 1 -- 172.21.15.112:0/2888475123 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7fc0480a91b0 con 0x5581042b5390 2024-09-21T03:37:48.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.613+0000 7fc06ce1b740 1 -- 172.21.15.112:0/2888475123 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc05801f040 con 0x5581042b5390 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc03f7fe640 1 -- 172.21.15.112:0/2888475123 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc0480b2080 con 0x5581042b5390 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc06ce1b740 10 client.16989 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc06ce1b740 1 -- 172.21.15.112:0/2888475123 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55810468ec00 con 0x5581042b5390 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc03f7fe640 1 -- 172.21.15.112:0/2888475123 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05801f040 con 0x5581042b5390 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc03f7fe640 1 -- 172.21.15.112:0/2888475123 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc0480b3420 con 0x5581042b5390 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc06ce1b740 10 client.16989 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc06ce1b740 10 client.16989 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc06ce1b740 20 client.16989 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x558104269da0 0x558104710680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc06ce1b740 4 client.16989 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc06ce1b740 1 -- 172.21.15.112:0/2888475123 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581045f18a0 con 0x558104269da0 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.614+0000 7fc05ffff640 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x558104269da0 0x558104710680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:48.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc03f7fe640 10 client.16989 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc05ffff640 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x558104269da0 0x558104710680 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b70339e38da19a6 server_cookie=55210b068432e14c in_seq=0 out_seq=0 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc03f7fe640 1 -- 172.21.15.112:0/2888475123 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581045f18a0 con 0x558104269da0 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc03f7fe640 10 client.16989 handle_command_reply: tid=0 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 1 client.16989 shutdown 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 1 -- 172.21.15.112:0/2888475123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042b5390 msgr2=0x558104691630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042b5390 0x558104691630 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc0480e50b0 tx=0x7fc0480096a0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 1 -- 172.21.15.112:0/2888475123 shutdown_connections 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x558104269da0 0x558104710680 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581042d6070 0x5581046fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc03f7fe640 0 client.16989 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581042b5390 0x558104691630 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 1 --2- 172.21.15.112:0/2888475123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581042ad1b0 0x558104691070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 1 -- 172.21.15.112:0/2888475123 >> 172.21.15.112:0/2888475123 conn(0x5581042ca460 msgr2=0x55810467e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:48.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 1 -- 172.21.15.112:0/2888475123 shutdown_connections 2024-09-21T03:37:48.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 1 -- 172.21.15.112:0/2888475123 wait complete. 2024-09-21T03:37:48.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.615+0000 7fc06ce1b740 20 client.16989 trim_cache size 0 max 16384 2024-09-21T03:37:48.624 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:37:48.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 -- 172.21.15.112:0/394651067 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc04c0b6680 msgr2=0x7fc04c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc04c0b6680 0x7fc04c0b8b90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc0500095d0 tx=0x7fc050009560 comp rx=0 tx=0).stop 2024-09-21T03:37:48.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 -- 172.21.15.112:0/394651067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060109570 msgr2=0x7fc06019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:48.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060109570 0x7fc06019ab70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e9f0 tx=0x7fc04800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 -- 172.21.15.112:0/394651067 shutdown_connections 2024-09-21T03:37:48.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc04c0b6680 0x7fc04c0b8b90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc060109ec0 0x7fc06019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060109570 0x7fc06019ab70 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 --2- 172.21.15.112:0/394651067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc060103c30 0x7fc06019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:48.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 -- 172.21.15.112:0/394651067 >> 172.21.15.112:0/394651067 conn(0x7fc06007b8d0 msgr2=0x7fc060107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:48.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 -- 172.21.15.112:0/394651067 shutdown_connections 2024-09-21T03:37:48.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:48.616+0000 7fc0668a7640 1 -- 172.21.15.112:0/394651067 wait complete. 2024-09-21T03:37:49.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 932413 10.65 MB/sec execute 349 sec latency 35.086 ms 2024-09-21T03:37:49.460 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:37:49.716 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:50.120 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:49 smithi112 ceph-mon[31442]: pgmap v588: 97 pgs: 97 active+clean; 704 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 216 op/s 2024-09-21T03:37:50.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:49 smithi203 ceph-mon[34723]: pgmap v588: 97 pgs: 97 active+clean; 704 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 216 op/s 2024-09-21T03:37:50.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:49 smithi187 ceph-mon[34603]: pgmap v588: 97 pgs: 97 active+clean; 704 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 216 op/s 2024-09-21T03:37:50.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.211+0000 7fa2db2b8640 1 -- 172.21.15.112:0/3977788585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4102a60 msgr2=0x7fa2d4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.211+0000 7fa2db2b8640 1 --2- 172.21.15.112:0/3977788585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4102a60 0x7fa2d4102e70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009900 tx=0x7fa2d002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.211+0000 7fa2db2b8640 1 -- 172.21.15.112:0/3977788585 shutdown_connections 2024-09-21T03:37:50.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.211+0000 7fa2db2b8640 1 --2- 172.21.15.112:0/3977788585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2d4108cf0 0x7fa2d410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.211+0000 7fa2db2b8640 1 --2- 172.21.15.112:0/3977788585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d41083a0 0x7fa2d41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.211+0000 7fa2db2b8640 1 --2- 172.21.15.112:0/3977788585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4102a60 0x7fa2d4102e70 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.211+0000 7fa2db2b8640 1 -- 172.21.15.112:0/3977788585 >> 172.21.15.112:0/3977788585 conn(0x7fa2d4079730 msgr2=0x7fa2d4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.211+0000 7fa2db2b8640 1 -- 172.21.15.112:0/3977788585 shutdown_connections 2024-09-21T03:37:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.211+0000 7fa2db2b8640 1 -- 172.21.15.112:0/3977788585 wait complete. 2024-09-21T03:37:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2db2b8640 1 Processor -- start 2024-09-21T03:37:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2db2b8640 1 -- start start 2024-09-21T03:37:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2db2b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4102a60 0x7fa2d41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2db2b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2d41083a0 0x7fa2d4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2db2b8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d4108cf0 0x7fa2d419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2db2b8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41118b0 con 0x7fa2d4102a60 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2db2b8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4111730 con 0x7fa2d41083a0 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2db2b8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4111a30 con 0x7fa2d4108cf0 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2d9054640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4102a60 0x7fa2d41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2d9054640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4102a60 0x7fa2d41982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45188/0 (socket says 172.21.15.112:45188) 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2d9054640 1 -- 172.21.15.112:0/1807773111 learned_addr learned my addr 172.21.15.112:0/1807773111 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.212+0000 7fa2d8853640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2d41083a0 0x7fa2d4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2d9855640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d4108cf0 0x7fa2d419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2d9054640 1 -- 172.21.15.112:0/1807773111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d4108cf0 msgr2=0x7fa2d419db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2d9054640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d4108cf0 0x7fa2d419db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2d9054640 1 -- 172.21.15.112:0/1807773111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2d41083a0 msgr2=0x7fa2d4198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2d9054640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2d41083a0 0x7fa2d4198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2d9054640 1 -- 172.21.15.112:0/1807773111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d419e230 con 0x7fa2d4102a60 2024-09-21T03:37:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2d9855640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d4108cf0 0x7fa2d419db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2d9054640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4102a60 0x7fa2d41982c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005940 tx=0x7fa2d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2ca7fc640 1 -- 172.21.15.112:0/1807773111 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d003f030 con 0x7fa2d4102a60 2024-09-21T03:37:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2ca7fc640 1 -- 172.21.15.112:0/1807773111 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d0003cb0 con 0x7fa2d4102a60 2024-09-21T03:37:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2db2b8640 1 -- 172.21.15.112:0/1807773111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d419e4c0 con 0x7fa2d4102a60 2024-09-21T03:37:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2ca7fc640 1 -- 172.21.15.112:0/1807773111 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0035070 con 0x7fa2d4102a60 2024-09-21T03:37:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.213+0000 7fa2db2b8640 1 -- 172.21.15.112:0/1807773111 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d41a4c80 con 0x7fa2d4102a60 2024-09-21T03:37:50.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.216+0000 7fa2ca7fc640 1 -- 172.21.15.112:0/1807773111 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa2d0042030 con 0x7fa2d4102a60 2024-09-21T03:37:50.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.216+0000 7fa2e182c740 1 Processor -- start 2024-09-21T03:37:50.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.216+0000 7fa2e182c740 1 -- start start 2024-09-21T03:37:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179326070 0x5591796cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591792fd1b0 0x5591796cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179305390 0x5591796cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2e182c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559179643ea0 con 0x559179305390 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2e182c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5591796332d0 con 0x559179326070 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2e182c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5591796370e0 con 0x5591792fd1b0 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9855640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179326070 0x5591796cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9855640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179326070 0x5591796cd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:38314/0 (socket says 172.21.15.112:38314) 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9855640 1 -- 172.21.15.112:0/1933776154 learned_addr learned my addr 172.21.15.112:0/1933776154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d8853640 1 --2- 172.21.15.112:0/1933776154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179305390 0x5591796cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9054640 1 --2- 172.21.15.112:0/1933776154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591792fd1b0 0x5591796cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9855640 1 -- 172.21.15.112:0/1933776154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591792fd1b0 msgr2=0x5591796cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9855640 1 --2- 172.21.15.112:0/1933776154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591792fd1b0 0x5591796cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9855640 1 -- 172.21.15.112:0/1933776154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179305390 msgr2=0x5591796cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9855640 1 --2- 172.21.15.112:0/1933776154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179305390 0x5591796cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9855640 1 -- 172.21.15.112:0/1933776154 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559179391650 con 0x559179326070 2024-09-21T03:37:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9054640 1 --2- 172.21.15.112:0/1933776154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591792fd1b0 0x5591796cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:37:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2ca7fc640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2c00b6680 0x7fa2c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9855640 1 --2- 172.21.15.112:0/1933776154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179326070 0x5591796cd2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00dd50 tx=0x7fa2cc00b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a14cdb78f7bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.217+0000 7fa2d9054640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2c00b6680 0x7fa2c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.218+0000 7fa2ca7fc640 1 -- 172.21.15.112:0/1807773111 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7fa2d0107fb0 con 0x7fa2d4102a60 2024-09-21T03:37:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.218+0000 7fa2bf7fe640 1 -- 172.21.15.112:0/1933776154 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc017780 con 0x559179326070 2024-09-21T03:37:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.218+0000 7fa2bf7fe640 1 -- 172.21.15.112:0/1933776154 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc01d070 con 0x559179326070 2024-09-21T03:37:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.218+0000 7fa2bf7fe640 1 -- 172.21.15.112:0/1933776154 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc018ce0 con 0x559179326070 2024-09-21T03:37:50.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2e182c740 1 -- 172.21.15.112:0/1933776154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179326070 msgr2=0x5591796cd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2d9054640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2c00b6680 0x7fa2c00b8b90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa2d01057e0 tx=0x7fa2d00044e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2e182c740 1 --2- 172.21.15.112:0/1933776154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179326070 0x5591796cd2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00dd50 tx=0x7fa2cc00b9a0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2e182c740 1 -- 172.21.15.112:0/1933776154 shutdown_connections 2024-09-21T03:37:50.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2e182c740 1 --2- 172.21.15.112:0/1933776154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179305390 0x5591796cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2e182c740 1 --2- 172.21.15.112:0/1933776154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591792fd1b0 0x5591796cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2e182c740 1 --2- 172.21.15.112:0/1933776154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179326070 0x5591796cd2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2e182c740 1 -- 172.21.15.112:0/1933776154 >> 172.21.15.112:0/1933776154 conn(0x55917931a460 msgr2=0x5591796cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2e182c740 1 -- 172.21.15.112:0/1933776154 shutdown_connections 2024-09-21T03:37:50.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2e182c740 1 -- 172.21.15.112:0/1933776154 wait complete. 2024-09-21T03:37:50.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2e182c740 1 -- start start 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.219+0000 7fa2e182c740 1 Processor -- start 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591792fd1b0 0x5591796d8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559179305390 0x5591796d8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 0x5591796e4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2e182c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559179494700 con 0x559179326070 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2e182c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5591795ce160 con 0x5591792fd1b0 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2e182c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5591796dbaa0 con 0x559179305390 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2e182c740 10 client.0 _ll_register_callbacks cb 0x5591795e79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2d9855640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591792fd1b0 0x5591796d8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2d8853640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559179305390 0x5591796d8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2d9855640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591792fd1b0 0x5591796d8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:38330/0 (socket says 172.21.15.112:38330) 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2d9855640 1 -- 172.21.15.112:0/2878033908 learned_addr learned my addr 172.21.15.112:0/2878033908 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.220+0000 7fa2d9855640 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 0x5591796e4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2d9855640 1 -- 172.21.15.112:0/2878033908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559179305390 msgr2=0x5591796d8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:50.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2d9855640 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559179305390 0x5591796d8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2d9855640 1 -- 172.21.15.112:0/2878033908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 msgr2=0x5591796e4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2d9855640 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 0x5591796e4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2d9855640 1 -- 172.21.15.112:0/2878033908 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591795fbc90 con 0x5591792fd1b0 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2d8853640 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559179305390 0x5591796d8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2d9855640 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591792fd1b0 0x5591796d8960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc01a020 tx=0x7fa2cc018880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8388b6327ac2d490 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2bcff9640 1 -- 172.21.15.112:0/2878033908 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0177c0 con 0x5591792fd1b0 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2bcff9640 1 -- 172.21.15.112:0/2878033908 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7fa2cc017960 con 0x5591792fd1b0 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2e182c740 1 -- 172.21.15.112:0/2878033908 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2cc01d070 con 0x5591792fd1b0 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2bcff9640 1 -- 172.21.15.112:0/2878033908 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa2cc017b60 con 0x5591792fd1b0 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2e182c740 10 client.26558 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2e182c740 1 -- 172.21.15.112:0/2878033908 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591796e77c0 con 0x5591792fd1b0 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.221+0000 7fa2bcff9640 1 -- 172.21.15.112:0/2878033908 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc01d070 con 0x5591792fd1b0 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.222+0000 7fa2bcff9640 1 -- 172.21.15.112:0/2878033908 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa2cc022ba0 con 0x5591792fd1b0 2024-09-21T03:37:50.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.222+0000 7fa2e182c740 10 client.26558 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.222+0000 7fa2e182c740 10 client.26558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.222+0000 7fa2e182c740 20 client.26558 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.222+0000 7fa2e182c740 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5591792e49f0 0x5591796eab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.222+0000 7fa2e182c740 4 client.26558 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.222+0000 7fa2e182c740 1 -- 172.21.15.112:0/2878033908 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559179645df0 con 0x5591792e49f0 2024-09-21T03:37:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.222+0000 7fa2d8853640 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5591792e49f0 0x5591796eab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.222+0000 7fa2bcff9640 10 client.26558 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:50.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.222+0000 7fa2d8853640 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5591792e49f0 0x5591796eab20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5b0dee7e74cbe2 server_cookie=594da052097bafe7 in_seq=0 out_seq=0 2024-09-21T03:37:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.223+0000 7fa2bcff9640 1 -- 172.21.15.112:0/2878033908 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559179645df0 con 0x5591792e49f0 2024-09-21T03:37:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.223+0000 7fa2bcff9640 10 client.26558 handle_command_reply: tid=0 2024-09-21T03:37:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.223+0000 7fa2e182c740 1 client.26558 shutdown 2024-09-21T03:37:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.223+0000 7fa2e182c740 1 -- 172.21.15.112:0/2878033908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591792fd1b0 msgr2=0x5591796d8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.223+0000 7fa2e182c740 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591792fd1b0 0x5591796d8960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc01a020 tx=0x7fa2cc018880 comp rx=0 tx=0).stop 2024-09-21T03:37:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.223+0000 7fa2e182c740 1 -- 172.21.15.112:0/2878033908 shutdown_connections 2024-09-21T03:37:50.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.223+0000 7fa2e182c740 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5591792e49f0 0x5591796eab20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.224+0000 7fa2e182c740 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 0x5591796e4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.224+0000 7fa2e182c740 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559179305390 0x5591796d8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.224+0000 7fa2e182c740 1 --2- 172.21.15.112:0/2878033908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591792fd1b0 0x5591796d8960 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.224+0000 7fa2e182c740 1 -- 172.21.15.112:0/2878033908 >> 172.21.15.112:0/2878033908 conn(0x55917931a460 msgr2=0x5591796d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.224+0000 7fa2bcff9640 0 client.26558 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.224+0000 7fa2e182c740 1 -- 172.21.15.112:0/2878033908 shutdown_connections 2024-09-21T03:37:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.224+0000 7fa2e182c740 1 -- 172.21.15.112:0/2878033908 wait complete. 2024-09-21T03:37:50.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.224+0000 7fa2e182c740 20 client.26558 trim_cache size 0 max 16384 2024-09-21T03:37:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2e182c740 1 Processor -- start 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2e182c740 1 -- start start 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591792fd1b0 0x5591796d7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179305390 0x5591796da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 0x5591796ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2e182c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41afd90 con 0x559179326070 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2e182c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41ad660 con 0x559179305390 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2e182c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41ab4d0 con 0x5591792fd1b0 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2d9855640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 0x5591796ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2d9855640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 0x5591796ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45212/0 (socket says 172.21.15.112:45212) 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2d9855640 1 -- 172.21.15.112:0/66383180 learned_addr learned my addr 172.21.15.112:0/66383180 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2d8853640 1 --2- 172.21.15.112:0/66383180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179305390 0x5591796da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.237+0000 7fa2d9855640 1 --2- 172.21.15.112:0/66383180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591792fd1b0 0x5591796d7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.238+0000 7fa2d9855640 1 -- 172.21.15.112:0/66383180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591792fd1b0 msgr2=0x5591796d7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.238+0000 7fa2d9855640 1 --2- 172.21.15.112:0/66383180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591792fd1b0 0x5591796d7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.238+0000 7fa2d9855640 1 -- 172.21.15.112:0/66383180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179305390 msgr2=0x5591796da590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.238+0000 7fa2d9855640 1 --2- 172.21.15.112:0/66383180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179305390 0x5591796da590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.238+0000 7fa2d9855640 1 -- 172.21.15.112:0/66383180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591796ce100 con 0x559179326070 2024-09-21T03:37:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.238+0000 7fa2d9855640 1 --2- 172.21.15.112:0/66383180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 0x5591796ca760 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc019e20 tx=0x7fa2cc00b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d75aa821a52bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.238+0000 7fa2beffd640 1 -- 172.21.15.112:0/66383180 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0177c0 con 0x559179326070 2024-09-21T03:37:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.238+0000 7fa2beffd640 1 -- 172.21.15.112:0/66383180 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc01d030 con 0x559179326070 2024-09-21T03:37:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.238+0000 7fa2beffd640 1 -- 172.21.15.112:0/66383180 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc023cf0 con 0x559179326070 2024-09-21T03:37:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.239+0000 7fa2e182c740 1 -- 172.21.15.112:0/66383180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 msgr2=0x5591796ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.239+0000 7fa2e182c740 1 --2- 172.21.15.112:0/66383180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 0x5591796ca760 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc019e20 tx=0x7fa2cc00b9a0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.239+0000 7fa2e182c740 1 -- 172.21.15.112:0/66383180 shutdown_connections 2024-09-21T03:37:50.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.239+0000 7fa2e182c740 1 --2- 172.21.15.112:0/66383180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559179326070 0x5591796ca760 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.239+0000 7fa2e182c740 1 --2- 172.21.15.112:0/66383180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179305390 0x5591796da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.239+0000 7fa2e182c740 1 --2- 172.21.15.112:0/66383180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5591792fd1b0 0x5591796d7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.239+0000 7fa2e182c740 1 -- 172.21.15.112:0/66383180 >> 172.21.15.112:0/66383180 conn(0x55917931a460 msgr2=0x5591796d9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.239+0000 7fa2e182c740 1 -- 172.21.15.112:0/66383180 shutdown_connections 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.239+0000 7fa2e182c740 1 -- 172.21.15.112:0/66383180 wait complete. 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.239+0000 7fa2e182c740 1 -- start start 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.239+0000 7fa2e182c740 1 Processor -- start 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591792fd1b0 0x5591796eb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179305390 0x5591796ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2e182c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559179326070 0x55917974d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2e182c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41abb20 con 0x5591792fd1b0 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2e182c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41acf90 con 0x559179305390 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2e182c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41ac1f0 con 0x559179326070 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2e182c740 10 client.0 _ll_register_callbacks cb 0x7fa2c40093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:50.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2d9855640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591792fd1b0 0x5591796eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2d9855640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591792fd1b0 0x5591796eb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45226/0 (socket says 172.21.15.112:45226) 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2d9855640 1 -- 172.21.15.112:0/3488221326 learned_addr learned my addr 172.21.15.112:0/3488221326 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2d8853640 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179305390 0x5591796ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2d9855640 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559179326070 0x55917974d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2d9855640 1 -- 172.21.15.112:0/3488221326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559179326070 msgr2=0x55917974d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2d9855640 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559179326070 0x55917974d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2d9855640 1 -- 172.21.15.112:0/3488221326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179305390 msgr2=0x5591796ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2d9855640 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179305390 0x5591796ebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2d9855640 1 -- 172.21.15.112:0/3488221326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917974d940 con 0x5591792fd1b0 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2d9855640 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591792fd1b0 0x5591796eb590 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc017580 tx=0x7fa2cc00d2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8ff21e68824c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.240+0000 7fa2bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2bcff9640 1 -- 172.21.15.112:0/3488221326 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc004e10 con 0x5591792fd1b0 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2bcff9640 1 -- 172.21.15.112:0/3488221326 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7fa2cc023a00 con 0x5591792fd1b0 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2e182c740 1 -- 172.21.15.112:0/3488221326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2cc01d030 con 0x5591792fd1b0 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2bcff9640 1 -- 172.21.15.112:0/3488221326 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa2cc017d00 con 0x5591792fd1b0 2024-09-21T03:37:50.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2e182c740 10 client.17007 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2e182c740 1 -- 172.21.15.112:0/3488221326 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559179754d10 con 0x5591792fd1b0 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2bcff9640 1 -- 172.21.15.112:0/3488221326 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc01d030 con 0x5591792fd1b0 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2bcff9640 1 -- 172.21.15.112:0/3488221326 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa2cc022940 con 0x5591792fd1b0 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2e182c740 10 client.17007 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2e182c740 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2e182c740 20 client.17007 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2e182c740 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5591792b9da0 0x559179757330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2e182c740 4 client.17007 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.241+0000 7fa2e182c740 1 -- 172.21.15.112:0/3488221326 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591796418a0 con 0x5591792b9da0 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2d8853640 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5591792b9da0 0x559179757330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2bcff9640 10 client.17007 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2d8853640 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5591792b9da0 0x559179757330 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d61dbc069bd3197f server_cookie=46af6391aea78ba9 in_seq=0 out_seq=0 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2bcff9640 1 -- 172.21.15.112:0/3488221326 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591796418a0 con 0x5591792b9da0 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2bcff9640 10 client.17007 handle_command_reply: tid=0 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2e182c740 1 client.17007 shutdown 2024-09-21T03:37:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2e182c740 1 -- 172.21.15.112:0/3488221326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591792fd1b0 msgr2=0x5591796eb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2e182c740 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591792fd1b0 0x5591796eb590 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc017580 tx=0x7fa2cc00d2d0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2e182c740 1 -- 172.21.15.112:0/3488221326 shutdown_connections 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2e182c740 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5591792b9da0 0x559179757330 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2e182c740 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559179326070 0x55917974d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2e182c740 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559179305390 0x5591796ebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2e182c740 1 --2- 172.21.15.112:0/3488221326 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5591792fd1b0 0x5591796eb590 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2bcff9640 0 client.17007 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.242+0000 7fa2e182c740 1 -- 172.21.15.112:0/3488221326 >> 172.21.15.112:0/3488221326 conn(0x55917931a460 msgr2=0x5591796cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2e182c740 1 -- 172.21.15.112:0/3488221326 shutdown_connections 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2e182c740 1 -- 172.21.15.112:0/3488221326 wait complete. 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2e182c740 20 client.17007 trim_cache size 0 max 16384 2024-09-21T03:37:50.249 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:37:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 -- 172.21.15.112:0/1807773111 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2c00b6680 msgr2=0x7fa2c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2c00b6680 0x7fa2c00b8b90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa2d01057e0 tx=0x7fa2d00044e0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 -- 172.21.15.112:0/1807773111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4102a60 msgr2=0x7fa2d41982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4102a60 0x7fa2d41982c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005940 tx=0x7fa2d00042f0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 -- 172.21.15.112:0/1807773111 shutdown_connections 2024-09-21T03:37:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa2c00b6680 0x7fa2c00b8b90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d4108cf0 0x7fa2d419db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2d41083a0 0x7fa2d4198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 --2- 172.21.15.112:0/1807773111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2d4102a60 0x7fa2d41982c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 -- 172.21.15.112:0/1807773111 >> 172.21.15.112:0/1807773111 conn(0x7fa2d4079730 msgr2=0x7fa2d41013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 -- 172.21.15.112:0/1807773111 shutdown_connections 2024-09-21T03:37:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:50.243+0000 7fa2db2b8640 1 -- 172.21.15.112:0/1807773111 wait complete. 2024-09-21T03:37:50.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 934731 10.66 MB/sec execute 350 sec latency 15.833 ms 2024-09-21T03:37:51.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 936243 10.65 MB/sec execute 351 sec latency 30.345 ms 2024-09-21T03:37:52.003 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:37:52.004 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:37:52.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:51 smithi203 ceph-mon[34723]: pgmap v589: 97 pgs: 97 active+clean; 709 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2024-09-21T03:37:52.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:51 smithi187 ceph-mon[34603]: pgmap v589: 97 pgs: 97 active+clean; 709 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2024-09-21T03:37:52.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 938377 10.66 MB/sec execute 352 sec latency 22.834 ms 2024-09-21T03:37:52.276 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:52.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:51 smithi112 ceph-mon[31442]: pgmap v589: 97 pgs: 97 active+clean; 709 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2024-09-21T03:37:52.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.823+0000 7f3a487f1640 1 -- 172.21.15.112:0/3687899299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a400760a0 msgr2=0x7f3a4010fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.823+0000 7f3a487f1640 1 --2- 172.21.15.112:0/3687899299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a400760a0 0x7f3a4010fc00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009900 tx=0x7f3a3002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.823+0000 7f3a487f1640 1 -- 172.21.15.112:0/3687899299 shutdown_connections 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.823+0000 7f3a487f1640 1 --2- 172.21.15.112:0/3687899299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a40110140 0x7f3a40112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.823+0000 7f3a487f1640 1 --2- 172.21.15.112:0/3687899299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a400760a0 0x7f3a4010fc00 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.823+0000 7f3a487f1640 1 --2- 172.21.15.112:0/3687899299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a40075750 0x7f3a40075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.823+0000 7f3a487f1640 1 -- 172.21.15.112:0/3687899299 >> 172.21.15.112:0/3687899299 conn(0x7f3a400ff610 msgr2=0x7f3a40101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.823+0000 7f3a487f1640 1 -- 172.21.15.112:0/3687899299 shutdown_connections 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.823+0000 7f3a487f1640 1 -- 172.21.15.112:0/3687899299 wait complete. 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a487f1640 1 Processor -- start 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a487f1640 1 -- start start 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a487f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a40075750 0x7f3a4019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a487f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a400760a0 0x7f3a4019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a487f1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a40110140 0x7f3a401a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a487f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a40115cb0 con 0x7f3a400760a0 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a487f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a40115b30 con 0x7f3a40075750 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a487f1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a40115e30 con 0x7f3a40110140 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a45d8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a400760a0 0x7f3a4019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a45d8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a400760a0 0x7f3a4019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45242/0 (socket says 172.21.15.112:45242) 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a45d8c640 1 -- 172.21.15.112:0/3146616898 learned_addr learned my addr 172.21.15.112:0/3146616898 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a4658d640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a40075750 0x7f3a4019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a46d8e640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a40110140 0x7f3a401a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a45d8c640 1 -- 172.21.15.112:0/3146616898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a40110140 msgr2=0x7f3a401a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a45d8c640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a40110140 0x7f3a401a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a45d8c640 1 -- 172.21.15.112:0/3146616898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a40075750 msgr2=0x7f3a4019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a45d8c640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a40075750 0x7f3a4019c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a45d8c640 1 -- 172.21.15.112:0/3146616898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a401a25f0 con 0x7f3a400760a0 2024-09-21T03:37:52.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.824+0000 7f3a4658d640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a40075750 0x7f3a4019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:52.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.825+0000 7f3a45d8c640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a400760a0 0x7f3a4019cc20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3a3002a930 tx=0x7f3a30031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.825+0000 7f3a377fe640 1 -- 172.21.15.112:0/3146616898 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3003f030 con 0x7f3a400760a0 2024-09-21T03:37:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.825+0000 7f3a487f1640 1 -- 172.21.15.112:0/3146616898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a401a2880 con 0x7f3a400760a0 2024-09-21T03:37:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.825+0000 7f3a377fe640 1 -- 172.21.15.112:0/3146616898 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a30004490 con 0x7f3a400760a0 2024-09-21T03:37:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.825+0000 7f3a377fe640 1 -- 172.21.15.112:0/3146616898 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30035070 con 0x7f3a400760a0 2024-09-21T03:37:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.825+0000 7f3a487f1640 1 -- 172.21.15.112:0/3146616898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a401a9040 con 0x7f3a400760a0 2024-09-21T03:37:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.827+0000 7f3a377fe640 1 -- 172.21.15.112:0/3146616898 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3a30042030 con 0x7f3a400760a0 2024-09-21T03:37:52.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.828+0000 7f3a4ed65740 1 Processor -- start 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a4ed65740 1 -- start start 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 0x562698bffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698827490 0x562698c00180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c006c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562698b6de60 con 0x562698850350 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562698b5d290 con 0x56269882f670 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a377fe640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a2c0b6680 0x7f3a2c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562698b610a0 con 0x562698827490 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a4658d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698827490 0x562698c00180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a45d8c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a4658d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698827490 0x562698c00180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46910/0 (socket says 172.21.15.112:46910) 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a4658d640 1 -- 172.21.15.112:0/3731770737 learned_addr learned my addr 172.21.15.112:0/3731770737 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a46d8e640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a2c0b6680 0x7f3a2c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.829+0000 7f3a46d8e640 1 --2- 172.21.15.112:0/3731770737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 0x562698bffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.830+0000 7f3a377fe640 1 -- 172.21.15.112:0/3146616898 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7f3a30107a00 con 0x7f3a400760a0 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.830+0000 7f3a4658d640 1 -- 172.21.15.112:0/3731770737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 msgr2=0x562698c006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.830+0000 7f3a4658d640 1 --2- 172.21.15.112:0/3731770737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.830+0000 7f3a4658d640 1 -- 172.21.15.112:0/3731770737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 msgr2=0x562698bffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.830+0000 7f3a4658d640 1 --2- 172.21.15.112:0/3731770737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 0x562698bffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.830+0000 7f3a4658d640 1 -- 172.21.15.112:0/3731770737 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626988bb930 con 0x562698827490 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.830+0000 7f3a4658d640 1 --2- 172.21.15.112:0/3731770737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698827490 0x562698c00180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c011540 tx=0x7f3a3c011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70868bd6e0752cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.830+0000 7f3a46d8e640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a2c0b6680 0x7f3a2c0b8b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3a38005fd0 tx=0x7f3a38004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.830+0000 7f3a34ff9640 1 -- 172.21.15.112:0/3731770737 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00f640 con 0x562698827490 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.830+0000 7f3a34ff9640 1 -- 172.21.15.112:0/3731770737 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3c00fc90 con 0x562698827490 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.830+0000 7f3a34ff9640 1 -- 172.21.15.112:0/3731770737 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c019c60 con 0x562698827490 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.831+0000 7f3a4ed65740 1 -- 172.21.15.112:0/3731770737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698827490 msgr2=0x562698c00180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.831+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/3731770737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698827490 0x562698c00180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c011540 tx=0x7f3a3c011a00 comp rx=0 tx=0).stop 2024-09-21T03:37:52.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.831+0000 7f3a4ed65740 1 -- 172.21.15.112:0/3731770737 shutdown_connections 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.831+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/3731770737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.831+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/3731770737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698827490 0x562698c00180 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.831+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/3731770737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 0x562698bffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.831+0000 7f3a4ed65740 1 -- 172.21.15.112:0/3731770737 >> 172.21.15.112:0/3731770737 conn(0x562698844740 msgr2=0x562698bfdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.831+0000 7f3a4ed65740 1 -- 172.21.15.112:0/3731770737 shutdown_connections 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.831+0000 7f3a4ed65740 1 -- 172.21.15.112:0/3731770737 wait complete. 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4ed65740 1 -- start start 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4ed65740 1 Processor -- start 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c0b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c0bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698850350 0x562698c17740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5626989be890 con 0x562698827490 2024-09-21T03:37:52.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562698af7d40 con 0x56269882f670 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562698c0e420 con 0x562698850350 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4658d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c0b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4ed65740 10 client.0 _ll_register_callbacks cb 0x562698b5bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4658d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c0b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45266/0 (socket says 172.21.15.112:45266) 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a4658d640 1 -- 172.21.15.112:0/1765372515 learned_addr learned my addr 172.21.15.112:0/1765372515 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a46d8e640 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698850350 0x562698c17740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.832+0000 7f3a45d8c640 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c0bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a4658d640 1 -- 172.21.15.112:0/1765372515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698850350 msgr2=0x562698c17740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a4658d640 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698850350 0x562698c17740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a4658d640 1 -- 172.21.15.112:0/1765372515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 msgr2=0x562698c0bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a4658d640 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c0bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a4658d640 1 -- 172.21.15.112:0/1765372515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562698c17cd0 con 0x562698827490 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a4658d640 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c0b810 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009350 tx=0x7f3a3c019c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75ca6bbaa1dd3be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:52.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a29ffb640 1 -- 172.21.15.112:0/1765372515 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c019e60 con 0x562698827490 2024-09-21T03:37:52.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a29ffb640 1 -- 172.21.15.112:0/1765372515 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7f3a3c00f850 con 0x562698827490 2024-09-21T03:37:52.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1765372515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a3c00fc90 con 0x562698827490 2024-09-21T03:37:52.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a29ffb640 1 -- 172.21.15.112:0/1765372515 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3a3c030050 con 0x562698827490 2024-09-21T03:37:52.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a4ed65740 10 client.17019 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:52.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.833+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1765372515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562698c1a5e0 con 0x562698827490 2024-09-21T03:37:52.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.834+0000 7f3a29ffb640 1 -- 172.21.15.112:0/1765372515 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3c00fc90 con 0x562698827490 2024-09-21T03:37:52.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.834+0000 7f3a29ffb640 1 -- 172.21.15.112:0/1765372515 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3a3c004920 con 0x562698827490 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.834+0000 7f3a4ed65740 10 client.17019 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.834+0000 7f3a4ed65740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.834+0000 7f3a4ed65740 20 client.17019 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.834+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56269880ecd0 0x562698c1d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.834+0000 7f3a4ed65740 4 client.17019 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.834+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1765372515 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562698b6fdb0 con 0x56269880ecd0 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.834+0000 7f3a45d8c640 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56269880ecd0 0x562698c1d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.835+0000 7f3a29ffb640 10 client.17019 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.835+0000 7f3a45d8c640 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56269880ecd0 0x562698c1d9f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f3c7218b5dfdfe server_cookie=356bfae8b29c7c73 in_seq=0 out_seq=0 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.836+0000 7f3a29ffb640 1 -- 172.21.15.112:0/1765372515 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562698b6fdb0 con 0x56269880ecd0 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.836+0000 7f3a29ffb640 10 client.17019 handle_command_reply: tid=0 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.836+0000 7f3a4ed65740 1 client.17019 shutdown 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1765372515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 msgr2=0x562698c0b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c0b810 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009350 tx=0x7f3a3c019c70 comp rx=0 tx=0).stop 2024-09-21T03:37:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1765372515 shutdown_connections 2024-09-21T03:37:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56269880ecd0 0x562698c1d9f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698850350 0x562698c17740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c0bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/1765372515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c0b810 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1765372515 >> 172.21.15.112:0/1765372515 conn(0x562698844740 msgr2=0x562698c02e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a29ffb640 0 client.17019 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1765372515 shutdown_connections 2024-09-21T03:37:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1765372515 wait complete. 2024-09-21T03:37:52.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.837+0000 7f3a4ed65740 20 client.17019 trim_cache size 0 max 16384 2024-09-21T03:37:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a4ed65740 1 Processor -- start 2024-09-21T03:37:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a4ed65740 1 -- start start 2024-09-21T03:37:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562698827490 0x562698c05460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56269882f670 0x562698c17640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 0x562698c0aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a401afcc0 con 0x562698850350 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a401ad590 con 0x562698827490 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a401ab440 con 0x56269882f670 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a46d8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 0x562698c0aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a46d8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 0x562698c0aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45274/0 (socket says 172.21.15.112:45274) 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a46d8e640 1 -- 172.21.15.112:0/1029977152 learned_addr learned my addr 172.21.15.112:0/1029977152 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a4658d640 1 --2- 172.21.15.112:0/1029977152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562698827490 0x562698c05460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a45d8c640 1 --2- 172.21.15.112:0/1029977152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56269882f670 0x562698c17640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a46d8e640 1 -- 172.21.15.112:0/1029977152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56269882f670 msgr2=0x562698c17640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a46d8e640 1 --2- 172.21.15.112:0/1029977152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56269882f670 0x562698c17640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a46d8e640 1 -- 172.21.15.112:0/1029977152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562698827490 msgr2=0x562698c05460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a46d8e640 1 --2- 172.21.15.112:0/1029977152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562698827490 0x562698c05460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a46d8e640 1 -- 172.21.15.112:0/1029977152 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562698bef520 con 0x562698850350 2024-09-21T03:37:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.855+0000 7f3a4658d640 1 --2- 172.21.15.112:0/1029977152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562698827490 0x562698c05460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:37:52.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.856+0000 7f3a46d8e640 1 --2- 172.21.15.112:0/1029977152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 0x562698c0aea0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800fba0 tx=0x7f3a38012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd9e8c4631b9fa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.856+0000 7f3a2bfff640 1 -- 172.21.15.112:0/1029977152 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38019040 con 0x562698850350 2024-09-21T03:37:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.856+0000 7f3a2bfff640 1 -- 172.21.15.112:0/1029977152 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3800fe10 con 0x562698850350 2024-09-21T03:37:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.856+0000 7f3a2bfff640 1 -- 172.21.15.112:0/1029977152 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38020d10 con 0x562698850350 2024-09-21T03:37:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.857+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1029977152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 msgr2=0x562698c0aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.857+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/1029977152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 0x562698c0aea0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800fba0 tx=0x7f3a38012040 comp rx=0 tx=0).stop 2024-09-21T03:37:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.857+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1029977152 shutdown_connections 2024-09-21T03:37:52.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.857+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/1029977152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698850350 0x562698c0aea0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.857+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/1029977152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56269882f670 0x562698c17640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.857+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/1029977152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562698827490 0x562698c05460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.857+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1029977152 >> 172.21.15.112:0/1029977152 conn(0x562698844740 msgr2=0x562698c04d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.857+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1029977152 shutdown_connections 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.857+0000 7f3a4ed65740 1 -- 172.21.15.112:0/1029977152 wait complete. 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.857+0000 7f3a4ed65740 1 -- start start 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.857+0000 7f3a4ed65740 1 Processor -- start 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c13ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c0f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4ed65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698850350 0x562698c0f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a401aba90 con 0x562698827490 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a401acec0 con 0x56269882f670 2024-09-21T03:37:52.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4ed65740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a401ac120 con 0x562698850350 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4ed65740 10 client.0 _ll_register_callbacks cb 0x7f3a30009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4658d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c13ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4658d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c13ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45288/0 (socket says 172.21.15.112:45288) 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4658d640 1 -- 172.21.15.112:0/26749274 learned_addr learned my addr 172.21.15.112:0/26749274 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a46d8e640 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698850350 0x562698c0f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a45d8c640 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c0f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4658d640 1 -- 172.21.15.112:0/26749274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698850350 msgr2=0x562698c0f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4658d640 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698850350 0x562698c0f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4658d640 1 -- 172.21.15.112:0/26749274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 msgr2=0x562698c0f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4658d640 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c0f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4658d640 1 -- 172.21.15.112:0/26749274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562698c0fad0 con 0x562698827490 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a45d8c640 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a4658d640 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c13ba0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c001eb0 tx=0x7f3a3c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3317157796724e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:52.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.858+0000 7f3a29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a29ffb640 1 -- 172.21.15.112:0/26749274 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00fea0 con 0x562698827490 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a29ffb640 1 -- 172.21.15.112:0/26749274 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7f3a3c00fa50 con 0x562698827490 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a4ed65740 1 -- 172.21.15.112:0/26749274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a3800fe10 con 0x562698827490 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a29ffb640 1 -- 172.21.15.112:0/26749274 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3a3c042050 con 0x562698827490 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a4ed65740 10 client.17025 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a4ed65740 1 -- 172.21.15.112:0/26749274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562698c87fe0 con 0x562698827490 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a29ffb640 1 -- 172.21.15.112:0/26749274 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3800fe10 con 0x562698827490 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a29ffb640 1 -- 172.21.15.112:0/26749274 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3a3c019ce0 con 0x562698827490 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a4ed65740 10 client.17025 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a4ed65740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a4ed65740 20 client.17025 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5626987e4080 0x562698c8a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:52.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a4ed65740 4 client.17025 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:37:52.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a4ed65740 1 -- 172.21.15.112:0/26749274 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562698b6b860 con 0x5626987e4080 2024-09-21T03:37:52.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.859+0000 7f3a45d8c640 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5626987e4080 0x562698c8a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:52.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.860+0000 7f3a29ffb640 10 client.17025 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:52.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.860+0000 7f3a45d8c640 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5626987e4080 0x562698c8a820 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88afca43775359c3 server_cookie=dac08d0743d70b8b in_seq=0 out_seq=0 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.861+0000 7f3a29ffb640 1 -- 172.21.15.112:0/26749274 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562698b6b860 con 0x5626987e4080 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.861+0000 7f3a29ffb640 10 client.17025 handle_command_reply: tid=0 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.861+0000 7f3a4ed65740 1 client.17025 shutdown 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.861+0000 7f3a4ed65740 1 -- 172.21.15.112:0/26749274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 msgr2=0x562698c13ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.861+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c13ba0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c001eb0 tx=0x7f3a3c009510 comp rx=0 tx=0).stop 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a4ed65740 1 -- 172.21.15.112:0/26749274 shutdown_connections 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5626987e4080 0x562698c8a820 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562698850350 0x562698c0f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56269882f670 0x562698c0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a29ffb640 0 client.17025 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a4ed65740 1 --2- 172.21.15.112:0/26749274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562698827490 0x562698c13ba0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a4ed65740 1 -- 172.21.15.112:0/26749274 >> 172.21.15.112:0/26749274 conn(0x562698844740 msgr2=0x562698c00c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a4ed65740 1 -- 172.21.15.112:0/26749274 shutdown_connections 2024-09-21T03:37:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a4ed65740 1 -- 172.21.15.112:0/26749274 wait complete. 2024-09-21T03:37:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a4ed65740 20 client.17025 trim_cache size 0 max 16384 2024-09-21T03:37:52.868 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:37:52.868 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:37:52.868 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "0a173e6c-39aa-46f9-8fd2-28b926e70bb6", 2024-09-21T03:37:52.868 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:37:52.868 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a487f1640 1 -- 172.21.15.112:0/3146616898 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a2c0b6680 msgr2=0x7f3a2c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a487f1640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a2c0b6680 0x7f3a2c0b8b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3a38005fd0 tx=0x7f3a38004a40 comp rx=0 tx=0).stop 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a487f1640 1 -- 172.21.15.112:0/3146616898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a400760a0 msgr2=0x7f3a4019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.862+0000 7f3a487f1640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a400760a0 0x7f3a4019cc20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3a3002a930 tx=0x7f3a30031a20 comp rx=0 tx=0).stop 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.863+0000 7f3a487f1640 1 -- 172.21.15.112:0/3146616898 shutdown_connections 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.863+0000 7f3a487f1640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3a2c0b6680 0x7f3a2c0b8b90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.863+0000 7f3a487f1640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a40110140 0x7f3a401a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.863+0000 7f3a487f1640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a400760a0 0x7f3a4019cc20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.863+0000 7f3a487f1640 1 --2- 172.21.15.112:0/3146616898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a40075750 0x7f3a4019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.863+0000 7f3a487f1640 1 -- 172.21.15.112:0/3146616898 >> 172.21.15.112:0/3146616898 conn(0x7f3a400ff610 msgr2=0x7f3a40109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.863+0000 7f3a487f1640 1 -- 172.21.15.112:0/3146616898 shutdown_connections 2024-09-21T03:37:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:52.863+0000 7f3a487f1640 1 -- 172.21.15.112:0/3146616898 wait complete. 2024-09-21T03:37:53.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 939873 10.64 MB/sec execute 353 sec latency 43.054 ms 2024-09-21T03:37:53.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:37:53.929 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:37:54.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:53 smithi112 ceph-mon[31442]: pgmap v590: 97 pgs: 97 active+clean; 711 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-09-21T03:37:54.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:53 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:37:54.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:53 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:53 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:37:54.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:53 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:37:54.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:53 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:37:54.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:53 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:53 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:53 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:53 smithi203 ceph-mon[34723]: pgmap v590: 97 pgs: 97 active+clean; 711 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-09-21T03:37:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:53 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:37:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:53 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:53 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:37:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:53 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:37:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:53 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:37:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:53 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:53 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:53 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:53 smithi187 ceph-mon[34603]: pgmap v590: 97 pgs: 97 active+clean; 711 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 227 op/s 2024-09-21T03:37:54.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:53 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:37:54.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:53 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:53 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:37:54.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:53 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:37:54.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:53 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:37:54.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:53 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:53 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:53 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:37:54.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 942016 10.65 MB/sec execute 354 sec latency 14.802 ms 2024-09-21T03:37:54.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.554+0000 7f545803f640 1 -- 172.21.15.112:0/965832698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54501094c0 msgr2=0x7f54501098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.554+0000 7f545803f640 1 --2- 172.21.15.112:0/965832698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54501094c0 0x7f54501098d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5440009900 tx=0x7f544002a410 comp rx=0 tx=0).stop 2024-09-21T03:37:54.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.554+0000 7f545803f640 1 -- 172.21.15.112:0/965832698 shutdown_connections 2024-09-21T03:37:54.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.554+0000 7f545803f640 1 --2- 172.21.15.112:0/965832698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5450109e10 0x7f5450110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.554+0000 7f545803f640 1 --2- 172.21.15.112:0/965832698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54501094c0 0x7f54501098d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.554+0000 7f545803f640 1 --2- 172.21.15.112:0/965832698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5450101f80 0x7f5450102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.554+0000 7f545803f640 1 -- 172.21.15.112:0/965832698 >> 172.21.15.112:0/965832698 conn(0x7f5450077cf0 msgr2=0x7f5450078130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:54.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.554+0000 7f545803f640 1 -- 172.21.15.112:0/965832698 shutdown_connections 2024-09-21T03:37:54.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.554+0000 7f545803f640 1 -- 172.21.15.112:0/965832698 wait complete. 2024-09-21T03:37:54.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.555+0000 7f545803f640 1 Processor -- start 2024-09-21T03:37:54.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.555+0000 7f545803f640 1 -- start start 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.555+0000 7f545803f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5450101f80 0x7f5450102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.555+0000 7f545803f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54501094c0 0x7f5450103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.555+0000 7f545803f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5450109e10 0x7f54501039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.555+0000 7f545803f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5450113ad0 con 0x7f5450109e10 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.555+0000 7f545803f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5450113950 con 0x7f54501094c0 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.555+0000 7f545803f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5450113c50 con 0x7f5450101f80 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f54565dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5450109e10 0x7f54501039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f5455ddb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5450101f80 0x7f5450102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f5455ddb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5450101f80 0x7f5450102f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46964/0 (socket says 172.21.15.112:46964) 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f54565dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5450109e10 0x7f54501039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45298/0 (socket says 172.21.15.112:45298) 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f5455ddb640 1 -- 172.21.15.112:0/291883915 learned_addr learned my addr 172.21.15.112:0/291883915 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f54555da640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54501094c0 0x7f5450103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f54555da640 1 -- 172.21.15.112:0/291883915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5450101f80 msgr2=0x7f5450102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f54555da640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5450101f80 0x7f5450102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f54555da640 1 -- 172.21.15.112:0/291883915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5450109e10 msgr2=0x7f54501039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f54555da640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5450109e10 0x7f54501039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f54555da640 1 -- 172.21.15.112:0/291883915 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54501a8780 con 0x7f54501094c0 2024-09-21T03:37:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f54555da640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54501094c0 0x7f5450103460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f544002a930 tx=0x7f5440004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.556+0000 7f5446ffd640 1 -- 172.21.15.112:0/291883915 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544003f070 con 0x7f54501094c0 2024-09-21T03:37:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.557+0000 7f545803f640 1 -- 172.21.15.112:0/291883915 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54501a8a10 con 0x7f54501094c0 2024-09-21T03:37:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.557+0000 7f5446ffd640 1 -- 172.21.15.112:0/291883915 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5440003ca0 con 0x7f54501094c0 2024-09-21T03:37:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.557+0000 7f5446ffd640 1 -- 172.21.15.112:0/291883915 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440035070 con 0x7f54501094c0 2024-09-21T03:37:54.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.557+0000 7f545803f640 1 -- 172.21.15.112:0/291883915 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54501a8f10 con 0x7f54501094c0 2024-09-21T03:37:54.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.559+0000 7f5446ffd640 1 -- 172.21.15.112:0/291883915 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f5440043400 con 0x7f54501094c0 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.559+0000 7f545e5b3740 1 Processor -- start 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.559+0000 7f545e5b3740 1 -- start start 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712efff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712efffb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec37390 0x55712f0000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f545e5b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55712ef75e50 con 0x55712ec58070 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f545e5b3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55712ef65280 con 0x55712ec37390 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f545e5b3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55712ef69090 con 0x55712ec2f1b0 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f54565dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712efff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f54565dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712efff650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45306/0 (socket says 172.21.15.112:45306) 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f54565dc640 1 -- 172.21.15.112:0/2286104289 learned_addr learned my addr 172.21.15.112:0/2286104289 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f5455ddb640 1 --2- 172.21.15.112:0/2286104289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712efffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f54555da640 1 --2- 172.21.15.112:0/2286104289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec37390 0x55712f0000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f5446ffd640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f543c0b6700 0x7f543c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f54565dc640 1 -- 172.21.15.112:0/2286104289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 msgr2=0x55712efffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:54.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f54565dc640 1 --2- 172.21.15.112:0/2286104289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712efffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f54565dc640 1 -- 172.21.15.112:0/2286104289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec37390 msgr2=0x55712f0000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:54.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f54565dc640 1 --2- 172.21.15.112:0/2286104289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec37390 0x55712f0000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.560+0000 7f54565dc640 1 -- 172.21.15.112:0/2286104289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55712ecc3650 con 0x55712ec58070 2024-09-21T03:37:54.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.561+0000 7f54565dc640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f543c0b6700 0x7f543c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.561+0000 7f5446ffd640 1 -- 172.21.15.112:0/291883915 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7f5440107bb0 con 0x7f54501094c0 2024-09-21T03:37:54.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.561+0000 7f54565dc640 1 --2- 172.21.15.112:0/2286104289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712efff650 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f544800b6b0 tx=0x7f544800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ea4ca712f4f7e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:54.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.561+0000 7f543bfff640 1 -- 172.21.15.112:0/2286104289 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448011440 con 0x55712ec58070 2024-09-21T03:37:54.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.561+0000 7f543bfff640 1 -- 172.21.15.112:0/2286104289 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5448015070 con 0x55712ec58070 2024-09-21T03:37:54.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.562+0000 7f543bfff640 1 -- 172.21.15.112:0/2286104289 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544801bb40 con 0x55712ec58070 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.562+0000 7f545e5b3740 1 -- 172.21.15.112:0/2286104289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 msgr2=0x55712efff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.562+0000 7f54565dc640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f543c0b6700 0x7f543c0b8c10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f544801c000 tx=0x7f5448010810 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.562+0000 7f545e5b3740 1 --2- 172.21.15.112:0/2286104289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712efff650 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f544800b6b0 tx=0x7f544800bab0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.562+0000 7f545e5b3740 1 -- 172.21.15.112:0/2286104289 shutdown_connections 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.562+0000 7f545e5b3740 1 --2- 172.21.15.112:0/2286104289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec37390 0x55712f0000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.562+0000 7f545e5b3740 1 --2- 172.21.15.112:0/2286104289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712efffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.562+0000 7f545e5b3740 1 --2- 172.21.15.112:0/2286104289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712efff650 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.562+0000 7f545e5b3740 1 -- 172.21.15.112:0/2286104289 >> 172.21.15.112:0/2286104289 conn(0x55712ec4c460 msgr2=0x55712effd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.563+0000 7f545e5b3740 1 -- 172.21.15.112:0/2286104289 shutdown_connections 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.563+0000 7f545e5b3740 1 -- 172.21.15.112:0/2286104289 wait complete. 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.563+0000 7f545e5b3740 1 -- start start 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.563+0000 7f545e5b3740 1 Processor -- start 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712f00ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec37390 0x55712f00b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712f0170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f545e5b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55712edc6700 con 0x55712ec58070 2024-09-21T03:37:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f545e5b3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55712ef00110 con 0x55712ec37390 2024-09-21T03:37:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f545e5b3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55712f00ddd0 con 0x55712ec2f1b0 2024-09-21T03:37:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f545e5b3740 10 client.0 _ll_register_callbacks cb 0x55712ef19990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54565dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712f0170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f5455ddb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712f00ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54555da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec37390 0x55712f00b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54565dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712f0170c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45314/0 (socket says 172.21.15.112:45314) 2024-09-21T03:37:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f5455ddb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712f00ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46984/0 (socket says 172.21.15.112:46984) 2024-09-21T03:37:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54565dc640 1 -- 172.21.15.112:0/3773710007 learned_addr learned my addr 172.21.15.112:0/3773710007 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54555da640 1 -- 172.21.15.112:0/3773710007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 msgr2=0x55712f00ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54555da640 1 --2- 172.21.15.112:0/3773710007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712f00ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54555da640 1 -- 172.21.15.112:0/3773710007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 msgr2=0x55712f0170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54555da640 1 --2- 172.21.15.112:0/3773710007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712f0170c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54555da640 1 -- 172.21.15.112:0/3773710007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55712ef2dc40 con 0x55712ec37390 2024-09-21T03:37:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54555da640 1 --2- 172.21.15.112:0/3773710007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec37390 0x55712f00b2d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54401052d0 tx=0x7f544003b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b38ec41ff5c7bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54397fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:37:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.564+0000 7f54397fa640 1 -- 172.21.15.112:0/3773710007 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440048070 con 0x55712ec37390 2024-09-21T03:37:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f54397fa640 1 -- 172.21.15.112:0/3773710007 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7f54400d6f60 con 0x55712ec37390 2024-09-21T03:37:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f545e5b3740 1 -- 172.21.15.112:0/3773710007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5448015070 con 0x55712ec37390 2024-09-21T03:37:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f54397fa640 1 -- 172.21.15.112:0/3773710007 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f54400ca0d0 con 0x55712ec37390 2024-09-21T03:37:54.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f545e5b3740 10 client.26591 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:54.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f545e5b3740 1 -- 172.21.15.112:0/3773710007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55712f019b60 con 0x55712ec37390 2024-09-21T03:37:54.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f54397fa640 1 -- 172.21.15.112:0/3773710007 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5448015070 con 0x55712ec37390 2024-09-21T03:37:54.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f54397fa640 1 -- 172.21.15.112:0/3773710007 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f544004b060 con 0x55712ec37390 2024-09-21T03:37:54.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f545e5b3740 10 client.26591 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:54.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f545e5b3740 10 client.26591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f545e5b3740 20 client.26591 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f545e5b3740 1 --2- 172.21.15.112:0/3773710007 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55712ec169f0 0x55712f01d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f545e5b3740 4 client.26591 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f545e5b3740 1 -- 172.21.15.112:0/3773710007 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55712ef77da0 con 0x55712ec169f0 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.565+0000 7f5455ddb640 1 --2- 172.21.15.112:0/3773710007 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55712ec169f0 0x55712f01d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.566+0000 7f54397fa640 10 client.26591 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.566+0000 7f5455ddb640 1 --2- 172.21.15.112:0/3773710007 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55712ec169f0 0x55712f01d0c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66d5532663382cb5 server_cookie=b535c60ce5e6add3 in_seq=0 out_seq=0 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f54397fa640 1 -- 172.21.15.112:0/3773710007 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55712ef77da0 con 0x55712ec169f0 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f54397fa640 10 client.26591 handle_command_reply: tid=0 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 1 client.26591 shutdown 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 1 -- 172.21.15.112:0/3773710007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec37390 msgr2=0x55712f00b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 1 --2- 172.21.15.112:0/3773710007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec37390 0x55712f00b2d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54401052d0 tx=0x7f544003b800 comp rx=0 tx=0).stop 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 1 -- 172.21.15.112:0/3773710007 shutdown_connections 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 1 --2- 172.21.15.112:0/3773710007 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55712ec169f0 0x55712f01d0c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 1 --2- 172.21.15.112:0/3773710007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712f0170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 1 --2- 172.21.15.112:0/3773710007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec37390 0x55712f00b2d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 1 --2- 172.21.15.112:0/3773710007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712f00ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f54397fa640 0 client.26591 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:54.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 1 -- 172.21.15.112:0/3773710007 >> 172.21.15.112:0/3773710007 conn(0x55712ec4c460 msgr2=0x55712f002830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:54.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 1 -- 172.21.15.112:0/3773710007 shutdown_connections 2024-09-21T03:37:54.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 1 -- 172.21.15.112:0/3773710007 wait complete. 2024-09-21T03:37:54.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.567+0000 7f545e5b3740 20 client.26591 trim_cache size 0 max 16384 2024-09-21T03:37:54.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.586+0000 7f545e5b3740 1 Processor -- start 2024-09-21T03:37:54.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.586+0000 7f545e5b3740 1 -- start start 2024-09-21T03:37:54.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.586+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712f00b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.586+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec37390 0x55712effd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.586+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec58070 0x55712f016fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.586+0000 7f545e5b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54501b40d0 con 0x55712ec37390 2024-09-21T03:37:54.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.586+0000 7f545e5b3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f54501b19a0 con 0x55712ec58070 2024-09-21T03:37:54.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.586+0000 7f545e5b3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f54501af810 con 0x55712ec2f1b0 2024-09-21T03:37:54.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.586+0000 7f54555da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec37390 0x55712effd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.586+0000 7f54555da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec37390 0x55712effd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45320/0 (socket says 172.21.15.112:45320) 2024-09-21T03:37:54.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f54555da640 1 -- 172.21.15.112:0/282139806 learned_addr learned my addr 172.21.15.112:0/282139806 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:54.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f5455ddb640 1 --2- 172.21.15.112:0/282139806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712f00b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f54565dc640 1 --2- 172.21.15.112:0/282139806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec58070 0x55712f016fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f54555da640 1 -- 172.21.15.112:0/282139806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 msgr2=0x55712f00b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f54555da640 1 --2- 172.21.15.112:0/282139806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712f00b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f54555da640 1 -- 172.21.15.112:0/282139806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec58070 msgr2=0x55712f016fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f54555da640 1 --2- 172.21.15.112:0/282139806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec58070 0x55712f016fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f54555da640 1 -- 172.21.15.112:0/282139806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55712f009cf0 con 0x55712ec37390 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f54555da640 1 --2- 172.21.15.112:0/282139806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec37390 0x55712effd0b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5440107f70 tx=0x7f5440049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8febf8b25a7b9a45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f543b7fe640 1 -- 172.21.15.112:0/282139806 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440048070 con 0x55712ec37390 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f543b7fe640 1 -- 172.21.15.112:0/282139806 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54400d2020 con 0x55712ec37390 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.587+0000 7f543b7fe640 1 -- 172.21.15.112:0/282139806 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54400cd1f0 con 0x55712ec37390 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.588+0000 7f545e5b3740 1 -- 172.21.15.112:0/282139806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec37390 msgr2=0x55712effd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.588+0000 7f545e5b3740 1 --2- 172.21.15.112:0/282139806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec37390 0x55712effd0b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5440107f70 tx=0x7f5440049000 comp rx=0 tx=0).stop 2024-09-21T03:37:54.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.588+0000 7f545e5b3740 1 -- 172.21.15.112:0/282139806 shutdown_connections 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.588+0000 7f545e5b3740 1 --2- 172.21.15.112:0/282139806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec58070 0x55712f016fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.588+0000 7f545e5b3740 1 --2- 172.21.15.112:0/282139806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec37390 0x55712effd0b0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.588+0000 7f545e5b3740 1 --2- 172.21.15.112:0/282139806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec2f1b0 0x55712f00b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.588+0000 7f545e5b3740 1 -- 172.21.15.112:0/282139806 >> 172.21.15.112:0/282139806 conn(0x55712ec4c460 msgr2=0x55712f00ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.588+0000 7f545e5b3740 1 -- 172.21.15.112:0/282139806 shutdown_connections 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.588+0000 7f545e5b3740 1 -- 172.21.15.112:0/282139806 wait complete. 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f545e5b3740 1 -- start start 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f545e5b3740 1 Processor -- start 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec2f1b0 0x55712f00efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec37390 0x55712f00f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f545e5b3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712f00fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f545e5b3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54501afe60 con 0x55712ec58070 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f545e5b3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f54501b12d0 con 0x55712ec2f1b0 2024-09-21T03:37:54.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f545e5b3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f54501b0530 con 0x55712ec37390 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f545e5b3740 10 client.0 _ll_register_callbacks cb 0x7f5440005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f54565dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712f00fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f54565dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712f00fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45330/0 (socket says 172.21.15.112:45330) 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f54565dc640 1 -- 172.21.15.112:0/1085474656 learned_addr learned my addr 172.21.15.112:0/1085474656 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f54555da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec37390 0x55712f00f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.589+0000 7f5455ddb640 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec2f1b0 0x55712f00efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f54565dc640 1 -- 172.21.15.112:0/1085474656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec37390 msgr2=0x55712f00f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f54565dc640 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec37390 0x55712f00f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f54565dc640 1 -- 172.21.15.112:0/1085474656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec2f1b0 msgr2=0x55712f00efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f54565dc640 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec2f1b0 0x55712f00efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f54565dc640 1 -- 172.21.15.112:0/1085474656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55712effd790 con 0x55712ec58070 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f54565dc640 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712f00fa60 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f544800b6b0 tx=0x7f5448015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2b61374d4b4721 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f54397fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:37:54.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f54397fa640 1 -- 172.21.15.112:0/1085474656 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448003cf0 con 0x55712ec58070 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f54397fa640 1 -- 172.21.15.112:0/1085474656 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8470+0+0 (secure 0 0 0) 0x7f5448005040 con 0x55712ec58070 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f545e5b3740 1 -- 172.21.15.112:0/1085474656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54400d2020 con 0x55712ec58070 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f54397fa640 1 -- 172.21.15.112:0/1085474656 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5448005240 con 0x55712ec58070 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f545e5b3740 10 client.17043 fetch_fsmap learned FSMap version 30 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.590+0000 7f545e5b3740 1 -- 172.21.15.112:0/1085474656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55712f087a80 con 0x55712ec58070 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.591+0000 7f54397fa640 1 -- 172.21.15.112:0/1085474656 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54400d2020 con 0x55712ec58070 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.591+0000 7f54397fa640 1 -- 172.21.15.112:0/1085474656 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f54480045a0 con 0x55712ec58070 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.591+0000 7f545e5b3740 10 client.17043 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.591+0000 7f545e5b3740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.591+0000 7f545e5b3740 20 client.17043 populate_metadata read hostname 'smithi112' 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.591+0000 7f545e5b3740 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55712ebebda0 0x55712f089f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.591+0000 7f545e5b3740 4 client.17043 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.591+0000 7f545e5b3740 1 -- 172.21.15.112:0/1085474656 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55712ef73850 con 0x55712ebebda0 2024-09-21T03:37:54.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.591+0000 7f5455ddb640 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55712ebebda0 0x55712f089f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:37:54.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f54397fa640 10 client.17043 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f5455ddb640 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55712ebebda0 0x55712f089f80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cac41e64f855560 server_cookie=3abbac8a038b3d6f in_seq=0 out_seq=0 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f54397fa640 1 -- 172.21.15.112:0/1085474656 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x55712ef73850 con 0x55712ebebda0 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f54397fa640 10 client.17043 handle_command_reply: tid=0 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f545e5b3740 1 client.17043 shutdown 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f545e5b3740 1 -- 172.21.15.112:0/1085474656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 msgr2=0x55712f00fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f545e5b3740 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712f00fa60 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f544800b6b0 tx=0x7f5448015000 comp rx=0 tx=0).stop 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f545e5b3740 1 -- 172.21.15.112:0/1085474656 shutdown_connections 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f545e5b3740 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55712ebebda0 0x55712f089f80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f545e5b3740 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55712ec58070 0x55712f00fa60 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f545e5b3740 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55712ec37390 0x55712f00f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f545e5b3740 1 --2- 172.21.15.112:0/1085474656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55712ec2f1b0 0x55712f00efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f54397fa640 0 client.17043 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:37:54.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f545e5b3740 1 -- 172.21.15.112:0/1085474656 >> 172.21.15.112:0/1085474656 conn(0x55712ec4c460 msgr2=0x55712f000680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:54.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f545e5b3740 1 -- 172.21.15.112:0/1085474656 shutdown_connections 2024-09-21T03:37:54.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.592+0000 7f545e5b3740 1 -- 172.21.15.112:0/1085474656 wait complete. 2024-09-21T03:37:54.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545e5b3740 20 client.17043 trim_cache size 0 max 16384 2024-09-21T03:37:54.599 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:37:54.599 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:37:54.599 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:37:54.599 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:37:54.599 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.599 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "0a173e6c-39aa-46f9-8fd2-28b926e70bb6", 2024-09-21T03:37:54.599 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.600 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.602 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.603 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.604 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.605 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.607 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:37:54.608 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 -- 172.21.15.112:0/291883915 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f543c0b6700 msgr2=0x7f543c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f543c0b6700 0x7f543c0b8c10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f544801c000 tx=0x7f5448010810 comp rx=0 tx=0).stop 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 -- 172.21.15.112:0/291883915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54501094c0 msgr2=0x7f5450103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54501094c0 0x7f5450103460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f544002a930 tx=0x7f5440004060 comp rx=0 tx=0).stop 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 -- 172.21.15.112:0/291883915 shutdown_connections 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f543c0b6700 0x7f543c0b8c10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5450109e10 0x7f54501039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54501094c0 0x7f5450103460 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 --2- 172.21.15.112:0/291883915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5450101f80 0x7f5450102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 -- 172.21.15.112:0/291883915 >> 172.21.15.112:0/291883915 conn(0x7f5450077cf0 msgr2=0x7f5450105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 -- 172.21.15.112:0/291883915 shutdown_connections 2024-09-21T03:37:54.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:37:54.593+0000 7f545803f640 1 -- 172.21.15.112:0/291883915 wait complete. 2024-09-21T03:37:55.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 944144 10.64 MB/sec execute 355 sec latency 16.056 ms 2024-09-21T03:37:55.384 INFO:tasks.cephfs.filesystem:scrub status for tag:0a173e6c-39aa-46f9-8fd2-28b926e70bb6 - {'path': '/', 'tag': '0a173e6c-39aa-46f9-8fd2-28b926e70bb6', 'options': 'recursive,force'} 2024-09-21T03:37:56.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:55 smithi203 ceph-mon[34723]: pgmap v591: 97 pgs: 97 active+clean; 711 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 224 op/s 2024-09-21T03:37:56.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:55 smithi187 ceph-mon[34603]: pgmap v591: 97 pgs: 97 active+clean; 711 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 224 op/s 2024-09-21T03:37:56.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 945724 10.65 MB/sec execute 356 sec latency 23.857 ms 2024-09-21T03:37:56.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:55 smithi112 ceph-mon[31442]: pgmap v591: 97 pgs: 97 active+clean; 711 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 224 op/s 2024-09-21T03:37:57.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 947538 10.64 MB/sec execute 357 sec latency 49.817 ms 2024-09-21T03:37:58.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:57 smithi203 ceph-mon[34723]: pgmap v592: 97 pgs: 97 active+clean; 723 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 323 op/s 2024-09-21T03:37:58.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:57 smithi187 ceph-mon[34603]: pgmap v592: 97 pgs: 97 active+clean; 723 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 323 op/s 2024-09-21T03:37:58.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 949184 10.64 MB/sec execute 358 sec latency 14.616 ms 2024-09-21T03:37:58.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:57 smithi112 ceph-mon[31442]: pgmap v592: 97 pgs: 97 active+clean; 723 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 323 op/s 2024-09-21T03:37:59.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 951070 10.63 MB/sec execute 359 sec latency 22.990 ms 2024-09-21T03:38:00.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:59 smithi203 ceph-mon[34723]: pgmap v593: 97 pgs: 97 active+clean; 723 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 227 op/s 2024-09-21T03:38:00.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:59 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:38:00.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:59 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:38:00.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:37:59 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:38:00.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:59 smithi187 ceph-mon[34603]: pgmap v593: 97 pgs: 97 active+clean; 723 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 227 op/s 2024-09-21T03:38:00.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:59 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:38:00.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:59 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:38:00.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:37:59 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:38:00.266 INFO:tasks.workunit.client.0.smithi112.stdout: 1 952862 10.63 MB/sec execute 360 sec latency 15.624 ms 2024-09-21T03:38:00.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:59 smithi112 ceph-mon[31442]: pgmap v593: 97 pgs: 97 active+clean; 723 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 227 op/s 2024-09-21T03:38:00.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:59 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:38:00.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:59 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:38:00.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:37:59 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:38:01.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 953702 10.62 MB/sec execute 361 sec latency 577.123 ms 2024-09-21T03:38:02.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:01 smithi203 ceph-mon[34723]: pgmap v594: 97 pgs: 97 active+clean; 728 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 298 op/s 2024-09-21T03:38:02.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:01 smithi187 ceph-mon[34603]: pgmap v594: 97 pgs: 97 active+clean; 728 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 298 op/s 2024-09-21T03:38:02.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 953702 10.59 MB/sec execute 362 sec latency 1577.238 ms 2024-09-21T03:38:02.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:01 smithi112 ceph-mon[31442]: pgmap v594: 97 pgs: 97 active+clean; 728 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 298 op/s 2024-09-21T03:38:03.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 955202 10.58 MB/sec execute 363 sec latency 1656.426 ms 2024-09-21T03:38:04.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:03 smithi203 ceph-mon[34723]: pgmap v595: 97 pgs: 97 active+clean; 729 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 224 op/s 2024-09-21T03:38:04.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:03 smithi187 ceph-mon[34603]: pgmap v595: 97 pgs: 97 active+clean; 729 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 224 op/s 2024-09-21T03:38:04.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 956627 10.57 MB/sec execute 364 sec latency 18.067 ms 2024-09-21T03:38:04.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:03 smithi112 ceph-mon[31442]: pgmap v595: 97 pgs: 97 active+clean; 729 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 224 op/s 2024-09-21T03:38:05.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 959038 10.58 MB/sec execute 365 sec latency 32.335 ms 2024-09-21T03:38:06.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:05 smithi203 ceph-mon[34723]: pgmap v596: 97 pgs: 97 active+clean; 732 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 242 op/s 2024-09-21T03:38:06.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:05 smithi203 ceph-mon[34723]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:38:06.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:05 smithi187 ceph-mon[34603]: pgmap v596: 97 pgs: 97 active+clean; 732 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 242 op/s 2024-09-21T03:38:06.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:05 smithi187 ceph-mon[34603]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:38:06.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 960344 10.57 MB/sec execute 366 sec latency 29.180 ms 2024-09-21T03:38:06.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:05 smithi112 ceph-mon[31442]: pgmap v596: 97 pgs: 97 active+clean; 732 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 242 op/s 2024-09-21T03:38:06.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:05 smithi112 ceph-mon[31442]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:38:07.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 962797 10.58 MB/sec execute 367 sec latency 51.051 ms 2024-09-21T03:38:08.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:07 smithi203 ceph-mon[34723]: pgmap v598: 97 pgs: 97 active+clean; 733 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-21T03:38:08.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:07 smithi187 ceph-mon[34603]: pgmap v598: 97 pgs: 97 active+clean; 733 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-21T03:38:08.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 963626 10.56 MB/sec execute 368 sec latency 72.676 ms 2024-09-21T03:38:08.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:07 smithi112 ceph-mon[31442]: pgmap v598: 97 pgs: 97 active+clean; 733 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-21T03:38:09.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 965827 10.56 MB/sec execute 369 sec latency 26.154 ms 2024-09-21T03:38:10.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:09 smithi203 ceph-mon[34723]: pgmap v599: 97 pgs: 97 active+clean; 733 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 265 op/s 2024-09-21T03:38:10.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:09 smithi187 ceph-mon[34603]: pgmap v599: 97 pgs: 97 active+clean; 733 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 265 op/s 2024-09-21T03:38:10.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 967121 10.55 MB/sec execute 370 sec latency 21.494 ms 2024-09-21T03:38:10.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:09 smithi112 ceph-mon[31442]: pgmap v599: 97 pgs: 97 active+clean; 733 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 265 op/s 2024-09-21T03:38:11.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 969292 10.56 MB/sec execute 371 sec latency 20.635 ms 2024-09-21T03:38:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:11 smithi203 ceph-mon[34723]: pgmap v600: 97 pgs: 97 active+clean; 740 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-09-21T03:38:12.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:11 smithi187 ceph-mon[34603]: pgmap v600: 97 pgs: 97 active+clean; 740 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-09-21T03:38:12.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 970533 10.55 MB/sec execute 372 sec latency 80.251 ms 2024-09-21T03:38:12.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:11 smithi112 ceph-mon[31442]: pgmap v600: 97 pgs: 97 active+clean; 740 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-09-21T03:38:13.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 972527 10.54 MB/sec execute 373 sec latency 21.372 ms 2024-09-21T03:38:14.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:13 smithi203 ceph-mon[34723]: pgmap v601: 97 pgs: 97 active+clean; 741 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-09-21T03:38:14.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:13 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:14.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:13 smithi187 ceph-mon[34603]: pgmap v601: 97 pgs: 97 active+clean; 741 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-09-21T03:38:14.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:13 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:14.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 974145 10.54 MB/sec execute 374 sec latency 13.701 ms 2024-09-21T03:38:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:13 smithi112 ceph-mon[31442]: pgmap v601: 97 pgs: 97 active+clean; 741 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-09-21T03:38:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:13 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:38:15.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:14 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:15.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:14 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:15.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:14 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:38:15.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:14 smithi203 ceph-mon[34723]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:38:15.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:14 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:15.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:14 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:15.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:14 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:38:15.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:14 smithi187 ceph-mon[34603]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:38:15.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 976390 10.55 MB/sec execute 375 sec latency 20.399 ms 2024-09-21T03:38:15.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:14 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:38:15.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:14 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:38:15.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:14 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:38:15.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:14 smithi112 ceph-mon[31442]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:38:16.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 977839 10.54 MB/sec execute 376 sec latency 17.825 ms 2024-09-21T03:38:16.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:16 smithi112 ceph-mon[31442]: pgmap v602: 97 pgs: 97 active+clean; 744 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-09-21T03:38:16.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:16 smithi203 ceph-mon[34723]: pgmap v602: 97 pgs: 97 active+clean; 744 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-09-21T03:38:16.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:16 smithi187 ceph-mon[34603]: pgmap v602: 97 pgs: 97 active+clean; 744 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 254 op/s 2024-09-21T03:38:17.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 980189 10.54 MB/sec execute 377 sec latency 47.927 ms 2024-09-21T03:38:18.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 981460 10.53 MB/sec execute 378 sec latency 19.926 ms 2024-09-21T03:38:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:18 smithi112 ceph-mon[31442]: pgmap v604: 97 pgs: 97 active+clean; 765 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 267 op/s 2024-09-21T03:38:18.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:18 smithi203 ceph-mon[34723]: pgmap v604: 97 pgs: 97 active+clean; 765 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 267 op/s 2024-09-21T03:38:18.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:18 smithi187 ceph-mon[34603]: pgmap v604: 97 pgs: 97 active+clean; 765 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 267 op/s 2024-09-21T03:38:19.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 983921 10.54 MB/sec execute 379 sec latency 20.152 ms 2024-09-21T03:38:20.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 985077 10.53 MB/sec execute 380 sec latency 14.344 ms 2024-09-21T03:38:20.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:20 smithi112 ceph-mon[31442]: pgmap v605: 97 pgs: 97 active+clean; 765 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-21T03:38:20.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:20 smithi203 ceph-mon[34723]: pgmap v605: 97 pgs: 97 active+clean; 765 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-21T03:38:20.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:20 smithi187 ceph-mon[34603]: pgmap v605: 97 pgs: 97 active+clean; 765 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 256 op/s 2024-09-21T03:38:21.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 987739 10.55 MB/sec execute 381 sec latency 18.077 ms 2024-09-21T03:38:22.267 INFO:tasks.workunit.client.0.smithi112.stdout: 1 989054 10.54 MB/sec execute 382 sec latency 37.117 ms 2024-09-21T03:38:22.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:22 smithi112 ceph-mon[31442]: pgmap v606: 97 pgs: 97 active+clean; 768 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 219 op/s 2024-09-21T03:38:22.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:22 smithi203 ceph-mon[34723]: pgmap v606: 97 pgs: 97 active+clean; 768 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 219 op/s 2024-09-21T03:38:22.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:22 smithi187 ceph-mon[34603]: pgmap v606: 97 pgs: 97 active+clean; 768 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 219 op/s 2024-09-21T03:38:23.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 991477 10.54 MB/sec execute 383 sec latency 17.592 ms 2024-09-21T03:38:24.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 993007 10.53 MB/sec execute 384 sec latency 20.385 ms 2024-09-21T03:38:24.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:24 smithi112 ceph-mon[31442]: pgmap v607: 97 pgs: 97 active+clean; 773 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 260 op/s 2024-09-21T03:38:24.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:24 smithi203 ceph-mon[34723]: pgmap v607: 97 pgs: 97 active+clean; 773 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 260 op/s 2024-09-21T03:38:24.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:24 smithi187 ceph-mon[34603]: pgmap v607: 97 pgs: 97 active+clean; 773 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 260 op/s 2024-09-21T03:38:25.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 995327 10.54 MB/sec execute 385 sec latency 11.784 ms 2024-09-21T03:38:25.386 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:38:25.663 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:26.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 997540 10.54 MB/sec execute 386 sec latency 17.245 ms 2024-09-21T03:38:26.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.257+0000 7f990a734640 1 -- 172.21.15.112:0/98012194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f990410de10 msgr2=0x7f9904110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:26.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.257+0000 7f990a734640 1 --2- 172.21.15.112:0/98012194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f990410de10 0x7f9904110300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009ca0 tx=0x7f98fc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.257+0000 7f990a734640 1 -- 172.21.15.112:0/98012194 shutdown_connections 2024-09-21T03:38:26.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.257+0000 7f990a734640 1 --2- 172.21.15.112:0/98012194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f990410de10 0x7f9904110300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.257+0000 7f990a734640 1 --2- 172.21.15.112:0/98012194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9904105300 0x7f990410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.257+0000 7f990a734640 1 --2- 172.21.15.112:0/98012194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99041049b0 0x7f9904104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.257+0000 7f990a734640 1 -- 172.21.15.112:0/98012194 >> 172.21.15.112:0/98012194 conn(0x7f9904079730 msgr2=0x7f9904079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:26.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.257+0000 7f990a734640 1 -- 172.21.15.112:0/98012194 shutdown_connections 2024-09-21T03:38:26.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.257+0000 7f990a734640 1 -- 172.21.15.112:0/98012194 wait complete. 2024-09-21T03:38:26.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f990a734640 1 Processor -- start 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f990a734640 1 -- start start 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f990a734640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99041049b0 0x7f990419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f990a734640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9904105300 0x7f990419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f990a734640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f990410de10 0x7f990419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f990a734640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9904113a90 con 0x7f99041049b0 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f990a734640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9904113910 con 0x7f9904105300 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f990a734640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9904113c10 con 0x7f990410de10 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f9903fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99041049b0 0x7f990419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f99037fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9904105300 0x7f990419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f99037fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9904105300 0x7f990419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33798/0 (socket says 172.21.15.112:33798) 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f9903fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99041049b0 0x7f990419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46580/0 (socket says 172.21.15.112:46580) 2024-09-21T03:38:26.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f99037fe640 1 -- 172.21.15.112:0/1078924161 learned_addr learned my addr 172.21.15.112:0/1078924161 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.258+0000 7f9908cd1640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f990410de10 0x7f990419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f9903fff640 1 -- 172.21.15.112:0/1078924161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f990410de10 msgr2=0x7f990419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f9903fff640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f990410de10 0x7f990419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f9903fff640 1 -- 172.21.15.112:0/1078924161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9904105300 msgr2=0x7f990419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f9903fff640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9904105300 0x7f990419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f9903fff640 1 -- 172.21.15.112:0/1078924161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99041a03d0 con 0x7f99041049b0 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f99037fe640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9904105300 0x7f990419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f9903fff640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99041049b0 0x7f990419a460 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e9c0 tx=0x7f98f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f99017fa640 1 -- 172.21.15.112:0/1078924161 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f400ccc0 con 0x7f99041049b0 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f990a734640 1 -- 172.21.15.112:0/1078924161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99041a06c0 con 0x7f99041049b0 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f99017fa640 1 -- 172.21.15.112:0/1078924161 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f40035f0 con 0x7f99041049b0 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f99017fa640 1 -- 172.21.15.112:0/1078924161 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f400ce60 con 0x7f99041049b0 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.259+0000 7f990a734640 1 -- 172.21.15.112:0/1078924161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99041a6e20 con 0x7f99041049b0 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9910ca8740 1 Processor -- start 2024-09-21T03:38:26.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9910ca8740 1 -- start start 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f99017fa640 1 -- 172.21.15.112:0/1078924161 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f98f4010650 con 0x7f99041049b0 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 0x559784afd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55978472d1b0 0x559784afd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784735390 0x559784afdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9910ca8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559784a73e50 con 0x559784735390 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9910ca8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559784a63280 con 0x55978472d1b0 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9910ca8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559784a67090 con 0x559784756070 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9908cd1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 0x559784afd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9903fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55978472d1b0 0x559784afd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f99037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784735390 0x559784afdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9908cd1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 0x559784afd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43252/0 (socket says 172.21.15.112:43252) 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f9908cd1640 1 -- 172.21.15.112:0/1739569598 learned_addr learned my addr 172.21.15.112:0/1739569598 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.262+0000 7f99037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784735390 0x559784afdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46590/0 (socket says 172.21.15.112:46590) 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f9908cd1640 1 -- 172.21.15.112:0/1739569598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55978472d1b0 msgr2=0x559784afd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:26.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f9908cd1640 1 --2- 172.21.15.112:0/1739569598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55978472d1b0 0x559784afd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f9908cd1640 1 -- 172.21.15.112:0/1739569598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784735390 msgr2=0x559784afdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f9908cd1640 1 --2- 172.21.15.112:0/1739569598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784735390 0x559784afdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f9908cd1640 1 -- 172.21.15.112:0/1739569598 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597847c1650 con 0x559784756070 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f99017fa640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98ec0b6680 0x7f98ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f9908cd1640 1 --2- 172.21.15.112:0/1739569598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 0x559784afd270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f98fc02f410 tx=0x7f98fc02f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a7dd98c8eb63220 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f9903fff640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98ec0b6680 0x7f98ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f98ea7fc640 1 -- 172.21.15.112:0/1739569598 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc031cf0 con 0x559784756070 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f98ea7fc640 1 -- 172.21.15.112:0/1739569598 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98fc036030 con 0x559784756070 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f98ea7fc640 1 -- 172.21.15.112:0/1739569598 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc040820 con 0x559784756070 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.263+0000 7f99017fa640 1 -- 172.21.15.112:0/1078924161 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f98f40e7aa0 con 0x7f99041049b0 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.264+0000 7f9910ca8740 1 -- 172.21.15.112:0/1739569598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 msgr2=0x559784afd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.264+0000 7f9910ca8740 1 --2- 172.21.15.112:0/1739569598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 0x559784afd270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f98fc02f410 tx=0x7f98fc02f440 comp rx=0 tx=0).stop 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.264+0000 7f9903fff640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98ec0b6680 0x7f98ec0b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f98f40e52d0 tx=0x7f98f40eb5f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.264+0000 7f9910ca8740 1 -- 172.21.15.112:0/1739569598 shutdown_connections 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.264+0000 7f9910ca8740 1 --2- 172.21.15.112:0/1739569598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784735390 0x559784afdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.264+0000 7f9910ca8740 1 --2- 172.21.15.112:0/1739569598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55978472d1b0 0x559784afd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.264+0000 7f9910ca8740 1 --2- 172.21.15.112:0/1739569598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 0x559784afd270 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.264+0000 7f9910ca8740 1 -- 172.21.15.112:0/1739569598 >> 172.21.15.112:0/1739569598 conn(0x55978474a460 msgr2=0x559784afb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.264+0000 7f9910ca8740 1 -- 172.21.15.112:0/1739569598 shutdown_connections 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.264+0000 7f9910ca8740 1 -- 172.21.15.112:0/1739569598 wait complete. 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.265+0000 7f9910ca8740 1 -- start start 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.265+0000 7f9910ca8740 1 Processor -- start 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.265+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55978472d1b0 0x559784b088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.265+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784735390 0x559784b08e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.265+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784756070 0x559784b14b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.265+0000 7f9910ca8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5597848c4700 con 0x559784756070 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.265+0000 7f9910ca8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5597849fe110 con 0x55978472d1b0 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.265+0000 7f9910ca8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559784b0b870 con 0x559784735390 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f9910ca8740 10 client.0 _ll_register_callbacks cb 0x559784a17990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f9908cd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784756070 0x559784b14b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f99037fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784735390 0x559784b08e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f99037fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784735390 0x559784b08e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43254/0 (socket says 172.21.15.112:43254) 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f9908cd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784756070 0x559784b14b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46592/0 (socket says 172.21.15.112:46592) 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f9908cd1640 1 -- 172.21.15.112:0/2766468266 learned_addr learned my addr 172.21.15.112:0/2766468266 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f9908cd1640 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55978472d1b0 0x559784b088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f99037fe640 1 -- 172.21.15.112:0/2766468266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55978472d1b0 msgr2=0x559784b088d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f99037fe640 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55978472d1b0 0x559784b088d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f99037fe640 1 -- 172.21.15.112:0/2766468266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784756070 msgr2=0x559784b14b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f99037fe640 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784756070 0x559784b14b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f99037fe640 1 -- 172.21.15.112:0/2766468266 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559784a2bc40 con 0x559784735390 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f9908cd1640 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55978472d1b0 0x559784b088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f99037fe640 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784735390 0x559784b08e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f98f8012cf0 tx=0x7f98f800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d85f0aec2e139c4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.266+0000 7f98e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f98e37fe640 1 -- 172.21.15.112:0/2766468266 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8012110 con 0x559784735390 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f98e37fe640 1 -- 172.21.15.112:0/2766468266 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f98f8004050 con 0x559784735390 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f9910ca8740 1 -- 172.21.15.112:0/2766468266 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98fc036030 con 0x559784735390 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f98e37fe640 1 -- 172.21.15.112:0/2766468266 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f98f80122b0 con 0x559784735390 2024-09-21T03:38:26.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f9910ca8740 10 client.26614 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f9910ca8740 1 -- 172.21.15.112:0/2766468266 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559784b17580 con 0x559784735390 2024-09-21T03:38:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f98e37fe640 1 -- 172.21.15.112:0/2766468266 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98fc036030 con 0x559784735390 2024-09-21T03:38:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f98e37fe640 1 -- 172.21.15.112:0/2766468266 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f98f8010430 con 0x559784735390 2024-09-21T03:38:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f9910ca8740 10 client.26614 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f9910ca8740 10 client.26614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:38:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f9910ca8740 20 client.26614 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f9910ca8740 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597847149f0 0x559784b1aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f9910ca8740 4 client.26614 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.267+0000 7f9910ca8740 1 -- 172.21.15.112:0/2766468266 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559784a75da0 con 0x5597847149f0 2024-09-21T03:38:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.268+0000 7f9908cd1640 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597847149f0 0x559784b1aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.268+0000 7f98e37fe640 10 client.26614 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.268+0000 7f9908cd1640 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597847149f0 0x559784b1aac0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=968372cec5c182c5 server_cookie=4c0aedb792d247a in_seq=0 out_seq=0 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f98e37fe640 1 -- 172.21.15.112:0/2766468266 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559784a75da0 con 0x5597847149f0 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f98e37fe640 10 client.26614 handle_command_reply: tid=0 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 1 client.26614 shutdown 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 1 -- 172.21.15.112:0/2766468266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784735390 msgr2=0x559784b08e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784735390 0x559784b08e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f98f8012cf0 tx=0x7f98f800f790 comp rx=0 tx=0).stop 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 1 -- 172.21.15.112:0/2766468266 shutdown_connections 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597847149f0 0x559784b1aac0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784756070 0x559784b14b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784735390 0x559784b08e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f98e37fe640 0 client.26614 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:26.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 1 --2- 172.21.15.112:0/2766468266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55978472d1b0 0x559784b088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 1 -- 172.21.15.112:0/2766468266 >> 172.21.15.112:0/2766468266 conn(0x55978474a460 msgr2=0x559784b00450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:26.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 1 -- 172.21.15.112:0/2766468266 shutdown_connections 2024-09-21T03:38:26.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 1 -- 172.21.15.112:0/2766468266 wait complete. 2024-09-21T03:38:26.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.269+0000 7f9910ca8740 20 client.26614 trim_cache size 0 max 16384 2024-09-21T03:38:26.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.289+0000 7f9910ca8740 1 Processor -- start 2024-09-21T03:38:26.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.289+0000 7f9910ca8740 1 -- start start 2024-09-21T03:38:26.295 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:26 smithi112 ceph-mon[31442]: pgmap v608: 97 pgs: 97 active+clean; 762 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2024-09-21T03:38:26.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55978472d1b0 0x559784b08d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 0x559784b14a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 0x559784b05ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9910ca8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99041afc50 con 0x55978472d1b0 2024-09-21T03:38:26.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9910ca8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99041ad520 con 0x559784735390 2024-09-21T03:38:26.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9910ca8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99041ab370 con 0x559784756070 2024-09-21T03:38:26.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9908cd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55978472d1b0 0x559784b08d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9908cd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55978472d1b0 0x559784b08d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46602/0 (socket says 172.21.15.112:46602) 2024-09-21T03:38:26.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9908cd1640 1 -- 172.21.15.112:0/2590829335 learned_addr learned my addr 172.21.15.112:0/2590829335 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:26.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f99037fe640 1 --2- 172.21.15.112:0/2590829335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 0x559784b14a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9908cd1640 1 --2- 172.21.15.112:0/2590829335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 0x559784b05ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9908cd1640 1 -- 172.21.15.112:0/2590829335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 msgr2=0x559784b05ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9908cd1640 1 --2- 172.21.15.112:0/2590829335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 0x559784b05ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9908cd1640 1 -- 172.21.15.112:0/2590829335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 msgr2=0x559784b14a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9908cd1640 1 --2- 172.21.15.112:0/2590829335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 0x559784b14a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f9908cd1640 1 -- 172.21.15.112:0/2590829335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559784b06dd0 con 0x55978472d1b0 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.290+0000 7f99037fe640 1 --2- 172.21.15.112:0/2590829335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 0x559784b14a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.291+0000 7f9908cd1640 1 --2- 172.21.15.112:0/2590829335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55978472d1b0 0x559784b08d90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f98fc003ae0 tx=0x7f98fc003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5cc5f9c0229640 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.291+0000 7f98e9ffb640 1 -- 172.21.15.112:0/2590829335 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc063070 con 0x55978472d1b0 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.291+0000 7f98e9ffb640 1 -- 172.21.15.112:0/2590829335 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98fc036040 con 0x55978472d1b0 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.291+0000 7f98e9ffb640 1 -- 172.21.15.112:0/2590829335 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc040bb0 con 0x55978472d1b0 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.292+0000 7f9910ca8740 1 -- 172.21.15.112:0/2590829335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55978472d1b0 msgr2=0x559784b08d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.292+0000 7f9910ca8740 1 --2- 172.21.15.112:0/2590829335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55978472d1b0 0x559784b08d90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f98fc003ae0 tx=0x7f98fc003b10 comp rx=0 tx=0).stop 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.292+0000 7f9910ca8740 1 -- 172.21.15.112:0/2590829335 shutdown_connections 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.292+0000 7f9910ca8740 1 --2- 172.21.15.112:0/2590829335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559784756070 0x559784b05ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.292+0000 7f9910ca8740 1 --2- 172.21.15.112:0/2590829335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 0x559784b14a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.292+0000 7f9910ca8740 1 --2- 172.21.15.112:0/2590829335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55978472d1b0 0x559784b08d90 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.292+0000 7f9910ca8740 1 -- 172.21.15.112:0/2590829335 >> 172.21.15.112:0/2590829335 conn(0x55978474a460 msgr2=0x559784b08680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.292+0000 7f9910ca8740 1 -- 172.21.15.112:0/2590829335 shutdown_connections 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.292+0000 7f9910ca8740 1 -- 172.21.15.112:0/2590829335 wait complete. 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.293+0000 7f9910ca8740 1 -- start start 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.293+0000 7f9910ca8740 1 Processor -- start 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55978472d1b0 0x559784b0ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 0x559784b0d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f9910ca8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784756070 0x559784b844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f9910ca8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99041ab9e0 con 0x559784756070 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f9910ca8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99041ace50 con 0x559784735390 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f9910ca8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99041ac0b0 con 0x55978472d1b0 2024-09-21T03:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f9910ca8740 10 client.0 _ll_register_callbacks cb 0x7f98f8009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f99037fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 0x559784b0d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f99037fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 0x559784b0d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33832/0 (socket says 172.21.15.112:33832) 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f99037fe640 1 -- 172.21.15.112:0/4013572765 learned_addr learned my addr 172.21.15.112:0/4013572765 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f9908cd1640 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784756070 0x559784b844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f9908cd1640 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55978472d1b0 0x559784b0ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f99037fe640 1 -- 172.21.15.112:0/4013572765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55978472d1b0 msgr2=0x559784b0ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f99037fe640 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55978472d1b0 0x559784b0ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f99037fe640 1 -- 172.21.15.112:0/4013572765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784756070 msgr2=0x559784b844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f99037fe640 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784756070 0x559784b844c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.294+0000 7f99037fe640 1 -- 172.21.15.112:0/4013572765 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559784b0da20 con 0x559784735390 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f99037fe640 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 0x559784b0d230 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98f8010850 tx=0x7f98f8010590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b19a3619404ce31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f98e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f98e37fe640 1 -- 172.21.15.112:0/4013572765 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8029040 con 0x559784735390 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f98e37fe640 1 -- 172.21.15.112:0/4013572765 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f98f8005110 con 0x559784735390 2024-09-21T03:38:26.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f9910ca8740 1 -- 172.21.15.112:0/4013572765 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98fc036040 con 0x559784735390 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f98e37fe640 1 -- 172.21.15.112:0/4013572765 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f98f8005310 con 0x559784735390 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f9910ca8740 10 client.26615 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f9910ca8740 1 -- 172.21.15.112:0/4013572765 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559784b859f0 con 0x559784735390 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f98e37fe640 1 -- 172.21.15.112:0/4013572765 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98fc036040 con 0x559784735390 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f98e37fe640 1 -- 172.21.15.112:0/4013572765 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f98f802c070 con 0x559784735390 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f9910ca8740 10 client.26615 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.295+0000 7f9910ca8740 10 client.26615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.296+0000 7f9910ca8740 20 client.26615 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.296+0000 7f9910ca8740 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597846e9da0 0x559784b88230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.296+0000 7f9910ca8740 4 client.26615 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.296+0000 7f9910ca8740 1 -- 172.21.15.112:0/4013572765 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559784a71850 con 0x5597846e9da0 2024-09-21T03:38:26.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.296+0000 7f9908cd1640 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597846e9da0 0x559784b88230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:26.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.296+0000 7f98e37fe640 10 client.26615 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:26.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.296+0000 7f9908cd1640 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597846e9da0 0x559784b88230 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b25360ee377b85 server_cookie=7fab5d8218649a6d in_seq=0 out_seq=0 2024-09-21T03:38:26.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.297+0000 7f98e37fe640 1 -- 172.21.15.112:0/4013572765 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x559784a71850 con 0x5597846e9da0 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.297+0000 7f98e37fe640 10 client.26615 handle_command_reply: tid=0 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.297+0000 7f9910ca8740 1 client.26615 shutdown 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.297+0000 7f9910ca8740 1 -- 172.21.15.112:0/4013572765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 msgr2=0x559784b0d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.297+0000 7f9910ca8740 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 0x559784b0d230 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98f8010850 tx=0x7f98f8010590 comp rx=0 tx=0).stop 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.298+0000 7f9910ca8740 1 -- 172.21.15.112:0/4013572765 shutdown_connections 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.298+0000 7f9910ca8740 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597846e9da0 0x559784b88230 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.298+0000 7f9910ca8740 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559784756070 0x559784b844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.298+0000 7f9910ca8740 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559784735390 0x559784b0d230 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.298+0000 7f9910ca8740 1 --2- 172.21.15.112:0/4013572765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55978472d1b0 0x559784b0ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.298+0000 7f98e37fe640 0 client.26615 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.298+0000 7f9910ca8740 1 -- 172.21.15.112:0/4013572765 >> 172.21.15.112:0/4013572765 conn(0x55978474a460 msgr2=0x559784afe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.298+0000 7f9910ca8740 1 -- 172.21.15.112:0/4013572765 shutdown_connections 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.298+0000 7f9910ca8740 1 -- 172.21.15.112:0/4013572765 wait complete. 2024-09-21T03:38:26.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.298+0000 7f9910ca8740 20 client.26615 trim_cache size 0 max 16384 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:38:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:38:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:38:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:38:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:38:26.314 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 -- 172.21.15.112:0/1078924161 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98ec0b6680 msgr2=0x7f98ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98ec0b6680 0x7f98ec0b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f98f40e52d0 tx=0x7f98f40eb5f0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 -- 172.21.15.112:0/1078924161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99041049b0 msgr2=0x7f990419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99041049b0 0x7f990419a460 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e9c0 tx=0x7f98f400ee80 comp rx=0 tx=0).stop 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 -- 172.21.15.112:0/1078924161 shutdown_connections 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98ec0b6680 0x7f98ec0b8b90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f990410de10 0x7f990419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9904105300 0x7f990419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 --2- 172.21.15.112:0/1078924161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99041049b0 0x7f990419a460 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 -- 172.21.15.112:0/1078924161 >> 172.21.15.112:0/1078924161 conn(0x7f9904079730 msgr2=0x7f99040ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 -- 172.21.15.112:0/1078924161 shutdown_connections 2024-09-21T03:38:26.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:26.299+0000 7f990a734640 1 -- 172.21.15.112:0/1078924161 wait complete. 2024-09-21T03:38:26.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:26 smithi203 ceph-mon[34723]: pgmap v608: 97 pgs: 97 active+clean; 762 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2024-09-21T03:38:26.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:26 smithi187 ceph-mon[34603]: pgmap v608: 97 pgs: 97 active+clean; 762 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2024-09-21T03:38:27.225 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:38:27.226 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:38:27.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 999207 10.54 MB/sec execute 387 sec latency 27.273 ms 2024-09-21T03:38:27.522 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:28.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.026+0000 7fbce52bf640 1 -- 172.21.15.112:0/3309765881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce01021f0 msgr2=0x7fbce0102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.026+0000 7fbce52bf640 1 --2- 172.21.15.112:0/3309765881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce01021f0 0x7fbce0102600 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009900 tx=0x7fbcd002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.026+0000 7fbce52bf640 1 -- 172.21.15.112:0/3309765881 shutdown_connections 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.026+0000 7fbce52bf640 1 --2- 172.21.15.112:0/3309765881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbce0108480 0x7fbce010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.026+0000 7fbce52bf640 1 --2- 172.21.15.112:0/3309765881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbce0107b30 0x7fbce0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.026+0000 7fbce52bf640 1 --2- 172.21.15.112:0/3309765881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce01021f0 0x7fbce0102600 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.026+0000 7fbce52bf640 1 -- 172.21.15.112:0/3309765881 >> 172.21.15.112:0/3309765881 conn(0x7fbce0077c60 msgr2=0x7fbce00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.026+0000 7fbce52bf640 1 -- 172.21.15.112:0/3309765881 shutdown_connections 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.026+0000 7fbce52bf640 1 -- 172.21.15.112:0/3309765881 wait complete. 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbce52bf640 1 Processor -- start 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbce52bf640 1 -- start start 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbce52bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbce01021f0 0x7fbce019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbce52bf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbce0107b30 0x7fbce019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbce52bf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0108480 0x7fbce019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbcdf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0108480 0x7fbce019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbcdf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0108480 0x7fbce019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46616/0 (socket says 172.21.15.112:46616) 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbcdf7fe640 1 -- 172.21.15.112:0/3894349609 learned_addr learned my addr 172.21.15.112:0/3894349609 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbcdeffd640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbce01021f0 0x7fbce019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01129c0 con 0x7fbce0108480 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0112840 con 0x7fbce0107b30 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0112b40 con 0x7fbce01021f0 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.027+0000 7fbcd7fff640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbce0107b30 0x7fbce019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.028+0000 7fbcdf7fe640 1 -- 172.21.15.112:0/3894349609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbce01021f0 msgr2=0x7fbce019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.028+0000 7fbcdf7fe640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbce01021f0 0x7fbce019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.028+0000 7fbcdf7fe640 1 -- 172.21.15.112:0/3894349609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbce0107b30 msgr2=0x7fbce019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.028+0000 7fbcdf7fe640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbce0107b30 0x7fbce019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.028+0000 7fbcdf7fe640 1 -- 172.21.15.112:0/3894349609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce01a0490 con 0x7fbce0108480 2024-09-21T03:38:28.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.028+0000 7fbcdf7fe640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0108480 0x7fbce019fd60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800eb10 tx=0x7fbcd800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.029+0000 7fbcdcff9640 1 -- 172.21.15.112:0/3894349609 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8004b20 con 0x7fbce0108480 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.029+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce01a0780 con 0x7fbce0108480 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.029+0000 7fbcdcff9640 1 -- 172.21.15.112:0/3894349609 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd8004cc0 con 0x7fbce0108480 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.029+0000 7fbcdcff9640 1 -- 172.21.15.112:0/3894349609 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8010670 con 0x7fbce0108480 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.029+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce01a6ee0 con 0x7fbce0108480 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.030+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce0102600 con 0x7fbce0108480 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.031+0000 7fbcdcff9640 1 -- 172.21.15.112:0/3894349609 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fbcd8005000 con 0x7fbce0108480 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.032+0000 7fbcdcff9640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbccc0b6680 0x7fbccc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.032+0000 7fbcdeffd640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbccc0b6680 0x7fbccc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.032+0000 7fbcdcff9640 1 -- 172.21.15.112:0/3894349609 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7fbcd80e7dc0 con 0x7fbce0108480 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.033+0000 7fbcdeffd640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbccc0b6680 0x7fbccc0b8b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0031440 tx=0x7fbcd0009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:28.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.036+0000 7fbcdcff9640 1 -- 172.21.15.112:0/3894349609 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbcd8056710 con 0x7fbce0108480 2024-09-21T03:38:28.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1001680 10.54 MB/sec execute 388 sec latency 34.273 ms 2024-09-21T03:38:28.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:28 smithi112 ceph-mon[31442]: pgmap v609: 97 pgs: 97 active+clean; 750 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 294 op/s 2024-09-21T03:38:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.318+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbce019b650 con 0x7fbce0108480 2024-09-21T03:38:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.318+0000 7fbcdcff9640 1 -- 172.21.15.112:0/3894349609 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7fbce019b650 con 0x7fbce0108480 2024-09-21T03:38:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:38:28.321 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"btime":"2024-09-21T03:35:32:357323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:35:32.307199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:38:28.321 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-09-21T03:38:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbccc0b6680 msgr2=0x7fbccc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbccc0b6680 0x7fbccc0b8b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0031440 tx=0x7fbcd0009560 comp rx=0 tx=0).stop 2024-09-21T03:38:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0108480 msgr2=0x7fbce019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0108480 0x7fbce019fd60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800eb10 tx=0x7fbcd800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 shutdown_connections 2024-09-21T03:38:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbccc0b6680 0x7fbccc0b8b90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0108480 0x7fbce019fd60 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbce0107b30 0x7fbce019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 --2- 172.21.15.112:0/3894349609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbce01021f0 0x7fbce019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 >> 172.21.15.112:0/3894349609 conn(0x7fbce0077c60 msgr2=0x7fbce010fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 shutdown_connections 2024-09-21T03:38:28.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:28.321+0000 7fbce52bf640 1 -- 172.21.15.112:0/3894349609 wait complete. 2024-09-21T03:38:28.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:28 smithi203 ceph-mon[34723]: pgmap v609: 97 pgs: 97 active+clean; 750 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 294 op/s 2024-09-21T03:38:28.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:28 smithi187 ceph-mon[34603]: pgmap v609: 97 pgs: 97 active+clean; 750 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 294 op/s 2024-09-21T03:38:29.232 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:38:29.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1003249 10.54 MB/sec execute 389 sec latency 18.295 ms 2024-09-21T03:38:29.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:29 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3894349609' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:29.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:29 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3894349609' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:29.537 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:29.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:29 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3894349609' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:38:30.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.061+0000 7f2d15842640 1 -- 172.21.15.112:0/1703605971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d10108cf0 msgr2=0x7f2d1010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.061+0000 7f2d15842640 1 --2- 172.21.15.112:0/1703605971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d10108cf0 0x7f2d1010f520 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009ca0 tx=0x7f2d0802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.061+0000 7f2d15842640 1 -- 172.21.15.112:0/1703605971 shutdown_connections 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.061+0000 7f2d15842640 1 --2- 172.21.15.112:0/1703605971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d10108cf0 0x7f2d1010f520 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.061+0000 7f2d15842640 1 --2- 172.21.15.112:0/1703605971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d101083a0 0x7f2d101087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.061+0000 7f2d15842640 1 --2- 172.21.15.112:0/1703605971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d10102a60 0x7f2d10102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.061+0000 7f2d15842640 1 -- 172.21.15.112:0/1703605971 >> 172.21.15.112:0/1703605971 conn(0x7f2d10079730 msgr2=0x7f2d10079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.061+0000 7f2d15842640 1 -- 172.21.15.112:0/1703605971 shutdown_connections 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.061+0000 7f2d15842640 1 -- 172.21.15.112:0/1703605971 wait complete. 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d15842640 1 Processor -- start 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d15842640 1 -- start start 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d15842640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d10102a60 0x7f2d1019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d15842640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d101083a0 0x7f2d1019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d15842640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d10108cf0 0x7f2d1019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d15842640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d10112960 con 0x7f2d10102a60 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d15842640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101127e0 con 0x7f2d10108cf0 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d15842640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d10112ae0 con 0x7f2d101083a0 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d0effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d10102a60 0x7f2d1019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d0effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d10102a60 0x7f2d1019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46624/0 (socket says 172.21.15.112:46624) 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d0effd640 1 -- 172.21.15.112:0/1827892770 learned_addr learned my addr 172.21.15.112:0/1827892770 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d067fc640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d101083a0 0x7f2d1019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.062+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d10108cf0 0x7f2d1019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.063+0000 7f2d0effd640 1 -- 172.21.15.112:0/1827892770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d101083a0 msgr2=0x7f2d1019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.063+0000 7f2d0effd640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d101083a0 0x7f2d1019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.063+0000 7f2d0effd640 1 -- 172.21.15.112:0/1827892770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d10108cf0 msgr2=0x7f2d1019bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.063+0000 7f2d0effd640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d10108cf0 0x7f2d1019bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.063+0000 7f2d0effd640 1 -- 172.21.15.112:0/1827892770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d101a6560 con 0x7f2d10102a60 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.063+0000 7f2d067fc640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d101083a0 0x7f2d1019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:30.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.063+0000 7f2d0effd640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d10102a60 0x7f2d1019b2d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000e9f0 tx=0x7f2d0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.067+0000 7f2d14840640 1 -- 172.21.15.112:0/1827892770 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0000cd80 con 0x7f2d10102a60 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.067+0000 7f2d15842640 1 -- 172.21.15.112:0/1827892770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d101a6850 con 0x7f2d10102a60 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.067+0000 7f2d14840640 1 -- 172.21.15.112:0/1827892770 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d000108e0 con 0x7f2d10102a60 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.067+0000 7f2d14840640 1 -- 172.21.15.112:0/1827892770 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00018980 con 0x7f2d10102a60 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.067+0000 7f2d15842640 1 -- 172.21.15.112:0/1827892770 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d101a6de0 con 0x7f2d10102a60 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d1bdb6740 1 Processor -- start 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d1bdb6740 1 -- start start 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d14840640 1 -- 172.21.15.112:0/1827892770 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2d00018b20 con 0x7f2d10102a60 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 0x55b0e1da06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1da0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1da1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b0e1d16ea0 con 0x55b0e19d01b0 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b0e1d062d0 con 0x55b0e19d8390 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b0e1d0a0e0 con 0x55b0e19f9070 2024-09-21T03:38:30.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d0effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1da0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d0f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 0x55b0e1da06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d0f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 0x55b0e1da06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43324/0 (socket says 172.21.15.112:43324) 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.070+0000 7f2d0f7fe640 1 -- 172.21.15.112:0/262456797 learned_addr learned my addr 172.21.15.112:0/262456797 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d067fc640 1 --2- 172.21.15.112:0/262456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1da1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d0f7fe640 1 -- 172.21.15.112:0/262456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 msgr2=0x55b0e1da1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/262456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1da1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d0f7fe640 1 -- 172.21.15.112:0/262456797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 msgr2=0x55b0e1da0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/262456797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1da0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d0f7fe640 1 -- 172.21.15.112:0/262456797 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e1a64650 con 0x55b0e19f9070 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/262456797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 0x55b0e1da06a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d0802f410 tx=0x7f2d0802f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe2b711bb3a9758 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d14840640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2cfc0b66b0 0x7f2cfc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d05ffb640 1 -- 172.21.15.112:0/262456797 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08031cf0 con 0x55b0e19f9070 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d0effd640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2cfc0b66b0 0x7f2cfc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d14840640 1 -- 172.21.15.112:0/1827892770 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f2d000e79e0 con 0x7f2d10102a60 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d05ffb640 1 -- 172.21.15.112:0/262456797 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d08036030 con 0x55b0e19f9070 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.071+0000 7f2d05ffb640 1 -- 172.21.15.112:0/262456797 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08004240 con 0x55b0e19f9070 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.072+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/262456797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 msgr2=0x55b0e1da06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.072+0000 7f2d0effd640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2cfc0b66b0 0x7f2cfc0b8bc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2d000e5210 tx=0x7f2d000ec040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.072+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/262456797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 0x55b0e1da06a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d0802f410 tx=0x7f2d0802f440 comp rx=0 tx=0).stop 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.072+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/262456797 shutdown_connections 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.072+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/262456797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1da1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.072+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/262456797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1da0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.072+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/262456797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 0x55b0e1da06a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.072+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/262456797 >> 172.21.15.112:0/262456797 conn(0x55b0e19ed460 msgr2=0x55b0e1d9e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.072+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/262456797 shutdown_connections 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.072+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/262456797 wait complete. 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d1bdb6740 1 -- start start 2024-09-21T03:38:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d1bdb6740 1 Processor -- start 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1dabd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1dac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 0x55b0e1db7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b0e1b67700 con 0x55b0e19d01b0 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b0e1ca1160 con 0x55b0e19d8390 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b0e1daeca0 con 0x55b0e19f9070 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d1bdb6740 10 client.0 _ll_register_callbacks cb 0x55b0e1cba9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d0f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1dabd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d0f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1dabd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46640/0 (socket says 172.21.15.112:46640) 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.073+0000 7f2d0f7fe640 1 -- 172.21.15.112:0/3944051815 learned_addr learned my addr 172.21.15.112:0/3944051815 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2d067fc640 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1dac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 0x55b0e1db7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2d0f7fe640 1 -- 172.21.15.112:0/3944051815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 msgr2=0x55b0e1db7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 0x55b0e1db7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2d0f7fe640 1 -- 172.21.15.112:0/3944051815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 msgr2=0x55b0e1dac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1dac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2d0f7fe640 1 -- 172.21.15.112:0/3944051815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e1ccec90 con 0x55b0e19d01b0 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1dabd30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009a60 tx=0x7f2d08003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f167e8d5f6048d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2ceeffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2ceeffd640 1 -- 172.21.15.112:0/3944051815 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08058060 con 0x55b0e19d01b0 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2ceeffd640 1 -- 172.21.15.112:0/3944051815 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f2d08031860 con 0x55b0e19d01b0 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/3944051815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d08036030 con 0x55b0e19d01b0 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2ceeffd640 1 -- 172.21.15.112:0/3944051815 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d0802f660 con 0x55b0e19d01b0 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2d1bdb6740 10 client.17079 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.074+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/3944051815 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b0e1dba980 con 0x55b0e19d01b0 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.075+0000 7f2ceeffd640 1 -- 172.21.15.112:0/3944051815 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d08036030 con 0x55b0e19d01b0 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.075+0000 7f2ceeffd640 1 -- 172.21.15.112:0/3944051815 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2d08043070 con 0x55b0e19d01b0 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.075+0000 7f2d1bdb6740 10 client.17079 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:30.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.075+0000 7f2d1bdb6740 10 client.17079 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.075+0000 7f2d1bdb6740 20 client.17079 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.075+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55b0e19b79f0 0x55b0e1dbdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.075+0000 7f2d1bdb6740 4 client.17079 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.075+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/3944051815 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0e1d18df0 con 0x55b0e19b79f0 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.075+0000 7f2d067fc640 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55b0e19b79f0 0x55b0e1dbdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.075+0000 7f2ceeffd640 10 client.17079 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.076+0000 7f2d067fc640 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55b0e19b79f0 0x55b0e1dbdce0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73cf74faa2d5bef2 server_cookie=be7008efe0f835e8 in_seq=0 out_seq=0 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2ceeffd640 1 -- 172.21.15.112:0/3944051815 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b0e1d18df0 con 0x55b0e19b79f0 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2ceeffd640 10 client.17079 handle_command_reply: tid=0 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 1 client.17079 shutdown 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/3944051815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 msgr2=0x55b0e1dabd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1dabd30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009a60 tx=0x7f2d08003d70 comp rx=0 tx=0).stop 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/3944051815 shutdown_connections 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55b0e19b79f0 0x55b0e1dbdce0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19f9070 0x55b0e1db7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2ceeffd640 0 client.17079 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1dac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/3944051815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1dabd30 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/3944051815 >> 172.21.15.112:0/3944051815 conn(0x55b0e19ed460 msgr2=0x55b0e1da3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/3944051815 shutdown_connections 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/3944051815 wait complete. 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.077+0000 7f2d1bdb6740 20 client.17079 trim_cache size 0 max 16384 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d1bdb6740 1 Processor -- start 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d1bdb6740 1 -- start start 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19b79f0 0x55b0e1da2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1db7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1dad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101b1e80 con 0x55b0e19d01b0 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101af750 con 0x55b0e19d8390 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101ad5c0 con 0x55b0e19b79f0 2024-09-21T03:38:30.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1db7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1db7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46654/0 (socket says 172.21.15.112:46654) 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d067fc640 1 -- 172.21.15.112:0/4017785926 learned_addr learned my addr 172.21.15.112:0/4017785926 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/4017785926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1dad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.091+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/4017785926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19b79f0 0x55b0e1da2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.092+0000 7f2d067fc640 1 -- 172.21.15.112:0/4017785926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19b79f0 msgr2=0x55b0e1da2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.092+0000 7f2d067fc640 1 --2- 172.21.15.112:0/4017785926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19b79f0 0x55b0e1da2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.092+0000 7f2d067fc640 1 -- 172.21.15.112:0/4017785926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 msgr2=0x55b0e1dad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.092+0000 7f2d067fc640 1 --2- 172.21.15.112:0/4017785926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1dad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.092+0000 7f2d067fc640 1 -- 172.21.15.112:0/4017785926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e1da3500 con 0x55b0e19d01b0 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.092+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/4017785926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19b79f0 0x55b0e1da2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.092+0000 7f2d067fc640 1 --2- 172.21.15.112:0/4017785926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1db7f00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2d1006aaa0 tx=0x7f2cf8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fc6ce3f0b3e4fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.092+0000 7f2d057fa640 1 -- 172.21.15.112:0/4017785926 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8005220 con 0x55b0e19d01b0 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.092+0000 7f2d057fa640 1 -- 172.21.15.112:0/4017785926 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf8004c50 con 0x55b0e19d01b0 2024-09-21T03:38:30.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.092+0000 7f2d057fa640 1 -- 172.21.15.112:0/4017785926 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf801f8f0 con 0x55b0e19d01b0 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.093+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/4017785926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 msgr2=0x55b0e1db7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.093+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/4017785926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1db7f00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2d1006aaa0 tx=0x7f2cf8005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.093+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/4017785926 shutdown_connections 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.093+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/4017785926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d8390 0x55b0e1dad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.093+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/4017785926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d01b0 0x55b0e1db7f00 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.093+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/4017785926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19b79f0 0x55b0e1da2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.093+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/4017785926 >> 172.21.15.112:0/4017785926 conn(0x55b0e19ed460 msgr2=0x55b0e1dad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.093+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/4017785926 shutdown_connections 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.093+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/4017785926 wait complete. 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d1bdb6740 1 -- start start 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d1bdb6740 1 Processor -- start 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19b79f0 0x55b0e1dbf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d01b0 0x55b0e1dbf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d1bdb6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d8390 0x55b0e1e27460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101adc10 con 0x55b0e19d8390 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101af080 con 0x55b0e19d01b0 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d1bdb6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101ae2e0 con 0x55b0e19b79f0 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d1bdb6740 10 client.0 _ll_register_callbacks cb 0x7f2d000eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d0f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d8390 0x55b0e1e27460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d067fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d01b0 0x55b0e1dbf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d067fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d01b0 0x55b0e1dbf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33880/0 (socket says 172.21.15.112:33880) 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d067fc640 1 -- 172.21.15.112:0/125919984 learned_addr learned my addr 172.21.15.112:0/125919984 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d0f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d8390 0x55b0e1e27460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46660/0 (socket says 172.21.15.112:46660) 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.094+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19b79f0 0x55b0e1dbf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2d067fc640 1 -- 172.21.15.112:0/125919984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19b79f0 msgr2=0x55b0e1dbf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2d067fc640 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19b79f0 0x55b0e1dbf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2d067fc640 1 -- 172.21.15.112:0/125919984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d8390 msgr2=0x55b0e1e27460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2d067fc640 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d8390 0x55b0e1e27460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2d067fc640 1 -- 172.21.15.112:0/125919984 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e1e279a0 con 0x55b0e19d01b0 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d8390 0x55b0e1e27460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2d067fc640 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d01b0 0x55b0e1dbf970 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8012480 tx=0x7f2cf801f590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740083dfcba37dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2ceeffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2ceeffd640 1 -- 172.21.15.112:0/125919984 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8029040 con 0x55b0e19d01b0 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2ceeffd640 1 -- 172.21.15.112:0/125919984 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f2cf8004930 con 0x55b0e19d01b0 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/125919984 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cf8004c50 con 0x55b0e19d01b0 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2ceeffd640 1 -- 172.21.15.112:0/125919984 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2cf803c050 con 0x55b0e19d01b0 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2d1bdb6740 10 client.26639 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.095+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/125919984 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b0e1e28bb0 con 0x55b0e19d01b0 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.096+0000 7f2ceeffd640 1 -- 172.21.15.112:0/125919984 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cf8004c50 con 0x55b0e19d01b0 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.096+0000 7f2ceeffd640 1 -- 172.21.15.112:0/125919984 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2cf801faf0 con 0x55b0e19d01b0 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.096+0000 7f2d1bdb6740 10 client.26639 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.096+0000 7f2d1bdb6740 10 client.26639 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.096+0000 7f2d1bdb6740 20 client.26639 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:30.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.096+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55b0e19f9070 0x55b0e1e2b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.096+0000 7f2d1bdb6740 4 client.26639 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.096+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/125919984 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0e1dac620 con 0x55b0e19f9070 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.096+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55b0e19f9070 0x55b0e1e2b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.096+0000 7f2ceeffd640 10 client.26639 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.097+0000 7f2d0f7fe640 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55b0e19f9070 0x55b0e1e2b2b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a693436d8228fd3 server_cookie=f774748c1f51246e in_seq=0 out_seq=0 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.097+0000 7f2ceeffd640 1 -- 172.21.15.112:0/125919984 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0e1dac620 con 0x55b0e19f9070 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.097+0000 7f2ceeffd640 10 client.26639 handle_command_reply: tid=0 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.097+0000 7f2d1bdb6740 1 client.26639 shutdown 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.097+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/125919984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d01b0 msgr2=0x55b0e1dbf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.097+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d01b0 0x55b0e1dbf970 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8012480 tx=0x7f2cf801f590 comp rx=0 tx=0).stop 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/125919984 shutdown_connections 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55b0e19f9070 0x55b0e1e2b2b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0e19d8390 0x55b0e1e27460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b0e19d01b0 0x55b0e1dbf970 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d1bdb6740 1 --2- 172.21.15.112:0/125919984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0e19b79f0 0x55b0e1dbf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/125919984 >> 172.21.15.112:0/125919984 conn(0x55b0e19ed460 msgr2=0x55b0e1dadb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:30.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2ceeffd640 0 client.26639 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:38:30.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/125919984 shutdown_connections 2024-09-21T03:38:30.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d1bdb6740 1 -- 172.21.15.112:0/125919984 wait complete. 2024-09-21T03:38:30.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d1bdb6740 20 client.26639 trim_cache size 0 max 16384 2024-09-21T03:38:30.115 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:38:30.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d15842640 1 -- 172.21.15.112:0/1827892770 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2cfc0b66b0 msgr2=0x7f2cfc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:30.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d15842640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2cfc0b66b0 0x7f2cfc0b8bc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2d000e5210 tx=0x7f2d000ec040 comp rx=0 tx=0).stop 2024-09-21T03:38:30.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.098+0000 7f2d15842640 1 -- 172.21.15.112:0/1827892770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d10102a60 msgr2=0x7f2d1019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:30.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.099+0000 7f2d15842640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d10102a60 0x7f2d1019b2d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000e9f0 tx=0x7f2d0000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.099+0000 7f2d15842640 1 -- 172.21.15.112:0/1827892770 shutdown_connections 2024-09-21T03:38:30.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.099+0000 7f2d15842640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2cfc0b66b0 0x7f2cfc0b8bc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.099+0000 7f2d15842640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d10108cf0 0x7f2d1019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.099+0000 7f2d15842640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d101083a0 0x7f2d1019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.099+0000 7f2d15842640 1 --2- 172.21.15.112:0/1827892770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d10102a60 0x7f2d1019b2d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:30.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.099+0000 7f2d15842640 1 -- 172.21.15.112:0/1827892770 >> 172.21.15.112:0/1827892770 conn(0x7f2d10079730 msgr2=0x7f2d10104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:30.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.099+0000 7f2d15842640 1 -- 172.21.15.112:0/1827892770 shutdown_connections 2024-09-21T03:38:30.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:30.099+0000 7f2d15842640 1 -- 172.21.15.112:0/1827892770 wait complete. 2024-09-21T03:38:30.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1005627 10.55 MB/sec execute 390 sec latency 19.379 ms 2024-09-21T03:38:30.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:30 smithi112 ceph-mon[31442]: pgmap v610: 97 pgs: 97 active+clean; 754 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-09-21T03:38:30.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:30 smithi203 ceph-mon[34723]: pgmap v610: 97 pgs: 97 active+clean; 754 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-09-21T03:38:30.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:30 smithi187 ceph-mon[34603]: pgmap v610: 97 pgs: 97 active+clean; 754 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-09-21T03:38:30.877 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:38:31.145 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:31.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1007109 10.54 MB/sec execute 391 sec latency 18.220 ms 2024-09-21T03:38:31.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.670+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/2780471800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d08104840 msgr2=0x7f0d08104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.670+0000 7f0d0fc0f640 1 --2- 172.21.15.112:0/2780471800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d08104840 0x7f0d08104c50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009ca0 tx=0x7f0d0002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.670+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/2780471800 shutdown_connections 2024-09-21T03:38:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.670+0000 7f0d0fc0f640 1 --2- 172.21.15.112:0/2780471800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d0810ddf0 0x7f0d08110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.670+0000 7f0d0fc0f640 1 --2- 172.21.15.112:0/2780471800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d08105270 0x7f0d0810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.670+0000 7f0d0fc0f640 1 --2- 172.21.15.112:0/2780471800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d08104840 0x7f0d08104c50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.670+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/2780471800 >> 172.21.15.112:0/2780471800 conn(0x7f0d08079730 msgr2=0x7f0d08079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.670+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/2780471800 shutdown_connections 2024-09-21T03:38:31.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.670+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/2780471800 wait complete. 2024-09-21T03:38:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.671+0000 7f0d0fc0f640 1 Processor -- start 2024-09-21T03:38:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.671+0000 7f0d0fc0f640 1 -- start start 2024-09-21T03:38:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.671+0000 7f0d0fc0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d08104840 0x7f0d0819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0fc0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d08105270 0x7f0d0819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0fc0f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d0810ddf0 0x7f0d0819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0fc0f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08113a90 con 0x7f0d08105270 2024-09-21T03:38:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0fc0f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08113910 con 0x7f0d0810ddf0 2024-09-21T03:38:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0fc0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08113c10 con 0x7f0d08104840 2024-09-21T03:38:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0d1aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d08105270 0x7f0d0819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0d9ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d08104840 0x7f0d0819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0d1aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d08105270 0x7f0d0819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46672/0 (socket says 172.21.15.112:46672) 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/3403906710 learned_addr learned my addr 172.21.15.112:0/3403906710 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0e1ac640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d0810ddf0 0x7f0d0819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/3403906710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d08104840 msgr2=0x7f0d0819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d08104840 0x7f0d0819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/3403906710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d0810ddf0 msgr2=0x7f0d0819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d0810ddf0 0x7f0d0819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/3403906710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d081a04f0 con 0x7f0d08105270 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0d9ab640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d08104840 0x7f0d0819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d08105270 0x7f0d0819ab20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00e9f0 tx=0x7f0cfc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d06ffd640 1 -- 172.21.15.112:0/3403906710 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc00ccc0 con 0x7f0d08105270 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d06ffd640 1 -- 172.21.15.112:0/3403906710 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cfc0108e0 con 0x7f0d08105270 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d06ffd640 1 -- 172.21.15.112:0/3403906710 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc018860 con 0x7f0d08105270 2024-09-21T03:38:31.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.672+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/3403906710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d081a07e0 con 0x7f0d08105270 2024-09-21T03:38:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.673+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/3403906710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d081a6f40 con 0x7f0d08105270 2024-09-21T03:38:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.675+0000 7f0d16183740 1 Processor -- start 2024-09-21T03:38:31.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d16183740 1 -- start start 2024-09-21T03:38:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d16183740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a28070 0x55a560dcf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d16183740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560dcfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d16183740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a07390 0x55a560dd0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d16183740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a560d45ea0 con 0x55a5609ff1b0 2024-09-21T03:38:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d16183740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a560d352d0 con 0x55a560a07390 2024-09-21T03:38:31.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d16183740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a560d390e0 con 0x55a560a28070 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d06ffd640 1 -- 172.21.15.112:0/3403906710 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0cfc018a00 con 0x7f0d08105270 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d0d9ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560dcfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d0d9ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560dcfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46682/0 (socket says 172.21.15.112:46682) 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d0e1ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a28070 0x55a560dcf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d0d9ab640 1 -- 172.21.15.112:0/2477074787 learned_addr learned my addr 172.21.15.112:0/2477074787 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/2477074787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a07390 0x55a560dd0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d0d9ab640 1 -- 172.21.15.112:0/2477074787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a28070 msgr2=0x55a560dcf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d0d9ab640 1 --2- 172.21.15.112:0/2477074787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a28070 0x55a560dcf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d0d9ab640 1 -- 172.21.15.112:0/2477074787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a07390 msgr2=0x55a560dd0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.676+0000 7f0d0d9ab640 1 --2- 172.21.15.112:0/2477074787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a07390 0x55a560dd0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.677+0000 7f0d0d9ab640 1 -- 172.21.15.112:0/2477074787 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a560a93650 con 0x55a5609ff1b0 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.677+0000 7f0d0e1ac640 1 --2- 172.21.15.112:0/2477074787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a28070 0x55a560dcf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:31.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.677+0000 7f0d0d9ab640 1 --2- 172.21.15.112:0/2477074787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560dcfbe0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0d0806bc00 tx=0x7f0d000094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de6c2385a18c6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:31.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.677+0000 7f0ceffff640 1 -- 172.21.15.112:0/2477074787 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00031650 con 0x55a5609ff1b0 2024-09-21T03:38:31.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.677+0000 7f0d06ffd640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0cf00b6680 0x7f0cf00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.677+0000 7f0ceffff640 1 -- 172.21.15.112:0/2477074787 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d00036040 con 0x55a5609ff1b0 2024-09-21T03:38:31.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.677+0000 7f0ceffff640 1 -- 172.21.15.112:0/2477074787 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00004470 con 0x55a5609ff1b0 2024-09-21T03:38:31.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.677+0000 7f0d06ffd640 1 -- 172.21.15.112:0/3403906710 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f0cfc0e7a50 con 0x7f0d08105270 2024-09-21T03:38:31.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.677+0000 7f0d0e1ac640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0cf00b6680 0x7f0cf00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.678+0000 7f0d16183740 1 -- 172.21.15.112:0/2477074787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 msgr2=0x55a560dcfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.678+0000 7f0d16183740 1 --2- 172.21.15.112:0/2477074787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560dcfbe0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0d0806bc00 tx=0x7f0d000094a0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.678+0000 7f0d0e1ac640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0cf00b6680 0x7f0cf00b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009ef0 tx=0x7f0cf8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.678+0000 7f0d16183740 1 -- 172.21.15.112:0/2477074787 shutdown_connections 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.678+0000 7f0d16183740 1 --2- 172.21.15.112:0/2477074787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a07390 0x55a560dd0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.678+0000 7f0d16183740 1 --2- 172.21.15.112:0/2477074787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560dcfbe0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.678+0000 7f0d16183740 1 --2- 172.21.15.112:0/2477074787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a28070 0x55a560dcf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.678+0000 7f0d16183740 1 -- 172.21.15.112:0/2477074787 >> 172.21.15.112:0/2477074787 conn(0x55a560a1c460 msgr2=0x55a560dcd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.678+0000 7f0d16183740 1 -- 172.21.15.112:0/2477074787 shutdown_connections 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.678+0000 7f0d16183740 1 -- 172.21.15.112:0/2477074787 wait complete. 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.679+0000 7f0d16183740 1 -- start start 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.679+0000 7f0d16183740 1 Processor -- start 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.679+0000 7f0d16183740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560ddad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.679+0000 7f0d16183740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a07390 0x55a560ddb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.679+0000 7f0d16183740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a28070 0x55a560de6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.679+0000 7f0d16183740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a560b96700 con 0x55a5609ff1b0 2024-09-21T03:38:31.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.679+0000 7f0d16183740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a560cd0160 con 0x55a560a07390 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.679+0000 7f0d16183740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a560dddca0 con 0x55a560a28070 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.679+0000 7f0d16183740 10 client.0 _ll_register_callbacks cb 0x55a560ce99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.679+0000 7f0d0d9ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560ddad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d0d1aa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a07390 0x55a560ddb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d0e1ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a28070 0x55a560de6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d0d9ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560ddad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46688/0 (socket says 172.21.15.112:46688) 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d0d9ab640 1 -- 172.21.15.112:0/3877614878 learned_addr learned my addr 172.21.15.112:0/3877614878 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/3877614878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a28070 msgr2=0x55a560de6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a28070 0x55a560de6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/3877614878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 msgr2=0x55a560ddad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560ddad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/3877614878 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a560cfdc90 con 0x55a560a07390 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d0d9ab640 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560ddad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a07390 0x55a560ddb240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0e5170 tx=0x7f0cfc00cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b797314d700c9f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:31.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0ced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0ced7fa640 1 -- 172.21.15.112:0/3877614878 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0ad6f0 con 0x55a560a07390 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0ced7fa640 1 -- 172.21.15.112:0/3877614878 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f0cfc0a9270 con 0x55a560a07390 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d16183740 1 -- 172.21.15.112:0/3877614878 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d00036040 con 0x55a560a07390 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0ced7fa640 1 -- 172.21.15.112:0/3877614878 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc0a9090 con 0x55a560a07390 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d16183740 10 client.26651 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.680+0000 7f0d16183740 1 -- 172.21.15.112:0/3877614878 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a560de99b0 con 0x55a560a07390 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.681+0000 7f0ced7fa640 1 -- 172.21.15.112:0/3877614878 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d00036040 con 0x55a560a07390 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.681+0000 7f0ced7fa640 1 -- 172.21.15.112:0/3877614878 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0cfc0aeba0 con 0x55a560a07390 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.681+0000 7f0d16183740 10 client.26651 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.681+0000 7f0d16183740 10 client.26651 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.681+0000 7f0d16183740 20 client.26651 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.681+0000 7f0d16183740 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a5609e69f0 0x55a560decd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.681+0000 7f0d16183740 4 client.26651 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.681+0000 7f0d16183740 1 -- 172.21.15.112:0/3877614878 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a560d47df0 con 0x55a5609e69f0 2024-09-21T03:38:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.681+0000 7f0d0d9ab640 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a5609e69f0 0x55a560decd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.681+0000 7f0ced7fa640 10 client.26651 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:38:31.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.681+0000 7f0d0d9ab640 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a5609e69f0 0x55a560decd90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7720593b7d050023 server_cookie=efd280a68ab22bc3 in_seq=0 out_seq=0 2024-09-21T03:38:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0ced7fa640 1 -- 172.21.15.112:0/3877614878 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a560d47df0 con 0x55a5609e69f0 2024-09-21T03:38:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0ced7fa640 10 client.26651 handle_command_reply: tid=0 2024-09-21T03:38:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0d16183740 1 client.26651 shutdown 2024-09-21T03:38:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0d16183740 1 -- 172.21.15.112:0/3877614878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a07390 msgr2=0x55a560ddb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0d16183740 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a07390 0x55a560ddb240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0e5170 tx=0x7f0cfc00cef0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0d16183740 1 -- 172.21.15.112:0/3877614878 shutdown_connections 2024-09-21T03:38:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0d16183740 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a5609e69f0 0x55a560decd90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0d16183740 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a28070 0x55a560de6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0d16183740 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a07390 0x55a560ddb240 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0d16183740 1 --2- 172.21.15.112:0/3877614878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a5609ff1b0 0x55a560ddad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0d16183740 1 -- 172.21.15.112:0/3877614878 >> 172.21.15.112:0/3877614878 conn(0x55a560a1c460 msgr2=0x55a560dd2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:31.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.682+0000 7f0ced7fa640 0 client.26651 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:38:31.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.683+0000 7f0d16183740 1 -- 172.21.15.112:0/3877614878 shutdown_connections 2024-09-21T03:38:31.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.683+0000 7f0d16183740 1 -- 172.21.15.112:0/3877614878 wait complete. 2024-09-21T03:38:31.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.683+0000 7f0d16183740 20 client.26651 trim_cache size 0 max 16384 2024-09-21T03:38:31.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d16183740 1 Processor -- start 2024-09-21T03:38:31.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d16183740 1 -- start start 2024-09-21T03:38:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d16183740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5609ff1b0 0x55a560de6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d16183740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a07390 0x55a560dd7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d16183740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a28070 0x55a560dd8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d16183740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081b2020 con 0x55a560a28070 2024-09-21T03:38:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d16183740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081af8f0 con 0x55a5609ff1b0 2024-09-21T03:38:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d16183740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081ad760 con 0x55a560a07390 2024-09-21T03:38:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d0d1aa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a07390 0x55a560dd7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d0d9ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5609ff1b0 0x55a560de6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d0e1ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a28070 0x55a560dd8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d0d1aa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a07390 0x55a560dd7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43390/0 (socket says 172.21.15.112:43390) 2024-09-21T03:38:31.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d0d9ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5609ff1b0 0x55a560de6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33924/0 (socket says 172.21.15.112:33924) 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.696+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/321605666 learned_addr learned my addr 172.21.15.112:0/321605666 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.697+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/321605666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5609ff1b0 msgr2=0x55a560de6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.697+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/321605666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5609ff1b0 0x55a560de6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.697+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/321605666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a28070 msgr2=0x55a560dd8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.697+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/321605666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a28070 0x55a560dd8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.697+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/321605666 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a560dcde80 con 0x55a560a07390 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.697+0000 7f0d0e1ac640 1 --2- 172.21.15.112:0/321605666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a28070 0x55a560dd8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.697+0000 7f0d0d9ab640 1 --2- 172.21.15.112:0/321605666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5609ff1b0 0x55a560de6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.697+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/321605666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a07390 0x55a560dd7ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0aa0e0 tx=0x7f0cfc0e5730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4640f0b58a2020d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.697+0000 7f0cef7fe640 1 -- 172.21.15.112:0/321605666 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0ad090 con 0x55a560a07390 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.697+0000 7f0cef7fe640 1 -- 172.21.15.112:0/321605666 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cfc0b2130 con 0x55a560a07390 2024-09-21T03:38:31.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.697+0000 7f0cef7fe640 1 -- 172.21.15.112:0/321605666 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0ad270 con 0x55a560a07390 2024-09-21T03:38:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.698+0000 7f0d16183740 1 -- 172.21.15.112:0/321605666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a07390 msgr2=0x55a560dd7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.698+0000 7f0d16183740 1 --2- 172.21.15.112:0/321605666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a07390 0x55a560dd7ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0aa0e0 tx=0x7f0cfc0e5730 comp rx=0 tx=0).stop 2024-09-21T03:38:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.698+0000 7f0d16183740 1 -- 172.21.15.112:0/321605666 shutdown_connections 2024-09-21T03:38:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.698+0000 7f0d16183740 1 --2- 172.21.15.112:0/321605666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a28070 0x55a560dd8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.698+0000 7f0d16183740 1 --2- 172.21.15.112:0/321605666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a560a07390 0x55a560dd7ef0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.698+0000 7f0d16183740 1 --2- 172.21.15.112:0/321605666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5609ff1b0 0x55a560de6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.698+0000 7f0d16183740 1 -- 172.21.15.112:0/321605666 >> 172.21.15.112:0/321605666 conn(0x55a560a1c460 msgr2=0x55a560dd9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.698+0000 7f0d16183740 1 -- 172.21.15.112:0/321605666 shutdown_connections 2024-09-21T03:38:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.698+0000 7f0d16183740 1 -- 172.21.15.112:0/321605666 wait complete. 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.698+0000 7f0d16183740 1 -- start start 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d16183740 1 Processor -- start 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d16183740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a5609ff1b0 0x55a560ddf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d16183740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a07390 0x55a560ddf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d16183740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a28070 0x55a560ddfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d16183740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081addb0 con 0x55a560a07390 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d16183740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081af220 con 0x55a560a28070 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d16183740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081ae480 con 0x55a5609ff1b0 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d16183740 10 client.0 _ll_register_callbacks cb 0x7f0cfc0eb0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d0d1aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a07390 0x55a560ddf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d0d9ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a5609ff1b0 0x55a560ddf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d0d1aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a07390 0x55a560ddf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46716/0 (socket says 172.21.15.112:46716) 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/893386832 learned_addr learned my addr 172.21.15.112:0/893386832 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d0e1ac640 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a28070 0x55a560ddfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/893386832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a5609ff1b0 msgr2=0x55a560ddf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a5609ff1b0 0x55a560ddf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/893386832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a28070 msgr2=0x55a560ddfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a28070 0x55a560ddfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.699+0000 7f0d0d1aa640 1 -- 172.21.15.112:0/893386832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a560e56880 con 0x55a560a07390 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.700+0000 7f0d0d9ab640 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a5609ff1b0 0x55a560ddf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.700+0000 7f0d0e1ac640 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a28070 0x55a560ddfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.700+0000 7f0d0d1aa640 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a07390 0x55a560ddf540 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0a9ef0 tx=0x7f0cfc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16db836befb49a51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.700+0000 7f0ced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.700+0000 7f0ced7fa640 1 -- 172.21.15.112:0/893386832 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0ad090 con 0x55a560a07390 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.700+0000 7f0ced7fa640 1 -- 172.21.15.112:0/893386832 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f0cfc0b3730 con 0x55a560a07390 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.700+0000 7f0d16183740 1 -- 172.21.15.112:0/893386832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a560d438a0 con 0x55a560a07390 2024-09-21T03:38:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.700+0000 7f0ced7fa640 1 -- 172.21.15.112:0/893386832 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc0ad430 con 0x55a560a07390 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.700+0000 7f0d16183740 10 client.17103 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.700+0000 7f0d16183740 1 -- 172.21.15.112:0/893386832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a560e57b00 con 0x55a560a07390 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.700+0000 7f0ced7fa640 1 -- 172.21.15.112:0/893386832 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a560d438a0 con 0x55a560a07390 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0ced7fa640 1 -- 172.21.15.112:0/893386832 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0cfc0b67d0 con 0x55a560a07390 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0d16183740 10 client.17103 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0d16183740 10 client.17103 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0d16183740 20 client.17103 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0d16183740 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a5609bbda0 0x55a560e59ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0d16183740 4 client.17103 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0d16183740 1 -- 172.21.15.112:0/893386832 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a560d40cd0 con 0x55a5609bbda0 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0d0d9ab640 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a5609bbda0 0x55a560e59ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0ced7fa640 10 client.17103 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0d0d9ab640 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a5609bbda0 0x55a560e59ff0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=789fcc9223e1ddd3 server_cookie=a80ca5168c24f592 in_seq=0 out_seq=0 2024-09-21T03:38:31.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0ced7fa640 1 -- 172.21.15.112:0/893386832 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a560d40cd0 con 0x55a5609bbda0 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0ced7fa640 10 client.17103 handle_command_reply: tid=0 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.701+0000 7f0d16183740 1 client.17103 shutdown 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0d16183740 1 -- 172.21.15.112:0/893386832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a07390 msgr2=0x55a560ddf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0d16183740 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a07390 0x55a560ddf540 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0a9ef0 tx=0x7f0cfc01f000 comp rx=0 tx=0).stop 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0d16183740 1 -- 172.21.15.112:0/893386832 shutdown_connections 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0d16183740 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55a5609bbda0 0x55a560e59ff0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0d16183740 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a560a28070 0x55a560ddfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0d16183740 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a560a07390 0x55a560ddf540 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0ced7fa640 0 client.17103 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0d16183740 1 --2- 172.21.15.112:0/893386832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a5609ff1b0 0x55a560ddf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0d16183740 1 -- 172.21.15.112:0/893386832 >> 172.21.15.112:0/893386832 conn(0x55a560a1c460 msgr2=0x55a560dd0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0d16183740 1 -- 172.21.15.112:0/893386832 shutdown_connections 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0d16183740 1 -- 172.21.15.112:0/893386832 wait complete. 2024-09-21T03:38:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.702+0000 7f0d16183740 20 client.17103 trim_cache size 0 max 16384 2024-09-21T03:38:31.709 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:38:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/3403906710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0cf00b6680 msgr2=0x7f0cf00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0cf00b6680 0x7f0cf00b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009ef0 tx=0x7f0cf8005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/3403906710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d08105270 msgr2=0x7f0d0819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d08105270 0x7f0d0819ab20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00e9f0 tx=0x7f0cfc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/3403906710 shutdown_connections 2024-09-21T03:38:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0cf00b6680 0x7f0cf00b8b90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d0810ddf0 0x7f0d0819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d08105270 0x7f0d0819ab20 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 --2- 172.21.15.112:0/3403906710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d08104840 0x7f0d0819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/3403906710 >> 172.21.15.112:0/3403906710 conn(0x7f0d08079730 msgr2=0x7f0d08107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/3403906710 shutdown_connections 2024-09-21T03:38:31.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:31.703+0000 7f0d0fc0f640 1 -- 172.21.15.112:0/3403906710 wait complete. 2024-09-21T03:38:32.268 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1009351 10.55 MB/sec execute 392 sec latency 27.233 ms 2024-09-21T03:38:32.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:32 smithi112 ceph-mon[31442]: pgmap v611: 97 pgs: 97 active+clean; 760 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 303 op/s 2024-09-21T03:38:32.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:32 smithi203 ceph-mon[34723]: pgmap v611: 97 pgs: 97 active+clean; 760 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 303 op/s 2024-09-21T03:38:32.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:32 smithi187 ceph-mon[34603]: pgmap v611: 97 pgs: 97 active+clean; 760 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 303 op/s 2024-09-21T03:38:32.678 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:38:32.942 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:33.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1011099 10.53 MB/sec execute 393 sec latency 56.169 ms 2024-09-21T03:38:33.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.465+0000 7f6ab4d62640 1 -- 172.21.15.112:0/3499361084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ab01052b0 msgr2=0x7f6ab010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.465+0000 7f6ab4d62640 1 --2- 172.21.15.112:0/3499361084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ab01052b0 0x7f6ab010d8f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009900 tx=0x7f6aa402a410 comp rx=0 tx=0).stop 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.465+0000 7f6ab4d62640 1 -- 172.21.15.112:0/3499361084 shutdown_connections 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.465+0000 7f6ab4d62640 1 --2- 172.21.15.112:0/3499361084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab010de30 0x7f6ab0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.465+0000 7f6ab4d62640 1 --2- 172.21.15.112:0/3499361084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ab01052b0 0x7f6ab010d8f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.465+0000 7f6ab4d62640 1 --2- 172.21.15.112:0/3499361084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0104880 0x7f6ab0104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.465+0000 7f6ab4d62640 1 -- 172.21.15.112:0/3499361084 >> 172.21.15.112:0/3499361084 conn(0x7f6ab0079730 msgr2=0x7f6ab0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.465+0000 7f6ab4d62640 1 -- 172.21.15.112:0/3499361084 shutdown_connections 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.465+0000 7f6ab4d62640 1 -- 172.21.15.112:0/3499361084 wait complete. 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6ab4d62640 1 Processor -- start 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6ab4d62640 1 -- start start 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6ab4d62640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0104880 0x7f6ab019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6ab4d62640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab01052b0 0x7f6ab019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6ab4d62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ab010de30 0x7f6ab019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6ab4d62640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0113a70 con 0x7f6ab010de30 2024-09-21T03:38:33.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6ab4d62640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01138f0 con 0x7f6ab0104880 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6ab4d62640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0113bf0 con 0x7f6ab01052b0 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aaed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ab010de30 0x7f6ab019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aadd9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab01052b0 0x7f6ab019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0104880 0x7f6ab019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aadd9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab01052b0 0x7f6ab019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43416/0 (socket says 172.21.15.112:43416) 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aaed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ab010de30 0x7f6ab019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46724/0 (socket says 172.21.15.112:46724) 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aadd9b640 1 -- 172.21.15.112:0/709012017 learned_addr learned my addr 172.21.15.112:0/709012017 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0104880 0x7f6ab019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33950/0 (socket says 172.21.15.112:33950) 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aadd9b640 1 -- 172.21.15.112:0/709012017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0104880 msgr2=0x7f6ab019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aadd9b640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0104880 0x7f6ab019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aadd9b640 1 -- 172.21.15.112:0/709012017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ab010de30 msgr2=0x7f6ab019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aadd9b640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ab010de30 0x7f6ab019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aadd9b640 1 -- 172.21.15.112:0/709012017 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab01a03f0 con 0x7f6ab01052b0 2024-09-21T03:38:33.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.466+0000 7f6aae59c640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0104880 0x7f6ab019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.467+0000 7f6aadd9b640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab01052b0 0x7f6ab019aa20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4032fd0 tx=0x7f6aa4031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.467+0000 7f6a9f7fe640 1 -- 172.21.15.112:0/709012017 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa403f030 con 0x7f6ab01052b0 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.467+0000 7f6ab4d62640 1 -- 172.21.15.112:0/709012017 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab01a0680 con 0x7f6ab01052b0 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.467+0000 7f6ab4d62640 1 -- 172.21.15.112:0/709012017 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab01a6e40 con 0x7f6ab01052b0 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.467+0000 7f6a9f7fe640 1 -- 172.21.15.112:0/709012017 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa4004490 con 0x7f6ab01052b0 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.467+0000 7f6a9f7fe640 1 -- 172.21.15.112:0/709012017 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4035070 con 0x7f6ab01052b0 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6abb2d6740 1 Processor -- start 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6a9f7fe640 1 -- 172.21.15.112:0/709012017 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f6aa4042030 con 0x7f6ab01052b0 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6abb2d6740 1 -- start start 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfeec070 0x55fce02936a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 0x55fce0293be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce0294120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fce0209ea0 con 0x55fcdfecb390 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fce01f92d0 con 0x55fcdfec31b0 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fce01fd0e0 con 0x55fcdfeec070 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6aadd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce0294120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6aaed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfeec070 0x55fce02936a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.470+0000 7f6aadd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce0294120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46740/0 (socket says 172.21.15.112:46740) 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6aadd9b640 1 -- 172.21.15.112:0/3502548123 learned_addr learned my addr 172.21.15.112:0/3502548123 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6aae59c640 1 --2- 172.21.15.112:0/3502548123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 0x55fce0293be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6a9f7fe640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6a980b6700 0x7f6a980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6aae59c640 1 -- 172.21.15.112:0/3502548123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfeec070 msgr2=0x55fce02936a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6aae59c640 1 --2- 172.21.15.112:0/3502548123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfeec070 0x55fce02936a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6aae59c640 1 -- 172.21.15.112:0/3502548123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 msgr2=0x55fce0294120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6aae59c640 1 --2- 172.21.15.112:0/3502548123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce0294120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6aae59c640 1 -- 172.21.15.112:0/3502548123 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcdff57650 con 0x55fcdfec31b0 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6aaed9d640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6a980b6700 0x7f6a980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6aadd9b640 1 --2- 172.21.15.112:0/3502548123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce0294120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6a9f7fe640 1 -- 172.21.15.112:0/709012017 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f6aa4107690 con 0x7f6ab01052b0 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6aae59c640 1 --2- 172.21.15.112:0/3502548123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 0x55fce0293be0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009fa0 tx=0x7f6aa00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f1d4ea3bdb8720a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6a9cff9640 1 -- 172.21.15.112:0/3502548123 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000f5a0 con 0x55fcdfec31b0 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6a9cff9640 1 -- 172.21.15.112:0/3502548123 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa000fbf0 con 0x55fcdfec31b0 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.471+0000 7f6a9cff9640 1 -- 172.21.15.112:0/3502548123 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0019c00 con 0x55fcdfec31b0 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.472+0000 7f6abb2d6740 1 -- 172.21.15.112:0/3502548123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 msgr2=0x55fce0293be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.472+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/3502548123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 0x55fce0293be0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009fa0 tx=0x7f6aa00118d0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.472+0000 7f6aaed9d640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6a980b6700 0x7f6a980b8c10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8005c40 tx=0x7f6aa8004ab0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.472+0000 7f6abb2d6740 1 -- 172.21.15.112:0/3502548123 shutdown_connections 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.472+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/3502548123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce0294120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.472+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/3502548123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 0x55fce0293be0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.472+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/3502548123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfeec070 0x55fce02936a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.472+0000 7f6abb2d6740 1 -- 172.21.15.112:0/3502548123 >> 172.21.15.112:0/3502548123 conn(0x55fcdfee0460 msgr2=0x55fce0291590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.472+0000 7f6abb2d6740 1 -- 172.21.15.112:0/3502548123 shutdown_connections 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.472+0000 7f6abb2d6740 1 -- 172.21.15.112:0/3502548123 wait complete. 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.473+0000 7f6abb2d6740 1 -- start start 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.473+0000 7f6abb2d6740 1 Processor -- start 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.473+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 0x55fce029ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.473+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfecb390 0x55fce029f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.473+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfeec070 0x55fce02ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.473+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fce005a700 con 0x55fcdfeec070 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.473+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fce0194160 con 0x55fcdfecb390 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.473+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fce02a1e80 con 0x55fcdfec31b0 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.473+0000 7f6abb2d6740 10 client.0 _ll_register_callbacks cb 0x55fce01ad9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aaed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfeec070 0x55fce02ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 0x55fce029ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 0x55fce029ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43442/0 (socket says 172.21.15.112:43442) 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aae59c640 1 -- 172.21.15.112:0/564401086 learned_addr learned my addr 172.21.15.112:0/564401086 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aaed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfeec070 0x55fce02ab170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46744/0 (socket says 172.21.15.112:46744) 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aadd9b640 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfecb390 0x55fce029f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aae59c640 1 -- 172.21.15.112:0/564401086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfecb390 msgr2=0x55fce029f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aae59c640 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfecb390 0x55fce029f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aae59c640 1 -- 172.21.15.112:0/564401086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfeec070 msgr2=0x55fce02ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aae59c640 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfeec070 0x55fce02ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aae59c640 1 -- 172.21.15.112:0/564401086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce01c1c90 con 0x55fcdfec31b0 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aadd9b640 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfecb390 0x55fce029f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aaed9d640 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfeec070 0x55fce02ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6aae59c640 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 0x55fce029ed70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0005890 tx=0x7f6aa00056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d9b72ebbabe6ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6a95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:38:33.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6a95ffb640 1 -- 172.21.15.112:0/564401086 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0028020 con 0x55fcdfec31b0 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6a95ffb640 1 -- 172.21.15.112:0/564401086 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f6aa0016040 con 0x55fcdfec31b0 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6abb2d6740 1 -- 172.21.15.112:0/564401086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa000fbf0 con 0x55fcdfec31b0 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.474+0000 7f6a95ffb640 1 -- 172.21.15.112:0/564401086 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6aa0033050 con 0x55fcdfec31b0 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.475+0000 7f6abb2d6740 10 client.26674 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.475+0000 7f6abb2d6740 1 -- 172.21.15.112:0/564401086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fce02adba0 con 0x55fcdfec31b0 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.475+0000 7f6a95ffb640 1 -- 172.21.15.112:0/564401086 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa000fbf0 con 0x55fcdfec31b0 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.475+0000 7f6a95ffb640 1 -- 172.21.15.112:0/564401086 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6aa0019c40 con 0x55fcdfec31b0 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.475+0000 7f6abb2d6740 10 client.26674 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.475+0000 7f6abb2d6740 10 client.26674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.475+0000 7f6abb2d6740 20 client.26674 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.475+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55fcdfeaa9f0 0x55fce02b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.475+0000 7f6abb2d6740 4 client.26674 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.475+0000 7f6abb2d6740 1 -- 172.21.15.112:0/564401086 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fce020bdf0 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.475+0000 7f6aadd9b640 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55fcdfeaa9f0 0x55fce02b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.476+0000 7f6aadd9b640 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55fcdfeaa9f0 0x55fce02b1010 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e184a5bc114cf2 server_cookie=75a25b4495a43035 in_seq=0 out_seq=0 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.476+0000 7f6a95ffb640 10 client.26674 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6a95ffb640 1 -- 172.21.15.112:0/564401086 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fce020bdf0 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6a95ffb640 10 client.26674 handle_command_reply: tid=0 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 1 client.26674 shutdown 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 1 -- 172.21.15.112:0/564401086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 msgr2=0x55fce029ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 0x55fce029ed70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0005890 tx=0x7f6aa00056d0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 1 -- 172.21.15.112:0/564401086 shutdown_connections 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55fcdfeaa9f0 0x55fce02b1010 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfeec070 0x55fce02ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6a95ffb640 0 client.26674 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfecb390 0x55fce029f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/564401086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 0x55fce029ed70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 1 -- 172.21.15.112:0/564401086 >> 172.21.15.112:0/564401086 conn(0x55fcdfee0460 msgr2=0x55fce0296830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:33.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 1 -- 172.21.15.112:0/564401086 shutdown_connections 2024-09-21T03:38:33.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 1 -- 172.21.15.112:0/564401086 wait complete. 2024-09-21T03:38:33.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.477+0000 7f6abb2d6740 20 client.26674 trim_cache size 0 max 16384 2024-09-21T03:38:33.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6abb2d6740 1 Processor -- start 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6abb2d6740 1 -- start start 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfeaa9f0 0x55fce0291600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 0x55fce02ab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce02967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01b1f60 con 0x55fcdfecb390 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01af830 con 0x55fcdfec31b0 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01ad6a0 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6aaed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce02967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfeaa9f0 0x55fce0291600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6aadd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 0x55fce02ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6aadd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 0x55fce02ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33990/0 (socket says 172.21.15.112:33990) 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6aadd9b640 1 -- 172.21.15.112:0/263277452 learned_addr learned my addr 172.21.15.112:0/263277452 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:33.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.492+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfeaa9f0 0x55fce0291600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:43452/0 (socket says 172.21.15.112:43452) 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.493+0000 7f6aaed9d640 1 -- 172.21.15.112:0/263277452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfeaa9f0 msgr2=0x55fce0291600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.493+0000 7f6aaed9d640 1 --2- 172.21.15.112:0/263277452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfeaa9f0 0x55fce0291600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.493+0000 7f6aaed9d640 1 -- 172.21.15.112:0/263277452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 msgr2=0x55fce02ab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.493+0000 7f6aaed9d640 1 --2- 172.21.15.112:0/263277452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 0x55fce02ab070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.493+0000 7f6aaed9d640 1 -- 172.21.15.112:0/263277452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce0295990 con 0x55fcdfecb390 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.493+0000 7f6aadd9b640 1 --2- 172.21.15.112:0/263277452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 0x55fce02ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.493+0000 7f6aaed9d640 1 --2- 172.21.15.112:0/263277452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce02967a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009fd0 tx=0x7f6aa8009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2966c6488f2d1839 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.493+0000 7f6a97fff640 1 -- 172.21.15.112:0/263277452 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa801a040 con 0x55fcdfecb390 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.493+0000 7f6a97fff640 1 -- 172.21.15.112:0/263277452 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa800fe00 con 0x55fcdfecb390 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.493+0000 7f6a97fff640 1 -- 172.21.15.112:0/263277452 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8005550 con 0x55fcdfecb390 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.494+0000 7f6abb2d6740 1 -- 172.21.15.112:0/263277452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 msgr2=0x55fce02967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.494+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/263277452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce02967a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009fd0 tx=0x7f6aa8009be0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.494+0000 7f6abb2d6740 1 -- 172.21.15.112:0/263277452 shutdown_connections 2024-09-21T03:38:33.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.494+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/263277452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce02967a0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.494+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/263277452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfec31b0 0x55fce02ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.494+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/263277452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfeaa9f0 0x55fce0291600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.494+0000 7f6abb2d6740 1 -- 172.21.15.112:0/263277452 >> 172.21.15.112:0/263277452 conn(0x55fcdfee0460 msgr2=0x55fce0290ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.494+0000 7f6abb2d6740 1 -- 172.21.15.112:0/263277452 shutdown_connections 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.494+0000 7f6abb2d6740 1 -- 172.21.15.112:0/263277452 wait complete. 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6abb2d6740 1 -- start start 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6abb2d6740 1 Processor -- start 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfeaa9f0 0x55fce02a3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 0x55fce02a3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6abb2d6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce02a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01adcf0 con 0x55fcdfecb390 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01af160 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6abb2d6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01ae3c0 con 0x55fcdfec31b0 2024-09-21T03:38:33.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6abb2d6740 10 client.0 _ll_register_callbacks cb 0x7f6aa410d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfeaa9f0 0x55fce02a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6aaed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce02a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6aadd9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 0x55fce02a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfeaa9f0 0x55fce02a3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34004/0 (socket says 172.21.15.112:34004) 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.495+0000 7f6aae59c640 1 -- 172.21.15.112:0/310739814 learned_addr learned my addr 172.21.15.112:0/310739814 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6aae59c640 1 -- 172.21.15.112:0/310739814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 msgr2=0x55fce02a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6aae59c640 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 0x55fce02a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6aae59c640 1 -- 172.21.15.112:0/310739814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 msgr2=0x55fce02a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6aae59c640 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce02a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6aae59c640 1 -- 172.21.15.112:0/310739814 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce031a8f0 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6aaed9d640 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce02a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6aae59c640 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfeaa9f0 0x55fce02a3100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0001eb0 tx=0x7f6aa0011dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6241168cb55c1724 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6a95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:38:33.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6a95ffb640 1 -- 172.21.15.112:0/310739814 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0028070 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6a95ffb640 1 -- 172.21.15.112:0/310739814 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f6aa0003a60 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6abb2d6740 1 -- 172.21.15.112:0/310739814 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa800fe00 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6a95ffb640 1 -- 172.21.15.112:0/310739814 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6aa0019410 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6abb2d6740 10 client.26675 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6abb2d6740 1 -- 172.21.15.112:0/310739814 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fce031bb70 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.496+0000 7f6a95ffb640 1 -- 172.21.15.112:0/310739814 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa800fe00 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.497+0000 7f6a95ffb640 1 -- 172.21.15.112:0/310739814 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6aa00199f0 con 0x55fcdfeaa9f0 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.497+0000 7f6abb2d6740 10 client.26675 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.497+0000 7f6abb2d6740 10 client.26675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.497+0000 7f6abb2d6740 20 client.26675 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.497+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55fcdfeec070 0x55fce031e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.497+0000 7f6abb2d6740 4 client.26675 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.497+0000 7f6abb2d6740 1 -- 172.21.15.112:0/310739814 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fce02078a0 con 0x55fcdfeec070 2024-09-21T03:38:33.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.497+0000 7f6aadd9b640 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55fcdfeec070 0x55fce031e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:33.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.497+0000 7f6a95ffb640 10 client.26675 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:33.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.497+0000 7f6aadd9b640 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55fcdfeec070 0x55fce031e220 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a082989139d688b server_cookie=4496a49ae9c7f11c in_seq=0 out_seq=0 2024-09-21T03:38:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6a95ffb640 1 -- 172.21.15.112:0/310739814 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fce02078a0 con 0x55fcdfeec070 2024-09-21T03:38:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6a95ffb640 10 client.26675 handle_command_reply: tid=0 2024-09-21T03:38:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6abb2d6740 1 client.26675 shutdown 2024-09-21T03:38:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6abb2d6740 1 -- 172.21.15.112:0/310739814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfeaa9f0 msgr2=0x55fce02a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfeaa9f0 0x55fce02a3100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0001eb0 tx=0x7f6aa0011dd0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6abb2d6740 1 -- 172.21.15.112:0/310739814 shutdown_connections 2024-09-21T03:38:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55fcdfeec070 0x55fce031e220 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fcdfecb390 0x55fce02a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fcdfec31b0 0x55fce02a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6abb2d6740 1 --2- 172.21.15.112:0/310739814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fcdfeaa9f0 0x55fce02a3100 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6abb2d6740 1 -- 172.21.15.112:0/310739814 >> 172.21.15.112:0/310739814 conn(0x55fcdfee0460 msgr2=0x55fce0296500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:33.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6a95ffb640 0 client.26675 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:33.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6abb2d6740 1 -- 172.21.15.112:0/310739814 shutdown_connections 2024-09-21T03:38:33.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.500+0000 7f6abb2d6740 1 -- 172.21.15.112:0/310739814 wait complete. 2024-09-21T03:38:33.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6abb2d6740 20 client.26675 trim_cache size 0 max 16384 2024-09-21T03:38:33.508 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:38:33.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 -- 172.21.15.112:0/709012017 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6a980b6700 msgr2=0x7f6a980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6a980b6700 0x7f6a980b8c10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8005c40 tx=0x7f6aa8004ab0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 -- 172.21.15.112:0/709012017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab01052b0 msgr2=0x7f6ab019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab01052b0 0x7f6ab019aa20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4032fd0 tx=0x7f6aa4031a20 comp rx=0 tx=0).stop 2024-09-21T03:38:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 -- 172.21.15.112:0/709012017 shutdown_connections 2024-09-21T03:38:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6a980b6700 0x7f6a980b8c10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ab010de30 0x7f6ab019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab01052b0 0x7f6ab019aa20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 --2- 172.21.15.112:0/709012017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0104880 0x7f6ab019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 -- 172.21.15.112:0/709012017 >> 172.21.15.112:0/709012017 conn(0x7f6ab0079730 msgr2=0x7f6ab0100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 -- 172.21.15.112:0/709012017 shutdown_connections 2024-09-21T03:38:33.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:33.501+0000 7f6ab4d62640 1 -- 172.21.15.112:0/709012017 wait complete. 2024-09-21T03:38:34.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1013040 10.54 MB/sec execute 394 sec latency 32.496 ms 2024-09-21T03:38:34.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:34 smithi203 ceph-mon[34723]: pgmap v612: 97 pgs: 97 active+clean; 761 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-09-21T03:38:34.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:34 smithi187 ceph-mon[34603]: pgmap v612: 97 pgs: 97 active+clean; 761 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-09-21T03:38:34.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:34 smithi112 ceph-mon[31442]: pgmap v612: 97 pgs: 97 active+clean; 761 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-09-21T03:38:35.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1014995 10.53 MB/sec execute 395 sec latency 21.973 ms 2024-09-21T03:38:35.732 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:38:35.733 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:38:35.992 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:36.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1016787 10.54 MB/sec execute 396 sec latency 14.404 ms 2024-09-21T03:38:36.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:36 smithi203 ceph-mon[34723]: pgmap v613: 97 pgs: 97 active+clean; 763 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2024-09-21T03:38:36.415 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:36 smithi112 ceph-mon[31442]: pgmap v613: 97 pgs: 97 active+clean; 763 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2024-09-21T03:38:36.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:36 smithi187 ceph-mon[34603]: pgmap v613: 97 pgs: 97 active+clean; 763 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2024-09-21T03:38:36.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.512+0000 7f3351c9f640 1 -- 172.21.15.112:0/1547400624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c1094b0 msgr2=0x7f334c1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.512+0000 7f3351c9f640 1 --2- 172.21.15.112:0/1547400624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c1094b0 0x7f334c1098c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3340009900 tx=0x7f334002a410 comp rx=0 tx=0).stop 2024-09-21T03:38:36.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.512+0000 7f3351c9f640 1 -- 172.21.15.112:0/1547400624 shutdown_connections 2024-09-21T03:38:36.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.512+0000 7f3351c9f640 1 --2- 172.21.15.112:0/1547400624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f334c109e00 0x7f334c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.512+0000 7f3351c9f640 1 --2- 172.21.15.112:0/1547400624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c1094b0 0x7f334c1098c0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.512+0000 7f3351c9f640 1 --2- 172.21.15.112:0/1547400624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334c103bc0 0x7f334c103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.512+0000 7f3351c9f640 1 -- 172.21.15.112:0/1547400624 >> 172.21.15.112:0/1547400624 conn(0x7f334c07b910 msgr2=0x7f334c07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.512+0000 7f3351c9f640 1 -- 172.21.15.112:0/1547400624 shutdown_connections 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.512+0000 7f3351c9f640 1 -- 172.21.15.112:0/1547400624 wait complete. 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.513+0000 7f3351c9f640 1 Processor -- start 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.513+0000 7f3351c9f640 1 -- start start 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f3351c9f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f334c103bc0 0x7f334c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f3351c9f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334c1094b0 0x7f334c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f3351c9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c109e00 0x7f334c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f3351c9f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f334c113ac0 con 0x7f334c109e00 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f3351c9f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f334c113940 con 0x7f334c1094b0 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f3351c9f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f334c113c40 con 0x7f334c103bc0 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c109e00 0x7f334c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c109e00 0x7f334c19fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43642/0 (socket says 172.21.15.112:43642) 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334c1094b0 0x7f334c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334bfff640 1 -- 172.21.15.112:0/3234673551 learned_addr learned my addr 172.21.15.112:0/3234673551 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334b7fe640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f334c103bc0 0x7f334c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334bfff640 1 -- 172.21.15.112:0/3234673551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f334c103bc0 msgr2=0x7f334c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334bfff640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f334c103bc0 0x7f334c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334bfff640 1 -- 172.21.15.112:0/3234673551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334c1094b0 msgr2=0x7f334c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334bfff640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334c1094b0 0x7f334c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334bfff640 1 -- 172.21.15.112:0/3234673551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c1a0440 con 0x7f334c109e00 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334affd640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334c1094b0 0x7f334c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.514+0000 7f334bfff640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c109e00 0x7f334c19fd10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f334400eb10 tx=0x7f334400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.515+0000 7f3348ff9640 1 -- 172.21.15.112:0/3234673551 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344004b20 con 0x7f334c109e00 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.515+0000 7f3351c9f640 1 -- 172.21.15.112:0/3234673551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c1a0730 con 0x7f334c109e00 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.515+0000 7f3348ff9640 1 -- 172.21.15.112:0/3234673551 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3344004cc0 con 0x7f334c109e00 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.515+0000 7f3348ff9640 1 -- 172.21.15.112:0/3234673551 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344010690 con 0x7f334c109e00 2024-09-21T03:38:36.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.515+0000 7f3351c9f640 1 -- 172.21.15.112:0/3234673551 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c0755f0 con 0x7f334c109e00 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.517+0000 7f3348ff9640 1 -- 172.21.15.112:0/3234673551 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f33440108f0 con 0x7f334c109e00 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f3358213740 1 Processor -- start 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f3358213740 1 -- start start 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f3358213740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a490350 0x55909a837430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f3358213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a467490 0x55909a837970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f3358213740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a46f670 0x55909a837eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f3358213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55909a7ade60 con 0x55909a467490 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f3358213740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55909a79d290 con 0x55909a46f670 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f3358213740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55909a7a10a0 con 0x55909a490350 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a467490 0x55909a837970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a467490 0x55909a837970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43646/0 (socket says 172.21.15.112:43646) 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f334b7fe640 1 -- 172.21.15.112:0/4153805354 learned_addr learned my addr 172.21.15.112:0/4153805354 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f334affd640 1 --2- 172.21.15.112:0/4153805354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a46f670 0x55909a837eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f334affd640 1 --2- 172.21.15.112:0/4153805354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a490350 0x55909a837430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f334b7fe640 1 -- 172.21.15.112:0/4153805354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a490350 msgr2=0x55909a837430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f334b7fe640 1 --2- 172.21.15.112:0/4153805354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a490350 0x55909a837430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f334b7fe640 1 -- 172.21.15.112:0/4153805354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a46f670 msgr2=0x55909a837eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.518+0000 7f334b7fe640 1 --2- 172.21.15.112:0/4153805354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a46f670 0x55909a837eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.519+0000 7f334b7fe640 1 -- 172.21.15.112:0/4153805354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909a4fb930 con 0x55909a467490 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.519+0000 7f3348ff9640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f33340b6700 0x7f33340b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.519+0000 7f334affd640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f33340b6700 0x7f33340b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.519+0000 7f334b7fe640 1 --2- 172.21.15.112:0/4153805354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a467490 0x55909a837970 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f333c009cf0 tx=0x7f333c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b0d1b5ca6f8932 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.519+0000 7f3348ff9640 1 -- 172.21.15.112:0/3234673551 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f33440e7b70 con 0x7f334c109e00 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.519+0000 7f3331ffb640 1 -- 172.21.15.112:0/4153805354 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c01e070 con 0x55909a467490 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.519+0000 7f3331ffb640 1 -- 172.21.15.112:0/4153805354 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c009de0 con 0x55909a467490 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.519+0000 7f3331ffb640 1 -- 172.21.15.112:0/4153805354 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c019450 con 0x55909a467490 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.520+0000 7f3358213740 1 -- 172.21.15.112:0/4153805354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a467490 msgr2=0x55909a837970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.520+0000 7f3358213740 1 --2- 172.21.15.112:0/4153805354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a467490 0x55909a837970 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f333c009cf0 tx=0x7f333c011810 comp rx=0 tx=0).stop 2024-09-21T03:38:36.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.520+0000 7f334affd640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f33340b6700 0x7f33340b8c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3340009230 tx=0x7f3340000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.520+0000 7f3358213740 1 -- 172.21.15.112:0/4153805354 shutdown_connections 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.520+0000 7f3358213740 1 --2- 172.21.15.112:0/4153805354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a46f670 0x55909a837eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.520+0000 7f3358213740 1 --2- 172.21.15.112:0/4153805354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a467490 0x55909a837970 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.520+0000 7f3358213740 1 --2- 172.21.15.112:0/4153805354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a490350 0x55909a837430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.520+0000 7f3358213740 1 -- 172.21.15.112:0/4153805354 >> 172.21.15.112:0/4153805354 conn(0x55909a484740 msgr2=0x55909a835320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.520+0000 7f3358213740 1 -- 172.21.15.112:0/4153805354 shutdown_connections 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.520+0000 7f3358213740 1 -- 172.21.15.112:0/4153805354 wait complete. 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.520+0000 7f3358213740 1 -- start start 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f3358213740 1 Processor -- start 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f3358213740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 0x55909a842ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f3358213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a46f670 0x55909a843530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f3358213740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a490350 0x55909a84ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f3358213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55909a5fe890 con 0x55909a46f670 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f3358213740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55909a737d40 con 0x55909a467490 2024-09-21T03:38:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f3358213740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55909a845c10 con 0x55909a490350 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 0x55909a842ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 0x55909a842ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:53854/0 (socket says 172.21.15.112:53854) 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f334b7fe640 1 -- 172.21.15.112:0/1447794424 learned_addr learned my addr 172.21.15.112:0/1447794424 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f3358213740 10 client.0 _ll_register_callbacks cb 0x55909a79bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.521+0000 7f334affd640 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a46f670 0x55909a843530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f334b7fe640 1 -- 172.21.15.112:0/1447794424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a490350 msgr2=0x55909a84ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f334b7fe640 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a490350 0x55909a84ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f334b7fe640 1 -- 172.21.15.112:0/1447794424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a46f670 msgr2=0x55909a843530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f334b7fe640 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a46f670 0x55909a843530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f334b7fe640 1 -- 172.21.15.112:0/1447794424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909a84f4f0 con 0x55909a467490 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f334b7fe640 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 0x55909a842ff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f333c0058c0 tx=0x7f333c0052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b718b3b8111de430 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f332affd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f332affd640 1 -- 172.21.15.112:0/1447794424 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c01e040 con 0x55909a467490 2024-09-21T03:38:36.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f332affd640 1 -- 172.21.15.112:0/1447794424 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f333c016070 con 0x55909a467490 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f3358213740 1 -- 172.21.15.112:0/1447794424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f333c009de0 con 0x55909a467490 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f332affd640 1 -- 172.21.15.112:0/1447794424 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f333c0054e0 con 0x55909a467490 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f3358213740 10 client.26687 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f3358213740 1 -- 172.21.15.112:0/1447794424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55909a851de0 con 0x55909a467490 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.522+0000 7f332affd640 1 -- 172.21.15.112:0/1447794424 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c009de0 con 0x55909a467490 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.523+0000 7f332affd640 1 -- 172.21.15.112:0/1447794424 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f333c021070 con 0x55909a467490 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.523+0000 7f3358213740 10 client.26687 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.523+0000 7f3358213740 10 client.26687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.523+0000 7f3358213740 20 client.26687 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.523+0000 7f3358213740 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55909a44ecd0 0x55909a8551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.523+0000 7f3358213740 4 client.26687 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.523+0000 7f3358213740 1 -- 172.21.15.112:0/1447794424 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55909a7afdb0 con 0x55909a44ecd0 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.523+0000 7f334affd640 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55909a44ecd0 0x55909a8551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.523+0000 7f334affd640 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55909a44ecd0 0x55909a8551f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56e64eb6f9075de server_cookie=a9fd9616e71de335 in_seq=0 out_seq=0 2024-09-21T03:38:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.523+0000 7f332affd640 10 client.26687 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:36.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f332affd640 1 -- 172.21.15.112:0/1447794424 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55909a7afdb0 con 0x55909a44ecd0 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f332affd640 10 client.26687 handle_command_reply: tid=0 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f3358213740 1 client.26687 shutdown 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f3358213740 1 -- 172.21.15.112:0/1447794424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 msgr2=0x55909a842ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f3358213740 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 0x55909a842ff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f333c0058c0 tx=0x7f333c0052f0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f3358213740 1 -- 172.21.15.112:0/1447794424 shutdown_connections 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f3358213740 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55909a44ecd0 0x55909a8551f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f3358213740 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a490350 0x55909a84ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f3358213740 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a46f670 0x55909a843530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f332affd640 0 client.26687 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f3358213740 1 --2- 172.21.15.112:0/1447794424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 0x55909a842ff0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.524+0000 7f3358213740 1 -- 172.21.15.112:0/1447794424 >> 172.21.15.112:0/1447794424 conn(0x55909a484740 msgr2=0x55909a83a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.525+0000 7f3358213740 1 -- 172.21.15.112:0/1447794424 shutdown_connections 2024-09-21T03:38:36.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.525+0000 7f3358213740 1 -- 172.21.15.112:0/1447794424 wait complete. 2024-09-21T03:38:36.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.525+0000 7f3358213740 20 client.26687 trim_cache size 0 max 16384 2024-09-21T03:38:36.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.544+0000 7f3358213740 1 Processor -- start 2024-09-21T03:38:36.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.544+0000 7f3358213740 1 -- start start 2024-09-21T03:38:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.544+0000 7f3358213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a84ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.544+0000 7f3358213740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 0x55909a83d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.544+0000 7f3358213740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a46f670 0x55909a83ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.544+0000 7f3358213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1afe70 con 0x55909a44ecd0 2024-09-21T03:38:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.544+0000 7f3358213740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ad740 con 0x55909a467490 2024-09-21T03:38:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.544+0000 7f3358213740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ab5d0 con 0x55909a46f670 2024-09-21T03:38:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a84ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a84ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43656/0 (socket says 172.21.15.112:43656) 2024-09-21T03:38:36.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334b7fe640 1 -- 172.21.15.112:0/1519305819 learned_addr learned my addr 172.21.15.112:0/1519305819 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334affd640 1 --2- 172.21.15.112:0/1519305819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 0x55909a83d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334bfff640 1 --2- 172.21.15.112:0/1519305819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a46f670 0x55909a83ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334b7fe640 1 -- 172.21.15.112:0/1519305819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a46f670 msgr2=0x55909a83ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334b7fe640 1 --2- 172.21.15.112:0/1519305819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a46f670 0x55909a83ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334b7fe640 1 -- 172.21.15.112:0/1519305819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 msgr2=0x55909a83d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334b7fe640 1 --2- 172.21.15.112:0/1519305819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 0x55909a83d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334b7fe640 1 -- 172.21.15.112:0/1519305819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909a843c60 con 0x55909a44ecd0 2024-09-21T03:38:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334affd640 1 --2- 172.21.15.112:0/1519305819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 0x55909a83d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:36.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.545+0000 7f334b7fe640 1 --2- 172.21.15.112:0/1519305819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a84ee00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f333c005010 tx=0x7f333c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d9b849bc3f3382 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.546+0000 7f33317fa640 1 -- 172.21.15.112:0/1519305819 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c01e070 con 0x55909a44ecd0 2024-09-21T03:38:36.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.546+0000 7f33317fa640 1 -- 172.21.15.112:0/1519305819 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c009910 con 0x55909a44ecd0 2024-09-21T03:38:36.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.546+0000 7f33317fa640 1 -- 172.21.15.112:0/1519305819 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c019bf0 con 0x55909a44ecd0 2024-09-21T03:38:36.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.547+0000 7f3358213740 1 -- 172.21.15.112:0/1519305819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 msgr2=0x55909a84ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.547+0000 7f3358213740 1 --2- 172.21.15.112:0/1519305819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a84ee00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f333c005010 tx=0x7f333c001e50 comp rx=0 tx=0).stop 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.547+0000 7f3358213740 1 -- 172.21.15.112:0/1519305819 shutdown_connections 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.547+0000 7f3358213740 1 --2- 172.21.15.112:0/1519305819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a46f670 0x55909a83ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.547+0000 7f3358213740 1 --2- 172.21.15.112:0/1519305819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a467490 0x55909a83d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.547+0000 7f3358213740 1 --2- 172.21.15.112:0/1519305819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a84ee00 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.547+0000 7f3358213740 1 -- 172.21.15.112:0/1519305819 >> 172.21.15.112:0/1519305819 conn(0x55909a484740 msgr2=0x55909a841670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.547+0000 7f3358213740 1 -- 172.21.15.112:0/1519305819 shutdown_connections 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.547+0000 7f3358213740 1 -- 172.21.15.112:0/1519305819 wait complete. 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.547+0000 7f3358213740 1 -- start start 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.547+0000 7f3358213740 1 Processor -- start 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f3358213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a846fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f3358213740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a467490 0x55909a847510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f3358213740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a46f670 0x55909a847a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f3358213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1abc20 con 0x55909a44ecd0 2024-09-21T03:38:36.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f3358213740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ad070 con 0x55909a46f670 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f3358213740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ac2d0 con 0x55909a467490 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f3358213740 10 client.0 _ll_register_callbacks cb 0x7f333c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a846fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334affd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a467490 0x55909a847510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a846fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43672/0 (socket says 172.21.15.112:43672) 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334b7fe640 1 -- 172.21.15.112:0/2951848320 learned_addr learned my addr 172.21.15.112:0/2951848320 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334bfff640 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a46f670 0x55909a847a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334b7fe640 1 -- 172.21.15.112:0/2951848320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a467490 msgr2=0x55909a847510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334b7fe640 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a467490 0x55909a847510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334b7fe640 1 -- 172.21.15.112:0/2951848320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a46f670 msgr2=0x55909a847a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334b7fe640 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a46f670 0x55909a847a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334b7fe640 1 -- 172.21.15.112:0/2951848320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909a8bec70 con 0x55909a44ecd0 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334bfff640 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a46f670 0x55909a847a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:36.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.548+0000 7f334affd640 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a467490 0x55909a847510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.549+0000 7f334b7fe640 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a846fd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f333c009820 tx=0x7f333c0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f34fcf282c28d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.549+0000 7f332affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.549+0000 7f332affd640 1 -- 172.21.15.112:0/2951848320 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c01e040 con 0x55909a44ecd0 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.549+0000 7f332affd640 1 -- 172.21.15.112:0/2951848320 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f333c019450 con 0x55909a44ecd0 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.549+0000 7f3358213740 1 -- 172.21.15.112:0/2951848320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f333c009910 con 0x55909a44ecd0 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.549+0000 7f332affd640 1 -- 172.21.15.112:0/2951848320 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f333c019650 con 0x55909a44ecd0 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.549+0000 7f3358213740 10 client.17139 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.549+0000 7f3358213740 1 -- 172.21.15.112:0/2951848320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55909a8bff10 con 0x55909a44ecd0 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.549+0000 7f332affd640 1 -- 172.21.15.112:0/2951848320 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c009910 con 0x55909a44ecd0 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.550+0000 7f332affd640 1 -- 172.21.15.112:0/2951848320 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f333c033070 con 0x55909a44ecd0 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.550+0000 7f3358213740 10 client.17139 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.550+0000 7f3358213740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.550+0000 7f3358213740 20 client.17139 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:36.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.550+0000 7f3358213740 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55909a490350 0x55909a8c2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:36.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.550+0000 7f3358213740 4 client.17139 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:38:36.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.550+0000 7f3358213740 1 -- 172.21.15.112:0/2951848320 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55909a7ab860 con 0x55909a490350 2024-09-21T03:38:36.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.550+0000 7f334affd640 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55909a490350 0x55909a8c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:36.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.550+0000 7f334affd640 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55909a490350 0x55909a8c2590 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a83ce20bc947f395 server_cookie=5acf453ad05897cd in_seq=0 out_seq=0 2024-09-21T03:38:36.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.550+0000 7f332affd640 10 client.17139 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.552+0000 7f332affd640 1 -- 172.21.15.112:0/2951848320 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55909a7ab860 con 0x55909a490350 2024-09-21T03:38:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.552+0000 7f332affd640 10 client.17139 handle_command_reply: tid=0 2024-09-21T03:38:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.552+0000 7f3358213740 1 client.17139 shutdown 2024-09-21T03:38:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.552+0000 7f3358213740 1 -- 172.21.15.112:0/2951848320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 msgr2=0x55909a846fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.552+0000 7f3358213740 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a846fd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f333c009820 tx=0x7f333c0092f0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.552+0000 7f3358213740 1 -- 172.21.15.112:0/2951848320 shutdown_connections 2024-09-21T03:38:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.552+0000 7f3358213740 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55909a490350 0x55909a8c2590 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.552+0000 7f3358213740 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55909a46f670 0x55909a847a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.553+0000 7f3358213740 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55909a467490 0x55909a847510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.553+0000 7f332affd640 0 client.17139 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.553+0000 7f3358213740 1 --2- 172.21.15.112:0/2951848320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55909a44ecd0 0x55909a846fd0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.553+0000 7f3358213740 1 -- 172.21.15.112:0/2951848320 >> 172.21.15.112:0/2951848320 conn(0x55909a484740 msgr2=0x55909a838460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.553+0000 7f3358213740 1 -- 172.21.15.112:0/2951848320 shutdown_connections 2024-09-21T03:38:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.553+0000 7f3358213740 1 -- 172.21.15.112:0/2951848320 wait complete. 2024-09-21T03:38:36.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.553+0000 7f3358213740 20 client.17139 trim_cache size 0 max 16384 2024-09-21T03:38:36.558 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:38:36.558 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:38:36.558 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "730258e8-6c54-4776-b8f7-7ce3b100573d", 2024-09-21T03:38:36.559 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:38:36.559 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:38:36.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.553+0000 7f3351c9f640 1 -- 172.21.15.112:0/3234673551 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f33340b6700 msgr2=0x7f33340b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.553+0000 7f3351c9f640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f33340b6700 0x7f33340b8c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3340009230 tx=0x7f3340000bc0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.553+0000 7f3351c9f640 1 -- 172.21.15.112:0/3234673551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c109e00 msgr2=0x7f334c19fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.553+0000 7f3351c9f640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c109e00 0x7f334c19fd10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f334400eb10 tx=0x7f334400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.554+0000 7f3351c9f640 1 -- 172.21.15.112:0/3234673551 shutdown_connections 2024-09-21T03:38:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.554+0000 7f3351c9f640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f33340b6700 0x7f33340b8c10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.554+0000 7f3351c9f640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f334c109e00 0x7f334c19fd10 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.554+0000 7f3351c9f640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f334c1094b0 0x7f334c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.554+0000 7f3351c9f640 1 --2- 172.21.15.112:0/3234673551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f334c103bc0 0x7f334c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.554+0000 7f3351c9f640 1 -- 172.21.15.112:0/3234673551 >> 172.21.15.112:0/3234673551 conn(0x7f334c07b910 msgr2=0x7f334c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.554+0000 7f3351c9f640 1 -- 172.21.15.112:0/3234673551 shutdown_connections 2024-09-21T03:38:36.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:36.554+0000 7f3351c9f640 1 -- 172.21.15.112:0/3234673551 wait complete. 2024-09-21T03:38:37.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1019235 10.54 MB/sec execute 397 sec latency 52.938 ms 2024-09-21T03:38:37.428 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:38:37.708 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:38:38.152 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:38 smithi112 ceph-mon[31442]: pgmap v614: 97 pgs: 97 active+clean; 769 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2024-09-21T03:38:38.152 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:38 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:38:38.152 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:38 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:38.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:38 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:38:38.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:38 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:38:38.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:38 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:38:38.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:38 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:38.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:38 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:38.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:38 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:38.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1020471 10.53 MB/sec execute 398 sec latency 41.427 ms 2024-09-21T03:38:38.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.269+0000 7f7fe00f5640 1 -- 172.21.15.112:0/1219222608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd8102ae0 msgr2=0x7f7fd8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.269+0000 7f7fe00f5640 1 --2- 172.21.15.112:0/1219222608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd8102ae0 0x7f7fd8102ef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009900 tx=0x7f7fd402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.269+0000 7f7fe00f5640 1 -- 172.21.15.112:0/1219222608 shutdown_connections 2024-09-21T03:38:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.269+0000 7f7fe00f5640 1 --2- 172.21.15.112:0/1219222608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd8108d20 0x7f7fd810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.269+0000 7f7fe00f5640 1 --2- 172.21.15.112:0/1219222608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd81083d0 0x7f7fd81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.269+0000 7f7fe00f5640 1 --2- 172.21.15.112:0/1219222608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd8102ae0 0x7f7fd8102ef0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.269+0000 7f7fe00f5640 1 -- 172.21.15.112:0/1219222608 >> 172.21.15.112:0/1219222608 conn(0x7f7fd8079790 msgr2=0x7f7fd8079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.270+0000 7f7fe00f5640 1 -- 172.21.15.112:0/1219222608 shutdown_connections 2024-09-21T03:38:38.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.271+0000 7f7fe00f5640 1 -- 172.21.15.112:0/1219222608 wait complete. 2024-09-21T03:38:38.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.271+0000 7f7fe00f5640 1 Processor -- start 2024-09-21T03:38:38.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fe00f5640 1 -- start start 2024-09-21T03:38:38.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fe00f5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd8102ae0 0x7f7fd8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fe00f5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd81083d0 0x7f7fd81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fe00f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd8108d20 0x7f7fd819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fe00f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8111920 con 0x7f7fd8108d20 2024-09-21T03:38:38.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fe00f5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81117a0 con 0x7f7fd8102ae0 2024-09-21T03:38:38.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fe00f5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8111aa0 con 0x7f7fd81083d0 2024-09-21T03:38:38.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fdde91640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd8102ae0 0x7f7fd8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fdd690640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd81083d0 0x7f7fd81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fdde91640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd8102ae0 0x7f7fd8198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:53886/0 (socket says 172.21.15.112:53886) 2024-09-21T03:38:38.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fdde91640 1 -- 172.21.15.112:0/2268426539 learned_addr learned my addr 172.21.15.112:0/2268426539 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:38.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.272+0000 7f7fde692640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd8108d20 0x7f7fd819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.273+0000 7f7fde692640 1 -- 172.21.15.112:0/2268426539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd81083d0 msgr2=0x7f7fd81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.273+0000 7f7fde692640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd81083d0 0x7f7fd81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.273+0000 7f7fde692640 1 -- 172.21.15.112:0/2268426539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd8102ae0 msgr2=0x7f7fd8198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.273+0000 7f7fde692640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd8102ae0 0x7f7fd8198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.273+0000 7f7fde692640 1 -- 172.21.15.112:0/2268426539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd819e1c0 con 0x7f7fd8108d20 2024-09-21T03:38:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.273+0000 7f7fdd690640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd81083d0 0x7f7fd81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:38:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.273+0000 7f7fde692640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd8108d20 0x7f7fd819db50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000cab0 tx=0x7f7fd000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.273+0000 7f7fceffd640 1 -- 172.21.15.112:0/2268426539 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd000ae20 con 0x7f7fd8108d20 2024-09-21T03:38:38.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.273+0000 7f7fceffd640 1 -- 172.21.15.112:0/2268426539 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd000f900 con 0x7f7fd8108d20 2024-09-21T03:38:38.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.273+0000 7f7fceffd640 1 -- 172.21.15.112:0/2268426539 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd00189f0 con 0x7f7fd8108d20 2024-09-21T03:38:38.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.273+0000 7f7fe00f5640 1 -- 172.21.15.112:0/2268426539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd819e4b0 con 0x7f7fd8108d20 2024-09-21T03:38:38.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.274+0000 7f7fe00f5640 1 -- 172.21.15.112:0/2268426539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd81a4cd0 con 0x7f7fd8108d20 2024-09-21T03:38:38.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.276+0000 7f7fe6669740 1 Processor -- start 2024-09-21T03:38:38.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fceffd640 1 -- 172.21.15.112:0/2268426539 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7fd000f450 con 0x7f7fd8108d20 2024-09-21T03:38:38.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fe6669740 1 -- start start 2024-09-21T03:38:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb7b070 0x55b13ef22270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb521b0 0x55b13ef227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb5a390 0x55b13ef22cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fe6669740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b13ee98e50 con 0x55b13eb521b0 2024-09-21T03:38:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fe6669740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b13ee88280 con 0x55b13eb7b070 2024-09-21T03:38:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fe6669740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b13ee8c090 con 0x55b13eb5a390 2024-09-21T03:38:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fdde91640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb521b0 0x55b13ef227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fdd690640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb5a390 0x55b13ef22cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fdde91640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb521b0 0x55b13ef227b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43704/0 (socket says 172.21.15.112:43704) 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fdde91640 1 -- 172.21.15.112:0/2119525293 learned_addr learned my addr 172.21.15.112:0/2119525293 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.277+0000 7f7fdd690640 1 --2- 172.21.15.112:0/2119525293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb7b070 0x55b13ef22270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fceffd640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7fc40b8970 0x7f7fc40bae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fdde91640 1 -- 172.21.15.112:0/2119525293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb5a390 msgr2=0x55b13ef22cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fdde91640 1 --2- 172.21.15.112:0/2119525293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb5a390 0x55b13ef22cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fdde91640 1 -- 172.21.15.112:0/2119525293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb7b070 msgr2=0x55b13ef22270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fdde91640 1 --2- 172.21.15.112:0/2119525293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb7b070 0x55b13ef22270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fdde91640 1 -- 172.21.15.112:0/2119525293 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13ebe6650 con 0x55b13eb521b0 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fde692640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7fc40b8970 0x7f7fc40bae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fceffd640 1 -- 172.21.15.112:0/2268426539 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f7fd00e7930 con 0x7f7fd8108d20 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fdde91640 1 --2- 172.21.15.112:0/2119525293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb521b0 0x55b13ef227b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006010 tx=0x7f7fd4004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e168b28c3ba4f9d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fc3fff640 1 -- 172.21.15.112:0/2119525293 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4040070 con 0x55b13eb521b0 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fc3fff640 1 -- 172.21.15.112:0/2119525293 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd40054d0 con 0x55b13eb521b0 2024-09-21T03:38:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.278+0000 7f7fc3fff640 1 -- 172.21.15.112:0/2119525293 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4035040 con 0x55b13eb521b0 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.279+0000 7f7fe6669740 1 -- 172.21.15.112:0/2119525293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb521b0 msgr2=0x55b13ef227b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.279+0000 7f7fde692640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7fc40b8970 0x7f7fc40bae80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00e5160 tx=0x7f7fd001aed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.279+0000 7f7fe6669740 1 --2- 172.21.15.112:0/2119525293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb521b0 0x55b13ef227b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006010 tx=0x7f7fd4004260 comp rx=0 tx=0).stop 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.279+0000 7f7fe6669740 1 -- 172.21.15.112:0/2119525293 shutdown_connections 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.279+0000 7f7fe6669740 1 --2- 172.21.15.112:0/2119525293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb5a390 0x55b13ef22cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.279+0000 7f7fe6669740 1 --2- 172.21.15.112:0/2119525293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb521b0 0x55b13ef227b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.279+0000 7f7fe6669740 1 --2- 172.21.15.112:0/2119525293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb7b070 0x55b13ef22270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.279+0000 7f7fe6669740 1 -- 172.21.15.112:0/2119525293 >> 172.21.15.112:0/2119525293 conn(0x55b13eb6f460 msgr2=0x55b13ef20160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.280+0000 7f7fe6669740 1 -- 172.21.15.112:0/2119525293 shutdown_connections 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.280+0000 7f7fe6669740 1 -- 172.21.15.112:0/2119525293 wait complete. 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.280+0000 7f7fe6669740 1 -- start start 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.280+0000 7f7fe6669740 1 Processor -- start 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb521b0 0x55b13ef2d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb5a390 0x55b13ef2de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb7b070 0x55b13ef39ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fe6669740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b13ece9700 con 0x55b13eb7b070 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fe6669740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b13ee23110 con 0x55b13eb521b0 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fe6669740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b13ef309f0 con 0x55b13eb5a390 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fe6669740 10 client.0 _ll_register_callbacks cb 0x55b13ee3c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fdd690640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb7b070 0x55b13ef39ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fdde91640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb5a390 0x55b13ef2de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fdd690640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb521b0 0x55b13ef2d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fdde91640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb5a390 0x55b13ef2de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:34062/0 (socket says 172.21.15.112:34062) 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fdde91640 1 -- 172.21.15.112:0/1301651392 learned_addr learned my addr 172.21.15.112:0/1301651392 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fdd690640 1 -- 172.21.15.112:0/1301651392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb5a390 msgr2=0x55b13ef2de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fdd690640 1 --2- 172.21.15.112:0/1301651392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb5a390 0x55b13ef2de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fdd690640 1 -- 172.21.15.112:0/1301651392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb7b070 msgr2=0x55b13ef39ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fdd690640 1 --2- 172.21.15.112:0/1301651392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb7b070 0x55b13ef39ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.281+0000 7f7fdd690640 1 -- 172.21.15.112:0/1301651392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13ee50c40 con 0x55b13eb521b0 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fdd690640 1 --2- 172.21.15.112:0/1301651392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb521b0 0x55b13ef2d950 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8079e00 tx=0x7f7fc8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec72351cca34b468 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fc17fa640 1 -- 172.21.15.112:0/1301651392 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8011070 con 0x55b13eb521b0 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fc17fa640 1 -- 172.21.15.112:0/1301651392 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f7fc8011210 con 0x55b13eb521b0 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fe6669740 1 -- 172.21.15.112:0/1301651392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd40054d0 con 0x55b13eb521b0 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fc17fa640 1 -- 172.21.15.112:0/1301651392 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7fc8004020 con 0x55b13eb521b0 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fe6669740 10 client.26705 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fe6669740 1 -- 172.21.15.112:0/1301651392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b13ef3c720 con 0x55b13eb521b0 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fc17fa640 1 -- 172.21.15.112:0/1301651392 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd40054d0 con 0x55b13eb521b0 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fc17fa640 1 -- 172.21.15.112:0/1301651392 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7fc8003c90 con 0x55b13eb521b0 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fe6669740 10 client.26705 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.282+0000 7f7fe6669740 10 client.26705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:38:38.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.283+0000 7f7fe6669740 20 client.26705 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:38.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.283+0000 7f7fe6669740 1 --2- 172.21.15.112:0/1301651392 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b13eb399f0 0x55b13ef3fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.283+0000 7f7fe6669740 4 client.26705 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:38:38.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.283+0000 7f7fe6669740 1 -- 172.21.15.112:0/1301651392 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b13ee9ada0 con 0x55b13eb399f0 2024-09-21T03:38:38.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.283+0000 7f7fdde91640 1 --2- 172.21.15.112:0/1301651392 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b13eb399f0 0x55b13ef3fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.283+0000 7f7fc17fa640 10 client.26705 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:38.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.283+0000 7f7fdde91640 1 --2- 172.21.15.112:0/1301651392 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b13eb399f0 0x55b13ef3fba0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c9f61e6b52d9bea server_cookie=df6f6f919c516be2 in_seq=0 out_seq=0 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.284+0000 7f7fc17fa640 1 -- 172.21.15.112:0/1301651392 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b13ee9ada0 con 0x55b13eb399f0 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.284+0000 7f7fc17fa640 10 client.26705 handle_command_reply: tid=0 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.284+0000 7f7fe6669740 1 client.26705 shutdown 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.284+0000 7f7fe6669740 1 -- 172.21.15.112:0/1301651392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb521b0 msgr2=0x55b13ef2d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.284+0000 7f7fe6669740 1 --2- 172.21.15.112:0/1301651392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb521b0 0x55b13ef2d950 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8079e00 tx=0x7f7fc8005f90 comp rx=0 tx=0).stop 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.284+0000 7f7fe6669740 1 -- 172.21.15.112:0/1301651392 shutdown_connections 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.284+0000 7f7fe6669740 1 --2- 172.21.15.112:0/1301651392 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b13eb399f0 0x55b13ef3fba0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.284+0000 7f7fc17fa640 0 client.26705 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.285+0000 7f7fe6669740 1 --2- 172.21.15.112:0/1301651392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb7b070 0x55b13ef39ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.285+0000 7f7fe6669740 1 --2- 172.21.15.112:0/1301651392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb5a390 0x55b13ef2de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.285+0000 7f7fe6669740 1 --2- 172.21.15.112:0/1301651392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb521b0 0x55b13ef2d950 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.285+0000 7f7fe6669740 1 -- 172.21.15.112:0/1301651392 >> 172.21.15.112:0/1301651392 conn(0x55b13eb6f460 msgr2=0x55b13ef25420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.285+0000 7f7fe6669740 1 -- 172.21.15.112:0/1301651392 shutdown_connections 2024-09-21T03:38:38.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.285+0000 7f7fe6669740 1 -- 172.21.15.112:0/1301651392 wait complete. 2024-09-21T03:38:38.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.285+0000 7f7fe6669740 20 client.26705 trim_cache size 0 max 16384 2024-09-21T03:38:38.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.304+0000 7f7fe6669740 1 Processor -- start 2024-09-21T03:38:38.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.304+0000 7f7fe6669740 1 -- start start 2024-09-21T03:38:38.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb399f0 0x55b13ef2f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb521b0 0x55b13ef39be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb5a390 0x55b13ef2ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fe6669740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81afd60 con 0x55b13eb399f0 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fe6669740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81ad630 con 0x55b13eb5a390 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fe6669740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81ab4a0 con 0x55b13eb521b0 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fdd690640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb399f0 0x55b13ef2f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fdd690640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb399f0 0x55b13ef2f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43722/0 (socket says 172.21.15.112:43722) 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fdd690640 1 -- 172.21.15.112:0/2466462801 learned_addr learned my addr 172.21.15.112:0/2466462801 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fdde91640 1 --2- 172.21.15.112:0/2466462801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb521b0 0x55b13ef39be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fdd690640 1 --2- 172.21.15.112:0/2466462801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb5a390 0x55b13ef2ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fdd690640 1 -- 172.21.15.112:0/2466462801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb521b0 msgr2=0x55b13ef39be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fdd690640 1 --2- 172.21.15.112:0/2466462801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb521b0 0x55b13ef39be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fdd690640 1 -- 172.21.15.112:0/2466462801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb5a390 msgr2=0x55b13ef2ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fdd690640 1 --2- 172.21.15.112:0/2466462801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb5a390 0x55b13ef2ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fdd690640 1 -- 172.21.15.112:0/2466462801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13ef2ec10 con 0x55b13eb399f0 2024-09-21T03:38:38.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fdd690640 1 --2- 172.21.15.112:0/2466462801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb399f0 0x55b13ef2f580 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009730 tx=0x7f7fc80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50a1ba761163058 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:38.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fc37fe640 1 -- 172.21.15.112:0/2466462801 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8030070 con 0x55b13eb399f0 2024-09-21T03:38:38.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fc37fe640 1 -- 172.21.15.112:0/2466462801 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc8022830 con 0x55b13eb399f0 2024-09-21T03:38:38.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.305+0000 7f7fc37fe640 1 -- 172.21.15.112:0/2466462801 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8005650 con 0x55b13eb399f0 2024-09-21T03:38:38.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.306+0000 7f7fe6669740 1 -- 172.21.15.112:0/2466462801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb399f0 msgr2=0x55b13ef2f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.306+0000 7f7fe6669740 1 --2- 172.21.15.112:0/2466462801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb399f0 0x55b13ef2f580 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009730 tx=0x7f7fc80096c0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.306+0000 7f7fe6669740 1 -- 172.21.15.112:0/2466462801 shutdown_connections 2024-09-21T03:38:38.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.306+0000 7f7fe6669740 1 --2- 172.21.15.112:0/2466462801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb5a390 0x55b13ef2ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.306+0000 7f7fe6669740 1 --2- 172.21.15.112:0/2466462801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb521b0 0x55b13ef39be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.306+0000 7f7fe6669740 1 --2- 172.21.15.112:0/2466462801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb399f0 0x55b13ef2f580 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.306+0000 7f7fe6669740 1 -- 172.21.15.112:0/2466462801 >> 172.21.15.112:0/2466462801 conn(0x55b13eb6f460 msgr2=0x55b13ef2ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.306+0000 7f7fe6669740 1 -- 172.21.15.112:0/2466462801 shutdown_connections 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.306+0000 7f7fe6669740 1 -- 172.21.15.112:0/2466462801 wait complete. 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.307+0000 7f7fe6669740 1 -- start start 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.307+0000 7f7fe6669740 1 Processor -- start 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.307+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb399f0 0x55b13ef36120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb521b0 0x55b13ef325e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fe6669740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb5a390 0x55b13ef32b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fe6669740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81abaf0 con 0x55b13eb5a390 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fe6669740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81acf60 con 0x55b13eb399f0 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fe6669740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81ac1c0 con 0x55b13eb521b0 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fe6669740 10 client.0 _ll_register_callbacks cb 0x7f7fc80118b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:38:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fdd690640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb5a390 0x55b13ef32b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fdd690640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb5a390 0x55b13ef32b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43730/0 (socket says 172.21.15.112:43730) 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fdde91640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb521b0 0x55b13ef325e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fdd690640 1 -- 172.21.15.112:0/305870106 learned_addr learned my addr 172.21.15.112:0/305870106 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fdd690640 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb399f0 0x55b13ef36120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fdd690640 1 -- 172.21.15.112:0/305870106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb521b0 msgr2=0x55b13ef325e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fdd690640 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb521b0 0x55b13ef325e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fdd690640 1 -- 172.21.15.112:0/305870106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb399f0 msgr2=0x55b13ef36120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fdd690640 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb399f0 0x55b13ef36120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fdd690640 1 -- 172.21.15.112:0/305870106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13ef33060 con 0x55b13eb5a390 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fdd690640 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb5a390 0x55b13ef32b20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8011820 tx=0x7f7fc8005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a7fdc2716bf09d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:38:38.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fc17fa640 1 -- 172.21.15.112:0/305870106 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8030070 con 0x55b13eb5a390 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fc17fa640 1 -- 172.21.15.112:0/305870106 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8470+0+0 (secure 0 0 0) 0x7f7fc80056e0 con 0x55b13eb5a390 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.308+0000 7f7fe6669740 1 -- 172.21.15.112:0/305870106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc8022830 con 0x55b13eb5a390 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fc17fa640 1 -- 172.21.15.112:0/305870106 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7fc80035f0 con 0x55b13eb5a390 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fe6669740 10 client.17154 fetch_fsmap learned FSMap version 30 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fe6669740 1 -- 172.21.15.112:0/305870106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b13efab1c0 con 0x55b13eb5a390 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fc17fa640 1 -- 172.21.15.112:0/305870106 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc8022830 con 0x55b13eb5a390 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fc17fa640 1 -- 172.21.15.112:0/305870106 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7fc8035070 con 0x55b13eb5a390 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fe6669740 10 client.17154 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fe6669740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fe6669740 20 client.17154 populate_metadata read hostname 'smithi112' 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fe6669740 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b13eb7b070 0x55b13efad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fe6669740 4 client.17154 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:38:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fe6669740 1 -- 172.21.15.112:0/305870106 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b13ee96850 con 0x55b13eb7b070 2024-09-21T03:38:38.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.309+0000 7f7fdde91640 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b13eb7b070 0x55b13efad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:38:38.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.310+0000 7f7fc17fa640 10 client.17154 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:38.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.310+0000 7f7fdde91640 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b13eb7b070 0x55b13efad8c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd54d680244d98a3 server_cookie=d2343db4feb0dd6d in_seq=0 out_seq=0 2024-09-21T03:38:38.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fc17fa640 1 -- 172.21.15.112:0/305870106 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x55b13ee96850 con 0x55b13eb7b070 2024-09-21T03:38:38.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fc17fa640 10 client.17154 handle_command_reply: tid=0 2024-09-21T03:38:38.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fe6669740 1 client.17154 shutdown 2024-09-21T03:38:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fe6669740 1 -- 172.21.15.112:0/305870106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb5a390 msgr2=0x55b13ef32b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fe6669740 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb5a390 0x55b13ef32b20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8011820 tx=0x7f7fc8005230 comp rx=0 tx=0).stop 2024-09-21T03:38:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fe6669740 1 -- 172.21.15.112:0/305870106 shutdown_connections 2024-09-21T03:38:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fe6669740 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b13eb7b070 0x55b13efad8c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fe6669740 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b13eb5a390 0x55b13ef32b20 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fc17fa640 0 client.17154 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fe6669740 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b13eb521b0 0x55b13ef325e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fe6669740 1 --2- 172.21.15.112:0/305870106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b13eb399f0 0x55b13ef36120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fe6669740 1 -- 172.21.15.112:0/305870106 >> 172.21.15.112:0/305870106 conn(0x55b13eb6f460 msgr2=0x55b13ef2f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fe6669740 1 -- 172.21.15.112:0/305870106 shutdown_connections 2024-09-21T03:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.311+0000 7f7fe6669740 1 -- 172.21.15.112:0/305870106 wait complete. 2024-09-21T03:38:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.312+0000 7f7fe6669740 20 client.17154 trim_cache size 0 max 16384 2024-09-21T03:38:38.318 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:38:38.318 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:38:38.318 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "730258e8-6c54-4776-b8f7-7ce3b100573d", 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.319 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.320 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.321 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.322 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:38:38.323 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.324 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:38:38.327 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:38:38.328 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:38:38.328 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:38:38.328 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:38:38.328 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:38:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.312+0000 7f7fe00f5640 1 -- 172.21.15.112:0/2268426539 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7fc40b8970 msgr2=0x7f7fc40bae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.312+0000 7f7fe00f5640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7fc40b8970 0x7f7fc40bae80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00e5160 tx=0x7f7fd001aed0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.312+0000 7f7fe00f5640 1 -- 172.21.15.112:0/2268426539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd8108d20 msgr2=0x7f7fd819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:38:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.312+0000 7f7fe00f5640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd8108d20 0x7f7fd819db50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000cab0 tx=0x7f7fd000cf70 comp rx=0 tx=0).stop 2024-09-21T03:38:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.313+0000 7f7fe00f5640 1 -- 172.21.15.112:0/2268426539 shutdown_connections 2024-09-21T03:38:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.313+0000 7f7fe00f5640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7fc40b8970 0x7f7fc40bae80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.313+0000 7f7fe00f5640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fd8108d20 0x7f7fd819db50 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.313+0000 7f7fe00f5640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fd81083d0 0x7f7fd81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.313+0000 7f7fe00f5640 1 --2- 172.21.15.112:0/2268426539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd8102ae0 0x7f7fd8198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:38:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.313+0000 7f7fe00f5640 1 -- 172.21.15.112:0/2268426539 >> 172.21.15.112:0/2268426539 conn(0x7f7fd8079790 msgr2=0x7f7fd81064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:38:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.313+0000 7f7fe00f5640 1 -- 172.21.15.112:0/2268426539 shutdown_connections 2024-09-21T03:38:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:38:38.313+0000 7f7fe00f5640 1 -- 172.21.15.112:0/2268426539 wait complete. 2024-09-21T03:38:38.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:38 smithi203 ceph-mon[34723]: pgmap v614: 97 pgs: 97 active+clean; 769 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2024-09-21T03:38:38.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:38 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:38:38.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:38 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:38.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:38 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:38:38.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:38 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:38:38.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:38 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:38:38.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:38 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:38.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:38 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:38.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:38 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:38.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:38 smithi187 ceph-mon[34603]: pgmap v614: 97 pgs: 97 active+clean; 769 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2024-09-21T03:38:38.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:38 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:38:38.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:38 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:38.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:38 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:38:38.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:38 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:38:38.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:38 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:38:38.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:38 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:38.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:38 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:38.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:38 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:38:39.130 INFO:tasks.cephfs.filesystem:scrub status for tag:730258e8-6c54-4776-b8f7-7ce3b100573d - {'path': '/', 'tag': '730258e8-6c54-4776-b8f7-7ce3b100573d', 'options': 'recursive,force'} 2024-09-21T03:38:39.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1023203 10.55 MB/sec execute 399 sec latency 27.202 ms 2024-09-21T03:38:40.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1024563 10.54 MB/sec execute 400 sec latency 19.896 ms 2024-09-21T03:38:40.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:40 smithi203 ceph-mon[34723]: pgmap v615: 97 pgs: 97 active+clean; 772 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-09-21T03:38:40.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:40 smithi187 ceph-mon[34603]: pgmap v615: 97 pgs: 97 active+clean; 772 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-09-21T03:38:40.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:40 smithi112 ceph-mon[31442]: pgmap v615: 97 pgs: 97 active+clean; 772 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-09-21T03:38:41.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1026930 10.55 MB/sec execute 401 sec latency 20.416 ms 2024-09-21T03:38:42.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1028113 10.53 MB/sec execute 402 sec latency 153.309 ms 2024-09-21T03:38:42.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:42 smithi203 ceph-mon[34723]: pgmap v616: 97 pgs: 97 active+clean; 778 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-09-21T03:38:42.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:42 smithi187 ceph-mon[34603]: pgmap v616: 97 pgs: 97 active+clean; 778 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-09-21T03:38:42.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:42 smithi112 ceph-mon[31442]: pgmap v616: 97 pgs: 97 active+clean; 778 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-09-21T03:38:43.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1030301 10.54 MB/sec execute 403 sec latency 70.557 ms 2024-09-21T03:38:44.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1031367 10.52 MB/sec execute 404 sec latency 21.202 ms 2024-09-21T03:38:44.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:44 smithi203 ceph-mon[34723]: pgmap v617: 97 pgs: 97 active+clean; 780 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-09-21T03:38:44.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:44 smithi187 ceph-mon[34603]: pgmap v617: 97 pgs: 97 active+clean; 780 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-09-21T03:38:44.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:44 smithi112 ceph-mon[31442]: pgmap v617: 97 pgs: 97 active+clean; 780 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-09-21T03:38:45.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1033921 10.54 MB/sec execute 405 sec latency 16.906 ms 2024-09-21T03:38:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:45 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:38:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:45 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:38:45.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:45 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:38:45.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:45 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:38:45.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:45 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:38:45.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:45 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:38:45.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:45 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:38:45.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:45 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:38:45.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:45 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:38:46.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1035162 10.52 MB/sec execute 406 sec latency 18.932 ms 2024-09-21T03:38:46.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:46 smithi203 ceph-mon[34723]: pgmap v618: 97 pgs: 97 active+clean; 785 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-09-21T03:38:46.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:46 smithi187 ceph-mon[34603]: pgmap v618: 97 pgs: 97 active+clean; 785 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-09-21T03:38:46.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:46 smithi112 ceph-mon[31442]: pgmap v618: 97 pgs: 97 active+clean; 785 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-09-21T03:38:47.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1037557 10.53 MB/sec execute 407 sec latency 29.498 ms 2024-09-21T03:38:48.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1038774 10.52 MB/sec execute 408 sec latency 23.734 ms 2024-09-21T03:38:48.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:48 smithi203 ceph-mon[34723]: pgmap v619: 97 pgs: 97 active+clean; 787 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-09-21T03:38:48.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:48 smithi187 ceph-mon[34603]: pgmap v619: 97 pgs: 97 active+clean; 787 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-09-21T03:38:48.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:48 smithi112 ceph-mon[31442]: pgmap v619: 97 pgs: 97 active+clean; 787 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-09-21T03:38:49.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1041121 10.53 MB/sec execute 409 sec latency 27.833 ms 2024-09-21T03:38:50.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1042483 10.52 MB/sec execute 410 sec latency 17.770 ms 2024-09-21T03:38:50.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:50 smithi203 ceph-mon[34723]: pgmap v620: 97 pgs: 97 active+clean; 790 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 228 op/s 2024-09-21T03:38:50.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:50 smithi187 ceph-mon[34603]: pgmap v620: 97 pgs: 97 active+clean; 790 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 228 op/s 2024-09-21T03:38:50.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:50 smithi112 ceph-mon[31442]: pgmap v620: 97 pgs: 97 active+clean; 790 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 228 op/s 2024-09-21T03:38:51.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1044756 10.52 MB/sec execute 411 sec latency 14.050 ms 2024-09-21T03:38:52.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1046241 10.51 MB/sec execute 412 sec latency 19.193 ms 2024-09-21T03:38:52.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:52 smithi203 ceph-mon[34723]: pgmap v621: 97 pgs: 97 active+clean; 797 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 278 op/s 2024-09-21T03:38:52.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:52 smithi187 ceph-mon[34603]: pgmap v621: 97 pgs: 97 active+clean; 797 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 278 op/s 2024-09-21T03:38:52.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:52 smithi112 ceph-mon[31442]: pgmap v621: 97 pgs: 97 active+clean; 797 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 278 op/s 2024-09-21T03:38:53.269 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1048273 10.52 MB/sec execute 413 sec latency 40.505 ms 2024-09-21T03:38:54.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1049604 10.51 MB/sec execute 414 sec latency 32.351 ms 2024-09-21T03:38:54.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:54 smithi203 ceph-mon[34723]: pgmap v622: 97 pgs: 97 active+clean; 800 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 249 op/s 2024-09-21T03:38:54.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:54 smithi187 ceph-mon[34603]: pgmap v622: 97 pgs: 97 active+clean; 800 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 249 op/s 2024-09-21T03:38:54.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:54 smithi112 ceph-mon[31442]: pgmap v622: 97 pgs: 97 active+clean; 800 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 249 op/s 2024-09-21T03:38:55.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1051802 10.51 MB/sec execute 415 sec latency 14.975 ms 2024-09-21T03:38:56.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1053284 10.50 MB/sec execute 416 sec latency 19.230 ms 2024-09-21T03:38:56.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:56 smithi203 ceph-mon[34723]: pgmap v623: 97 pgs: 97 active+clean; 800 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 223 op/s 2024-09-21T03:38:56.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:56 smithi187 ceph-mon[34603]: pgmap v623: 97 pgs: 97 active+clean; 800 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 223 op/s 2024-09-21T03:38:56.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:56 smithi112 ceph-mon[31442]: pgmap v623: 97 pgs: 97 active+clean; 800 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 223 op/s 2024-09-21T03:38:57.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1055429 10.51 MB/sec execute 417 sec latency 17.157 ms 2024-09-21T03:38:58.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1056881 10.50 MB/sec execute 418 sec latency 56.626 ms 2024-09-21T03:38:58.352 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:38:58 smithi112 ceph-mon[31442]: pgmap v624: 97 pgs: 97 active+clean; 807 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-09-21T03:38:58.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:38:58 smithi203 ceph-mon[34723]: pgmap v624: 97 pgs: 97 active+clean; 807 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-09-21T03:38:58.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:38:58 smithi187 ceph-mon[34603]: pgmap v624: 97 pgs: 97 active+clean; 807 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-09-21T03:38:59.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1058990 10.51 MB/sec execute 419 sec latency 40.812 ms 2024-09-21T03:39:00.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1060634 10.49 MB/sec execute 420 sec latency 20.227 ms 2024-09-21T03:39:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:00 smithi203 ceph-mon[34723]: pgmap v625: 97 pgs: 97 active+clean; 807 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 226 op/s 2024-09-21T03:39:00.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:00 smithi187 ceph-mon[34603]: pgmap v625: 97 pgs: 97 active+clean; 807 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 226 op/s 2024-09-21T03:39:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:00 smithi112 ceph-mon[31442]: pgmap v625: 97 pgs: 97 active+clean; 807 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 226 op/s 2024-09-21T03:39:01.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1062346 10.50 MB/sec execute 421 sec latency 27.826 ms 2024-09-21T03:39:02.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1063653 10.49 MB/sec execute 422 sec latency 17.952 ms 2024-09-21T03:39:02.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:02 smithi203 ceph-mon[34723]: pgmap v626: 97 pgs: 97 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-09-21T03:39:02.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:02 smithi187 ceph-mon[34603]: pgmap v626: 97 pgs: 97 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-09-21T03:39:02.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:02 smithi112 ceph-mon[31442]: pgmap v626: 97 pgs: 97 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 303 op/s 2024-09-21T03:39:03.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1065912 10.50 MB/sec execute 423 sec latency 48.607 ms 2024-09-21T03:39:04.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1067194 10.48 MB/sec execute 424 sec latency 39.458 ms 2024-09-21T03:39:04.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:04 smithi203 ceph-mon[34723]: pgmap v627: 97 pgs: 97 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2024-09-21T03:39:04.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:04 smithi187 ceph-mon[34603]: pgmap v627: 97 pgs: 97 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2024-09-21T03:39:04.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:04 smithi112 ceph-mon[31442]: pgmap v627: 97 pgs: 97 active+clean; 817 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2024-09-21T03:39:05.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1069554 10.49 MB/sec execute 425 sec latency 15.383 ms 2024-09-21T03:39:05.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:05 smithi203 ceph-mon[34723]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:39:05.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:05 smithi187 ceph-mon[34603]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:39:05.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:05 smithi112 ceph-mon[31442]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:39:06.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1070288 10.47 MB/sec execute 426 sec latency 352.500 ms 2024-09-21T03:39:06.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:06 smithi203 ceph-mon[34723]: pgmap v629: 97 pgs: 97 active+clean; 815 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-09-21T03:39:06.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:06 smithi187 ceph-mon[34603]: pgmap v629: 97 pgs: 97 active+clean; 815 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-09-21T03:39:06.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:06 smithi112 ceph-mon[31442]: pgmap v629: 97 pgs: 97 active+clean; 815 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-09-21T03:39:07.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1072887 10.48 MB/sec execute 427 sec latency 17.887 ms 2024-09-21T03:39:08.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1073888 10.47 MB/sec execute 428 sec latency 26.303 ms 2024-09-21T03:39:08.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:08 smithi203 ceph-mon[34723]: pgmap v630: 97 pgs: 97 active+clean; 806 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 305 op/s 2024-09-21T03:39:08.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:08 smithi187 ceph-mon[34603]: pgmap v630: 97 pgs: 97 active+clean; 806 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 305 op/s 2024-09-21T03:39:08.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:08 smithi112 ceph-mon[31442]: pgmap v630: 97 pgs: 97 active+clean; 806 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 305 op/s 2024-09-21T03:39:09.131 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:39:09.270 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1076323 10.48 MB/sec execute 429 sec latency 52.699 ms 2024-09-21T03:39:09.400 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.933+0000 7f450e37b640 1 -- 172.21.15.112:0/3092913155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4508104840 msgr2=0x7f4508104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.933+0000 7f450e37b640 1 --2- 172.21.15.112:0/3092913155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4508104840 0x7f4508104c50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009900 tx=0x7f44f802a410 comp rx=0 tx=0).stop 2024-09-21T03:39:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.933+0000 7f450e37b640 1 -- 172.21.15.112:0/3092913155 shutdown_connections 2024-09-21T03:39:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.933+0000 7f450e37b640 1 --2- 172.21.15.112:0/3092913155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f450810ddf0 0x7f4508110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.933+0000 7f450e37b640 1 --2- 172.21.15.112:0/3092913155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4508105270 0x7f450810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.933+0000 7f450e37b640 1 --2- 172.21.15.112:0/3092913155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4508104840 0x7f4508104c50 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.933+0000 7f450e37b640 1 -- 172.21.15.112:0/3092913155 >> 172.21.15.112:0/3092913155 conn(0x7f4508079730 msgr2=0x7f4508079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.933+0000 7f450e37b640 1 -- 172.21.15.112:0/3092913155 shutdown_connections 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.933+0000 7f450e37b640 1 -- 172.21.15.112:0/3092913155 wait complete. 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f450e37b640 1 Processor -- start 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f450e37b640 1 -- start start 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f450e37b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4508104840 0x7f450819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f450e37b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4508105270 0x7f450819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f450e37b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f450810ddf0 0x7f450819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f450e37b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4508113a90 con 0x7f4508104840 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f450e37b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4508113910 con 0x7f450810ddf0 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f450e37b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4508113c10 con 0x7f4508105270 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f450c918640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f450810ddf0 0x7f450819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f450c918640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f450810ddf0 0x7f450819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34732/0 (socket says 172.21.15.112:34732) 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f450c918640 1 -- 172.21.15.112:0/3159807038 learned_addr learned my addr 172.21.15.112:0/3159807038 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.934+0000 7f44fffff640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4508105270 0x7f450819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.935+0000 7f4507fff640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4508104840 0x7f450819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.935+0000 7f44fffff640 1 -- 172.21.15.112:0/3159807038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f450810ddf0 msgr2=0x7f450819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.935+0000 7f44fffff640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f450810ddf0 0x7f450819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.935+0000 7f44fffff640 1 -- 172.21.15.112:0/3159807038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4508104840 msgr2=0x7f450819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.935+0000 7f44fffff640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4508104840 0x7f450819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.935+0000 7f44fffff640 1 -- 172.21.15.112:0/3159807038 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45081a04c0 con 0x7f4508105270 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.936+0000 7f44fffff640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4508105270 0x7f450819ab20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f450806ad30 tx=0x7f44f000ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.937+0000 7f4505ffb640 1 -- 172.21.15.112:0/3159807038 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0020070 con 0x7f4508105270 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.937+0000 7f4505ffb640 1 -- 172.21.15.112:0/3159807038 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44f000ccc0 con 0x7f4508105270 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.937+0000 7f450e37b640 1 -- 172.21.15.112:0/3159807038 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45081a07b0 con 0x7f4508105270 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.937+0000 7f450e37b640 1 -- 172.21.15.112:0/3159807038 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4508102630 con 0x7f4508105270 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.938+0000 7f4505ffb640 1 -- 172.21.15.112:0/3159807038 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0020070 con 0x7f4508105270 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.940+0000 7f45148ef740 1 Processor -- start 2024-09-21T03:39:09.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.940+0000 7f45148ef740 1 -- start start 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.940+0000 7f4505ffb640 1 -- 172.21.15.112:0/3159807038 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f44f0023070 con 0x7f4508105270 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.940+0000 7f45148ef740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e84e070 0x563b0ebf5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.940+0000 7f45148ef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e8251b0 0x563b0ebf5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.940+0000 7f45148ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e82d390 0x563b0ebf60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.940+0000 7f45148ef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563b0eb6be50 con 0x563b0e8251b0 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.940+0000 7f45148ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563b0eb5b280 con 0x563b0e82d390 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.940+0000 7f45148ef740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563b0eb5f090 con 0x563b0e84e070 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f4507fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e8251b0 0x563b0ebf5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f450c918640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e84e070 0x563b0ebf5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f4507fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e8251b0 0x563b0ebf5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55086/0 (socket says 172.21.15.112:55086) 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f4507fff640 1 -- 172.21.15.112:0/3055876166 learned_addr learned my addr 172.21.15.112:0/3055876166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:09.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f44fffff640 1 --2- 172.21.15.112:0/3055876166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e82d390 0x563b0ebf60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f4507fff640 1 -- 172.21.15.112:0/3055876166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e84e070 msgr2=0x563b0ebf5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f4507fff640 1 --2- 172.21.15.112:0/3055876166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e84e070 0x563b0ebf5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f4507fff640 1 -- 172.21.15.112:0/3055876166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e82d390 msgr2=0x563b0ebf60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f4507fff640 1 --2- 172.21.15.112:0/3055876166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e82d390 0x563b0ebf60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f4507fff640 1 -- 172.21.15.112:0/3055876166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0e8b9650 con 0x563b0e8251b0 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f44fffff640 1 --2- 172.21.15.112:0/3055876166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e82d390 0x563b0ebf60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f4505ffb640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44f40b68b0 0x7f44f40b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f4507fff640 1 --2- 172.21.15.112:0/3055876166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e8251b0 0x563b0ebf5b90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f450806bc00 tx=0x7f44f8032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3ccc391017e88c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f450c918640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44f40b68b0 0x7f44f40b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.941+0000 7f4505ffb640 1 -- 172.21.15.112:0/3159807038 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f44f00eb2b0 con 0x7f4508105270 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.942+0000 7f44fe7fc640 1 -- 172.21.15.112:0/3055876166 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f803f070 con 0x563b0e8251b0 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.942+0000 7f44fe7fc640 1 -- 172.21.15.112:0/3055876166 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44f8004030 con 0x563b0e8251b0 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.942+0000 7f44fe7fc640 1 -- 172.21.15.112:0/3055876166 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f8035070 con 0x563b0e8251b0 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f45148ef740 1 -- 172.21.15.112:0/3055876166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e8251b0 msgr2=0x563b0ebf5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f450c918640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44f40b68b0 0x7f44f40b8dc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f45000049d0 tx=0x7f4500009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f45148ef740 1 --2- 172.21.15.112:0/3055876166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e8251b0 0x563b0ebf5b90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f450806bc00 tx=0x7f44f8032ba0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f45148ef740 1 -- 172.21.15.112:0/3055876166 shutdown_connections 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f45148ef740 1 --2- 172.21.15.112:0/3055876166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e82d390 0x563b0ebf60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f45148ef740 1 --2- 172.21.15.112:0/3055876166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e8251b0 0x563b0ebf5b90 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f45148ef740 1 --2- 172.21.15.112:0/3055876166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e84e070 0x563b0ebf5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f45148ef740 1 -- 172.21.15.112:0/3055876166 >> 172.21.15.112:0/3055876166 conn(0x563b0e842460 msgr2=0x563b0ebf3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f45148ef740 1 -- 172.21.15.112:0/3055876166 shutdown_connections 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f45148ef740 1 -- 172.21.15.112:0/3055876166 wait complete. 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f45148ef740 1 -- start start 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.943+0000 7f45148ef740 1 Processor -- start 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f45148ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e8251b0 0x563b0ec00cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f45148ef740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e82d390 0x563b0ec01230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f45148ef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e84e070 0x563b0ec0d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f45148ef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563b0e9bc700 con 0x563b0e84e070 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f45148ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563b0eaf6110 con 0x563b0e8251b0 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f45148ef740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563b0ec03dd0 con 0x563b0e82d390 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f45148ef740 10 client.0 _ll_register_callbacks cb 0x563b0eb0f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f44fffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e82d390 0x563b0ec01230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f450c918640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e84e070 0x563b0ec0d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f4507fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e8251b0 0x563b0ec00cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f450c918640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e84e070 0x563b0ec0d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55102/0 (socket says 172.21.15.112:55102) 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f4507fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e8251b0 0x563b0ec00cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34746/0 (socket says 172.21.15.112:34746) 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f450c918640 1 -- 172.21.15.112:0/902111361 learned_addr learned my addr 172.21.15.112:0/902111361 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f450c918640 1 -- 172.21.15.112:0/902111361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e82d390 msgr2=0x563b0ec01230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f450c918640 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e82d390 0x563b0ec01230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f450c918640 1 -- 172.21.15.112:0/902111361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e8251b0 msgr2=0x563b0ec00cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f450c918640 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e8251b0 0x563b0ec00cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f450c918640 1 -- 172.21.15.112:0/902111361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b0eb23c40 con 0x563b0e84e070 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.944+0000 7f4507fff640 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e8251b0 0x563b0ec00cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.945+0000 7f450c918640 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e84e070 0x563b0ec0d110 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f450000fb00 tx=0x7f4500004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0e53bd5cff9040 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.945+0000 7f44e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.945+0000 7f44e77fe640 1 -- 172.21.15.112:0/902111361 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450002e060 con 0x563b0e84e070 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.945+0000 7f44e77fe640 1 -- 172.21.15.112:0/902111361 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f4500018030 con 0x563b0e84e070 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.945+0000 7f45148ef740 1 -- 172.21.15.112:0/902111361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44f8004030 con 0x563b0e84e070 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.945+0000 7f44e77fe640 1 -- 172.21.15.112:0/902111361 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4500004b20 con 0x563b0e84e070 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.945+0000 7f45148ef740 10 client.17166 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.945+0000 7f45148ef740 1 -- 172.21.15.112:0/902111361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b0ec0fb70 con 0x563b0e84e070 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.945+0000 7f44e77fe640 1 -- 172.21.15.112:0/902111361 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44f8004030 con 0x563b0e84e070 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.946+0000 7f44e77fe640 1 -- 172.21.15.112:0/902111361 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4500005140 con 0x563b0e84e070 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.946+0000 7f45148ef740 10 client.17166 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.946+0000 7f45148ef740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:39:09.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.946+0000 7f45148ef740 20 client.17166 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:09.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.946+0000 7f45148ef740 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563b0e80c9f0 0x563b0ec130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.946+0000 7f45148ef740 4 client.17166 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:09.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.946+0000 7f45148ef740 1 -- 172.21.15.112:0/902111361 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b0eb6dda0 con 0x563b0e80c9f0 2024-09-21T03:39:09.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.946+0000 7f4507fff640 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563b0e80c9f0 0x563b0ec130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.946+0000 7f4507fff640 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563b0e80c9f0 0x563b0ec130d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9a11c34fa2ed4d server_cookie=468c1b7f16933eac in_seq=0 out_seq=0 2024-09-21T03:39:09.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.946+0000 7f44e77fe640 10 client.17166 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f44e77fe640 1 -- 172.21.15.112:0/902111361 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b0eb6dda0 con 0x563b0e80c9f0 2024-09-21T03:39:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f44e77fe640 10 client.17166 handle_command_reply: tid=0 2024-09-21T03:39:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f45148ef740 1 client.17166 shutdown 2024-09-21T03:39:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f45148ef740 1 -- 172.21.15.112:0/902111361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e84e070 msgr2=0x563b0ec0d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f45148ef740 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e84e070 0x563b0ec0d110 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f450000fb00 tx=0x7f4500004ab0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f45148ef740 1 -- 172.21.15.112:0/902111361 shutdown_connections 2024-09-21T03:39:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f45148ef740 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563b0e80c9f0 0x563b0ec130d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f44e77fe640 0 client.17166 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:09.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f45148ef740 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e84e070 0x563b0ec0d110 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f45148ef740 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e82d390 0x563b0ec01230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f45148ef740 1 --2- 172.21.15.112:0/902111361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e8251b0 0x563b0ec00cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.947+0000 7f45148ef740 1 -- 172.21.15.112:0/902111361 >> 172.21.15.112:0/902111361 conn(0x563b0e842460 msgr2=0x563b0ebf87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.948+0000 7f45148ef740 1 -- 172.21.15.112:0/902111361 shutdown_connections 2024-09-21T03:39:09.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.948+0000 7f45148ef740 1 -- 172.21.15.112:0/902111361 wait complete. 2024-09-21T03:39:09.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.948+0000 7f45148ef740 20 client.17166 trim_cache size 0 max 16384 2024-09-21T03:39:09.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.965+0000 7f45148ef740 1 Processor -- start 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.965+0000 7f45148ef740 1 -- start start 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f45148ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 0x563b0ebf6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f45148ef740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e8251b0 0x563b0ebf2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f45148ef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ebf30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f45148ef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f45081b86c0 con 0x563b0e82d390 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f45148ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4508073630 con 0x563b0e80c9f0 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f45148ef740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45080714a0 con 0x563b0e8251b0 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f450c918640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ebf30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f450c918640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ebf30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55114/0 (socket says 172.21.15.112:55114) 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f450c918640 1 -- 172.21.15.112:0/3782439722 learned_addr learned my addr 172.21.15.112:0/3782439722 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f44fffff640 1 --2- 172.21.15.112:0/3782439722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e8251b0 0x563b0ebf2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f4507fff640 1 --2- 172.21.15.112:0/3782439722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 0x563b0ebf6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f450c918640 1 -- 172.21.15.112:0/3782439722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e8251b0 msgr2=0x563b0ebf2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f450c918640 1 --2- 172.21.15.112:0/3782439722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e8251b0 0x563b0ebf2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f450c918640 1 -- 172.21.15.112:0/3782439722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 msgr2=0x563b0ebf6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:09.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f450c918640 1 --2- 172.21.15.112:0/3782439722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 0x563b0ebf6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f450c918640 1 -- 172.21.15.112:0/3782439722 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0ebf4300 con 0x563b0e82d390 2024-09-21T03:39:09.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f4507fff640 1 --2- 172.21.15.112:0/3782439722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 0x563b0ebf6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:09.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f450c918640 1 --2- 172.21.15.112:0/3782439722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ebf30b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4500019040 tx=0x7f4500005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab775847a2c190a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f44fdffb640 1 -- 172.21.15.112:0/3782439722 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500005890 con 0x563b0e82d390 2024-09-21T03:39:09.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f44fdffb640 1 -- 172.21.15.112:0/3782439722 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4500004cc0 con 0x563b0e82d390 2024-09-21T03:39:09.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.966+0000 7f44fdffb640 1 -- 172.21.15.112:0/3782439722 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450001e9e0 con 0x563b0e82d390 2024-09-21T03:39:09.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.967+0000 7f45148ef740 1 -- 172.21.15.112:0/3782439722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 msgr2=0x563b0ebf30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.967+0000 7f45148ef740 1 --2- 172.21.15.112:0/3782439722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ebf30b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4500019040 tx=0x7f4500005c50 comp rx=0 tx=0).stop 2024-09-21T03:39:09.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.967+0000 7f45148ef740 1 -- 172.21.15.112:0/3782439722 shutdown_connections 2024-09-21T03:39:09.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.967+0000 7f45148ef740 1 --2- 172.21.15.112:0/3782439722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ebf30b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.967+0000 7f45148ef740 1 --2- 172.21.15.112:0/3782439722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e8251b0 0x563b0ebf2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.967+0000 7f45148ef740 1 --2- 172.21.15.112:0/3782439722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 0x563b0ebf6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.967+0000 7f45148ef740 1 -- 172.21.15.112:0/3782439722 >> 172.21.15.112:0/3782439722 conn(0x563b0e842460 msgr2=0x563b0ebf6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.967+0000 7f45148ef740 1 -- 172.21.15.112:0/3782439722 shutdown_connections 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.967+0000 7f45148ef740 1 -- 172.21.15.112:0/3782439722 wait complete. 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f45148ef740 1 -- start start 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f45148ef740 1 Processor -- start 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f45148ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 0x563b0ec099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f45148ef740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e8251b0 0x563b0ec04eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f45148ef740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ec05550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f45148ef740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4508071af0 con 0x563b0e82d390 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f45148ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4508072f60 con 0x563b0e80c9f0 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f45148ef740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45080721c0 con 0x563b0e8251b0 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f45148ef740 10 client.0 _ll_register_callbacks cb 0x7f4500009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:09.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f450c918640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ec05550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f450c918640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ec05550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55120/0 (socket says 172.21.15.112:55120) 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f450c918640 1 -- 172.21.15.112:0/3726779108 learned_addr learned my addr 172.21.15.112:0/3726779108 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.968+0000 7f44fffff640 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e8251b0 0x563b0ec04eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f4507fff640 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 0x563b0ec099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f450c918640 1 -- 172.21.15.112:0/3726779108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e8251b0 msgr2=0x563b0ec04eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f450c918640 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e8251b0 0x563b0ec04eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f450c918640 1 -- 172.21.15.112:0/3726779108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 msgr2=0x563b0ec099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f450c918640 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 0x563b0ec099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f450c918640 1 -- 172.21.15.112:0/3726779108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b0ec05ce0 con 0x563b0e82d390 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f4507fff640 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 0x563b0ec099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f450c918640 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ec05550 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4500005600 tx=0x7f450001e270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1831caf679412939 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f44e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f44e77fe640 1 -- 172.21.15.112:0/3726779108 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500040020 con 0x563b0e82d390 2024-09-21T03:39:09.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f44e77fe640 1 -- 172.21.15.112:0/3726779108 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f4500005140 con 0x563b0e82d390 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f45148ef740 1 -- 172.21.15.112:0/3726779108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4500004cc0 con 0x563b0e82d390 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f44e77fe640 1 -- 172.21.15.112:0/3726779108 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f450004b050 con 0x563b0e82d390 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f45148ef740 10 client.17172 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.969+0000 7f45148ef740 1 -- 172.21.15.112:0/3726779108 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b0ec7db50 con 0x563b0e82d390 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.970+0000 7f44e77fe640 1 -- 172.21.15.112:0/3726779108 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4500004cc0 con 0x563b0e82d390 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.970+0000 7f44e77fe640 1 -- 172.21.15.112:0/3726779108 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f450003a070 con 0x563b0e82d390 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.970+0000 7f45148ef740 10 client.17172 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.970+0000 7f45148ef740 10 client.17172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.970+0000 7f45148ef740 20 client.17172 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.970+0000 7f45148ef740 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563b0e84e070 0x563b0ec801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.970+0000 7f45148ef740 4 client.17172 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.970+0000 7f45148ef740 1 -- 172.21.15.112:0/3726779108 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b0eb69850 con 0x563b0e84e070 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.970+0000 7f4507fff640 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563b0e84e070 0x563b0ec801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:09.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.970+0000 7f44e77fe640 10 client.17172 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.970+0000 7f4507fff640 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563b0e84e070 0x563b0ec801b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce15b5782e264091 server_cookie=e81603de5a5b785e in_seq=0 out_seq=0 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f44e77fe640 1 -- 172.21.15.112:0/3726779108 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x563b0eb69850 con 0x563b0e84e070 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f44e77fe640 10 client.17172 handle_command_reply: tid=0 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 1 client.17172 shutdown 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 1 -- 172.21.15.112:0/3726779108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 msgr2=0x563b0ec05550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ec05550 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4500005600 tx=0x7f450001e270 comp rx=0 tx=0).stop 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 1 -- 172.21.15.112:0/3726779108 shutdown_connections 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563b0e84e070 0x563b0ec801b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563b0e82d390 0x563b0ec05550 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0e8251b0 0x563b0ec04eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 1 --2- 172.21.15.112:0/3726779108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b0e80c9f0 0x563b0ec099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f44e77fe640 0 client.17172 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 1 -- 172.21.15.112:0/3726779108 >> 172.21.15.112:0/3726779108 conn(0x563b0e842460 msgr2=0x563b0ebf6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 1 -- 172.21.15.112:0/3726779108 shutdown_connections 2024-09-21T03:39:09.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 1 -- 172.21.15.112:0/3726779108 wait complete. 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.971+0000 7f45148ef740 20 client.17172 trim_cache size 0 max 16384 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:39:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:09.988 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:09.988 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:39:09.988 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:39:09.988 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:39:09.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 -- 172.21.15.112:0/3159807038 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44f40b68b0 msgr2=0x7f44f40b8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44f40b68b0 0x7f44f40b8dc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f45000049d0 tx=0x7f4500009500 comp rx=0 tx=0).stop 2024-09-21T03:39:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 -- 172.21.15.112:0/3159807038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4508105270 msgr2=0x7f450819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4508105270 0x7f450819ab20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f450806ad30 tx=0x7f44f000ed90 comp rx=0 tx=0).stop 2024-09-21T03:39:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 -- 172.21.15.112:0/3159807038 shutdown_connections 2024-09-21T03:39:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f44f40b68b0 0x7f44f40b8dc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f450810ddf0 0x7f450819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4508105270 0x7f450819ab20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 --2- 172.21.15.112:0/3159807038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4508104840 0x7f450819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 -- 172.21.15.112:0/3159807038 >> 172.21.15.112:0/3159807038 conn(0x7f4508079730 msgr2=0x7f4508107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 -- 172.21.15.112:0/3159807038 shutdown_connections 2024-09-21T03:39:09.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:09.972+0000 7f450e37b640 1 -- 172.21.15.112:0/3159807038 wait complete. 2024-09-21T03:39:10.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1077242 10.46 MB/sec execute 430 sec latency 43.290 ms 2024-09-21T03:39:10.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:10 smithi203 ceph-mon[34723]: pgmap v631: 97 pgs: 97 active+clean; 805 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 305 op/s 2024-09-21T03:39:10.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:10 smithi203 ceph-mon[34723]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:39:10.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:10 smithi187 ceph-mon[34603]: pgmap v631: 97 pgs: 97 active+clean; 805 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 305 op/s 2024-09-21T03:39:10.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:10 smithi187 ceph-mon[34603]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:39:10.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:10 smithi112 ceph-mon[31442]: pgmap v631: 97 pgs: 97 active+clean; 805 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 305 op/s 2024-09-21T03:39:10.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:10 smithi112 ceph-mon[31442]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:39:10.769 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:39:10.770 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:39:11.031 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:11.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1079716 10.47 MB/sec execute 431 sec latency 21.058 ms 2024-09-21T03:39:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.572+0000 7f8e49812640 1 -- 172.21.15.112:0/1270504669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e441083a0 msgr2=0x7f8e441087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.572+0000 7f8e49812640 1 --2- 172.21.15.112:0/1270504669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e441083a0 0x7f8e441087b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009900 tx=0x7f8e2c02a410 comp rx=0 tx=0).stop 2024-09-21T03:39:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.572+0000 7f8e49812640 1 -- 172.21.15.112:0/1270504669 shutdown_connections 2024-09-21T03:39:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.572+0000 7f8e49812640 1 --2- 172.21.15.112:0/1270504669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44108cf0 0x7f8e4410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.572+0000 7f8e49812640 1 --2- 172.21.15.112:0/1270504669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e441083a0 0x7f8e441087b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.572+0000 7f8e49812640 1 --2- 172.21.15.112:0/1270504669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e44102a60 0x7f8e44102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.572+0000 7f8e49812640 1 -- 172.21.15.112:0/1270504669 >> 172.21.15.112:0/1270504669 conn(0x7f8e44079730 msgr2=0x7f8e44079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.572+0000 7f8e49812640 1 -- 172.21.15.112:0/1270504669 shutdown_connections 2024-09-21T03:39:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.572+0000 7f8e49812640 1 -- 172.21.15.112:0/1270504669 wait complete. 2024-09-21T03:39:11.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e49812640 1 Processor -- start 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e49812640 1 -- start start 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e49812640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44102a60 0x7f8e4419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e49812640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e441083a0 0x7f8e4419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e49812640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e44108cf0 0x7f8e4419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e49812640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44112960 con 0x7f8e441083a0 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e49812640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e441127e0 con 0x7f8e44108cf0 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e49812640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44112ae0 con 0x7f8e44102a60 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e441083a0 0x7f8e4419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e441083a0 0x7f8e4419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55130/0 (socket says 172.21.15.112:55130) 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e427fc640 1 -- 172.21.15.112:0/1080663339 learned_addr learned my addr 172.21.15.112:0/1080663339 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e42ffd640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44102a60 0x7f8e4419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.573+0000 7f8e437fe640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e44108cf0 0x7f8e4419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.574+0000 7f8e427fc640 1 -- 172.21.15.112:0/1080663339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44102a60 msgr2=0x7f8e4419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.574+0000 7f8e427fc640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44102a60 0x7f8e4419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.574+0000 7f8e427fc640 1 -- 172.21.15.112:0/1080663339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e44108cf0 msgr2=0x7f8e4419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.574+0000 7f8e427fc640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e44108cf0 0x7f8e4419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.574+0000 7f8e427fc640 1 -- 172.21.15.112:0/1080663339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e441a0450 con 0x7f8e441083a0 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.574+0000 7f8e427fc640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e441083a0 0x7f8e4419aa80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c032fd0 tx=0x7f8e2c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.574+0000 7f8e48810640 1 -- 172.21.15.112:0/1080663339 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c03f030 con 0x7f8e441083a0 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.574+0000 7f8e48810640 1 -- 172.21.15.112:0/1080663339 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e2c004490 con 0x7f8e441083a0 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.574+0000 7f8e49812640 1 -- 172.21.15.112:0/1080663339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e441a06e0 con 0x7f8e441083a0 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.574+0000 7f8e48810640 1 -- 172.21.15.112:0/1080663339 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c035070 con 0x7f8e441083a0 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.574+0000 7f8e49812640 1 -- 172.21.15.112:0/1080663339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e441a6ea0 con 0x7f8e441083a0 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.575+0000 7f8e49812640 1 -- 172.21.15.112:0/1080663339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e44103ce0 con 0x7f8e441083a0 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.577+0000 7f8e48810640 1 -- 172.21.15.112:0/1080663339 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8e2c042030 con 0x7f8e441083a0 2024-09-21T03:39:11.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.577+0000 7f8e48810640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8e300b6680 0x7f8e300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:11.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.578+0000 7f8e42ffd640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8e300b6680 0x7f8e300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:11.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.578+0000 7f8e48810640 1 -- 172.21.15.112:0/1080663339 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f8e2c107a00 con 0x7f8e441083a0 2024-09-21T03:39:11.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.578+0000 7f8e42ffd640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8e300b6680 0x7f8e300b8b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009bd0 tx=0x7f8e3c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:11.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.582+0000 7f8e48810640 1 -- 172.21.15.112:0/1080663339 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8e2c0d2360 con 0x7f8e441083a0 2024-09-21T03:39:11.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.806+0000 7f8e49812640 1 -- 172.21.15.112:0/1080663339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e44102e70 con 0x7f8e441083a0 2024-09-21T03:39:11.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.806+0000 7f8e48810640 1 -- 172.21.15.112:0/1080663339 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f8e2c0d72a0 con 0x7f8e441083a0 2024-09-21T03:39:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:39:11.809 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"btime":"2024-09-21T03:35:32:357323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:35:32.307199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:39:11.809 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-09-21T03:39:11.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 -- 172.21.15.112:0/1080663339 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8e300b6680 msgr2=0x7f8e300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8e300b6680 0x7f8e300b8b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009bd0 tx=0x7f8e3c005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:11.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 -- 172.21.15.112:0/1080663339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e441083a0 msgr2=0x7f8e4419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:11.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e441083a0 0x7f8e4419aa80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c032fd0 tx=0x7f8e2c031a20 comp rx=0 tx=0).stop 2024-09-21T03:39:11.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 -- 172.21.15.112:0/1080663339 shutdown_connections 2024-09-21T03:39:11.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f8e300b6680 0x7f8e300b8b90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e44108cf0 0x7f8e4419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e441083a0 0x7f8e4419aa80 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 --2- 172.21.15.112:0/1080663339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e44102a60 0x7f8e4419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:11.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 -- 172.21.15.112:0/1080663339 >> 172.21.15.112:0/1080663339 conn(0x7f8e44079730 msgr2=0x7f8e4406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:11.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 -- 172.21.15.112:0/1080663339 shutdown_connections 2024-09-21T03:39:11.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:11.809+0000 7f8e49812640 1 -- 172.21.15.112:0/1080663339 wait complete. 2024-09-21T03:39:12.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1080553 10.46 MB/sec execute 432 sec latency 83.034 ms 2024-09-21T03:39:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:12 smithi203 ceph-mon[34723]: pgmap v633: 97 pgs: 97 active+clean; 811 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 322 op/s 2024-09-21T03:39:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:12 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1080663339' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:12.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:12 smithi187 ceph-mon[34603]: pgmap v633: 97 pgs: 97 active+clean; 811 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 322 op/s 2024-09-21T03:39:12.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:12 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1080663339' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:12.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:12 smithi112 ceph-mon[31442]: pgmap v633: 97 pgs: 97 active+clean; 811 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 322 op/s 2024-09-21T03:39:12.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:12 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1080663339' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:12.598 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:39:12.852 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:13.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1082769 10.46 MB/sec execute 433 sec latency 15.910 ms 2024-09-21T03:39:13.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.391+0000 7fcf89fc6640 1 -- 172.21.15.112:0/81055370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf84108cf0 msgr2=0x7fcf8410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.391+0000 7fcf89fc6640 1 --2- 172.21.15.112:0/81055370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf84108cf0 0x7fcf8410f520 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009ca0 tx=0x7fcf7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.391+0000 7fcf89fc6640 1 -- 172.21.15.112:0/81055370 shutdown_connections 2024-09-21T03:39:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.391+0000 7fcf89fc6640 1 --2- 172.21.15.112:0/81055370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf84108cf0 0x7fcf8410f520 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.391+0000 7fcf89fc6640 1 --2- 172.21.15.112:0/81055370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf841083a0 0x7fcf841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.391+0000 7fcf89fc6640 1 --2- 172.21.15.112:0/81055370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf84102a60 0x7fcf84102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.391+0000 7fcf89fc6640 1 -- 172.21.15.112:0/81055370 >> 172.21.15.112:0/81055370 conn(0x7fcf84079730 msgr2=0x7fcf84079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.391+0000 7fcf89fc6640 1 -- 172.21.15.112:0/81055370 shutdown_connections 2024-09-21T03:39:13.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.391+0000 7fcf89fc6640 1 -- 172.21.15.112:0/81055370 wait complete. 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.392+0000 7fcf89fc6640 1 Processor -- start 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.392+0000 7fcf89fc6640 1 -- start start 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.392+0000 7fcf89fc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf84102a60 0x7fcf84199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.392+0000 7fcf89fc6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf841083a0 0x7fcf841998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.392+0000 7fcf89fc6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf84108cf0 0x7fcf8419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.392+0000 7fcf89fc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcf84112960 con 0x7fcf84102a60 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.392+0000 7fcf89fc6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841127e0 con 0x7fcf841083a0 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.392+0000 7fcf89fc6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf84112ae0 con 0x7fcf84108cf0 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf837fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf84102a60 0x7fcf84199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf837fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf84102a60 0x7fcf84199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55146/0 (socket says 172.21.15.112:55146) 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf837fe640 1 -- 172.21.15.112:0/200961307 learned_addr learned my addr 172.21.15.112:0/200961307 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:13.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf83fff640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf84108cf0 0x7fcf8419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf82ffd640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf841083a0 0x7fcf841998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf837fe640 1 -- 172.21.15.112:0/200961307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf84108cf0 msgr2=0x7fcf8419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf837fe640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf84108cf0 0x7fcf8419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf837fe640 1 -- 172.21.15.112:0/200961307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf841083a0 msgr2=0x7fcf841998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf837fe640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf841083a0 0x7fcf841998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf837fe640 1 -- 172.21.15.112:0/200961307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8419f280 con 0x7fcf84102a60 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf837fe640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf84102a60 0x7fcf84199310 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400e9f0 tx=0x7fcf7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf80ff9640 1 -- 172.21.15.112:0/200961307 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400cd80 con 0x7fcf84102a60 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf80ff9640 1 -- 172.21.15.112:0/200961307 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf740108e0 con 0x7fcf84102a60 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf80ff9640 1 -- 172.21.15.112:0/200961307 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74018980 con 0x7fcf84102a60 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf89fc6640 1 -- 172.21.15.112:0/200961307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8419f570 con 0x7fcf84102a60 2024-09-21T03:39:13.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.393+0000 7fcf89fc6640 1 -- 172.21.15.112:0/200961307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf841a5cd0 con 0x7fcf84102a60 2024-09-21T03:39:13.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf9053a740 1 Processor -- start 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf80ff9640 1 -- 172.21.15.112:0/200961307 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fcf74010430 con 0x7fcf84102a60 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf9053a740 1 -- start start 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f80b070 0x55e34fbb26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f7e21b0 0x55e34fbb2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbb3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf9053a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e34fb28ea0 con 0x55e34f80b070 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf9053a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e34fb182d0 con 0x55e34f7e21b0 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf9053a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e34fb1c0e0 con 0x55e34f7ea390 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f80b070 0x55e34fbb26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf837fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f7e21b0 0x55e34fbb2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f80b070 0x55e34fbb26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55150/0 (socket says 172.21.15.112:55150) 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf83fff640 1 -- 172.21.15.112:0/2644971173 learned_addr learned my addr 172.21.15.112:0/2644971173 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:13.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.396+0000 7fcf82ffd640 1 --2- 172.21.15.112:0/2644971173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbb3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.397+0000 7fcf83fff640 1 -- 172.21.15.112:0/2644971173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 msgr2=0x55e34fbb3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:13.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.397+0000 7fcf83fff640 1 --2- 172.21.15.112:0/2644971173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbb3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.397+0000 7fcf83fff640 1 -- 172.21.15.112:0/2644971173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f7e21b0 msgr2=0x55e34fbb2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:13.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.397+0000 7fcf83fff640 1 --2- 172.21.15.112:0/2644971173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f7e21b0 0x55e34fbb2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.397+0000 7fcf83fff640 1 -- 172.21.15.112:0/2644971173 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e34f876650 con 0x55e34f80b070 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.397+0000 7fcf82ffd640 1 --2- 172.21.15.112:0/2644971173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbb3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.397+0000 7fcf80ff9640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcf6c0b6680 0x7fcf6c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.397+0000 7fcf83fff640 1 --2- 172.21.15.112:0/2644971173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f80b070 0x55e34fbb26a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c02f410 tx=0x7fcf7c02f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd79e1bd23c30f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.398+0000 7fcf837fe640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcf6c0b6680 0x7fcf6c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.398+0000 7fcf80ff9640 1 -- 172.21.15.112:0/200961307 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fcf740e7ad0 con 0x7fcf84102a60 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.398+0000 7fcf69ffb640 1 -- 172.21.15.112:0/2644971173 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c031cf0 con 0x55e34f80b070 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.398+0000 7fcf69ffb640 1 -- 172.21.15.112:0/2644971173 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf7c036030 con 0x55e34f80b070 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.398+0000 7fcf69ffb640 1 -- 172.21.15.112:0/2644971173 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c0042a0 con 0x55e34f80b070 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.399+0000 7fcf9053a740 1 -- 172.21.15.112:0/2644971173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f80b070 msgr2=0x55e34fbb26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.399+0000 7fcf837fe640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcf6c0b6680 0x7fcf6c0b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcf740e5300 tx=0x7fcf740eb620 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.399+0000 7fcf9053a740 1 --2- 172.21.15.112:0/2644971173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f80b070 0x55e34fbb26a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c02f410 tx=0x7fcf7c02f440 comp rx=0 tx=0).stop 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.399+0000 7fcf9053a740 1 -- 172.21.15.112:0/2644971173 shutdown_connections 2024-09-21T03:39:13.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.399+0000 7fcf9053a740 1 --2- 172.21.15.112:0/2644971173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbb3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.399+0000 7fcf9053a740 1 --2- 172.21.15.112:0/2644971173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f7e21b0 0x55e34fbb2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.399+0000 7fcf9053a740 1 --2- 172.21.15.112:0/2644971173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f80b070 0x55e34fbb26a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.399+0000 7fcf9053a740 1 -- 172.21.15.112:0/2644971173 >> 172.21.15.112:0/2644971173 conn(0x55e34f7ff460 msgr2=0x55e34fbb0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.399+0000 7fcf9053a740 1 -- 172.21.15.112:0/2644971173 shutdown_connections 2024-09-21T03:39:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.399+0000 7fcf9053a740 1 -- 172.21.15.112:0/2644971173 wait complete. 2024-09-21T03:39:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf9053a740 1 -- start start 2024-09-21T03:39:13.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf9053a740 1 Processor -- start 2024-09-21T03:39:13.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbbde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbbe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbcc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf9053a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e34f979700 con 0x55e34f7e21b0 2024-09-21T03:39:13.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf9053a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e34fab3160 con 0x55e34f80b070 2024-09-21T03:39:13.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf9053a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e34fbc2eb0 con 0x55e34f7ea390 2024-09-21T03:39:13.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf9053a740 10 client.0 _ll_register_callbacks cb 0x55e34facc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbbde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbbde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55152/0 (socket says 172.21.15.112:55152) 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf83fff640 1 -- 172.21.15.112:0/1750600216 learned_addr learned my addr 172.21.15.112:0/1750600216 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf82ffd640 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbbe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.400+0000 7fcf83fff640 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbcc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf83fff640 1 -- 172.21.15.112:0/1750600216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 msgr2=0x55e34fbbe340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf83fff640 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbbe340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf83fff640 1 -- 172.21.15.112:0/1750600216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 msgr2=0x55e34fbcc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf83fff640 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbcc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf83fff640 1 -- 172.21.15.112:0/1750600216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e34fae0c90 con 0x55e34f7e21b0 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf83fff640 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbbde00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009bf0 tx=0x7fcf7c003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78aaaabe36e12c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:39:13.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf62ffd640 1 -- 172.21.15.112:0/1750600216 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c058060 con 0x55e34f7e21b0 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf62ffd640 1 -- 172.21.15.112:0/1750600216 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fcf7c02f660 con 0x55e34f7e21b0 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf9053a740 1 -- 172.21.15.112:0/1750600216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf7c036030 con 0x55e34f7e21b0 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf62ffd640 1 -- 172.21.15.112:0/1750600216 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf7c031920 con 0x55e34f7e21b0 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf9053a740 10 client.17190 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf9053a740 1 -- 172.21.15.112:0/1750600216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e34fbcec60 con 0x55e34f7e21b0 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.401+0000 7fcf62ffd640 1 -- 172.21.15.112:0/1750600216 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf7c036030 con 0x55e34f7e21b0 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.402+0000 7fcf62ffd640 1 -- 172.21.15.112:0/1750600216 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcf7c043070 con 0x55e34f7e21b0 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.402+0000 7fcf9053a740 10 client.17190 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.402+0000 7fcf9053a740 10 client.17190 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.402+0000 7fcf9053a740 20 client.17190 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.402+0000 7fcf9053a740 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55e34f7c99f0 0x55e34fbd1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.402+0000 7fcf9053a740 4 client.17190 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.402+0000 7fcf9053a740 1 -- 172.21.15.112:0/1750600216 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e34fb2adf0 con 0x55e34f7c99f0 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.402+0000 7fcf82ffd640 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55e34f7c99f0 0x55e34fbd1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.402+0000 7fcf62ffd640 10 client.17190 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.402+0000 7fcf82ffd640 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55e34f7c99f0 0x55e34fbd1fc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb9e40a296d59a4b server_cookie=1f057f69c7ca741d in_seq=0 out_seq=0 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf62ffd640 1 -- 172.21.15.112:0/1750600216 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e34fb2adf0 con 0x55e34f7c99f0 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf62ffd640 10 client.17190 handle_command_reply: tid=0 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf9053a740 1 client.17190 shutdown 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf9053a740 1 -- 172.21.15.112:0/1750600216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 msgr2=0x55e34fbbde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf9053a740 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbbde00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009bf0 tx=0x7fcf7c003c90 comp rx=0 tx=0).stop 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf9053a740 1 -- 172.21.15.112:0/1750600216 shutdown_connections 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf9053a740 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55e34f7c99f0 0x55e34fbd1fc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf9053a740 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbcc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf9053a740 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbbe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf62ffd640 0 client.17190 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf9053a740 1 --2- 172.21.15.112:0/1750600216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbbde00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf9053a740 1 -- 172.21.15.112:0/1750600216 >> 172.21.15.112:0/1750600216 conn(0x55e34f7ff460 msgr2=0x55e34fbb5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:13.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.403+0000 7fcf9053a740 1 -- 172.21.15.112:0/1750600216 shutdown_connections 2024-09-21T03:39:13.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.404+0000 7fcf9053a740 1 -- 172.21.15.112:0/1750600216 wait complete. 2024-09-21T03:39:13.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.404+0000 7fcf9053a740 20 client.17190 trim_cache size 0 max 16384 2024-09-21T03:39:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf9053a740 1 Processor -- start 2024-09-21T03:39:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf9053a740 1 -- start start 2024-09-21T03:39:13.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbb5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbcc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbbaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf9053a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841afc30 con 0x55e34f7e21b0 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf9053a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841ad500 con 0x55e34f80b070 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf9053a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841ab370 con 0x55e34f7ea390 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbb5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbb5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55166/0 (socket says 172.21.15.112:55166) 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf83fff640 1 -- 172.21.15.112:0/1492207575 learned_addr learned my addr 172.21.15.112:0/1492207575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf82ffd640 1 --2- 172.21.15.112:0/1492207575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbcc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.417+0000 7fcf83fff640 1 --2- 172.21.15.112:0/1492207575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbbaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.418+0000 7fcf83fff640 1 -- 172.21.15.112:0/1492207575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 msgr2=0x55e34fbcc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.418+0000 7fcf83fff640 1 --2- 172.21.15.112:0/1492207575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbcc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.418+0000 7fcf83fff640 1 -- 172.21.15.112:0/1492207575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 msgr2=0x55e34fbbaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.418+0000 7fcf83fff640 1 --2- 172.21.15.112:0/1492207575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbbaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.418+0000 7fcf83fff640 1 -- 172.21.15.112:0/1492207575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e34fbc09b0 con 0x55e34f7e21b0 2024-09-21T03:39:13.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.418+0000 7fcf83fff640 1 --2- 172.21.15.112:0/1492207575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbb5eb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c02ade0 tx=0x7fcf7c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0e81655e6a0de6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:13.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.418+0000 7fcf697fa640 1 -- 172.21.15.112:0/1492207575 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c058030 con 0x55e34f7e21b0 2024-09-21T03:39:13.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.418+0000 7fcf697fa640 1 -- 172.21.15.112:0/1492207575 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf7c036070 con 0x55e34f7e21b0 2024-09-21T03:39:13.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.418+0000 7fcf697fa640 1 -- 172.21.15.112:0/1492207575 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c02fd70 con 0x55e34f7e21b0 2024-09-21T03:39:13.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.419+0000 7fcf9053a740 1 -- 172.21.15.112:0/1492207575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 msgr2=0x55e34fbb5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.419+0000 7fcf9053a740 1 --2- 172.21.15.112:0/1492207575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbb5eb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c02ade0 tx=0x7fcf7c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.419+0000 7fcf9053a740 1 -- 172.21.15.112:0/1492207575 shutdown_connections 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.419+0000 7fcf9053a740 1 --2- 172.21.15.112:0/1492207575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbbaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.419+0000 7fcf9053a740 1 --2- 172.21.15.112:0/1492207575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7ea390 0x55e34fbcc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.419+0000 7fcf9053a740 1 --2- 172.21.15.112:0/1492207575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7e21b0 0x55e34fbb5eb0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.419+0000 7fcf9053a740 1 -- 172.21.15.112:0/1492207575 >> 172.21.15.112:0/1492207575 conn(0x55e34f7ff460 msgr2=0x55e34fbb57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.419+0000 7fcf9053a740 1 -- 172.21.15.112:0/1492207575 shutdown_connections 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.419+0000 7fcf9053a740 1 -- 172.21.15.112:0/1492207575 wait complete. 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.419+0000 7fcf9053a740 1 -- start start 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf9053a740 1 Processor -- start 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7e21b0 0x55e34fbd59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7ea390 0x55e34fbd2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf9053a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbd35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf9053a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841ab9c0 con 0x55e34f7ea390 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf9053a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841ace30 con 0x55e34f80b070 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf9053a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841ac090 con 0x55e34f7e21b0 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf9053a740 10 client.0 _ll_register_callbacks cb 0x7fcf740eb4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf82ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7ea390 0x55e34fbd2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf82ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7ea390 0x55e34fbd2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55180/0 (socket says 172.21.15.112:55180) 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf82ffd640 1 -- 172.21.15.112:0/4122139931 learned_addr learned my addr 172.21.15.112:0/4122139931 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf83fff640 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbd35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf83fff640 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7e21b0 0x55e34fbd59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf82ffd640 1 -- 172.21.15.112:0/4122139931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7e21b0 msgr2=0x55e34fbd59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf82ffd640 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7e21b0 0x55e34fbd59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf82ffd640 1 -- 172.21.15.112:0/4122139931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 msgr2=0x55e34fbd35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf82ffd640 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbd35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf82ffd640 1 -- 172.21.15.112:0/4122139931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e34fc3b740 con 0x55e34f7ea390 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.420+0000 7fcf83fff640 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7e21b0 0x55e34fbd59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:39:13.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf82ffd640 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7ea390 0x55e34fbd2eb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x55e34fb18390 tx=0x7fcf78009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe98c84f46e0f19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf62ffd640 1 -- 172.21.15.112:0/4122139931 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78012650 con 0x55e34f7ea390 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf62ffd640 1 -- 172.21.15.112:0/4122139931 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fcf780127f0 con 0x55e34f7ea390 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf9053a740 1 -- 172.21.15.112:0/4122139931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf7c036070 con 0x55e34f7ea390 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf62ffd640 1 -- 172.21.15.112:0/4122139931 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf78003e50 con 0x55e34f7ea390 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf9053a740 10 client.17196 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf9053a740 1 -- 172.21.15.112:0/4122139931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e34fc3c900 con 0x55e34f7ea390 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf62ffd640 1 -- 172.21.15.112:0/4122139931 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf7c036070 con 0x55e34f7ea390 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf62ffd640 1 -- 172.21.15.112:0/4122139931 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcf78025b20 con 0x55e34f7ea390 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf9053a740 10 client.17196 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf9053a740 10 client.17196 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf9053a740 20 client.17196 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf9053a740 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55e34f79eda0 0x55e34fc3f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf9053a740 4 client.17196 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:39:13.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.421+0000 7fcf9053a740 1 -- 172.21.15.112:0/4122139931 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e34fb268a0 con 0x55e34f79eda0 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.422+0000 7fcf83fff640 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55e34f79eda0 0x55e34fc3f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.422+0000 7fcf62ffd640 10 client.17196 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.422+0000 7fcf83fff640 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55e34f79eda0 0x55e34fc3f0f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd689133428a22dd server_cookie=99d6b291d572bd8c in_seq=0 out_seq=0 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.422+0000 7fcf62ffd640 1 -- 172.21.15.112:0/4122139931 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e34fb268a0 con 0x55e34f79eda0 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.422+0000 7fcf62ffd640 10 client.17196 handle_command_reply: tid=0 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.422+0000 7fcf9053a740 1 client.17196 shutdown 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.422+0000 7fcf9053a740 1 -- 172.21.15.112:0/4122139931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7ea390 msgr2=0x55e34fbd2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.422+0000 7fcf9053a740 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7ea390 0x55e34fbd2eb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x55e34fb18390 tx=0x7fcf78009f90 comp rx=0 tx=0).stop 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf9053a740 1 -- 172.21.15.112:0/4122139931 shutdown_connections 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf9053a740 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55e34f79eda0 0x55e34fc3f0f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf9053a740 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e34f80b070 0x55e34fbd35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf9053a740 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e34f7ea390 0x55e34fbd2eb0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf62ffd640 0 client.17196 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:39:13.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf9053a740 1 --2- 172.21.15.112:0/4122139931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e34f7e21b0 0x55e34fbd59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf9053a740 1 -- 172.21.15.112:0/4122139931 >> 172.21.15.112:0/4122139931 conn(0x55e34f7ff460 msgr2=0x55e34fbbf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:13.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf9053a740 1 -- 172.21.15.112:0/4122139931 shutdown_connections 2024-09-21T03:39:13.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf9053a740 1 -- 172.21.15.112:0/4122139931 wait complete. 2024-09-21T03:39:13.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf9053a740 20 client.17196 trim_cache size 0 max 16384 2024-09-21T03:39:13.431 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:39:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf89fc6640 1 -- 172.21.15.112:0/200961307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcf6c0b6680 msgr2=0x7fcf6c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf89fc6640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcf6c0b6680 0x7fcf6c0b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcf740e5300 tx=0x7fcf740eb620 comp rx=0 tx=0).stop 2024-09-21T03:39:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf89fc6640 1 -- 172.21.15.112:0/200961307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf84102a60 msgr2=0x7fcf84199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf89fc6640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf84102a60 0x7fcf84199310 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400e9f0 tx=0x7fcf7400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf89fc6640 1 -- 172.21.15.112:0/200961307 shutdown_connections 2024-09-21T03:39:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf89fc6640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcf6c0b6680 0x7fcf6c0b8b90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf89fc6640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf84108cf0 0x7fcf8419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf89fc6640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf841083a0 0x7fcf841998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf89fc6640 1 --2- 172.21.15.112:0/200961307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf84102a60 0x7fcf84199310 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.423+0000 7fcf89fc6640 1 -- 172.21.15.112:0/200961307 >> 172.21.15.112:0/200961307 conn(0x7fcf84079730 msgr2=0x7fcf841043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:13.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.424+0000 7fcf89fc6640 1 -- 172.21.15.112:0/200961307 shutdown_connections 2024-09-21T03:39:13.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:13.424+0000 7fcf89fc6640 1 -- 172.21.15.112:0/200961307 wait complete. 2024-09-21T03:39:14.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1083897 10.45 MB/sec execute 434 sec latency 54.552 ms 2024-09-21T03:39:14.324 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:39:14.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:14 smithi203 ceph-mon[34723]: pgmap v634: 97 pgs: 97 active+clean; 813 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 331 op/s 2024-09-21T03:39:14.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:14 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:14.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:14 smithi187 ceph-mon[34603]: pgmap v634: 97 pgs: 97 active+clean; 813 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 331 op/s 2024-09-21T03:39:14.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:14 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:14 smithi112 ceph-mon[31442]: pgmap v634: 97 pgs: 97 active+clean; 813 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 331 op/s 2024-09-21T03:39:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:14 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:39:15.240 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:15.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1085124 10.44 MB/sec execute 435 sec latency 136.413 ms 2024-09-21T03:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.766+0000 7f881ce35640 1 -- 172.21.15.112:0/3274210944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8818101f70 msgr2=0x7f8818102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.766+0000 7f881ce35640 1 --2- 172.21.15.112:0/3274210944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8818101f70 0x7f8818102380 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8808009900 tx=0x7f880802a410 comp rx=0 tx=0).stop 2024-09-21T03:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.766+0000 7f881ce35640 1 -- 172.21.15.112:0/3274210944 shutdown_connections 2024-09-21T03:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.766+0000 7f881ce35640 1 --2- 172.21.15.112:0/3274210944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8818109e00 0x7f8818110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.766+0000 7f881ce35640 1 --2- 172.21.15.112:0/3274210944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88181094b0 0x7f88181098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.766+0000 7f881ce35640 1 --2- 172.21.15.112:0/3274210944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8818101f70 0x7f8818102380 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.766+0000 7f881ce35640 1 -- 172.21.15.112:0/3274210944 >> 172.21.15.112:0/3274210944 conn(0x7f8818077c60 msgr2=0x7f88180780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.766+0000 7f881ce35640 1 -- 172.21.15.112:0/3274210944 shutdown_connections 2024-09-21T03:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.767+0000 7f881ce35640 1 -- 172.21.15.112:0/3274210944 wait complete. 2024-09-21T03:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.767+0000 7f881ce35640 1 Processor -- start 2024-09-21T03:39:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f881ce35640 1 -- start start 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f881ce35640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8818101f70 0x7f881819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f881ce35640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88181094b0 0x7f881819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f881ce35640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8818109e00 0x7f881819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f881ce35640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8818113ac0 con 0x7f88181094b0 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f881ce35640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8818113940 con 0x7f8818101f70 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f881ce35640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8818113c40 con 0x7f8818109e00 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f880dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88181094b0 0x7f881819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f881659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8818101f70 0x7f881819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f880dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88181094b0 0x7f881819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46818/0 (socket says 172.21.15.112:46818) 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f880dd9b640 1 -- 172.21.15.112:0/2894941414 learned_addr learned my addr 172.21.15.112:0/2894941414 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f881659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8818101f70 0x7f881819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40966/0 (socket says 172.21.15.112:40966) 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f8816d9d640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8818109e00 0x7f881819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f880dd9b640 1 -- 172.21.15.112:0/2894941414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8818109e00 msgr2=0x7f881819fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f880dd9b640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8818109e00 0x7f881819fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f880dd9b640 1 -- 172.21.15.112:0/2894941414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8818101f70 msgr2=0x7f881819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f880dd9b640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8818101f70 0x7f881819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f880dd9b640 1 -- 172.21.15.112:0/2894941414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88181a0480 con 0x7f88181094b0 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.768+0000 7f880dd9b640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88181094b0 0x7f881819aab0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f880000e9c0 tx=0x7f880000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.769+0000 7f880ffff640 1 -- 172.21.15.112:0/2894941414 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880000ccc0 con 0x7f88181094b0 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.769+0000 7f880ffff640 1 -- 172.21.15.112:0/2894941414 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88000054d0 con 0x7f88181094b0 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.769+0000 7f881ce35640 1 -- 172.21.15.112:0/2894941414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88181a0770 con 0x7f88181094b0 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.769+0000 7f880ffff640 1 -- 172.21.15.112:0/2894941414 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800010430 con 0x7f88181094b0 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.769+0000 7f881ce35640 1 -- 172.21.15.112:0/2894941414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88181a6ed0 con 0x7f88181094b0 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.771+0000 7f88233a9740 1 Processor -- start 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.771+0000 7f88233a9740 1 -- start start 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.771+0000 7f880ffff640 1 -- 172.21.15.112:0/2894941414 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f88000105d0 con 0x7f88181094b0 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f88233a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e080a5070 0x562e0844c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f88233a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e0807c1b0 0x562e0844cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f88233a9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e0844d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f88233a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e083c2ea0 con 0x562e080a5070 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f88233a9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562e083b22d0 con 0x562e0807c1b0 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f88233a9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562e083b60e0 con 0x562e08084390 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f8816d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e080a5070 0x562e0844c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f880dd9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e0844d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f8816d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e080a5070 0x562e0844c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46824/0 (socket says 172.21.15.112:46824) 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f8816d9d640 1 -- 172.21.15.112:0/1236587359 learned_addr learned my addr 172.21.15.112:0/1236587359 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f881659c640 1 --2- 172.21.15.112:0/1236587359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e0807c1b0 0x562e0844cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f8816d9d640 1 -- 172.21.15.112:0/1236587359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 msgr2=0x562e0844d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f8816d9d640 1 --2- 172.21.15.112:0/1236587359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e0844d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f8816d9d640 1 -- 172.21.15.112:0/1236587359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e0807c1b0 msgr2=0x562e0844cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f8816d9d640 1 --2- 172.21.15.112:0/1236587359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e0807c1b0 0x562e0844cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f8816d9d640 1 -- 172.21.15.112:0/1236587359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e08110650 con 0x562e080a5070 2024-09-21T03:39:15.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f880dd9b640 1 --2- 172.21.15.112:0/1236587359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e0844d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f880ffff640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88040b6680 0x7f88040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.772+0000 7f8816d9d640 1 --2- 172.21.15.112:0/1236587359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e080a5070 0x562e0844c6a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8810009f90 tx=0x7f8810010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3c889d7768d92a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.773+0000 7f881659c640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88040b6680 0x7f88040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.773+0000 7f880cd99640 1 -- 172.21.15.112:0/1236587359 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810011700 con 0x562e080a5070 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.773+0000 7f880cd99640 1 -- 172.21.15.112:0/1236587359 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8810011d50 con 0x562e080a5070 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.773+0000 7f880cd99640 1 -- 172.21.15.112:0/1236587359 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810019da0 con 0x562e080a5070 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.773+0000 7f880ffff640 1 -- 172.21.15.112:0/2894941414 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f88000e7a80 con 0x7f88181094b0 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f88233a9740 1 -- 172.21.15.112:0/1236587359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e080a5070 msgr2=0x562e0844c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f881659c640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88040b6680 0x7f88040b8b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8808031440 tx=0x7f8808009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f88233a9740 1 --2- 172.21.15.112:0/1236587359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e080a5070 0x562e0844c6a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8810009f90 tx=0x7f8810010860 comp rx=0 tx=0).stop 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f88233a9740 1 -- 172.21.15.112:0/1236587359 shutdown_connections 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f88233a9740 1 --2- 172.21.15.112:0/1236587359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e0844d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f88233a9740 1 --2- 172.21.15.112:0/1236587359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e0807c1b0 0x562e0844cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f88233a9740 1 --2- 172.21.15.112:0/1236587359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e080a5070 0x562e0844c6a0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f88233a9740 1 -- 172.21.15.112:0/1236587359 >> 172.21.15.112:0/1236587359 conn(0x562e08099460 msgr2=0x562e0844a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f88233a9740 1 -- 172.21.15.112:0/1236587359 shutdown_connections 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f88233a9740 1 -- 172.21.15.112:0/1236587359 wait complete. 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f88233a9740 1 -- start start 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.774+0000 7f88233a9740 1 Processor -- start 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f88233a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e08457db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f88233a9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e084582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f88233a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e080a5070 0x562e084641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f88233a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e08213700 con 0x562e0807c1b0 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f88233a9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562e0834d160 con 0x562e080a5070 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f88233a9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562e0845ae80 con 0x562e08084390 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f88233a9740 10 client.0 _ll_register_callbacks cb 0x562e083669e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:15.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f881659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e08457db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f881659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e08457db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46834/0 (socket says 172.21.15.112:46834) 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f881659c640 1 -- 172.21.15.112:0/4030047072 learned_addr learned my addr 172.21.15.112:0/4030047072 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f880dd9b640 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e084582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f8816d9d640 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e080a5070 0x562e084641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f881659c640 1 -- 172.21.15.112:0/4030047072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 msgr2=0x562e084582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f881659c640 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e084582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f881659c640 1 -- 172.21.15.112:0/4030047072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e080a5070 msgr2=0x562e084641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f881659c640 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e080a5070 0x562e084641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f881659c640 1 -- 172.21.15.112:0/4030047072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e0837ac90 con 0x562e0807c1b0 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f881659c640 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e08457db0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8808033000 tx=0x7f8808009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f974e64aedacaaa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f87fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.775+0000 7f87fdffb640 1 -- 172.21.15.112:0/4030047072 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880804d070 con 0x562e0807c1b0 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f87fdffb640 1 -- 172.21.15.112:0/4030047072 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f8808005030 con 0x562e0807c1b0 2024-09-21T03:39:15.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f88233a9740 1 -- 172.21.15.112:0/4030047072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8810011d50 con 0x562e0807c1b0 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f87fdffb640 1 -- 172.21.15.112:0/4030047072 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8808059050 con 0x562e0807c1b0 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f88233a9740 10 client.17208 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f88233a9740 1 -- 172.21.15.112:0/4030047072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e08466bc0 con 0x562e0807c1b0 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f87fdffb640 1 -- 172.21.15.112:0/4030047072 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8810011d50 con 0x562e0807c1b0 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f87fdffb640 1 -- 172.21.15.112:0/4030047072 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f88080035f0 con 0x562e0807c1b0 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f88233a9740 10 client.17208 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f88233a9740 10 client.17208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f88233a9740 20 client.17208 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f88233a9740 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562e080639f0 0x562e0846a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f88233a9740 4 client.17208 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f88233a9740 1 -- 172.21.15.112:0/4030047072 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e083c4df0 con 0x562e080639f0 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.776+0000 7f880dd9b640 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562e080639f0 0x562e0846a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.777+0000 7f880dd9b640 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562e080639f0 0x562e0846a090 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=963c363f3d2c9281 server_cookie=6691e24a4cf8f8d8 in_seq=0 out_seq=0 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.777+0000 7f87fdffb640 10 client.17208 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f87fdffb640 1 -- 172.21.15.112:0/4030047072 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562e083c4df0 con 0x562e080639f0 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f87fdffb640 10 client.17208 handle_command_reply: tid=0 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 1 client.17208 shutdown 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 1 -- 172.21.15.112:0/4030047072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 msgr2=0x562e08457db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e08457db0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8808033000 tx=0x7f8808009260 comp rx=0 tx=0).stop 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 1 -- 172.21.15.112:0/4030047072 shutdown_connections 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562e080639f0 0x562e0846a090 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e080a5070 0x562e084641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e084582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 1 --2- 172.21.15.112:0/4030047072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e08457db0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f87fdffb640 0 client.17208 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 1 -- 172.21.15.112:0/4030047072 >> 172.21.15.112:0/4030047072 conn(0x562e08099460 msgr2=0x562e0844f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 1 -- 172.21.15.112:0/4030047072 shutdown_connections 2024-09-21T03:39:15.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 1 -- 172.21.15.112:0/4030047072 wait complete. 2024-09-21T03:39:15.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.778+0000 7f88233a9740 20 client.17208 trim_cache size 0 max 16384 2024-09-21T03:39:15.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.793+0000 7f88233a9740 1 Processor -- start 2024-09-21T03:39:15.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.793+0000 7f88233a9740 1 -- start start 2024-09-21T03:39:15.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f88233a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e0844dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f88233a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e08084390 0x562e084640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f88233a9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e080a5070 0x562e08454fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f88233a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88181b1f10 con 0x562e0807c1b0 2024-09-21T03:39:15.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f88233a9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88181af7e0 con 0x562e08084390 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f88233a9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88181ad650 con 0x562e080a5070 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f881659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e0844dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f881659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e0844dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46840/0 (socket says 172.21.15.112:46840) 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f881659c640 1 -- 172.21.15.112:0/2742957203 learned_addr learned my addr 172.21.15.112:0/2742957203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f8816d9d640 1 --2- 172.21.15.112:0/2742957203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e080a5070 0x562e08454fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f880dd9b640 1 --2- 172.21.15.112:0/2742957203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e08084390 0x562e084640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f881659c640 1 -- 172.21.15.112:0/2742957203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e080a5070 msgr2=0x562e08454fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f881659c640 1 --2- 172.21.15.112:0/2742957203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e080a5070 0x562e08454fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f881659c640 1 -- 172.21.15.112:0/2742957203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e08084390 msgr2=0x562e084640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f881659c640 1 --2- 172.21.15.112:0/2742957203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e08084390 0x562e084640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.794+0000 7f881659c640 1 -- 172.21.15.112:0/2742957203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e08459070 con 0x562e0807c1b0 2024-09-21T03:39:15.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.795+0000 7f881659c640 1 --2- 172.21.15.112:0/2742957203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e0844dde0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f880802a930 tx=0x7f88080044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fbab49af66c7f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.795+0000 7f87fffff640 1 -- 172.21.15.112:0/2742957203 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880804d070 con 0x562e0807c1b0 2024-09-21T03:39:15.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.795+0000 7f87fffff640 1 -- 172.21.15.112:0/2742957203 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88080054e0 con 0x562e0807c1b0 2024-09-21T03:39:15.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.795+0000 7f87fffff640 1 -- 172.21.15.112:0/2742957203 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808035040 con 0x562e0807c1b0 2024-09-21T03:39:15.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.796+0000 7f88233a9740 1 -- 172.21.15.112:0/2742957203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 msgr2=0x562e0844dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.796+0000 7f88233a9740 1 --2- 172.21.15.112:0/2742957203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e0844dde0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f880802a930 tx=0x7f88080044c0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.796+0000 7f88233a9740 1 -- 172.21.15.112:0/2742957203 shutdown_connections 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.796+0000 7f88233a9740 1 --2- 172.21.15.112:0/2742957203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e080a5070 0x562e08454fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.796+0000 7f88233a9740 1 --2- 172.21.15.112:0/2742957203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e08084390 0x562e084640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.796+0000 7f88233a9740 1 --2- 172.21.15.112:0/2742957203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e0844dde0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.796+0000 7f88233a9740 1 -- 172.21.15.112:0/2742957203 >> 172.21.15.112:0/2742957203 conn(0x562e08099460 msgr2=0x562e0844d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.796+0000 7f88233a9740 1 -- 172.21.15.112:0/2742957203 shutdown_connections 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.796+0000 7f88233a9740 1 -- 172.21.15.112:0/2742957203 wait complete. 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.796+0000 7f88233a9740 1 -- start start 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.796+0000 7f88233a9740 1 Processor -- start 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f88233a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e0845c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f88233a9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e0845c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f88233a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e080a5070 0x562e084d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f88233a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88181adca0 con 0x562e0807c1b0 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f88233a9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88181af110 con 0x562e080a5070 2024-09-21T03:39:15.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f88233a9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88181ae370 con 0x562e08084390 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f88233a9740 10 client.0 _ll_register_callbacks cb 0x7f8808032f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f880dd9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e0845c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f880dd9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e0845c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:37394/0 (socket says 172.21.15.112:37394) 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f880dd9b640 1 -- 172.21.15.112:0/1683238347 learned_addr learned my addr 172.21.15.112:0/1683238347 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f881659c640 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e0845c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f8816d9d640 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e080a5070 0x562e084d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f880dd9b640 1 -- 172.21.15.112:0/1683238347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e080a5070 msgr2=0x562e084d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f880dd9b640 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e080a5070 0x562e084d3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f880dd9b640 1 -- 172.21.15.112:0/1683238347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 msgr2=0x562e0845c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f880dd9b640 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e0845c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.797+0000 7f880dd9b640 1 -- 172.21.15.112:0/1683238347 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e084d3fd0 con 0x562e08084390 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f880dd9b640 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e0845c9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f88000ad600 tx=0x7f88000ad790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=895f62c2051e2a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:15.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f87fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:39:15.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f87fdffb640 1 -- 172.21.15.112:0/1683238347 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88000ef040 con 0x562e08084390 2024-09-21T03:39:15.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f87fdffb640 1 -- 172.21.15.112:0/1683238347 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f88000a9b70 con 0x562e08084390 2024-09-21T03:39:15.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f88233a9740 1 -- 172.21.15.112:0/1683238347 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88080054e0 con 0x562e08084390 2024-09-21T03:39:15.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f87fdffb640 1 -- 172.21.15.112:0/1683238347 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88000ad070 con 0x562e08084390 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f88233a9740 10 client.26773 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f88233a9740 1 -- 172.21.15.112:0/1683238347 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e084d5120 con 0x562e08084390 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f87fdffb640 1 -- 172.21.15.112:0/1683238347 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88080054e0 con 0x562e08084390 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f87fdffb640 1 -- 172.21.15.112:0/1683238347 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8800005000 con 0x562e08084390 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f88233a9740 10 client.26773 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.798+0000 7f88233a9740 10 client.26773 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.799+0000 7f88233a9740 20 client.26773 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.799+0000 7f88233a9740 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562e08038da0 0x562e084d7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.799+0000 7f88233a9740 4 client.26773 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.799+0000 7f88233a9740 1 -- 172.21.15.112:0/1683238347 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e083c08a0 con 0x562e08038da0 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.799+0000 7f881659c640 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562e08038da0 0x562e084d7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.799+0000 7f881659c640 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562e08038da0 0x562e084d7890 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e1b3df6249673e2 server_cookie=b7e451a07dc783b6 in_seq=0 out_seq=0 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.799+0000 7f87fdffb640 10 client.26773 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:39:15.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.799+0000 7f87fdffb640 1 -- 172.21.15.112:0/1683238347 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e083c08a0 con 0x562e08038da0 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.799+0000 7f87fdffb640 10 client.26773 handle_command_reply: tid=0 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.799+0000 7f88233a9740 1 client.26773 shutdown 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f88233a9740 1 -- 172.21.15.112:0/1683238347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 msgr2=0x562e0845c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f88233a9740 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e0845c9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f88000ad600 tx=0x7f88000ad790 comp rx=0 tx=0).stop 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f88233a9740 1 -- 172.21.15.112:0/1683238347 shutdown_connections 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f88233a9740 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562e08038da0 0x562e084d7890 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f88233a9740 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e080a5070 0x562e084d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f87fdffb640 0 client.26773 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f88233a9740 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e08084390 0x562e0845c9c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f88233a9740 1 --2- 172.21.15.112:0/1683238347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e0807c1b0 0x562e0845c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f88233a9740 1 -- 172.21.15.112:0/1683238347 >> 172.21.15.112:0/1683238347 conn(0x562e08099460 msgr2=0x562e0844d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f88233a9740 1 -- 172.21.15.112:0/1683238347 shutdown_connections 2024-09-21T03:39:15.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f88233a9740 1 -- 172.21.15.112:0/1683238347 wait complete. 2024-09-21T03:39:15.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.800+0000 7f88233a9740 20 client.26773 trim_cache size 0 max 16384 2024-09-21T03:39:15.806 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:39:15.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 -- 172.21.15.112:0/2894941414 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88040b6680 msgr2=0x7f88040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88040b6680 0x7f88040b8b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8808031440 tx=0x7f8808009560 comp rx=0 tx=0).stop 2024-09-21T03:39:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 -- 172.21.15.112:0/2894941414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88181094b0 msgr2=0x7f881819aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88181094b0 0x7f881819aab0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f880000e9c0 tx=0x7f880000ee80 comp rx=0 tx=0).stop 2024-09-21T03:39:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 -- 172.21.15.112:0/2894941414 shutdown_connections 2024-09-21T03:39:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f88040b6680 0x7f88040b8b90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8818109e00 0x7f881819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88181094b0 0x7f881819aab0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 --2- 172.21.15.112:0/2894941414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8818101f70 0x7f881819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 -- 172.21.15.112:0/2894941414 >> 172.21.15.112:0/2894941414 conn(0x7f8818077c60 msgr2=0x7f8818107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 -- 172.21.15.112:0/2894941414 shutdown_connections 2024-09-21T03:39:15.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:15.801+0000 7f881ce35640 1 -- 172.21.15.112:0/2894941414 wait complete. 2024-09-21T03:39:16.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1087237 10.45 MB/sec execute 436 sec latency 20.005 ms 2024-09-21T03:39:16.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:16 smithi112 ceph-mon[31442]: pgmap v635: 97 pgs: 97 active+clean; 814 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 264 op/s 2024-09-21T03:39:16.399 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:16 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:16.399 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:16 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:16.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:16 smithi203 ceph-mon[34723]: pgmap v635: 97 pgs: 97 active+clean; 814 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 264 op/s 2024-09-21T03:39:16.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:16 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:16.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:16 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:16.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:16 smithi187 ceph-mon[34603]: pgmap v635: 97 pgs: 97 active+clean; 814 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 264 op/s 2024-09-21T03:39:16.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:16 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:16.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:16 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:16.581 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:39:16.847 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:17.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1088414 10.43 MB/sec execute 437 sec latency 20.126 ms 2024-09-21T03:39:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.500+0000 7f7420f46640 1 -- 172.21.15.112:0/1578627887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c103b90 msgr2=0x7f741c103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.500+0000 7f7420f46640 1 --2- 172.21.15.112:0/1578627887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c103b90 0x7f741c103fa0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f740c009900 tx=0x7f740c02a410 comp rx=0 tx=0).stop 2024-09-21T03:39:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.500+0000 7f7420f46640 1 -- 172.21.15.112:0/1578627887 shutdown_connections 2024-09-21T03:39:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.500+0000 7f7420f46640 1 --2- 172.21.15.112:0/1578627887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f741c109e20 0x7f741c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.500+0000 7f7420f46640 1 --2- 172.21.15.112:0/1578627887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c1094d0 0x7f741c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.500+0000 7f7420f46640 1 --2- 172.21.15.112:0/1578627887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c103b90 0x7f741c103fa0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.500+0000 7f7420f46640 1 -- 172.21.15.112:0/1578627887 >> 172.21.15.112:0/1578627887 conn(0x7f741c0ff600 msgr2=0x7f741c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.500+0000 7f7420f46640 1 -- 172.21.15.112:0/1578627887 shutdown_connections 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.500+0000 7f7420f46640 1 -- 172.21.15.112:0/1578627887 wait complete. 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f7420f46640 1 Processor -- start 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f7420f46640 1 -- start start 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f7420f46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c103b90 0x7f741c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f7420f46640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f741c1094d0 0x7f741c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f7420f46640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c109e20 0x7f741c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f7420f46640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f741c113ae0 con 0x7f741c103b90 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f7420f46640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f741c113960 con 0x7f741c109e20 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f7420f46640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f741c113c60 con 0x7f741c1094d0 2024-09-21T03:39:17.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f741a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c103b90 0x7f741c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f741a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c103b90 0x7f741c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46854/0 (socket says 172.21.15.112:46854) 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f741a59c640 1 -- 172.21.15.112:0/4018029874 learned_addr learned my addr 172.21.15.112:0/4018029874 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f7413fff640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f741c1094d0 0x7f741c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f741ad9d640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c109e20 0x7f741c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f741ad9d640 1 -- 172.21.15.112:0/4018029874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f741c1094d0 msgr2=0x7f741c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f741ad9d640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f741c1094d0 0x7f741c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f741ad9d640 1 -- 172.21.15.112:0/4018029874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c103b90 msgr2=0x7f741c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f741ad9d640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c103b90 0x7f741c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.501+0000 7f741ad9d640 1 -- 172.21.15.112:0/4018029874 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741c1a03e0 con 0x7f741c109e20 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.502+0000 7f7413fff640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f741c1094d0 0x7f741c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.502+0000 7f741a59c640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c103b90 0x7f741c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.502+0000 7f741ad9d640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c109e20 0x7f741c19fcb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7414009fd0 tx=0x7f741400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.502+0000 7f74137fe640 1 -- 172.21.15.112:0/4018029874 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741400ce20 con 0x7f741c109e20 2024-09-21T03:39:17.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.502+0000 7f7420f46640 1 -- 172.21.15.112:0/4018029874 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f741c1a06d0 con 0x7f741c109e20 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.502+0000 7f74137fe640 1 -- 172.21.15.112:0/4018029874 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7414005020 con 0x7f741c109e20 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.502+0000 7f74137fe640 1 -- 172.21.15.112:0/4018029874 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414018670 con 0x7f741c109e20 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.502+0000 7f7420f46640 1 -- 172.21.15.112:0/4018029874 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f741c1a6e30 con 0x7f741c109e20 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.504+0000 7f74274ba740 1 Processor -- start 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f74137fe640 1 -- 172.21.15.112:0/4018029874 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f74140054d0 con 0x7f741c109e20 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f74274ba740 1 -- start start 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f74274ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1c6070 0x563c1b56d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f74274ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b56dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f74274ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1a5390 0x563c1b56e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f74274ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563c1b4e3ea0 con 0x563c1b1a5390 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f74274ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563c1b4d32d0 con 0x563c1b19d1b0 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f74274ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563c1b4d70e0 con 0x563c1b1c6070 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f7413fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1c6070 0x563c1b56d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f741a59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b56dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f741a59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b56dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41034/0 (socket says 172.21.15.112:41034) 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f7413fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1c6070 0x563c1b56d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:37426/0 (socket says 172.21.15.112:37426) 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.505+0000 7f741a59c640 1 -- 172.21.15.112:0/2994070238 learned_addr learned my addr 172.21.15.112:0/2994070238 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f7413fff640 1 --2- 172.21.15.112:0/2994070238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1a5390 0x563c1b56e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f74137fe640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74080b6680 0x7f74080b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f741a59c640 1 -- 172.21.15.112:0/2994070238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1c6070 msgr2=0x563c1b56d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f741a59c640 1 --2- 172.21.15.112:0/2994070238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1c6070 0x563c1b56d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f741a59c640 1 -- 172.21.15.112:0/2994070238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1a5390 msgr2=0x563c1b56e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f741a59c640 1 --2- 172.21.15.112:0/2994070238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1a5390 0x563c1b56e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f741a59c640 1 -- 172.21.15.112:0/2994070238 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1b231650 con 0x563c1b19d1b0 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f741ad9d640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74080b6680 0x7f74080b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f741a59c640 1 --2- 172.21.15.112:0/2994070238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b56dbe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f740c000c00 tx=0x7f740c005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=405daa4a893a6d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f74137fe640 1 -- 172.21.15.112:0/4018029874 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f74140e7d50 con 0x7f741c109e20 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f7410ff9640 1 -- 172.21.15.112:0/2994070238 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c035040 con 0x563c1b19d1b0 2024-09-21T03:39:17.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f7410ff9640 1 -- 172.21.15.112:0/2994070238 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f740c0317a0 con 0x563c1b19d1b0 2024-09-21T03:39:17.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.506+0000 7f7410ff9640 1 -- 172.21.15.112:0/2994070238 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c0049c0 con 0x563c1b19d1b0 2024-09-21T03:39:17.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.507+0000 7f74274ba740 1 -- 172.21.15.112:0/2994070238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 msgr2=0x563c1b56dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.507+0000 7f741ad9d640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74080b6680 0x7f74080b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f74140aade0 tx=0x7f74140aa260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:17.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.507+0000 7f74274ba740 1 --2- 172.21.15.112:0/2994070238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b56dbe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f740c000c00 tx=0x7f740c005ed0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.507+0000 7f74274ba740 1 -- 172.21.15.112:0/2994070238 shutdown_connections 2024-09-21T03:39:17.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.507+0000 7f74274ba740 1 --2- 172.21.15.112:0/2994070238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1a5390 0x563c1b56e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.507+0000 7f74274ba740 1 --2- 172.21.15.112:0/2994070238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b56dbe0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.507+0000 7f74274ba740 1 --2- 172.21.15.112:0/2994070238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1c6070 0x563c1b56d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.507+0000 7f74274ba740 1 -- 172.21.15.112:0/2994070238 >> 172.21.15.112:0/2994070238 conn(0x563c1b1ba460 msgr2=0x563c1b56b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.507+0000 7f74274ba740 1 -- 172.21.15.112:0/2994070238 shutdown_connections 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.507+0000 7f74274ba740 1 -- 172.21.15.112:0/2994070238 wait complete. 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f74274ba740 1 -- start start 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f74274ba740 1 Processor -- start 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f74274ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b578ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f74274ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b5791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f74274ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1c6070 0x563c1b579720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f74274ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563c1b334700 con 0x563c1b1c6070 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f74274ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563c1b46e160 con 0x563c1b19d1b0 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f74274ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563c1b57be20 con 0x563c1b1a5390 2024-09-21T03:39:17.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f74274ba740 10 client.0 _ll_register_callbacks cb 0x563c1b4879e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f7413fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b578ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f7413fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b578ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41040/0 (socket says 172.21.15.112:41040) 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f7413fff640 1 -- 172.21.15.112:0/3370073074 learned_addr learned my addr 172.21.15.112:0/3370073074 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f7413fff640 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1c6070 0x563c1b579720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.508+0000 7f741a59c640 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b5791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f7413fff640 1 -- 172.21.15.112:0/3370073074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 msgr2=0x563c1b5791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f7413fff640 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b5791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f7413fff640 1 -- 172.21.15.112:0/3370073074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1c6070 msgr2=0x563c1b579720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f7413fff640 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1c6070 0x563c1b579720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f7413fff640 1 -- 172.21.15.112:0/3370073074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1b49bc90 con 0x563c1b19d1b0 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f7413fff640 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b578ca0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f740400f750 tx=0x7f7404004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6991100ebf27c438 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f7401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f7401ffb640 1 -- 172.21.15.112:0/3370073074 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404011070 con 0x563c1b19d1b0 2024-09-21T03:39:17.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f7401ffb640 1 -- 172.21.15.112:0/3370073074 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f7404011210 con 0x563c1b19d1b0 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f74274ba740 1 -- 172.21.15.112:0/3370073074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f740c0317a0 con 0x563c1b19d1b0 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f7401ffb640 1 -- 172.21.15.112:0/3370073074 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7404004020 con 0x563c1b19d1b0 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f74274ba740 10 client.26783 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f74274ba740 1 -- 172.21.15.112:0/3370073074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c1b587b00 con 0x563c1b19d1b0 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.509+0000 7f7401ffb640 1 -- 172.21.15.112:0/3370073074 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f740c0317a0 con 0x563c1b19d1b0 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.510+0000 7f7401ffb640 1 -- 172.21.15.112:0/3370073074 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f740401d020 con 0x563c1b19d1b0 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.510+0000 7f74274ba740 10 client.26783 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.510+0000 7f74274ba740 10 client.26783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.510+0000 7f74274ba740 20 client.26783 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.510+0000 7f74274ba740 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563c1b1849f0 0x563c1b58b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.510+0000 7f74274ba740 4 client.26783 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.510+0000 7f74274ba740 1 -- 172.21.15.112:0/3370073074 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c1b4e5df0 con 0x563c1b1849f0 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.510+0000 7f741a59c640 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563c1b1849f0 0x563c1b58b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.510+0000 7f741a59c640 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563c1b1849f0 0x563c1b58b060 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d2de64d4c2b5c8 server_cookie=4c4431a1711d5203 in_seq=0 out_seq=0 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.510+0000 7f7401ffb640 10 client.26783 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.511+0000 7f7401ffb640 1 -- 172.21.15.112:0/3370073074 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c1b4e5df0 con 0x563c1b1849f0 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.511+0000 7f7401ffb640 10 client.26783 handle_command_reply: tid=0 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.511+0000 7f74274ba740 1 client.26783 shutdown 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f74274ba740 1 -- 172.21.15.112:0/3370073074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 msgr2=0x563c1b578ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f74274ba740 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b578ca0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f740400f750 tx=0x7f7404004730 comp rx=0 tx=0).stop 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f74274ba740 1 -- 172.21.15.112:0/3370073074 shutdown_connections 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f74274ba740 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563c1b1849f0 0x563c1b58b060 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f74274ba740 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1c6070 0x563c1b579720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f74274ba740 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b5791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f74274ba740 1 --2- 172.21.15.112:0/3370073074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b578ca0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f7401ffb640 0 client.26783 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f74274ba740 1 -- 172.21.15.112:0/3370073074 >> 172.21.15.112:0/3370073074 conn(0x563c1b1ba460 msgr2=0x563c1b570830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:17.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f74274ba740 1 -- 172.21.15.112:0/3370073074 shutdown_connections 2024-09-21T03:39:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f74274ba740 1 -- 172.21.15.112:0/3370073074 wait complete. 2024-09-21T03:39:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.512+0000 7f74274ba740 20 client.26783 trim_cache size 0 max 16384 2024-09-21T03:39:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f74274ba740 1 Processor -- start 2024-09-21T03:39:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f74274ba740 1 -- start start 2024-09-21T03:39:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f74274ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b56f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f74274ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b57a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f74274ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1c6070 0x563c1b575e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f74274ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f741c1b1f20 con 0x563c1b1c6070 2024-09-21T03:39:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f74274ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f741c1af7f0 con 0x563c1b19d1b0 2024-09-21T03:39:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f74274ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f741c1ad660 con 0x563c1b1a5390 2024-09-21T03:39:17.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f7413fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1c6070 0x563c1b575e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f7413fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b56f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f741a59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b57a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f7413fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b56f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41056/0 (socket says 172.21.15.112:41056) 2024-09-21T03:39:17.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f7413fff640 1 -- 172.21.15.112:0/1264690731 learned_addr learned my addr 172.21.15.112:0/1264690731 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:17.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f741a59c640 1 -- 172.21.15.112:0/1264690731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 msgr2=0x563c1b56f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:17.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f741a59c640 1 --2- 172.21.15.112:0/1264690731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b56f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f741a59c640 1 -- 172.21.15.112:0/1264690731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1c6070 msgr2=0x563c1b575e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:17.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f741a59c640 1 --2- 172.21.15.112:0/1264690731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1c6070 0x563c1b575e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.532+0000 7f741a59c640 1 -- 172.21.15.112:0/1264690731 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1b570500 con 0x563c1b1a5390 2024-09-21T03:39:17.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.533+0000 7f741a59c640 1 --2- 172.21.15.112:0/1264690731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b57a8d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f740c005410 tx=0x7f740c005510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0f5c753cc0e0aee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.533+0000 7f7403fff640 1 -- 172.21.15.112:0/1264690731 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c035040 con 0x563c1b1a5390 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.533+0000 7f7403fff640 1 -- 172.21.15.112:0/1264690731 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f740c0319d0 con 0x563c1b1a5390 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.533+0000 7f7403fff640 1 -- 172.21.15.112:0/1264690731 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c0035f0 con 0x563c1b1a5390 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.534+0000 7f74274ba740 1 -- 172.21.15.112:0/1264690731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 msgr2=0x563c1b57a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.534+0000 7f74274ba740 1 --2- 172.21.15.112:0/1264690731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b57a8d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f740c005410 tx=0x7f740c005510 comp rx=0 tx=0).stop 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.534+0000 7f74274ba740 1 -- 172.21.15.112:0/1264690731 shutdown_connections 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.534+0000 7f74274ba740 1 --2- 172.21.15.112:0/1264690731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b1c6070 0x563c1b575e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.534+0000 7f74274ba740 1 --2- 172.21.15.112:0/1264690731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b57a8d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.534+0000 7f74274ba740 1 --2- 172.21.15.112:0/1264690731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b19d1b0 0x563c1b56f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.534+0000 7f74274ba740 1 -- 172.21.15.112:0/1264690731 >> 172.21.15.112:0/1264690731 conn(0x563c1b1ba460 msgr2=0x563c1b57a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.534+0000 7f74274ba740 1 -- 172.21.15.112:0/1264690731 shutdown_connections 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.534+0000 7f74274ba740 1 -- 172.21.15.112:0/1264690731 wait complete. 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.534+0000 7f74274ba740 1 -- start start 2024-09-21T03:39:17.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f74274ba740 1 Processor -- start 2024-09-21T03:39:17.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f74274ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b19d1b0 0x563c1b57d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f74274ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b57d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f74274ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b1c6070 0x563c1b5f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f74274ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f741c1adcb0 con 0x563c1b19d1b0 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f74274ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f741c1af120 con 0x563c1b1c6070 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f74274ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f741c1ae380 con 0x563c1b1a5390 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f74274ba740 10 client.0 _ll_register_callbacks cb 0x7f74040118b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f7413fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b19d1b0 0x563c1b57d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f7413fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b19d1b0 0x563c1b57d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46886/0 (socket says 172.21.15.112:46886) 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f7413fff640 1 -- 172.21.15.112:0/3738114410 learned_addr learned my addr 172.21.15.112:0/3738114410 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f741a59c640 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b57d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f7413fff640 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b1c6070 0x563c1b5f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f7413fff640 1 -- 172.21.15.112:0/3738114410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 msgr2=0x563c1b57d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f7413fff640 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b57d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f7413fff640 1 -- 172.21.15.112:0/3738114410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b1c6070 msgr2=0x563c1b5f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f7413fff640 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b1c6070 0x563c1b5f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f7413fff640 1 -- 172.21.15.112:0/3738114410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1b5f9370 con 0x563c1b19d1b0 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f741a59c640 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b57d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.535+0000 7f7413fff640 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b19d1b0 0x563c1b57d460 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f740400ee30 tx=0x7f7404009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81b521152e7c9a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f7401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f7401ffb640 1 -- 172.21.15.112:0/3738114410 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740402f040 con 0x563c1b19d1b0 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f7401ffb640 1 -- 172.21.15.112:0/3738114410 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f740400e720 con 0x563c1b19d1b0 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f74274ba740 1 -- 172.21.15.112:0/3738114410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f740c0319d0 con 0x563c1b19d1b0 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f7401ffb640 1 -- 172.21.15.112:0/3738114410 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f740400e920 con 0x563c1b19d1b0 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f74274ba740 10 client.17232 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f74274ba740 1 -- 172.21.15.112:0/3738114410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c1b5fa470 con 0x563c1b19d1b0 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f7401ffb640 1 -- 172.21.15.112:0/3738114410 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f740c0319d0 con 0x563c1b19d1b0 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f7401ffb640 1 -- 172.21.15.112:0/3738114410 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f740401d070 con 0x563c1b19d1b0 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f74274ba740 10 client.17232 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f74274ba740 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:39:17.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f74274ba740 20 client.17232 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f74274ba740 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563c1b159da0 0x563c1b5fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f74274ba740 4 client.17232 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.536+0000 7f74274ba740 1 -- 172.21.15.112:0/3738114410 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c1b4e18a0 con 0x563c1b159da0 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f741a59c640 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563c1b159da0 0x563c1b5fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f741a59c640 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563c1b159da0 0x563c1b5fcc60 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed0fa1938636f470 server_cookie=3ca9034d058c1cc7 in_seq=0 out_seq=0 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f7401ffb640 10 client.17232 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f7401ffb640 1 -- 172.21.15.112:0/3738114410 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c1b4e18a0 con 0x563c1b159da0 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f7401ffb640 10 client.17232 handle_command_reply: tid=0 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f74274ba740 1 client.17232 shutdown 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f74274ba740 1 -- 172.21.15.112:0/3738114410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b19d1b0 msgr2=0x563c1b57d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f74274ba740 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b19d1b0 0x563c1b57d460 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f740400ee30 tx=0x7f7404009eb0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f74274ba740 1 -- 172.21.15.112:0/3738114410 shutdown_connections 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f74274ba740 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x563c1b159da0 0x563c1b5fcc60 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f74274ba740 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c1b1c6070 0x563c1b5f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f74274ba740 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563c1b1a5390 0x563c1b57d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f74274ba740 1 --2- 172.21.15.112:0/3738114410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c1b19d1b0 0x563c1b57d460 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f74274ba740 1 -- 172.21.15.112:0/3738114410 >> 172.21.15.112:0/3738114410 conn(0x563c1b1ba460 msgr2=0x563c1b56ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:17.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f7401ffb640 0 client.17232 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:17.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f74274ba740 1 -- 172.21.15.112:0/3738114410 shutdown_connections 2024-09-21T03:39:17.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.537+0000 7f74274ba740 1 -- 172.21.15.112:0/3738114410 wait complete. 2024-09-21T03:39:17.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f74274ba740 20 client.17232 trim_cache size 0 max 16384 2024-09-21T03:39:17.544 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:39:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 -- 172.21.15.112:0/4018029874 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74080b6680 msgr2=0x7f74080b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74080b6680 0x7f74080b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f74140aade0 tx=0x7f74140aa260 comp rx=0 tx=0).stop 2024-09-21T03:39:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 -- 172.21.15.112:0/4018029874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c109e20 msgr2=0x7f741c19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c109e20 0x7f741c19fcb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7414009fd0 tx=0x7f741400ef10 comp rx=0 tx=0).stop 2024-09-21T03:39:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 -- 172.21.15.112:0/4018029874 shutdown_connections 2024-09-21T03:39:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74080b6680 0x7f74080b8b90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f741c109e20 0x7f741c19fcb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f741c1094d0 0x7f741c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 --2- 172.21.15.112:0/4018029874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f741c103b90 0x7f741c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 -- 172.21.15.112:0/4018029874 >> 172.21.15.112:0/4018029874 conn(0x7f741c0ff600 msgr2=0x7f741c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 -- 172.21.15.112:0/4018029874 shutdown_connections 2024-09-21T03:39:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:17.538+0000 7f7420f46640 1 -- 172.21.15.112:0/4018029874 wait complete. 2024-09-21T03:39:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:17 smithi203 ceph-mon[34723]: pgmap v636: 97 pgs: 97 active+clean; 824 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 225 op/s 2024-09-21T03:39:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:17 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:17 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:17 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:17 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:17 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:17 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:17 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:17 smithi187 ceph-mon[34603]: pgmap v636: 97 pgs: 97 active+clean; 824 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 225 op/s 2024-09-21T03:39:18.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:17 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:17 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:17 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:17 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:17 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:18.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:17 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:18.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:17 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1090719 10.44 MB/sec execute 438 sec latency 16.821 ms 2024-09-21T03:39:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:17 smithi112 ceph-mon[31442]: pgmap v636: 97 pgs: 97 active+clean; 824 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 225 op/s 2024-09-21T03:39:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:17 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:17 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:17 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:17 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:17 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:39:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:17 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:39:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:17 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:39:19.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1091945 10.43 MB/sec execute 439 sec latency 51.324 ms 2024-09-21T03:39:19.386 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:39:19.386 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:39:19.649 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:20.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:19 smithi112 ceph-mon[31442]: pgmap v637: 97 pgs: 97 active+clean; 824 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 224 op/s 2024-09-21T03:39:20.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.143+0000 7f75693d8640 1 -- 172.21.15.112:0/3722327732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75641083a0 msgr2=0x7f75641087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.143+0000 7f75693d8640 1 --2- 172.21.15.112:0/3722327732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75641083a0 0x7f75641087b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7558009900 tx=0x7f755802a410 comp rx=0 tx=0).stop 2024-09-21T03:39:20.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.143+0000 7f75693d8640 1 -- 172.21.15.112:0/3722327732 shutdown_connections 2024-09-21T03:39:20.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.143+0000 7f75693d8640 1 --2- 172.21.15.112:0/3722327732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7564108cf0 0x7f756410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.143+0000 7f75693d8640 1 --2- 172.21.15.112:0/3722327732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75641083a0 0x7f75641087b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.143+0000 7f75693d8640 1 --2- 172.21.15.112:0/3722327732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f7564102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.143+0000 7f75693d8640 1 -- 172.21.15.112:0/3722327732 >> 172.21.15.112:0/3722327732 conn(0x7f7564079730 msgr2=0x7f7564079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:20.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.143+0000 7f75693d8640 1 -- 172.21.15.112:0/3722327732 shutdown_connections 2024-09-21T03:39:20.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.143+0000 7f75693d8640 1 -- 172.21.15.112:0/3722327732 wait complete. 2024-09-21T03:39:20.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75693d8640 1 Processor -- start 2024-09-21T03:39:20.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75693d8640 1 -- start start 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75693d8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f756419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75693d8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75641083a0 0x7f756419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75693d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564108cf0 0x7f756419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75693d8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7564113a60 con 0x7f7564108cf0 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75693d8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75641138e0 con 0x7f7564102a60 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75693d8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7564113be0 con 0x7f75641083a0 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75627fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75641083a0 0x7f756419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564108cf0 0x7f756419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f756419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75627fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75641083a0 0x7f756419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:37478/0 (socket says 172.21.15.112:37478) 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75627fc640 1 -- 172.21.15.112:0/2022304156 learned_addr learned my addr 172.21.15.112:0/2022304156 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.144+0000 7f75637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564108cf0 0x7f756419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46902/0 (socket says 172.21.15.112:46902) 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f75627fc640 1 -- 172.21.15.112:0/2022304156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 msgr2=0x7f756419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f75627fc640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f756419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f75627fc640 1 -- 172.21.15.112:0/2022304156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564108cf0 msgr2=0x7f756419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f75627fc640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564108cf0 0x7f756419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f75627fc640 1 -- 172.21.15.112:0/2022304156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75641a03c0 con 0x7f75641083a0 2024-09-21T03:39:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f75637fe640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564108cf0 0x7f756419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f7562ffd640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f756419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f75627fc640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75641083a0 0x7f756419a9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7558032fd0 tx=0x7f7558031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f754bfff640 1 -- 172.21.15.112:0/2022304156 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755803f030 con 0x7f75641083a0 2024-09-21T03:39:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f75693d8640 1 -- 172.21.15.112:0/2022304156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75641a0650 con 0x7f75641083a0 2024-09-21T03:39:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f754bfff640 1 -- 172.21.15.112:0/2022304156 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7558004490 con 0x7f75641083a0 2024-09-21T03:39:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f754bfff640 1 -- 172.21.15.112:0/2022304156 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558035070 con 0x7f75641083a0 2024-09-21T03:39:20.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.145+0000 7f75693d8640 1 -- 172.21.15.112:0/2022304156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75641a6e10 con 0x7f75641083a0 2024-09-21T03:39:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.148+0000 7f756f94c740 1 Processor -- start 2024-09-21T03:39:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.148+0000 7f756f94c740 1 -- start start 2024-09-21T03:39:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.148+0000 7f754bfff640 1 -- 172.21.15.112:0/2022304156 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7558042030 con 0x7f75641083a0 2024-09-21T03:39:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.148+0000 7f756f94c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0e6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.148+0000 7f756f94c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d16490 0x55a52a0e6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.148+0000 7f756f94c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 0x55a52a0e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.148+0000 7f756f94c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a52a05ce60 con 0x55a529d1e670 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.148+0000 7f756f94c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a52a04c290 con 0x55a529d3f350 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.148+0000 7f756f94c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a52a0500a0 con 0x55a529d16490 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d16490 0x55a52a0e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f75627fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 0x55a52a0e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d16490 0x55a52a0e6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:37488/0 (socket says 172.21.15.112:37488) 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f7562ffd640 1 -- 172.21.15.112:0/507489169 learned_addr learned my addr 172.21.15.112:0/507489169 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f75637fe640 1 --2- 172.21.15.112:0/507489169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f7562ffd640 1 -- 172.21.15.112:0/507489169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 msgr2=0x55a52a0e6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f7562ffd640 1 --2- 172.21.15.112:0/507489169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0e6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f7562ffd640 1 -- 172.21.15.112:0/507489169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 msgr2=0x55a52a0e6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f7562ffd640 1 --2- 172.21.15.112:0/507489169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 0x55a52a0e6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f7562ffd640 1 -- 172.21.15.112:0/507489169 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a529daa930 con 0x55a529d16490 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f754bfff640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 0x7f754c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f7562ffd640 1 --2- 172.21.15.112:0/507489169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d16490 0x55a52a0e6970 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7554011620 tx=0x7f7554011ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ac8f74b14ccc4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:20.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f75637fe640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 0x7f754c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f754bfff640 1 -- 172.21.15.112:0/2022304156 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f75581076f0 con 0x7f75641083a0 2024-09-21T03:39:20.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f75497fa640 1 -- 172.21.15.112:0/507489169 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755401e070 con 0x55a529d16490 2024-09-21T03:39:20.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f75497fa640 1 -- 172.21.15.112:0/507489169 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755400fb10 con 0x55a529d16490 2024-09-21T03:39:20.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.149+0000 7f75497fa640 1 -- 172.21.15.112:0/507489169 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554019bc0 con 0x55a529d16490 2024-09-21T03:39:20.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.150+0000 7f756f94c740 1 -- 172.21.15.112:0/507489169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d16490 msgr2=0x55a52a0e6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.150+0000 7f756f94c740 1 --2- 172.21.15.112:0/507489169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d16490 0x55a52a0e6970 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7554011620 tx=0x7f7554011ae0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.150+0000 7f75637fe640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 0x7f754c0b8b90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f755c009fd0 tx=0x7f755c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:20.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.150+0000 7f756f94c740 1 -- 172.21.15.112:0/507489169 shutdown_connections 2024-09-21T03:39:20.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.150+0000 7f756f94c740 1 --2- 172.21.15.112:0/507489169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 0x55a52a0e6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.150+0000 7f756f94c740 1 --2- 172.21.15.112:0/507489169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d16490 0x55a52a0e6970 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.150+0000 7f756f94c740 1 --2- 172.21.15.112:0/507489169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0e6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.150+0000 7f756f94c740 1 -- 172.21.15.112:0/507489169 >> 172.21.15.112:0/507489169 conn(0x55a529d33740 msgr2=0x55a52a0e4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:20.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.150+0000 7f756f94c740 1 -- 172.21.15.112:0/507489169 shutdown_connections 2024-09-21T03:39:20.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.150+0000 7f756f94c740 1 -- 172.21.15.112:0/507489169 wait complete. 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f756f94c740 1 -- start start 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f756f94c740 1 Processor -- start 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f756f94c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d16490 0x55a52a0f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f756f94c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 0x55a52a0f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f756f94c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d3f350 0x55a52a102270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f756f94c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a529ead890 con 0x55a529d1e670 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f756f94c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a529fe6d40 con 0x55a529d16490 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f756f94c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a52a0f8f40 con 0x55a529d3f350 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f756f94c740 10 client.0 _ll_register_callbacks cb 0x55a52a04ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f75627fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 0x55a52a0f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d16490 0x55a52a0f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f75627fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 0x55a52a0f68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46912/0 (socket says 172.21.15.112:46912) 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d16490 0x55a52a0f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41116/0 (socket says 172.21.15.112:41116) 2024-09-21T03:39:20.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f75627fc640 1 -- 172.21.15.112:0/1162913989 learned_addr learned my addr 172.21.15.112:0/1162913989 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.151+0000 7f75637fe640 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d3f350 0x55a52a102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f75627fc640 1 -- 172.21.15.112:0/1162913989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d3f350 msgr2=0x55a52a102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f75627fc640 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d3f350 0x55a52a102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f75627fc640 1 -- 172.21.15.112:0/1162913989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d16490 msgr2=0x55a52a0f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f75627fc640 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d16490 0x55a52a0f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f75627fc640 1 -- 172.21.15.112:0/1162913989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a52a102800 con 0x55a529d1e670 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f75637fe640 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d3f350 0x55a52a102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f7562ffd640 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d16490 0x55a52a0f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f75627fc640 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 0x55a52a0f68f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7558104ed0 tx=0x7f755803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89025ec5d130cdab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f7546ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f7546ff5640 1 -- 172.21.15.112:0/1162913989 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558049070 con 0x55a529d1e670 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f7546ff5640 1 -- 172.21.15.112:0/1162913989 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f75580d6d00 con 0x55a529d1e670 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f756f94c740 1 -- 172.21.15.112:0/1162913989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f755400fb10 con 0x55a529d1e670 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f7546ff5640 1 -- 172.21.15.112:0/1162913989 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f75580c94c0 con 0x55a529d1e670 2024-09-21T03:39:20.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f756f94c740 10 client.17244 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.152+0000 7f756f94c740 1 -- 172.21.15.112:0/1162913989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a52a105110 con 0x55a529d1e670 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.153+0000 7f7546ff5640 1 -- 172.21.15.112:0/1162913989 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755400fb10 con 0x55a529d1e670 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.153+0000 7f7546ff5640 1 -- 172.21.15.112:0/1162913989 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7558109ce0 con 0x55a529d1e670 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.153+0000 7f756f94c740 10 client.17244 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.153+0000 7f756f94c740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.153+0000 7f756f94c740 20 client.17244 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.153+0000 7f756f94c740 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a529cfdcd0 0x55a52a108560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.153+0000 7f756f94c740 4 client.17244 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.153+0000 7f756f94c740 1 -- 172.21.15.112:0/1162913989 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a52a05edb0 con 0x55a529cfdcd0 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.153+0000 7f7562ffd640 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a529cfdcd0 0x55a52a108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.154+0000 7f7562ffd640 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a529cfdcd0 0x55a52a108560 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=101651c9924e12e6 server_cookie=441403236b651d8b in_seq=0 out_seq=0 2024-09-21T03:39:20.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.154+0000 7f7546ff5640 10 client.17244 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:20.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f7546ff5640 1 -- 172.21.15.112:0/1162913989 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a52a05edb0 con 0x55a529cfdcd0 2024-09-21T03:39:20.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f7546ff5640 10 client.17244 handle_command_reply: tid=0 2024-09-21T03:39:20.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 1 client.17244 shutdown 2024-09-21T03:39:20.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 1 -- 172.21.15.112:0/1162913989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 msgr2=0x55a52a0f68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 0x55a52a0f68f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7558104ed0 tx=0x7f755803b000 comp rx=0 tx=0).stop 2024-09-21T03:39:20.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 1 -- 172.21.15.112:0/1162913989 shutdown_connections 2024-09-21T03:39:20.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a529cfdcd0 0x55a52a108560 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d3f350 0x55a52a102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d1e670 0x55a52a0f68f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 1 --2- 172.21.15.112:0/1162913989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d16490 0x55a52a0f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 1 -- 172.21.15.112:0/1162913989 >> 172.21.15.112:0/1162913989 conn(0x55a529d33740 msgr2=0x55a52a0e9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:20.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f7546ff5640 0 client.17244 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:20.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 1 -- 172.21.15.112:0/1162913989 shutdown_connections 2024-09-21T03:39:20.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 1 -- 172.21.15.112:0/1162913989 wait complete. 2024-09-21T03:39:20.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.155+0000 7f756f94c740 20 client.17244 trim_cache size 0 max 16384 2024-09-21T03:39:20.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:19 smithi203 ceph-mon[34723]: pgmap v637: 97 pgs: 97 active+clean; 824 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 224 op/s 2024-09-21T03:39:20.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f756f94c740 1 Processor -- start 2024-09-21T03:39:20.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f756f94c740 1 -- start start 2024-09-21T03:39:20.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f756f94c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a102170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f756f94c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d1e670 0x55a52a0f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f756f94c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f756f94c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75641afd10 con 0x55a529d16490 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f756f94c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75641ad5e0 con 0x55a529d3f350 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f756f94c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75641ab450 con 0x55a529d1e670 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f75637fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46922/0 (socket says 172.21.15.112:46922) 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f7562ffd640 1 -- 172.21.15.112:0/1437083257 learned_addr learned my addr 172.21.15.112:0/1437083257 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.173+0000 7f75627fc640 1 --2- 172.21.15.112:0/1437083257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d1e670 0x55a52a0f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.174+0000 7f7562ffd640 1 -- 172.21.15.112:0/1437083257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d1e670 msgr2=0x55a52a0f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.174+0000 7f7562ffd640 1 --2- 172.21.15.112:0/1437083257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d1e670 0x55a52a0f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.174+0000 7f7562ffd640 1 -- 172.21.15.112:0/1437083257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 msgr2=0x55a52a0f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:20.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.174+0000 7f7562ffd640 1 --2- 172.21.15.112:0/1437083257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.174+0000 7f7562ffd640 1 -- 172.21.15.112:0/1437083257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a52a0f8e20 con 0x55a529d16490 2024-09-21T03:39:20.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.174+0000 7f7562ffd640 1 --2- 172.21.15.112:0/1437083257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a102170 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7554019750 tx=0x7f7554019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3088ef281f807 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:20.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.174+0000 7f7548ff9640 1 -- 172.21.15.112:0/1437083257 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755401e040 con 0x55a529d16490 2024-09-21T03:39:20.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.174+0000 7f7548ff9640 1 -- 172.21.15.112:0/1437083257 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7554005040 con 0x55a529d16490 2024-09-21T03:39:20.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.174+0000 7f7548ff9640 1 -- 172.21.15.112:0/1437083257 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554019a30 con 0x55a529d16490 2024-09-21T03:39:20.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.175+0000 7f756f94c740 1 -- 172.21.15.112:0/1437083257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 msgr2=0x55a52a102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.175+0000 7f756f94c740 1 --2- 172.21.15.112:0/1437083257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a102170 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7554019750 tx=0x7f7554019780 comp rx=0 tx=0).stop 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.175+0000 7f756f94c740 1 -- 172.21.15.112:0/1437083257 shutdown_connections 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.175+0000 7f756f94c740 1 --2- 172.21.15.112:0/1437083257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.175+0000 7f756f94c740 1 --2- 172.21.15.112:0/1437083257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d1e670 0x55a52a0f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.175+0000 7f756f94c740 1 --2- 172.21.15.112:0/1437083257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a102170 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.175+0000 7f756f94c740 1 -- 172.21.15.112:0/1437083257 >> 172.21.15.112:0/1437083257 conn(0x55a529d33740 msgr2=0x55a52a0f1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.175+0000 7f756f94c740 1 -- 172.21.15.112:0/1437083257 shutdown_connections 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.175+0000 7f756f94c740 1 -- 172.21.15.112:0/1437083257 wait complete. 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f756f94c740 1 -- start start 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f756f94c740 1 Processor -- start 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f756f94c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a0fa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f756f94c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d1e670 0x55a52a0fa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f756f94c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0face0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f756f94c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75641abaa0 con 0x55a529d16490 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f756f94c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75641acf10 con 0x55a529d3f350 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f756f94c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75641ac170 con 0x55a529d1e670 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f756f94c740 10 client.0 _ll_register_callbacks cb 0x7f7558005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a0fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a0fa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46926/0 (socket says 172.21.15.112:46926) 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f75627fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d1e670 0x55a52a0fa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f7562ffd640 1 -- 172.21.15.112:0/1364090432 learned_addr learned my addr 172.21.15.112:0/1364090432 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.176+0000 7f75637fe640 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f75627fc640 1 -- 172.21.15.112:0/1364090432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 msgr2=0x55a52a0face0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f75627fc640 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0face0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f75627fc640 1 -- 172.21.15.112:0/1364090432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 msgr2=0x55a52a0fa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f75627fc640 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a0fa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f75627fc640 1 -- 172.21.15.112:0/1364090432 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a52a171fd0 con 0x55a529d1e670 2024-09-21T03:39:20.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f7562ffd640 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a0fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:20.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f75627fc640 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d1e670 0x55a52a0fa7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7558005820 tx=0x7f7558009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6e29a5a32806afb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:20.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f7546ff5640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:39:20.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f7546ff5640 1 -- 172.21.15.112:0/1364090432 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580f4020 con 0x55a529d1e670 2024-09-21T03:39:20.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f7546ff5640 1 -- 172.21.15.112:0/1364090432 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f75580cc2f0 con 0x55a529d1e670 2024-09-21T03:39:20.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f756f94c740 1 -- 172.21.15.112:0/1364090432 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7554005040 con 0x55a529d1e670 2024-09-21T03:39:20.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f7546ff5640 1 -- 172.21.15.112:0/1364090432 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7558049070 con 0x55a529d1e670 2024-09-21T03:39:20.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f756f94c740 10 client.26809 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:20.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.177+0000 7f756f94c740 1 -- 172.21.15.112:0/1364090432 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a52a173290 con 0x55a529d1e670 2024-09-21T03:39:20.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.178+0000 7f7546ff5640 1 -- 172.21.15.112:0/1364090432 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7554005040 con 0x55a529d1e670 2024-09-21T03:39:20.183 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:19 smithi187 ceph-mon[34603]: pgmap v637: 97 pgs: 97 active+clean; 824 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 224 op/s 2024-09-21T03:39:20.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.178+0000 7f7546ff5640 1 -- 172.21.15.112:0/1364090432 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f755804c060 con 0x55a529d1e670 2024-09-21T03:39:20.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.178+0000 7f756f94c740 10 client.26809 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:20.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.178+0000 7f756f94c740 10 client.26809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:39:20.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.178+0000 7f756f94c740 20 client.26809 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:20.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.178+0000 7f756f94c740 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a529cd3080 0x55a52a175910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.178+0000 7f756f94c740 4 client.26809 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.178+0000 7f756f94c740 1 -- 172.21.15.112:0/1364090432 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a52a05a860 con 0x55a529cd3080 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.178+0000 7f7562ffd640 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a529cd3080 0x55a52a175910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.178+0000 7f7546ff5640 10 client.26809 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.178+0000 7f7562ffd640 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a529cd3080 0x55a52a175910 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=357263587be7bb34 server_cookie=79f7394043c4537f in_seq=0 out_seq=0 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f7546ff5640 1 -- 172.21.15.112:0/1364090432 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a52a05a860 con 0x55a529cd3080 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f7546ff5640 10 client.26809 handle_command_reply: tid=0 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 1 client.26809 shutdown 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 1 -- 172.21.15.112:0/1364090432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d1e670 msgr2=0x55a52a0fa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d1e670 0x55a52a0fa7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7558005820 tx=0x7f7558009260 comp rx=0 tx=0).stop 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 1 -- 172.21.15.112:0/1364090432 shutdown_connections 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55a529cd3080 0x55a52a175910 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a529d3f350 0x55a52a0face0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a529d1e670 0x55a52a0fa7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 1 --2- 172.21.15.112:0/1364090432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a529d16490 0x55a52a0fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f7546ff5640 0 client.26809 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:20.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 1 -- 172.21.15.112:0/1364090432 >> 172.21.15.112:0/1364090432 conn(0x55a529d33740 msgr2=0x55a52a0f4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:20.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 1 -- 172.21.15.112:0/1364090432 shutdown_connections 2024-09-21T03:39:20.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 1 -- 172.21.15.112:0/1364090432 wait complete. 2024-09-21T03:39:20.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.180+0000 7f756f94c740 20 client.26809 trim_cache size 0 max 16384 2024-09-21T03:39:20.187 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:39:20.187 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:39:20.187 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "7227cba3-c722-4732-8f72-ff30c3ee5d5a", 2024-09-21T03:39:20.187 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:39:20.187 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:39:20.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 -- 172.21.15.112:0/2022304156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 msgr2=0x7f754c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 0x7f754c0b8b90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f755c009fd0 tx=0x7f755c009500 comp rx=0 tx=0).stop 2024-09-21T03:39:20.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 -- 172.21.15.112:0/2022304156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75641083a0 msgr2=0x7f756419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:20.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75641083a0 0x7f756419a9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7558032fd0 tx=0x7f7558031a20 comp rx=0 tx=0).stop 2024-09-21T03:39:20.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 -- 172.21.15.112:0/2022304156 shutdown_connections 2024-09-21T03:39:20.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f754c0b6680 0x7f754c0b8b90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7564108cf0 0x7f756419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75641083a0 0x7f756419a9f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 --2- 172.21.15.112:0/2022304156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7564102a60 0x7f756419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:20.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 -- 172.21.15.112:0/2022304156 >> 172.21.15.112:0/2022304156 conn(0x7f7564079730 msgr2=0x7f7564104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:20.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 -- 172.21.15.112:0/2022304156 shutdown_connections 2024-09-21T03:39:20.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:20.181+0000 7f75693d8640 1 -- 172.21.15.112:0/2022304156 wait complete. 2024-09-21T03:39:20.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1094060 10.44 MB/sec execute 440 sec latency 45.213 ms 2024-09-21T03:39:20.968 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:39:21.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:20 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:39:21.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:20 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:21.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:20 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:39:21.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:20 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:21.228 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:21.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1095189 10.42 MB/sec execute 441 sec latency 73.333 ms 2024-09-21T03:39:21.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:20 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:39:21.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:20 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:21.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.746+0000 7f0c4829b640 1 -- 172.21.15.112:0/1779310063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40103b80 msgr2=0x7f0c40103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.746+0000 7f0c4829b640 1 --2- 172.21.15.112:0/1779310063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40103b80 0x7f0c40103f90 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009900 tx=0x7f0c3c02a410 comp rx=0 tx=0).stop 2024-09-21T03:39:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.746+0000 7f0c4829b640 1 -- 172.21.15.112:0/1779310063 shutdown_connections 2024-09-21T03:39:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.746+0000 7f0c4829b640 1 --2- 172.21.15.112:0/1779310063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c40109e10 0x7f0c40110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.746+0000 7f0c4829b640 1 --2- 172.21.15.112:0/1779310063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c401094c0 0x7f0c401098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.746+0000 7f0c4829b640 1 --2- 172.21.15.112:0/1779310063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40103b80 0x7f0c40103f90 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.746+0000 7f0c4829b640 1 -- 172.21.15.112:0/1779310063 >> 172.21.15.112:0/1779310063 conn(0x7f0c400ff5d0 msgr2=0x7f0c40101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.746+0000 7f0c4829b640 1 -- 172.21.15.112:0/1779310063 shutdown_connections 2024-09-21T03:39:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.746+0000 7f0c4829b640 1 -- 172.21.15.112:0/1779310063 wait complete. 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.747+0000 7f0c4829b640 1 Processor -- start 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.747+0000 7f0c4829b640 1 -- start start 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.747+0000 7f0c4829b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40103b80 0x7f0c40075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.747+0000 7f0c4829b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c401094c0 0x7f0c40075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.747+0000 7f0c4829b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c40109e10 0x7f0c40076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.747+0000 7f0c4829b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40113ad0 con 0x7f0c40103b80 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.747+0000 7f0c4829b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40113950 con 0x7f0c401094c0 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.747+0000 7f0c4829b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40113c50 con 0x7f0c40109e10 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c46037640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40103b80 0x7f0c40075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c46037640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40103b80 0x7f0c40075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46944/0 (socket says 172.21.15.112:46944) 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c46838640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c40109e10 0x7f0c40076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c45836640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c401094c0 0x7f0c40075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c46037640 1 -- 172.21.15.112:0/3516573134 learned_addr learned my addr 172.21.15.112:0/3516573134 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c45836640 1 -- 172.21.15.112:0/3516573134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c40109e10 msgr2=0x7f0c40076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c45836640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c40109e10 0x7f0c40076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c45836640 1 -- 172.21.15.112:0/3516573134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40103b80 msgr2=0x7f0c40075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c45836640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40103b80 0x7f0c40075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c45836640 1 -- 172.21.15.112:0/3516573134 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c401a8740 con 0x7f0c401094c0 2024-09-21T03:39:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c45836640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c401094c0 0x7f0c40075c50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c4006ca10 tx=0x7f0c3000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c377fe640 1 -- 172.21.15.112:0/3516573134 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3000cc40 con 0x7f0c401094c0 2024-09-21T03:39:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c377fe640 1 -- 172.21.15.112:0/3516573134 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c3000cde0 con 0x7f0c401094c0 2024-09-21T03:39:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c4829b640 1 -- 172.21.15.112:0/3516573134 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c401a8a30 con 0x7f0c401094c0 2024-09-21T03:39:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c377fe640 1 -- 172.21.15.112:0/3516573134 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30018780 con 0x7f0c401094c0 2024-09-21T03:39:21.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.748+0000 7f0c4829b640 1 -- 172.21.15.112:0/3516573134 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c401a8fc0 con 0x7f0c401094c0 2024-09-21T03:39:21.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.751+0000 7f0c377fe640 1 -- 172.21.15.112:0/3516573134 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0c30010a90 con 0x7f0c401094c0 2024-09-21T03:39:21.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.751+0000 7f0c4e80f740 1 Processor -- start 2024-09-21T03:39:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.751+0000 7f0c4e80f740 1 -- start start 2024-09-21T03:39:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c757070 0x55df3cafe650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c72e1b0 0x55df3cafeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3caff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55df3ca74e50 con 0x55df3c72e1b0 2024-09-21T03:39:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55df3ca64280 con 0x55df3c757070 2024-09-21T03:39:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55df3ca68090 con 0x55df3c736390 2024-09-21T03:39:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c46037640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c72e1b0 0x55df3cafeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c46838640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c757070 0x55df3cafe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c46037640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c72e1b0 0x55df3cafeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46956/0 (socket says 172.21.15.112:46956) 2024-09-21T03:39:21.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c46037640 1 -- 172.21.15.112:0/4248869444 learned_addr learned my addr 172.21.15.112:0/4248869444 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c45836640 1 --2- 172.21.15.112:0/4248869444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3caff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c377fe640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c2c0b6700 0x7f0c2c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c46037640 1 -- 172.21.15.112:0/4248869444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 msgr2=0x55df3caff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c46037640 1 --2- 172.21.15.112:0/4248869444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3caff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c46037640 1 -- 172.21.15.112:0/4248869444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c757070 msgr2=0x55df3cafe650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c46037640 1 --2- 172.21.15.112:0/4248869444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c757070 0x55df3cafe650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c46037640 1 -- 172.21.15.112:0/4248869444 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3c7c2650 con 0x55df3c72e1b0 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c46838640 1 --2- 172.21.15.112:0/4248869444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c757070 0x55df3cafe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c46838640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c2c0b6700 0x7f0c2c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.752+0000 7f0c377fe640 1 -- 172.21.15.112:0/3516573134 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f0c300e7920 con 0x7f0c401094c0 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.753+0000 7f0c46037640 1 --2- 172.21.15.112:0/4248869444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c72e1b0 0x55df3cafeb90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c000c00 tx=0x7f0c3c031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb981543acdff0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.753+0000 7f0c34d9a640 1 -- 172.21.15.112:0/4248869444 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c040070 con 0x55df3c72e1b0 2024-09-21T03:39:21.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.753+0000 7f0c34d9a640 1 -- 172.21.15.112:0/4248869444 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c3c004030 con 0x55df3c72e1b0 2024-09-21T03:39:21.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.753+0000 7f0c46838640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c2c0b6700 0x7f0c2c0b8c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0c38005fd0 tx=0x7f0c38004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:21.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.753+0000 7f0c34d9a640 1 -- 172.21.15.112:0/4248869444 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c035040 con 0x55df3c72e1b0 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.754+0000 7f0c4e80f740 1 -- 172.21.15.112:0/4248869444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c72e1b0 msgr2=0x55df3cafeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.754+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/4248869444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c72e1b0 0x55df3cafeb90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c000c00 tx=0x7f0c3c031b10 comp rx=0 tx=0).stop 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.754+0000 7f0c4e80f740 1 -- 172.21.15.112:0/4248869444 shutdown_connections 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.754+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/4248869444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3caff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.754+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/4248869444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c72e1b0 0x55df3cafeb90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.754+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/4248869444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c757070 0x55df3cafe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.754+0000 7f0c4e80f740 1 -- 172.21.15.112:0/4248869444 >> 172.21.15.112:0/4248869444 conn(0x55df3c74b460 msgr2=0x55df3cafc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.754+0000 7f0c4e80f740 1 -- 172.21.15.112:0/4248869444 shutdown_connections 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.754+0000 7f0c4e80f740 1 -- 172.21.15.112:0/4248869444 wait complete. 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.754+0000 7f0c4e80f740 1 -- start start 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.754+0000 7f0c4e80f740 1 Processor -- start 2024-09-21T03:39:21.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb09cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3cb0a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 0x55df3cb0a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55df3c8c5700 con 0x55df3c757070 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55df3c9ff110 con 0x55df3c72e1b0 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55df3cb0cdd0 con 0x55df3c736390 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c4e80f740 10 client.0 _ll_register_callbacks cb 0x55df3ca18990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c46838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 0x55df3cb0a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c46037640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb09cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c45836640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3cb0a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c46037640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb09cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41160/0 (socket says 172.21.15.112:41160) 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c46037640 1 -- 172.21.15.112:0/2428777854 learned_addr learned my addr 172.21.15.112:0/2428777854 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:21.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c46037640 1 -- 172.21.15.112:0/2428777854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 msgr2=0x55df3cb0a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c46037640 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3cb0a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c46037640 1 -- 172.21.15.112:0/2428777854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 msgr2=0x55df3cb0a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c46037640 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 0x55df3cb0a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c46037640 1 -- 172.21.15.112:0/2428777854 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3ca2cc40 con 0x55df3c72e1b0 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c45836640 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3cb0a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c46838640 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 0x55df3cb0a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c46037640 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb09cd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0044c0 tx=0x7f0c3c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43578a7b432c443a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.755+0000 7f0c29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c29ffb640 1 -- 172.21.15.112:0/2428777854 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c040070 con 0x55df3c72e1b0 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c29ffb640 1 -- 172.21.15.112:0/2428777854 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f0c3c0314a0 con 0x55df3c72e1b0 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c4e80f740 1 -- 172.21.15.112:0/2428777854 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c3c004030 con 0x55df3c72e1b0 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c29ffb640 1 -- 172.21.15.112:0/2428777854 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c3c056050 con 0x55df3c72e1b0 2024-09-21T03:39:21.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c4e80f740 10 client.26819 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c4e80f740 1 -- 172.21.15.112:0/2428777854 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df3cb18a80 con 0x55df3c72e1b0 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c29ffb640 1 -- 172.21.15.112:0/2428777854 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c3c004030 con 0x55df3c72e1b0 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c29ffb640 1 -- 172.21.15.112:0/2428777854 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0c3c0035f0 con 0x55df3c72e1b0 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c4e80f740 10 client.26819 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c4e80f740 10 client.26819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c4e80f740 20 client.26819 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df3c7159f0 0x55df3cb1bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c4e80f740 4 client.26819 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.756+0000 7f0c4e80f740 1 -- 172.21.15.112:0/2428777854 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df3ca76da0 con 0x55df3c7159f0 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.757+0000 7f0c45836640 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df3c7159f0 0x55df3cb1bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.757+0000 7f0c45836640 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df3c7159f0 0x55df3cb1bde0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff4e93b10274d52 server_cookie=e815fcddd6c430b9 in_seq=0 out_seq=0 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.757+0000 7f0c29ffb640 10 client.26819 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c29ffb640 1 -- 172.21.15.112:0/2428777854 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df3ca76da0 con 0x55df3c7159f0 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c29ffb640 10 client.26819 handle_command_reply: tid=0 2024-09-21T03:39:21.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 1 client.26819 shutdown 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 1 -- 172.21.15.112:0/2428777854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 msgr2=0x55df3cb09cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb09cd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0044c0 tx=0x7f0c3c005d50 comp rx=0 tx=0).stop 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 1 -- 172.21.15.112:0/2428777854 shutdown_connections 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df3c7159f0 0x55df3cb1bde0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 0x55df3cb0a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3cb0a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c29ffb640 0 client.26819 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/2428777854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb09cd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 1 -- 172.21.15.112:0/2428777854 >> 172.21.15.112:0/2428777854 conn(0x55df3c74b460 msgr2=0x55df3cb017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 1 -- 172.21.15.112:0/2428777854 shutdown_connections 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 1 -- 172.21.15.112:0/2428777854 wait complete. 2024-09-21T03:39:21.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.758+0000 7f0c4e80f740 20 client.26819 trim_cache size 0 max 16384 2024-09-21T03:39:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c4e80f740 1 Processor -- start 2024-09-21T03:39:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c4e80f740 1 -- start start 2024-09-21T03:39:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb08bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3cb0b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 0x55df3cafbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401b40b0 con 0x55df3c757070 2024-09-21T03:39:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401b1980 con 0x55df3c72e1b0 2024-09-21T03:39:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401af7f0 con 0x55df3c736390 2024-09-21T03:39:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c46838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 0x55df3cafbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c46838640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 0x55df3cafbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46980/0 (socket says 172.21.15.112:46980) 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c45836640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3cb0b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c46838640 1 -- 172.21.15.112:0/3779293727 learned_addr learned my addr 172.21.15.112:0/3779293727 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c46037640 1 --2- 172.21.15.112:0/3779293727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb08bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c46838640 1 -- 172.21.15.112:0/3779293727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 msgr2=0x55df3cb0b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c46838640 1 --2- 172.21.15.112:0/3779293727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3cb0b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c46838640 1 -- 172.21.15.112:0/3779293727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 msgr2=0x55df3cb08bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c46838640 1 --2- 172.21.15.112:0/3779293727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb08bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.777+0000 7f0c46838640 1 -- 172.21.15.112:0/3779293727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3cafd370 con 0x55df3c757070 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.778+0000 7f0c46838640 1 --2- 172.21.15.112:0/3779293727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 0x55df3cafbb70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0c38020b40 tx=0x7f0c38013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dae482be35012e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.778+0000 7f0c2bfff640 1 -- 172.21.15.112:0/3779293727 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38004d20 con 0x55df3c757070 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.778+0000 7f0c2bfff640 1 -- 172.21.15.112:0/3779293727 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c38005020 con 0x55df3c757070 2024-09-21T03:39:21.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.778+0000 7f0c2bfff640 1 -- 172.21.15.112:0/3779293727 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3801d690 con 0x55df3c757070 2024-09-21T03:39:21.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.778+0000 7f0c4e80f740 1 -- 172.21.15.112:0/3779293727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 msgr2=0x55df3cafbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.778+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/3779293727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 0x55df3cafbb70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0c38020b40 tx=0x7f0c38013030 comp rx=0 tx=0).stop 2024-09-21T03:39:21.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 -- 172.21.15.112:0/3779293727 shutdown_connections 2024-09-21T03:39:21.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/3779293727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c757070 0x55df3cafbb70 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/3779293727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c736390 0x55df3cb0b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/3779293727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb08bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 -- 172.21.15.112:0/3779293727 >> 172.21.15.112:0/3779293727 conn(0x55df3c74b460 msgr2=0x55df3cb0b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:21.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 -- 172.21.15.112:0/3779293727 shutdown_connections 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 -- 172.21.15.112:0/3779293727 wait complete. 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 -- start start 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 Processor -- start 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb1ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c736390 0x55df3cb1d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c757070 0x55df3cb1d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401afe40 con 0x55df3c736390 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401b12b0 con 0x55df3c72e1b0 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401b0510 con 0x55df3c757070 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.779+0000 7f0c4e80f740 10 client.0 _ll_register_callbacks cb 0x7f0c38009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c46037640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb1ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c45836640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c736390 0x55df3cb1d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c46037640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb1ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41180/0 (socket says 172.21.15.112:41180) 2024-09-21T03:39:21.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c46037640 1 -- 172.21.15.112:0/1987331005 learned_addr learned my addr 172.21.15.112:0/1987331005 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c46838640 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c757070 0x55df3cb1d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c46037640 1 -- 172.21.15.112:0/1987331005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c757070 msgr2=0x55df3cb1d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c46037640 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c757070 0x55df3cb1d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c46037640 1 -- 172.21.15.112:0/1987331005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c736390 msgr2=0x55df3cb1d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c46037640 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c736390 0x55df3cb1d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c46037640 1 -- 172.21.15.112:0/1987331005 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3cb85590 con 0x55df3c72e1b0 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c45836640 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c736390 0x55df3cb1d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c46037640 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb1ce20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c001eb0 tx=0x7f0c3c0051d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2bbf3cb1818c254 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c29ffb640 1 -- 172.21.15.112:0/1987331005 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c040040 con 0x55df3c72e1b0 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c4e80f740 1 -- 172.21.15.112:0/1987331005 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c38005020 con 0x55df3c72e1b0 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c29ffb640 1 -- 172.21.15.112:0/1987331005 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f0c3c003e30 con 0x55df3c72e1b0 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c29ffb640 1 -- 172.21.15.112:0/1987331005 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c3c06b050 con 0x55df3c72e1b0 2024-09-21T03:39:21.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c4e80f740 10 client.26825 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.780+0000 7f0c4e80f740 1 -- 172.21.15.112:0/1987331005 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df3cb86800 con 0x55df3c72e1b0 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.781+0000 7f0c29ffb640 1 -- 172.21.15.112:0/1987331005 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c3c05f020 con 0x55df3c72e1b0 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.781+0000 7f0c29ffb640 1 -- 172.21.15.112:0/1987331005 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0c3c004770 con 0x55df3c72e1b0 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.781+0000 7f0c4e80f740 10 client.26825 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.781+0000 7f0c4e80f740 10 client.26825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.781+0000 7f0c4e80f740 20 client.26825 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.781+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df3c6eada0 0x55df3cb88f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.781+0000 7f0c4e80f740 4 client.26825 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.781+0000 7f0c4e80f740 1 -- 172.21.15.112:0/1987331005 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df3ca72850 con 0x55df3c6eada0 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.781+0000 7f0c45836640 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df3c6eada0 0x55df3cb88f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.781+0000 7f0c45836640 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df3c6eada0 0x55df3cb88f00 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51610f3430d75b1a server_cookie=d62e66e628151e51 in_seq=0 out_seq=0 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.781+0000 7f0c29ffb640 10 client.26825 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c29ffb640 1 -- 172.21.15.112:0/1987331005 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5114 (crc 0 0 0) 0x55df3ca72850 con 0x55df3c6eada0 2024-09-21T03:39:21.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c29ffb640 10 client.26825 handle_command_reply: tid=0 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c4e80f740 1 client.26825 shutdown 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c4e80f740 1 -- 172.21.15.112:0/1987331005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 msgr2=0x55df3cb1ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb1ce20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c001eb0 tx=0x7f0c3c0051d0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c4e80f740 1 -- 172.21.15.112:0/1987331005 shutdown_connections 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55df3c6eada0 0x55df3cb88f00 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df3c757070 0x55df3cb1d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55df3c736390 0x55df3cb1d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c4e80f740 1 --2- 172.21.15.112:0/1987331005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df3c72e1b0 0x55df3cb1ce20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c29ffb640 0 client.26825 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c4e80f740 1 -- 172.21.15.112:0/1987331005 >> 172.21.15.112:0/1987331005 conn(0x55df3c74b460 msgr2=0x55df3cb03760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c4e80f740 1 -- 172.21.15.112:0/1987331005 shutdown_connections 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.782+0000 7f0c4e80f740 1 -- 172.21.15.112:0/1987331005 wait complete. 2024-09-21T03:39:21.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4e80f740 20 client.26825 trim_cache size 0 max 16384 2024-09-21T03:39:21.791 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:39:21.791 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:39:21.791 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:39:21.791 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:39:21.791 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.791 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "7227cba3-c722-4732-8f72-ff30c3ee5d5a", 2024-09-21T03:39:21.791 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:39:21.791 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:39:21.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.791 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.792 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.793 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:39:21.794 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.796 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.797 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.798 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.799 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.800 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:39:21.801 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:39:21.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 -- 172.21.15.112:0/3516573134 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c2c0b6700 msgr2=0x7f0c2c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c2c0b6700 0x7f0c2c0b8c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0c38005fd0 tx=0x7f0c38004a40 comp rx=0 tx=0).stop 2024-09-21T03:39:21.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 -- 172.21.15.112:0/3516573134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c401094c0 msgr2=0x7f0c40075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:21.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c401094c0 0x7f0c40075c50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c4006ca10 tx=0x7f0c3000ee50 comp rx=0 tx=0).stop 2024-09-21T03:39:21.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 -- 172.21.15.112:0/3516573134 shutdown_connections 2024-09-21T03:39:21.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f0c2c0b6700 0x7f0c2c0b8c10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c40109e10 0x7f0c40076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c401094c0 0x7f0c40075c50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 --2- 172.21.15.112:0/3516573134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c40103b80 0x7f0c40075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:21.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 -- 172.21.15.112:0/3516573134 >> 172.21.15.112:0/3516573134 conn(0x7f0c400ff5d0 msgr2=0x7f0c401019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:21.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 -- 172.21.15.112:0/3516573134 shutdown_connections 2024-09-21T03:39:21.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:21.783+0000 7f0c4829b640 1 -- 172.21.15.112:0/3516573134 wait complete. 2024-09-21T03:39:22.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:21 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:39:22.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:21 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:22.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:21 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:39:22.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:21 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:39:22.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:21 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:22.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:21 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:22.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:21 smithi112 ceph-mon[31442]: pgmap v638: 97 pgs: 97 active+clean; 832 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 272 op/s 2024-09-21T03:39:22.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:21 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:39:22.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:21 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:22.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:21 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:39:22.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:21 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:39:22.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:21 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:22.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:21 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:22.079 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:21 smithi203 ceph-mon[34723]: pgmap v638: 97 pgs: 97 active+clean; 832 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 272 op/s 2024-09-21T03:39:22.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:21 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:39:22.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:21 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:22.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:21 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:39:22.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:21 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:39:22.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:21 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:22.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:21 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:39:22.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:21 smithi187 ceph-mon[34603]: pgmap v638: 97 pgs: 97 active+clean; 832 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 272 op/s 2024-09-21T03:39:22.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1097687 10.43 MB/sec execute 442 sec latency 17.444 ms 2024-09-21T03:39:22.562 INFO:tasks.cephfs.filesystem:scrub status for tag:7227cba3-c722-4732-8f72-ff30c3ee5d5a - {'path': '/', 'tag': '7227cba3-c722-4732-8f72-ff30c3ee5d5a', 'options': 'recursive,force'} 2024-09-21T03:39:23.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1098544 10.41 MB/sec execute 443 sec latency 161.967 ms 2024-09-21T03:39:23.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:23 smithi203 ceph-mon[34723]: pgmap v639: 97 pgs: 97 active+clean; 832 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 209 op/s 2024-09-21T03:39:24.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:23 smithi187 ceph-mon[34603]: pgmap v639: 97 pgs: 97 active+clean; 832 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 209 op/s 2024-09-21T03:39:24.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1100789 10.42 MB/sec execute 444 sec latency 44.481 ms 2024-09-21T03:39:24.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:23 smithi112 ceph-mon[31442]: pgmap v639: 97 pgs: 97 active+clean; 832 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 209 op/s 2024-09-21T03:39:25.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1101741 10.41 MB/sec execute 445 sec latency 138.310 ms 2024-09-21T03:39:26.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:25 smithi203 ceph-mon[34723]: pgmap v640: 97 pgs: 97 active+clean; 835 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-09-21T03:39:26.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:25 smithi187 ceph-mon[34603]: pgmap v640: 97 pgs: 97 active+clean; 835 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-09-21T03:39:26.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1104058 10.41 MB/sec execute 446 sec latency 20.179 ms 2024-09-21T03:39:26.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:25 smithi112 ceph-mon[31442]: pgmap v640: 97 pgs: 97 active+clean; 835 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-09-21T03:39:27.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1105520 10.40 MB/sec execute 447 sec latency 18.739 ms 2024-09-21T03:39:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:27 smithi203 ceph-mon[34723]: pgmap v641: 97 pgs: 97 active+clean; 803 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 305 op/s 2024-09-21T03:39:28.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:27 smithi187 ceph-mon[34603]: pgmap v641: 97 pgs: 97 active+clean; 803 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 305 op/s 2024-09-21T03:39:28.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1107867 10.41 MB/sec execute 448 sec latency 32.619 ms 2024-09-21T03:39:28.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:27 smithi112 ceph-mon[31442]: pgmap v641: 97 pgs: 97 active+clean; 803 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 305 op/s 2024-09-21T03:39:29.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1108972 10.40 MB/sec execute 449 sec latency 57.591 ms 2024-09-21T03:39:30.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:29 smithi203 ceph-mon[34723]: pgmap v642: 97 pgs: 97 active+clean; 807 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 250 op/s 2024-09-21T03:39:30.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:29 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:39:30.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:29 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:39:30.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:29 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:39:30.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:29 smithi187 ceph-mon[34603]: pgmap v642: 97 pgs: 97 active+clean; 807 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 250 op/s 2024-09-21T03:39:30.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:29 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:39:30.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:29 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:39:30.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:29 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:39:30.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1111233 10.40 MB/sec execute 450 sec latency 46.723 ms 2024-09-21T03:39:30.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:29 smithi112 ceph-mon[31442]: pgmap v642: 97 pgs: 97 active+clean; 807 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 250 op/s 2024-09-21T03:39:30.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:29 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:39:30.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:29 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:39:30.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:29 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:39:31.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1112687 10.40 MB/sec execute 451 sec latency 17.231 ms 2024-09-21T03:39:32.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:31 smithi203 ceph-mon[34723]: pgmap v643: 97 pgs: 97 active+clean; 810 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2024-09-21T03:39:32.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:31 smithi187 ceph-mon[34603]: pgmap v643: 97 pgs: 97 active+clean; 810 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2024-09-21T03:39:32.271 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1115309 10.41 MB/sec execute 452 sec latency 17.865 ms 2024-09-21T03:39:32.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:31 smithi112 ceph-mon[31442]: pgmap v643: 97 pgs: 97 active+clean; 810 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2024-09-21T03:39:33.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1116602 10.40 MB/sec execute 453 sec latency 13.611 ms 2024-09-21T03:39:34.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:33 smithi203 ceph-mon[34723]: pgmap v644: 97 pgs: 97 active+clean; 810 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 224 op/s 2024-09-21T03:39:34.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:33 smithi187 ceph-mon[34603]: pgmap v644: 97 pgs: 97 active+clean; 810 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 224 op/s 2024-09-21T03:39:34.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1119070 10.41 MB/sec execute 454 sec latency 21.574 ms 2024-09-21T03:39:34.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:33 smithi112 ceph-mon[31442]: pgmap v644: 97 pgs: 97 active+clean; 810 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 224 op/s 2024-09-21T03:39:35.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1120299 10.40 MB/sec execute 455 sec latency 34.295 ms 2024-09-21T03:39:36.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:35 smithi203 ceph-mon[34723]: pgmap v645: 97 pgs: 97 active+clean; 814 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 247 op/s 2024-09-21T03:39:36.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:35 smithi187 ceph-mon[34603]: pgmap v645: 97 pgs: 97 active+clean; 814 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 247 op/s 2024-09-21T03:39:36.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1122689 10.41 MB/sec execute 456 sec latency 13.741 ms 2024-09-21T03:39:36.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:35 smithi112 ceph-mon[31442]: pgmap v645: 97 pgs: 97 active+clean; 814 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 247 op/s 2024-09-21T03:39:37.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1124109 10.40 MB/sec execute 457 sec latency 17.096 ms 2024-09-21T03:39:38.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:37 smithi203 ceph-mon[34723]: pgmap v646: 97 pgs: 97 active+clean; 821 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 279 op/s 2024-09-21T03:39:38.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:37 smithi187 ceph-mon[34603]: pgmap v646: 97 pgs: 97 active+clean; 821 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 279 op/s 2024-09-21T03:39:38.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1126345 10.40 MB/sec execute 458 sec latency 14.437 ms 2024-09-21T03:39:38.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:37 smithi112 ceph-mon[31442]: pgmap v646: 97 pgs: 97 active+clean; 821 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 279 op/s 2024-09-21T03:39:39.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1127792 10.39 MB/sec execute 459 sec latency 54.015 ms 2024-09-21T03:39:40.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:39 smithi203 ceph-mon[34723]: pgmap v647: 97 pgs: 97 active+clean; 827 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-21T03:39:40.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:39 smithi187 ceph-mon[34603]: pgmap v647: 97 pgs: 97 active+clean; 827 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-21T03:39:40.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1129871 10.40 MB/sec execute 460 sec latency 34.058 ms 2024-09-21T03:39:40.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:39 smithi112 ceph-mon[31442]: pgmap v647: 97 pgs: 97 active+clean; 827 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-21T03:39:41.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1131707 10.39 MB/sec execute 461 sec latency 18.019 ms 2024-09-21T03:39:42.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:41 smithi203 ceph-mon[34723]: pgmap v648: 97 pgs: 97 active+clean; 828 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 239 op/s 2024-09-21T03:39:42.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:41 smithi187 ceph-mon[34603]: pgmap v648: 97 pgs: 97 active+clean; 828 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 239 op/s 2024-09-21T03:39:42.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1133690 10.40 MB/sec execute 462 sec latency 12.461 ms 2024-09-21T03:39:42.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:41 smithi112 ceph-mon[31442]: pgmap v648: 97 pgs: 97 active+clean; 828 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 239 op/s 2024-09-21T03:39:43.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1136120 10.40 MB/sec execute 463 sec latency 13.889 ms 2024-09-21T03:39:44.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:43 smithi203 ceph-mon[34723]: pgmap v649: 97 pgs: 97 active+clean; 832 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 225 op/s 2024-09-21T03:39:44.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:43 smithi187 ceph-mon[34603]: pgmap v649: 97 pgs: 97 active+clean; 832 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 225 op/s 2024-09-21T03:39:44.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1137480 10.39 MB/sec execute 464 sec latency 19.619 ms 2024-09-21T03:39:44.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:43 smithi112 ceph-mon[31442]: pgmap v649: 97 pgs: 97 active+clean; 832 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 225 op/s 2024-09-21T03:39:45.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1139839 10.40 MB/sec execute 465 sec latency 29.681 ms 2024-09-21T03:39:46.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:45 smithi203 ceph-mon[34723]: pgmap v650: 97 pgs: 97 active+clean; 835 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 234 op/s 2024-09-21T03:39:46.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:45 smithi187 ceph-mon[34603]: pgmap v650: 97 pgs: 97 active+clean; 835 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 234 op/s 2024-09-21T03:39:46.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1141114 10.39 MB/sec execute 466 sec latency 18.894 ms 2024-09-21T03:39:46.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:45 smithi112 ceph-mon[31442]: pgmap v650: 97 pgs: 97 active+clean; 835 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 234 op/s 2024-09-21T03:39:47.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1143563 10.40 MB/sec execute 467 sec latency 18.409 ms 2024-09-21T03:39:48.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:47 smithi203 ceph-mon[34723]: pgmap v651: 97 pgs: 97 active+clean; 845 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 296 op/s 2024-09-21T03:39:48.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:47 smithi187 ceph-mon[34603]: pgmap v651: 97 pgs: 97 active+clean; 845 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 296 op/s 2024-09-21T03:39:48.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1144723 10.39 MB/sec execute 468 sec latency 15.226 ms 2024-09-21T03:39:48.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:47 smithi112 ceph-mon[31442]: pgmap v651: 97 pgs: 97 active+clean; 845 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 296 op/s 2024-09-21T03:39:49.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1146938 10.39 MB/sec execute 469 sec latency 35.291 ms 2024-09-21T03:39:50.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:49 smithi203 ceph-mon[34723]: pgmap v652: 97 pgs: 97 active+clean; 845 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 232 op/s 2024-09-21T03:39:50.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:49 smithi187 ceph-mon[34603]: pgmap v652: 97 pgs: 97 active+clean; 845 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 232 op/s 2024-09-21T03:39:50.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1148176 10.38 MB/sec execute 470 sec latency 26.626 ms 2024-09-21T03:39:50.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:49 smithi112 ceph-mon[31442]: pgmap v652: 97 pgs: 97 active+clean; 845 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 232 op/s 2024-09-21T03:39:51.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1150801 10.40 MB/sec execute 471 sec latency 18.717 ms 2024-09-21T03:39:52.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:51 smithi203 ceph-mon[34723]: pgmap v653: 97 pgs: 97 active+clean; 852 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-09-21T03:39:52.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:51 smithi187 ceph-mon[34603]: pgmap v653: 97 pgs: 97 active+clean; 852 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-09-21T03:39:52.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1151959 10.38 MB/sec execute 472 sec latency 21.528 ms 2024-09-21T03:39:52.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:51 smithi112 ceph-mon[31442]: pgmap v653: 97 pgs: 97 active+clean; 852 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-09-21T03:39:52.564 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:39:52.836 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:53.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1154453 10.39 MB/sec execute 473 sec latency 16.399 ms 2024-09-21T03:39:53.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.360+0000 7fe3330aa640 1 -- 172.21.15.112:0/3091260105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c102a60 msgr2=0x7fe32c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.360+0000 7fe3330aa640 1 --2- 172.21.15.112:0/3091260105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c102a60 0x7fe32c102e70 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe320009900 tx=0x7fe32002a410 comp rx=0 tx=0).stop 2024-09-21T03:39:53.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.360+0000 7fe3330aa640 1 -- 172.21.15.112:0/3091260105 shutdown_connections 2024-09-21T03:39:53.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.360+0000 7fe3330aa640 1 --2- 172.21.15.112:0/3091260105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe32c108cf0 0x7fe32c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.360+0000 7fe3330aa640 1 --2- 172.21.15.112:0/3091260105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe32c1083a0 0x7fe32c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.360+0000 7fe3330aa640 1 --2- 172.21.15.112:0/3091260105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c102a60 0x7fe32c102e70 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.360+0000 7fe3330aa640 1 -- 172.21.15.112:0/3091260105 >> 172.21.15.112:0/3091260105 conn(0x7fe32c079730 msgr2=0x7fe32c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.360+0000 7fe3330aa640 1 -- 172.21.15.112:0/3091260105 shutdown_connections 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.360+0000 7fe3330aa640 1 -- 172.21.15.112:0/3091260105 wait complete. 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.361+0000 7fe3330aa640 1 Processor -- start 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.361+0000 7fe3330aa640 1 -- start start 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.361+0000 7fe3330aa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe32c102a60 0x7fe32c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.361+0000 7fe3330aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c1083a0 0x7fe32c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.361+0000 7fe3330aa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe32c108cf0 0x7fe32c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.361+0000 7fe3330aa640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c112960 con 0x7fe32c1083a0 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.361+0000 7fe3330aa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1127e0 con 0x7fe32c108cf0 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.361+0000 7fe3330aa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c112ae0 con 0x7fe32c102a60 2024-09-21T03:39:53.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.361+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c1083a0 0x7fe32c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.361+0000 7fe330e46640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe32c102a60 0x7fe32c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.361+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c1083a0 0x7fe32c1998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54594/0 (socket says 172.21.15.112:54594) 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe32bfff640 1 -- 172.21.15.112:0/1154255485 learned_addr learned my addr 172.21.15.112:0/1154255485 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe331647640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe32c108cf0 0x7fe32c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe32bfff640 1 -- 172.21.15.112:0/1154255485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe32c102a60 msgr2=0x7fe32c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe32bfff640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe32c102a60 0x7fe32c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe32bfff640 1 -- 172.21.15.112:0/1154255485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe32c108cf0 msgr2=0x7fe32c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe32bfff640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe32c108cf0 0x7fe32c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe32bfff640 1 -- 172.21.15.112:0/1154255485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c19f2c0 con 0x7fe32c1083a0 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe330e46640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe32c102a60 0x7fe32c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe32bfff640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c1083a0 0x7fe32c1998f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe318005fd0 tx=0x7fe31800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe329ffb640 1 -- 172.21.15.112:0/1154255485 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318017070 con 0x7fe32c1083a0 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe3330aa640 1 -- 172.21.15.112:0/1154255485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32c19f5b0 con 0x7fe32c1083a0 2024-09-21T03:39:53.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.362+0000 7fe329ffb640 1 -- 172.21.15.112:0/1154255485 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe318009c30 con 0x7fe32c1083a0 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.363+0000 7fe329ffb640 1 -- 172.21.15.112:0/1154255485 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318003d30 con 0x7fe32c1083a0 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.363+0000 7fe3330aa640 1 -- 172.21.15.112:0/1154255485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe32c1a5d10 con 0x7fe32c1083a0 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.365+0000 7fe329ffb640 1 -- 172.21.15.112:0/1154255485 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe318003ed0 con 0x7fe32c1083a0 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.365+0000 7fe33961e740 1 Processor -- start 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.365+0000 7fe33961e740 1 -- start start 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe33961e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c98070 0x55c44603f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe33961e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 0x55c44603f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe33961e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c77390 0x55c44603fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe33961e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c445fb5e50 con 0x55c445c6f1b0 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe33961e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c445fa5280 con 0x55c445c77390 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe33961e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c445fa9090 con 0x55c445c98070 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe330e46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 0x55c44603f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe331647640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c98070 0x55c44603f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe330e46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 0x55c44603f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54604/0 (socket says 172.21.15.112:54604) 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe330e46640 1 -- 172.21.15.112:0/2281116298 learned_addr learned my addr 172.21.15.112:0/2281116298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe32bfff640 1 --2- 172.21.15.112:0/2281116298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c77390 0x55c44603fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe329ffb640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe31c0b6680 0x7fe31c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe331647640 1 -- 172.21.15.112:0/2281116298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c77390 msgr2=0x55c44603fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe331647640 1 --2- 172.21.15.112:0/2281116298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c77390 0x55c44603fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe331647640 1 -- 172.21.15.112:0/2281116298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 msgr2=0x55c44603f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe331647640 1 --2- 172.21.15.112:0/2281116298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 0x55c44603f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe331647640 1 -- 172.21.15.112:0/2281116298 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c445d03650 con 0x55c445c98070 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.366+0000 7fe331647640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe31c0b6680 0x7fe31c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.367+0000 7fe329ffb640 1 -- 172.21.15.112:0/1154255485 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fe3180e7340 con 0x7fe32c1083a0 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.367+0000 7fe331647640 1 --2- 172.21.15.112:0/2281116298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c98070 0x55c44603f270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe324005f10 tx=0x7fe324010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec79c910ad1b54f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.367+0000 7fe316ffd640 1 -- 172.21.15.112:0/2281116298 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324004b20 con 0x55c445c98070 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.367+0000 7fe316ffd640 1 -- 172.21.15.112:0/2281116298 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe32401ddc0 con 0x55c445c98070 2024-09-21T03:39:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.367+0000 7fe316ffd640 1 -- 172.21.15.112:0/2281116298 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3240055e0 con 0x55c445c98070 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.368+0000 7fe33961e740 1 -- 172.21.15.112:0/2281116298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c98070 msgr2=0x55c44603f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.368+0000 7fe331647640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe31c0b6680 0x7fe31c0b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe324021000 tx=0x7fe32401e000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.368+0000 7fe33961e740 1 --2- 172.21.15.112:0/2281116298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c98070 0x55c44603f270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe324005f10 tx=0x7fe324010860 comp rx=0 tx=0).stop 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.368+0000 7fe33961e740 1 -- 172.21.15.112:0/2281116298 shutdown_connections 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.368+0000 7fe33961e740 1 --2- 172.21.15.112:0/2281116298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c77390 0x55c44603fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.368+0000 7fe33961e740 1 --2- 172.21.15.112:0/2281116298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 0x55c44603f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.368+0000 7fe33961e740 1 --2- 172.21.15.112:0/2281116298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c98070 0x55c44603f270 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.368+0000 7fe33961e740 1 -- 172.21.15.112:0/2281116298 >> 172.21.15.112:0/2281116298 conn(0x55c445c8c460 msgr2=0x55c44603d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.368+0000 7fe33961e740 1 -- 172.21.15.112:0/2281116298 shutdown_connections 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.368+0000 7fe33961e740 1 -- 172.21.15.112:0/2281116298 wait complete. 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.369+0000 7fe33961e740 1 -- start start 2024-09-21T03:39:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.369+0000 7fe33961e740 1 Processor -- start 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe33961e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c6f1b0 0x55c44604caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe33961e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c77390 0x55c44604d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe33961e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 0x55c446058f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe33961e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c445e06700 con 0x55c445c98070 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe33961e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c445f40110 con 0x55c445c77390 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe33961e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c44604fb80 con 0x55c445c6f1b0 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe33961e740 10 client.0 _ll_register_callbacks cb 0x55c445f59990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe330e46640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c6f1b0 0x55c44604caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe331647640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 0x55c446058f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe330e46640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c6f1b0 0x55c44604caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60200/0 (socket says 172.21.15.112:60200) 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe331647640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 0x55c446058f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54618/0 (socket says 172.21.15.112:54618) 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe32bfff640 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c77390 0x55c44604d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe330e46640 1 -- 172.21.15.112:0/2105718528 learned_addr learned my addr 172.21.15.112:0/2105718528 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe330e46640 1 -- 172.21.15.112:0/2105718528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c77390 msgr2=0x55c44604d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe330e46640 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c77390 0x55c44604d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe330e46640 1 -- 172.21.15.112:0/2105718528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 msgr2=0x55c446058f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe330e46640 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 0x55c446058f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe330e46640 1 -- 172.21.15.112:0/2105718528 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c445f6dc40 con 0x55c445c6f1b0 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe32bfff640 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c77390 0x55c44604d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe331647640 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 0x55c446058f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe330e46640 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c6f1b0 0x55c44604caf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe320031870 tx=0x7fe3200318a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=394a42510f675c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.370+0000 7fe30bfff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe30bfff640 1 -- 172.21.15.112:0/2105718528 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320035040 con 0x55c445c6f1b0 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe30bfff640 1 -- 172.21.15.112:0/2105718528 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fe320004240 con 0x55c445c6f1b0 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe33961e740 1 -- 172.21.15.112:0/2105718528 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe32401ddc0 con 0x55c445c6f1b0 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe30bfff640 1 -- 172.21.15.112:0/2105718528 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe320004440 con 0x55c445c6f1b0 2024-09-21T03:39:53.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe33961e740 10 client.26839 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe33961e740 1 -- 172.21.15.112:0/2105718528 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c446047080 con 0x55c445c6f1b0 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe30bfff640 1 -- 172.21.15.112:0/2105718528 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe32401ddc0 con 0x55c445c6f1b0 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe30bfff640 1 -- 172.21.15.112:0/2105718528 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe320004c60 con 0x55c445c6f1b0 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe33961e740 10 client.26839 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe33961e740 10 client.26839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe33961e740 20 client.26839 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.371+0000 7fe33961e740 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c445c569f0 0x55c4460613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.372+0000 7fe33961e740 4 client.26839 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.372+0000 7fe33961e740 1 -- 172.21.15.112:0/2105718528 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c445fb7da0 con 0x55c445c569f0 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.372+0000 7fe32bfff640 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c445c569f0 0x55c4460613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.372+0000 7fe30bfff640 10 client.26839 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.372+0000 7fe32bfff640 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c445c569f0 0x55c4460613b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd0b8dfb0f0292e6 server_cookie=3eaa1c6029b258f9 in_seq=0 out_seq=0 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe30bfff640 1 -- 172.21.15.112:0/2105718528 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c445fb7da0 con 0x55c445c569f0 2024-09-21T03:39:53.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe30bfff640 10 client.26839 handle_command_reply: tid=0 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 1 client.26839 shutdown 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 1 -- 172.21.15.112:0/2105718528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c6f1b0 msgr2=0x55c44604caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c6f1b0 0x55c44604caf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe320031870 tx=0x7fe3200318a0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 1 -- 172.21.15.112:0/2105718528 shutdown_connections 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c445c569f0 0x55c4460613b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 0x55c446058f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c77390 0x55c44604d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe30bfff640 0 client.26839 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 1 --2- 172.21.15.112:0/2105718528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c6f1b0 0x55c44604caf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 1 -- 172.21.15.112:0/2105718528 >> 172.21.15.112:0/2105718528 conn(0x55c445c8c460 msgr2=0x55c44603b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 1 -- 172.21.15.112:0/2105718528 shutdown_connections 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 1 -- 172.21.15.112:0/2105718528 wait complete. 2024-09-21T03:39:53.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.373+0000 7fe33961e740 20 client.26839 trim_cache size 0 max 16384 2024-09-21T03:39:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.393+0000 7fe33961e740 1 Processor -- start 2024-09-21T03:39:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.393+0000 7fe33961e740 1 -- start start 2024-09-21T03:39:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.393+0000 7fe33961e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 0x55c44604b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.393+0000 7fe33961e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 0x55c446044af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.393+0000 7fe33961e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c98070 0x55c446049ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.393+0000 7fe33961e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1afcb0 con 0x55c445c6f1b0 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.393+0000 7fe33961e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1ad580 con 0x55c445c98070 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.393+0000 7fe33961e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1ab3f0 con 0x55c445c77390 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe330e46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 0x55c44604b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe330e46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 0x55c44604b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54620/0 (socket says 172.21.15.112:54620) 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe330e46640 1 -- 172.21.15.112:0/3296487249 learned_addr learned my addr 172.21.15.112:0/3296487249 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe32bfff640 1 --2- 172.21.15.112:0/3296487249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 0x55c446044af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe331647640 1 --2- 172.21.15.112:0/3296487249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c98070 0x55c446049ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe330e46640 1 -- 172.21.15.112:0/3296487249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 msgr2=0x55c446044af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe330e46640 1 --2- 172.21.15.112:0/3296487249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 0x55c446044af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe330e46640 1 -- 172.21.15.112:0/3296487249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c98070 msgr2=0x55c446049ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:53.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe330e46640 1 --2- 172.21.15.112:0/3296487249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c98070 0x55c446049ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe330e46640 1 -- 172.21.15.112:0/3296487249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44603dae0 con 0x55c445c6f1b0 2024-09-21T03:39:53.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe330e46640 1 --2- 172.21.15.112:0/3296487249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 0x55c44604b9e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe320033000 tx=0x7fe320031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f4f3f83fa70261 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:53.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe3167fc640 1 -- 172.21.15.112:0/3296487249 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320035070 con 0x55c445c6f1b0 2024-09-21T03:39:53.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.394+0000 7fe3167fc640 1 -- 172.21.15.112:0/3296487249 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe320056030 con 0x55c445c6f1b0 2024-09-21T03:39:53.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.395+0000 7fe3167fc640 1 -- 172.21.15.112:0/3296487249 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320003fd0 con 0x55c445c6f1b0 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.395+0000 7fe33961e740 1 -- 172.21.15.112:0/3296487249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 msgr2=0x55c44604b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.395+0000 7fe33961e740 1 --2- 172.21.15.112:0/3296487249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 0x55c44604b9e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe320033000 tx=0x7fe320031400 comp rx=0 tx=0).stop 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.395+0000 7fe33961e740 1 -- 172.21.15.112:0/3296487249 shutdown_connections 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.395+0000 7fe33961e740 1 --2- 172.21.15.112:0/3296487249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c98070 0x55c446049ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.395+0000 7fe33961e740 1 --2- 172.21.15.112:0/3296487249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 0x55c446044af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.395+0000 7fe33961e740 1 --2- 172.21.15.112:0/3296487249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c6f1b0 0x55c44604b9e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.395+0000 7fe33961e740 1 -- 172.21.15.112:0/3296487249 >> 172.21.15.112:0/3296487249 conn(0x55c445c8c460 msgr2=0x55c4460443e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.395+0000 7fe33961e740 1 -- 172.21.15.112:0/3296487249 shutdown_connections 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.395+0000 7fe33961e740 1 -- 172.21.15.112:0/3296487249 wait complete. 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.396+0000 7fe33961e740 1 -- start start 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.396+0000 7fe33961e740 1 Processor -- start 2024-09-21T03:39:53.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.396+0000 7fe33961e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c6f1b0 0x55c446064ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.396+0000 7fe33961e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 0x55c44605ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.396+0000 7fe33961e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 0x55c44605d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.396+0000 7fe33961e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1aba40 con 0x55c445c98070 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.396+0000 7fe33961e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1aceb0 con 0x55c445c6f1b0 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.396+0000 7fe33961e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1ac110 con 0x55c445c77390 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.396+0000 7fe33961e740 10 client.0 _ll_register_callbacks cb 0x7fe320009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 0x55c44605ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 0x55c44605ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60228/0 (socket says 172.21.15.112:60228) 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe32bfff640 1 -- 172.21.15.112:0/1343568343 learned_addr learned my addr 172.21.15.112:0/1343568343 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe331647640 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 0x55c44605d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe330e46640 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c6f1b0 0x55c446064ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe32bfff640 1 -- 172.21.15.112:0/1343568343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c6f1b0 msgr2=0x55c446064ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe32bfff640 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c6f1b0 0x55c446064ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe32bfff640 1 -- 172.21.15.112:0/1343568343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 msgr2=0x55c44605d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe32bfff640 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 0x55c44605d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe32bfff640 1 -- 172.21.15.112:0/1343568343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44605da20 con 0x55c445c77390 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe32bfff640 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 0x55c44605ce40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe3180e4b20 tx=0x7fe318092f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e14e00948adbe1ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe30bfff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe30bfff640 1 -- 172.21.15.112:0/1343568343 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3180ac930 con 0x55c445c77390 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe33961e740 1 -- 172.21.15.112:0/1343568343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe320056030 con 0x55c445c77390 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe30bfff640 1 -- 172.21.15.112:0/1343568343 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fe3180a9430 con 0x55c445c77390 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.397+0000 7fe30bfff640 1 -- 172.21.15.112:0/1343568343 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3180accd0 con 0x55c445c77390 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.398+0000 7fe33961e740 10 client.26845 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.398+0000 7fe33961e740 1 -- 172.21.15.112:0/1343568343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c4460d0310 con 0x55c445c77390 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.398+0000 7fe30bfff640 1 -- 172.21.15.112:0/1343568343 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe320056030 con 0x55c445c77390 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.398+0000 7fe30bfff640 1 -- 172.21.15.112:0/1343568343 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe3180a8b10 con 0x55c445c77390 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.398+0000 7fe33961e740 10 client.26845 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:53.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.398+0000 7fe33961e740 10 client.26845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.398+0000 7fe33961e740 20 client.26845 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.398+0000 7fe33961e740 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c445c2bda0 0x55c4460d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.398+0000 7fe33961e740 4 client.26845 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.398+0000 7fe33961e740 1 -- 172.21.15.112:0/1343568343 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c445fb3850 con 0x55c445c2bda0 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.398+0000 7fe330e46640 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c445c2bda0 0x55c4460d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe30bfff640 10 client.26845 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe330e46640 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c445c2bda0 0x55c4460d2a00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7dabb1ab2fc02a6 server_cookie=b24f91bcc2969bba in_seq=0 out_seq=0 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe30bfff640 1 -- 172.21.15.112:0/1343568343 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x55c445fb3850 con 0x55c445c2bda0 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe30bfff640 10 client.26845 handle_command_reply: tid=0 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe33961e740 1 client.26845 shutdown 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe33961e740 1 -- 172.21.15.112:0/1343568343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 msgr2=0x55c44605ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe33961e740 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 0x55c44605ce40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe3180e4b20 tx=0x7fe318092f90 comp rx=0 tx=0).stop 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe33961e740 1 -- 172.21.15.112:0/1343568343 shutdown_connections 2024-09-21T03:39:53.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe33961e740 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c445c2bda0 0x55c4460d2a00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe30bfff640 0 client.26845 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:39:53.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe33961e740 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c445c98070 0x55c44605d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe33961e740 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c445c77390 0x55c44605ce40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe33961e740 1 --2- 172.21.15.112:0/1343568343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c445c6f1b0 0x55c446064ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.399+0000 7fe33961e740 1 -- 172.21.15.112:0/1343568343 >> 172.21.15.112:0/1343568343 conn(0x55c445c8c460 msgr2=0x55c4460402a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:53.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.400+0000 7fe33961e740 1 -- 172.21.15.112:0/1343568343 shutdown_connections 2024-09-21T03:39:53.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.400+0000 7fe33961e740 1 -- 172.21.15.112:0/1343568343 wait complete. 2024-09-21T03:39:53.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.400+0000 7fe33961e740 20 client.26845 trim_cache size 0 max 16384 2024-09-21T03:39:53.406 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:39:53.407 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.409 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.410 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:39:53.411 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.413 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.414 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:39:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:39:53.416 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:39:53.417 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:39:53.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.400+0000 7fe3330aa640 1 -- 172.21.15.112:0/1154255485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe31c0b6680 msgr2=0x7fe31c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.400+0000 7fe3330aa640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe31c0b6680 0x7fe31c0b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe324021000 tx=0x7fe32401e000 comp rx=0 tx=0).stop 2024-09-21T03:39:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.401+0000 7fe3330aa640 1 -- 172.21.15.112:0/1154255485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c1083a0 msgr2=0x7fe32c1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.401+0000 7fe3330aa640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c1083a0 0x7fe32c1998f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe318005fd0 tx=0x7fe31800dde0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.401+0000 7fe3330aa640 1 -- 172.21.15.112:0/1154255485 shutdown_connections 2024-09-21T03:39:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.401+0000 7fe3330aa640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fe31c0b6680 0x7fe31c0b8b90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.401+0000 7fe3330aa640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe32c108cf0 0x7fe32c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.401+0000 7fe3330aa640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c1083a0 0x7fe32c1998f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.401+0000 7fe3330aa640 1 --2- 172.21.15.112:0/1154255485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe32c102a60 0x7fe32c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.401+0000 7fe3330aa640 1 -- 172.21.15.112:0/1154255485 >> 172.21.15.112:0/1154255485 conn(0x7fe32c079730 msgr2=0x7fe32c1013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.401+0000 7fe3330aa640 1 -- 172.21.15.112:0/1154255485 shutdown_connections 2024-09-21T03:39:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:53.401+0000 7fe3330aa640 1 -- 172.21.15.112:0/1154255485 wait complete. 2024-09-21T03:39:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:53 smithi203 ceph-mon[34723]: pgmap v654: 97 pgs: 97 active+clean; 855 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 254 op/s 2024-09-21T03:39:54.177 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:53 smithi112 ceph-mon[31442]: pgmap v654: 97 pgs: 97 active+clean; 855 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 254 op/s 2024-09-21T03:39:54.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:53 smithi187 ceph-mon[34603]: pgmap v654: 97 pgs: 97 active+clean; 855 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.1 MiB/s wr, 254 op/s 2024-09-21T03:39:54.209 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:39:54.209 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:39:54.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1155572 10.38 MB/sec execute 474 sec latency 15.026 ms 2024-09-21T03:39:54.505 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.995+0000 7f071b2f4640 1 -- 172.21.15.112:0/2803065083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07141049b0 msgr2=0x7f0714104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.995+0000 7f071b2f4640 1 --2- 172.21.15.112:0/2803065083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07141049b0 0x7f0714104dc0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0710009900 tx=0x7f071002a410 comp rx=0 tx=0).stop 2024-09-21T03:39:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.995+0000 7f071b2f4640 1 -- 172.21.15.112:0/2803065083 shutdown_connections 2024-09-21T03:39:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.995+0000 7f071b2f4640 1 --2- 172.21.15.112:0/2803065083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f071410de10 0x7f0714110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.995+0000 7f071b2f4640 1 --2- 172.21.15.112:0/2803065083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0714105300 0x7f071410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.995+0000 7f071b2f4640 1 --2- 172.21.15.112:0/2803065083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07141049b0 0x7f0714104dc0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.995+0000 7f071b2f4640 1 -- 172.21.15.112:0/2803065083 >> 172.21.15.112:0/2803065083 conn(0x7f0714079730 msgr2=0x7f0714079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.995+0000 7f071b2f4640 1 -- 172.21.15.112:0/2803065083 shutdown_connections 2024-09-21T03:39:54.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.996+0000 7f071b2f4640 1 -- 172.21.15.112:0/2803065083 wait complete. 2024-09-21T03:39:55.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.996+0000 7f071b2f4640 1 Processor -- start 2024-09-21T03:39:55.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.996+0000 7f071b2f4640 1 -- start start 2024-09-21T03:39:55.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.996+0000 7f071b2f4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07141049b0 0x7f071419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f071b2f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0714105300 0x7f071419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f071b2f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071410de10 0x7f071419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f071b2f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0714113ae0 con 0x7f071410de10 2024-09-21T03:39:55.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f071b2f4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0714113960 con 0x7f0714105300 2024-09-21T03:39:55.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f071b2f4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0714113c60 con 0x7f07141049b0 2024-09-21T03:39:55.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f0719090640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07141049b0 0x7f071419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:55.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f0719090640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07141049b0 0x7f071419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60248/0 (socket says 172.21.15.112:60248) 2024-09-21T03:39:55.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f0719090640 1 -- 172.21.15.112:0/2328253239 learned_addr learned my addr 172.21.15.112:0/2328253239 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:55.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f0719891640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071410de10 0x7f071419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:55.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f0719090640 1 -- 172.21.15.112:0/2328253239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0714105300 msgr2=0x7f071419aa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:39:55.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f0719090640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0714105300 0x7f071419aa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.997+0000 7f0719090640 1 -- 172.21.15.112:0/2328253239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071410de10 msgr2=0x7f071419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.998+0000 7f0719090640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071410de10 0x7f071419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.998+0000 7f0719090640 1 -- 172.21.15.112:0/2328253239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07141a0460 con 0x7f07141049b0 2024-09-21T03:39:55.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.998+0000 7f0719891640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071410de10 0x7f071419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:39:55.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.998+0000 7f0719090640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07141049b0 0x7f071419a4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0710005ab0 tx=0x7f0710003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:55.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.998+0000 7f070a7fc640 1 -- 172.21.15.112:0/2328253239 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071003f070 con 0x7f07141049b0 2024-09-21T03:39:55.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.998+0000 7f070a7fc640 1 -- 172.21.15.112:0/2328253239 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0710031b50 con 0x7f07141049b0 2024-09-21T03:39:55.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.998+0000 7f070a7fc640 1 -- 172.21.15.112:0/2328253239 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710035030 con 0x7f07141049b0 2024-09-21T03:39:55.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.998+0000 7f071b2f4640 1 -- 172.21.15.112:0/2328253239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07141a06f0 con 0x7f07141049b0 2024-09-21T03:39:55.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.998+0000 7f071b2f4640 1 -- 172.21.15.112:0/2328253239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07141025d0 con 0x7f07141049b0 2024-09-21T03:39:55.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:54.999+0000 7f071b2f4640 1 -- 172.21.15.112:0/2328253239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f071406b560 con 0x7f07141049b0 2024-09-21T03:39:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.001+0000 7f070a7fc640 1 -- 172.21.15.112:0/2328253239 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0710042070 con 0x7f07141049b0 2024-09-21T03:39:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.002+0000 7f070a7fc640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07000b6680 0x7f07000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.002+0000 7f070a7fc640 1 -- 172.21.15.112:0/2328253239 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f0710108690 con 0x7f07141049b0 2024-09-21T03:39:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.003+0000 7f071888f640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07000b6680 0x7f07000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.003+0000 7f071888f640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07000b6680 0x7f07000b8b90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0714079fb0 tx=0x7f0704005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.005+0000 7f070a7fc640 1 -- 172.21.15.112:0/2328253239 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07100cec10 con 0x7f07141049b0 2024-09-21T03:39:55.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.245+0000 7f071b2f4640 1 -- 172.21.15.112:0/2328253239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0714104dc0 con 0x7f07141049b0 2024-09-21T03:39:55.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.246+0000 7f070a7fc640 1 -- 172.21.15.112:0/2328253239 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f07100d2ff0 con 0x7f07141049b0 2024-09-21T03:39:55.248 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:39:55.248 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"btime":"2024-09-21T03:35:32:357323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:35:32.307199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:39:55.249 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-09-21T03:39:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 -- 172.21.15.112:0/2328253239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07000b6680 msgr2=0x7f07000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07000b6680 0x7f07000b8b90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0714079fb0 tx=0x7f0704005f90 comp rx=0 tx=0).stop 2024-09-21T03:39:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 -- 172.21.15.112:0/2328253239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07141049b0 msgr2=0x7f071419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07141049b0 0x7f071419a4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0710005ab0 tx=0x7f0710003b90 comp rx=0 tx=0).stop 2024-09-21T03:39:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 -- 172.21.15.112:0/2328253239 shutdown_connections 2024-09-21T03:39:55.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f07000b6680 0x7f07000b8b90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f071410de10 0x7f071419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0714105300 0x7f071419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 --2- 172.21.15.112:0/2328253239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07141049b0 0x7f071419a4f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 -- 172.21.15.112:0/2328253239 >> 172.21.15.112:0/2328253239 conn(0x7f0714079730 msgr2=0x7f07141007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 -- 172.21.15.112:0/2328253239 shutdown_connections 2024-09-21T03:39:55.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:55.248+0000 7f071b2f4640 1 -- 172.21.15.112:0/2328253239 wait complete. 2024-09-21T03:39:55.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1157891 10.39 MB/sec execute 475 sec latency 40.144 ms 2024-09-21T03:39:56.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:39:56.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:55 smithi203 ceph-mon[34723]: pgmap v655: 97 pgs: 97 active+clean; 857 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 248 op/s 2024-09-21T03:39:56.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:55 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2328253239' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:56.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:55 smithi187 ceph-mon[34603]: pgmap v655: 97 pgs: 97 active+clean; 857 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 248 op/s 2024-09-21T03:39:56.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:55 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2328253239' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:56.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1158907 10.37 MB/sec execute 476 sec latency 22.065 ms 2024-09-21T03:39:56.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:55 smithi112 ceph-mon[31442]: pgmap v655: 97 pgs: 97 active+clean; 857 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 248 op/s 2024-09-21T03:39:56.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:55 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2328253239' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:39:56.317 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.829+0000 7fc3b26ac640 1 -- 172.21.15.112:0/3268055621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ac1030f0 msgr2=0x7fc3ac107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.829+0000 7fc3b26ac640 1 --2- 172.21.15.112:0/3268055621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ac1030f0 0x7fc3ac107620 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc394009930 tx=0x7fc39402a420 comp rx=0 tx=0).stop 2024-09-21T03:39:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.829+0000 7fc3b26ac640 1 -- 172.21.15.112:0/3268055621 shutdown_connections 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.829+0000 7fc3b26ac640 1 --2- 172.21.15.112:0/3268055621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ac107be0 0x7fc3ac1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.829+0000 7fc3b26ac640 1 --2- 172.21.15.112:0/3268055621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ac1030f0 0x7fc3ac107620 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.829+0000 7fc3b26ac640 1 --2- 172.21.15.112:0/3268055621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3ac1027a0 0x7fc3ac102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.829+0000 7fc3b26ac640 1 -- 172.21.15.112:0/3268055621 >> 172.21.15.112:0/3268055621 conn(0x7fc3ac079730 msgr2=0x7fc3ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.829+0000 7fc3b26ac640 1 -- 172.21.15.112:0/3268055621 shutdown_connections 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.829+0000 7fc3b26ac640 1 -- 172.21.15.112:0/3268055621 wait complete. 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3b26ac640 1 Processor -- start 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3b26ac640 1 -- start start 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3b26ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ac1027a0 0x7fc3ac1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3b26ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ac1030f0 0x7fc3ac1a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3b26ac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3ac107be0 0x7fc3ac1a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3b26ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac11a060 con 0x7fc3ac1027a0 2024-09-21T03:39:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3b26ac640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac119ee0 con 0x7fc3ac107be0 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3b26ac640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac11a1e0 con 0x7fc3ac1030f0 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ac1027a0 0x7fc3ac1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ac1027a0 0x7fc3ac1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49418/0 (socket says 172.21.15.112:49418) 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3a3fff640 1 -- 172.21.15.112:0/2507739371 learned_addr learned my addr 172.21.15.112:0/2507739371 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3ac107be0 0x7fc3ac1a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3abfff640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ac1030f0 0x7fc3ac1a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3a3fff640 1 -- 172.21.15.112:0/2507739371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ac1030f0 msgr2=0x7fc3ac1a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3a3fff640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ac1030f0 0x7fc3ac1a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3a3fff640 1 -- 172.21.15.112:0/2507739371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3ac107be0 msgr2=0x7fc3ac1a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3a3fff640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3ac107be0 0x7fc3ac1a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3a3fff640 1 -- 172.21.15.112:0/2507739371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac1a6aa0 con 0x7fc3ac1027a0 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.830+0000 7fc3a3fff640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ac1027a0 0x7fc3ac1a0b30 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00d950 tx=0x7fc39c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.831+0000 7fc3a9ffb640 1 -- 172.21.15.112:0/2507739371 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c017020 con 0x7fc3ac1027a0 2024-09-21T03:39:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.831+0000 7fc3b26ac640 1 -- 172.21.15.112:0/2507739371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac1a6d90 con 0x7fc3ac1027a0 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.831+0000 7fc3a9ffb640 1 -- 172.21.15.112:0/2507739371 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc39c009c30 con 0x7fc3ac1027a0 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.831+0000 7fc3a9ffb640 1 -- 172.21.15.112:0/2507739371 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c00f6b0 con 0x7fc3ac1027a0 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.831+0000 7fc3b26ac640 1 -- 172.21.15.112:0/2507739371 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac1ad4f0 con 0x7fc3ac1027a0 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.834+0000 7fc3b8c20740 1 Processor -- start 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.834+0000 7fc3b8c20740 1 -- start start 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.834+0000 7fc3a9ffb640 1 -- 172.21.15.112:0/2507739371 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc39c01a020 con 0x7fc3ac1027a0 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 0x55c2034782c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203478800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030b0390 0x55c203478d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c2033eeea0 con 0x55c2030a81b0 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c2033de2d0 con 0x55c2030b0390 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c2033e20e0 con 0x55c2030d1070 2024-09-21T03:39:56.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203478800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b0c49640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 0x55c2034782c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b0c49640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 0x55c2034782c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52570/0 (socket says 172.21.15.112:52570) 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b0c49640 1 -- 172.21.15.112:0/610031097 learned_addr learned my addr 172.21.15.112:0/610031097 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3abfff640 1 --2- 172.21.15.112:0/610031097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030b0390 0x55c203478d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b0c49640 1 -- 172.21.15.112:0/610031097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030b0390 msgr2=0x55c203478d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/610031097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030b0390 0x55c203478d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b0c49640 1 -- 172.21.15.112:0/610031097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 msgr2=0x55c203478800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/610031097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203478800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b0c49640 1 -- 172.21.15.112:0/610031097 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c20313c650 con 0x55c2030d1070 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3a9ffb640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3980b6680 0x7fc3980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/610031097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 0x55c2034782c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400ae10 tx=0x7fc3a400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d9f2380bf2c08cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3a3fff640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3980b6680 0x7fc3980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.835+0000 7fc3a9ffb640 1 -- 172.21.15.112:0/2507739371 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fc39c0e7380 con 0x7fc3ac1027a0 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.836+0000 7fc3a27fc640 1 -- 172.21.15.112:0/610031097 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4003b70 con 0x55c2030d1070 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.836+0000 7fc3a27fc640 1 -- 172.21.15.112:0/610031097 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3a4003d10 con 0x55c2030d1070 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.836+0000 7fc3a27fc640 1 -- 172.21.15.112:0/610031097 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4010a00 con 0x55c2030d1070 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.836+0000 7fc3b8c20740 1 -- 172.21.15.112:0/610031097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 msgr2=0x55c2034782c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.836+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/610031097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 0x55c2034782c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400ae10 tx=0x7fc3a400f790 comp rx=0 tx=0).stop 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.836+0000 7fc3a3fff640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3980b6680 0x7fc3980b8b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0e4bb0 tx=0x7fc39c0eaed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.836+0000 7fc3b8c20740 1 -- 172.21.15.112:0/610031097 shutdown_connections 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.836+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/610031097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030b0390 0x55c203478d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.836+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/610031097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203478800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.836+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/610031097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 0x55c2034782c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.836+0000 7fc3b8c20740 1 -- 172.21.15.112:0/610031097 >> 172.21.15.112:0/610031097 conn(0x55c2030c5460 msgr2=0x55c2034761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.837+0000 7fc3b8c20740 1 -- 172.21.15.112:0/610031097 shutdown_connections 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.837+0000 7fc3b8c20740 1 -- 172.21.15.112:0/610031097 wait complete. 2024-09-21T03:39:56.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.837+0000 7fc3b8c20740 1 -- start start 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.837+0000 7fc3b8c20740 1 Processor -- start 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.837+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c2034839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 0x55c203483f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030d1070 0x55c20348fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c20323f700 con 0x55c2030a81b0 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c203379160 con 0x55c2030d1070 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c203486aa0 con 0x55c2030b0390 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3b8c20740 10 client.0 _ll_register_callbacks cb 0x55c2033929e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3b0c49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c2034839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3b0c49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c2034839d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49436/0 (socket says 172.21.15.112:49436) 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3b0c49640 1 -- 172.21.15.112:0/3761184034 learned_addr learned my addr 172.21.15.112:0/3761184034 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3abfff640 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 0x55c203483f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030d1070 0x55c20348fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3abfff640 1 -- 172.21.15.112:0/3761184034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030d1070 msgr2=0x55c20348fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3abfff640 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030d1070 0x55c20348fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3abfff640 1 -- 172.21.15.112:0/3761184034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 msgr2=0x55c2034839d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3abfff640 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c2034839d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3abfff640 1 -- 172.21.15.112:0/3761184034 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2033a6c90 con 0x55c2030b0390 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3abfff640 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 0x55c203483f10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc39402a940 tx=0x7fc3940317f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c4eca4de04ec003 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc38b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3761184034 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394035040 con 0x55c2030b0390 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3761184034 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fc394004240 con 0x55c2030b0390 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.838+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3761184034 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3a4003d10 con 0x55c2030b0390 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3761184034 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc394004440 con 0x55c2030b0390 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc3b8c20740 10 client.26860 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3761184034 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2034927e0 con 0x55c2030b0390 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3761184034 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3a4003d10 con 0x55c2030b0390 2024-09-21T03:39:56.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3761184034 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc39403c640 con 0x55c2030b0390 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc3b8c20740 10 client.26860 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc3b8c20740 10 client.26860 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc3b8c20740 20 client.26860 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55c20308f9f0 0x55c203495c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc3b8c20740 4 client.26860 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3761184034 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2033f0df0 con 0x55c20308f9f0 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.839+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55c20308f9f0 0x55c203495c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.840+0000 7fc38b7fe640 10 client.26860 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.840+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55c20308f9f0 0x55c203495c40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=626d27bccee49f31 server_cookie=d41619549d5cf9f6 in_seq=0 out_seq=0 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3761184034 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c2033f0df0 con 0x55c20308f9f0 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc38b7fe640 10 client.26860 handle_command_reply: tid=0 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 1 client.26860 shutdown 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3761184034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 msgr2=0x55c203483f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 0x55c203483f10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc39402a940 tx=0x7fc3940317f0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3761184034 shutdown_connections 2024-09-21T03:39:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55c20308f9f0 0x55c203495c40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030d1070 0x55c20348fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 0x55c203483f10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc38b7fe640 0 client.26860 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:39:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3761184034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c2034839d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3761184034 >> 172.21.15.112:0/3761184034 conn(0x55c2030c5460 msgr2=0x55c20347b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3761184034 shutdown_connections 2024-09-21T03:39:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3761184034 wait complete. 2024-09-21T03:39:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.841+0000 7fc3b8c20740 20 client.26860 trim_cache size 0 max 16384 2024-09-21T03:39:56.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b8c20740 1 Processor -- start 2024-09-21T03:39:56.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b8c20740 1 -- start start 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203483e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 0x55c203475760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030d1070 0x55c203475ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac1b8670 con 0x55c2030a81b0 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac1b5f40 con 0x55c2030d1070 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac1b3db0 con 0x55c2030b0390 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b0c49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203483e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b0c49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203483e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49446/0 (socket says 172.21.15.112:49446) 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b0c49640 1 -- 172.21.15.112:0/2720669153 learned_addr learned my addr 172.21.15.112:0/2720669153 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3abfff640 1 --2- 172.21.15.112:0/2720669153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 0x55c203475760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/2720669153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030d1070 0x55c203475ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3abfff640 1 -- 172.21.15.112:0/2720669153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030d1070 msgr2=0x55c203475ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3abfff640 1 --2- 172.21.15.112:0/2720669153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030d1070 0x55c203475ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3abfff640 1 -- 172.21.15.112:0/2720669153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 msgr2=0x55c203483e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3abfff640 1 --2- 172.21.15.112:0/2720669153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203483e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3abfff640 1 -- 172.21.15.112:0/2720669153 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c203481ed0 con 0x55c2030b0390 2024-09-21T03:39:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.855+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/2720669153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203483e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.856+0000 7fc3abfff640 1 --2- 172.21.15.112:0/2720669153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 0x55c203475760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc394031440 tx=0x7fc39403cd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbe8a352dcd68386 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.856+0000 7fc3a1ffb640 1 -- 172.21.15.112:0/2720669153 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394035070 con 0x55c2030b0390 2024-09-21T03:39:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.856+0000 7fc3a1ffb640 1 -- 172.21.15.112:0/2720669153 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc394005520 con 0x55c2030b0390 2024-09-21T03:39:56.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.856+0000 7fc3a1ffb640 1 -- 172.21.15.112:0/2720669153 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394000e60 con 0x55c2030b0390 2024-09-21T03:39:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.857+0000 7fc3b8c20740 1 -- 172.21.15.112:0/2720669153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 msgr2=0x55c203475760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.857+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/2720669153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 0x55c203475760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc394031440 tx=0x7fc39403cd40 comp rx=0 tx=0).stop 2024-09-21T03:39:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.857+0000 7fc3b8c20740 1 -- 172.21.15.112:0/2720669153 shutdown_connections 2024-09-21T03:39:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.857+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/2720669153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030d1070 0x55c203475ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.857+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/2720669153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030b0390 0x55c203475760 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.857+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/2720669153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203483e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.857+0000 7fc3b8c20740 1 -- 172.21.15.112:0/2720669153 >> 172.21.15.112:0/2720669153 conn(0x55c2030c5460 msgr2=0x55c203483780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:56.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.857+0000 7fc3b8c20740 1 -- 172.21.15.112:0/2720669153 shutdown_connections 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.857+0000 7fc3b8c20740 1 -- 172.21.15.112:0/2720669153 wait complete. 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.857+0000 7fc3b8c20740 1 -- start start 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.857+0000 7fc3b8c20740 1 Processor -- start 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203488090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030b0390 0x55c2034885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b8c20740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 0x55c2034ff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac1b4400 con 0x55c2030a81b0 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac1b5870 con 0x55c2030b0390 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b8c20740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac1b4ad0 con 0x55c2030d1070 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b8c20740 10 client.0 _ll_register_callbacks cb 0x7fc394005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b0c49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203488090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b0c49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203488090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49454/0 (socket says 172.21.15.112:49454) 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030b0390 0x55c2034885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b0c49640 1 -- 172.21.15.112:0/3923335986 learned_addr learned my addr 172.21.15.112:0/3923335986 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 0x55c2034ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3abfff640 1 -- 172.21.15.112:0/3923335986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 msgr2=0x55c2034ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3abfff640 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 0x55c2034ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3abfff640 1 -- 172.21.15.112:0/3923335986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 msgr2=0x55c203488090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3abfff640 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203488090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3abfff640 1 -- 172.21.15.112:0/3923335986 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2034ffb40 con 0x55c2030b0390 2024-09-21T03:39:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203488090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc3abfff640 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030b0390 0x55c2034885d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55c203492730 tx=0x7fc394000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22145b5844a115b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.858+0000 7fc38b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:39:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.859+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3923335986 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394035070 con 0x55c2030b0390 2024-09-21T03:39:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.859+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3923335986 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc394005520 con 0x55c2030b0390 2024-09-21T03:39:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.859+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3923335986 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fc39403c440 con 0x55c2030b0390 2024-09-21T03:39:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.862+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3923335986 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc3940319a0 con 0x55c2030b0390 2024-09-21T03:39:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.862+0000 7fc3b8c20740 10 client.26864 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.862+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3923335986 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c203500c60 con 0x55c2030b0390 2024-09-21T03:39:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.862+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3923335986 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc394005520 con 0x55c2030b0390 2024-09-21T03:39:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.862+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3923335986 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc3940322d0 con 0x55c2030b0390 2024-09-21T03:39:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.862+0000 7fc3b8c20740 10 client.26864 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.862+0000 7fc3b8c20740 10 client.26864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:39:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.862+0000 7fc3b8c20740 20 client.26864 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.862+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55c203064da0 0x55c2035032b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.862+0000 7fc3b8c20740 4 client.26864 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:39:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.862+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3923335986 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2033ec8a0 con 0x55c203064da0 2024-09-21T03:39:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.863+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55c203064da0 0x55c2035032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.863+0000 7fc38b7fe640 10 client.26864 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:39:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.863+0000 7fc3b0c49640 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55c203064da0 0x55c2035032b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17cf274be2fd9a0b server_cookie=6eff07dc16bf93b1 in_seq=0 out_seq=0 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.863+0000 7fc38b7fe640 1 -- 172.21.15.112:0/3923335986 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2033ec8a0 con 0x55c203064da0 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.863+0000 7fc38b7fe640 10 client.26864 handle_command_reply: tid=0 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.863+0000 7fc3b8c20740 1 client.26864 shutdown 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.863+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3923335986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030b0390 msgr2=0x55c2034885d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.863+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030b0390 0x55c2034885d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55c203492730 tx=0x7fc394000bc0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3923335986 shutdown_connections 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55c203064da0 0x55c2035032b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2030d1070 0x55c2034ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2030b0390 0x55c2034885d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc38b7fe640 0 client.26864 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b8c20740 1 --2- 172.21.15.112:0/3923335986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c2030a81b0 0x55c203488090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3923335986 >> 172.21.15.112:0/3923335986 conn(0x55c2030c5460 msgr2=0x55c203484e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:56.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3923335986 shutdown_connections 2024-09-21T03:39:56.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b8c20740 1 -- 172.21.15.112:0/3923335986 wait complete. 2024-09-21T03:39:56.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b8c20740 20 client.26864 trim_cache size 0 max 16384 2024-09-21T03:39:56.868 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:39:56.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b26ac640 1 -- 172.21.15.112:0/2507739371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3980b6680 msgr2=0x7fc3980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b26ac640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3980b6680 0x7fc3980b8b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0e4bb0 tx=0x7fc39c0eaed0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b26ac640 1 -- 172.21.15.112:0/2507739371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ac1027a0 msgr2=0x7fc3ac1a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b26ac640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ac1027a0 0x7fc3ac1a0b30 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00d950 tx=0x7fc39c00de10 comp rx=0 tx=0).stop 2024-09-21T03:39:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b26ac640 1 -- 172.21.15.112:0/2507739371 shutdown_connections 2024-09-21T03:39:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b26ac640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3980b6680 0x7fc3980b8b90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b26ac640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3ac107be0 0x7fc3ac1a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b26ac640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ac1030f0 0x7fc3ac1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b26ac640 1 --2- 172.21.15.112:0/2507739371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3ac1027a0 0x7fc3ac1a0b30 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b26ac640 1 -- 172.21.15.112:0/2507739371 >> 172.21.15.112:0/2507739371 conn(0x7fc3ac079730 msgr2=0x7fc3ac1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.864+0000 7fc3b26ac640 1 -- 172.21.15.112:0/2507739371 shutdown_connections 2024-09-21T03:39:56.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:56.865+0000 7fc3b26ac640 1 -- 172.21.15.112:0/2507739371 wait complete. 2024-09-21T03:39:57.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1161276 10.38 MB/sec execute 477 sec latency 22.323 ms 2024-09-21T03:39:57.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:39:57.942 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:58.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:57 smithi203 ceph-mon[34723]: pgmap v656: 97 pgs: 97 active+clean; 866 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 306 op/s 2024-09-21T03:39:58.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:57 smithi187 ceph-mon[34603]: pgmap v656: 97 pgs: 97 active+clean; 866 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 306 op/s 2024-09-21T03:39:58.230 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:57 smithi112 ceph-mon[31442]: pgmap v656: 97 pgs: 97 active+clean; 866 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.9 MiB/s wr, 306 op/s 2024-09-21T03:39:58.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1162349 10.37 MB/sec execute 478 sec latency 33.755 ms 2024-09-21T03:39:58.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.465+0000 7ff86e4b8640 1 -- 172.21.15.112:0/1249691415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff868102a60 msgr2=0x7ff868102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.465+0000 7ff86e4b8640 1 --2- 172.21.15.112:0/1249691415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff868102a60 0x7ff868102e70 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff858009900 tx=0x7ff85802a410 comp rx=0 tx=0).stop 2024-09-21T03:39:58.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.465+0000 7ff86e4b8640 1 -- 172.21.15.112:0/1249691415 shutdown_connections 2024-09-21T03:39:58.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.465+0000 7ff86e4b8640 1 --2- 172.21.15.112:0/1249691415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff868108cf0 0x7ff86810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.465+0000 7ff86e4b8640 1 --2- 172.21.15.112:0/1249691415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8681083a0 0x7ff8681087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.465+0000 7ff86e4b8640 1 --2- 172.21.15.112:0/1249691415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff868102a60 0x7ff868102e70 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.465+0000 7ff86e4b8640 1 -- 172.21.15.112:0/1249691415 >> 172.21.15.112:0/1249691415 conn(0x7ff868079730 msgr2=0x7ff868079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.465+0000 7ff86e4b8640 1 -- 172.21.15.112:0/1249691415 shutdown_connections 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.465+0000 7ff86e4b8640 1 -- 172.21.15.112:0/1249691415 wait complete. 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff86e4b8640 1 Processor -- start 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff86e4b8640 1 -- start start 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff86e4b8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff868102a60 0x7ff8681990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff86e4b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8681083a0 0x7ff868199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff86e4b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff868108cf0 0x7ff868199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff86e4b8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff868112960 con 0x7ff8681083a0 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff86e4b8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681127e0 con 0x7ff868108cf0 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff86e4b8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff868112ae0 con 0x7ff868102a60 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff867fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff868102a60 0x7ff8681990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff85ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8681083a0 0x7ff868199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff867fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff868102a60 0x7ff8681990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52594/0 (socket says 172.21.15.112:52594) 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff867fff640 1 -- 172.21.15.112:0/3697366016 learned_addr learned my addr 172.21.15.112:0/3697366016 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.466+0000 7ff86ca55640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff868108cf0 0x7ff868199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.467+0000 7ff85ffff640 1 -- 172.21.15.112:0/3697366016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff868102a60 msgr2=0x7ff8681990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.467+0000 7ff85ffff640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff868102a60 0x7ff8681990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.467+0000 7ff85ffff640 1 -- 172.21.15.112:0/3697366016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff868108cf0 msgr2=0x7ff868199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.467+0000 7ff85ffff640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff868108cf0 0x7ff868199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.467+0000 7ff85ffff640 1 -- 172.21.15.112:0/3697366016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8681a8740 con 0x7ff8681083a0 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.467+0000 7ff867fff640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff868102a60 0x7ff8681990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.467+0000 7ff85ffff640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8681083a0 0x7ff868199600 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff85000e9c0 tx=0x7ff85000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.467+0000 7ff865ffb640 1 -- 172.21.15.112:0/3697366016 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85000ccc0 con 0x7ff8681083a0 2024-09-21T03:39:58.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.467+0000 7ff86e4b8640 1 -- 172.21.15.112:0/3697366016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8681a89d0 con 0x7ff8681083a0 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.467+0000 7ff865ffb640 1 -- 172.21.15.112:0/3697366016 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff850005020 con 0x7ff8681083a0 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.467+0000 7ff865ffb640 1 -- 172.21.15.112:0/3697366016 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850004a00 con 0x7ff8681083a0 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.468+0000 7ff86e4b8640 1 -- 172.21.15.112:0/3697366016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8681a8f10 con 0x7ff8681083a0 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.470+0000 7ff874a2c740 1 Processor -- start 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.470+0000 7ff874a2c740 1 -- start start 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 0x557745ece6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745afe1b0 0x557745ecebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b06390 0x557745ecf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff874a2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557745e44ea0 con 0x557745b06390 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff874a2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557745e342d0 con 0x557745afe1b0 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff874a2c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557745e380e0 con 0x557745b27070 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff865ffb640 1 -- 172.21.15.112:0/3697366016 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff8500051d0 con 0x7ff8681083a0 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff86ca55640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 0x557745ece6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff86ca55640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 0x557745ece6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52600/0 (socket says 172.21.15.112:52600) 2024-09-21T03:39:58.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff86ca55640 1 -- 172.21.15.112:0/844382575 learned_addr learned my addr 172.21.15.112:0/844382575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff85ffff640 1 --2- 172.21.15.112:0/844382575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b06390 0x557745ecf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff867fff640 1 --2- 172.21.15.112:0/844382575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745afe1b0 0x557745ecebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff86ca55640 1 -- 172.21.15.112:0/844382575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745afe1b0 msgr2=0x557745ecebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff86ca55640 1 --2- 172.21.15.112:0/844382575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745afe1b0 0x557745ecebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff86ca55640 1 -- 172.21.15.112:0/844382575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b06390 msgr2=0x557745ecf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff86ca55640 1 --2- 172.21.15.112:0/844382575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b06390 0x557745ecf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff86ca55640 1 -- 172.21.15.112:0/844382575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557745b92650 con 0x557745b27070 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.471+0000 7ff86ca55640 1 --2- 172.21.15.112:0/844382575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 0x557745ece6a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff860007f40 tx=0x7ff86000f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8076f39b010d0b1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.472+0000 7ff865ffb640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff8540b6680 0x7ff8540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.472+0000 7ff85e7fc640 1 -- 172.21.15.112:0/844382575 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860010900 con 0x557745b27070 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.472+0000 7ff85e7fc640 1 -- 172.21.15.112:0/844382575 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8600183f0 con 0x557745b27070 2024-09-21T03:39:58.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.472+0000 7ff85e7fc640 1 -- 172.21.15.112:0/844382575 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860017600 con 0x557745b27070 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.472+0000 7ff867fff640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff8540b6680 0x7ff8540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.472+0000 7ff865ffb640 1 -- 172.21.15.112:0/3697366016 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7ff8500e7680 con 0x7ff8681083a0 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.473+0000 7ff874a2c740 1 -- 172.21.15.112:0/844382575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 msgr2=0x557745ece6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.473+0000 7ff867fff640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff8540b6680 0x7ff8540b8b90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff858009230 tx=0x7ff8580058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.473+0000 7ff874a2c740 1 --2- 172.21.15.112:0/844382575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 0x557745ece6a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff860007f40 tx=0x7ff86000f850 comp rx=0 tx=0).stop 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.473+0000 7ff874a2c740 1 -- 172.21.15.112:0/844382575 shutdown_connections 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.473+0000 7ff874a2c740 1 --2- 172.21.15.112:0/844382575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b06390 0x557745ecf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.473+0000 7ff874a2c740 1 --2- 172.21.15.112:0/844382575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745afe1b0 0x557745ecebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.473+0000 7ff874a2c740 1 --2- 172.21.15.112:0/844382575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 0x557745ece6a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.473+0000 7ff874a2c740 1 -- 172.21.15.112:0/844382575 >> 172.21.15.112:0/844382575 conn(0x557745b1b460 msgr2=0x557745ecc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.473+0000 7ff874a2c740 1 -- 172.21.15.112:0/844382575 shutdown_connections 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.473+0000 7ff874a2c740 1 -- 172.21.15.112:0/844382575 wait complete. 2024-09-21T03:39:58.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.473+0000 7ff874a2c740 1 -- start start 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff874a2c740 1 Processor -- start 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745ee2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b06390 0x557745ee3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 0x557745ee8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff874a2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557745c95700 con 0x557745afe1b0 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff874a2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557745dcf160 con 0x557745b06390 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff874a2c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557745edce80 con 0x557745b27070 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff874a2c740 10 client.0 _ll_register_callbacks cb 0x557745de89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff867fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745ee2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff867fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745ee2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49492/0 (socket says 172.21.15.112:49492) 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff867fff640 1 -- 172.21.15.112:0/2329988228 learned_addr learned my addr 172.21.15.112:0/2329988228 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff86ca55640 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 0x557745ee8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.474+0000 7ff85ffff640 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b06390 0x557745ee3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff86ca55640 1 -- 172.21.15.112:0/2329988228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b06390 msgr2=0x557745ee3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:58.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff86ca55640 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b06390 0x557745ee3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff86ca55640 1 -- 172.21.15.112:0/2329988228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 msgr2=0x557745ee2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:58.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff86ca55640 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745ee2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff86ca55640 1 -- 172.21.15.112:0/2329988228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557745dfcc90 con 0x557745b27070 2024-09-21T03:39:58.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff867fff640 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745ee2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:58.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff85ffff640 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b06390 0x557745ee3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:58.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff86ca55640 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 0x557745ee8460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff860000f80 tx=0x7ff860007320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=103b3344fc59733f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:58.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff8477fe640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:39:58.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff8477fe640 1 -- 172.21.15.112:0/2329988228 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860026020 con 0x557745b27070 2024-09-21T03:39:58.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff8477fe640 1 -- 172.21.15.112:0/2329988228 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7ff860003970 con 0x557745b27070 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff874a2c740 1 -- 172.21.15.112:0/2329988228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8600183f0 con 0x557745b27070 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff8477fe640 1 -- 172.21.15.112:0/2329988228 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff860031050 con 0x557745b27070 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff874a2c740 10 client.26878 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff874a2c740 1 -- 172.21.15.112:0/2329988228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557745ef13b0 con 0x557745b27070 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.475+0000 7ff8477fe640 1 -- 172.21.15.112:0/2329988228 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8600183f0 con 0x557745b27070 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.476+0000 7ff8477fe640 1 -- 172.21.15.112:0/2329988228 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff860004040 con 0x557745b27070 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.476+0000 7ff874a2c740 10 client.26878 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.476+0000 7ff874a2c740 10 client.26878 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.476+0000 7ff874a2c740 20 client.26878 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.476+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557745ae59f0 0x557745ef4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.476+0000 7ff874a2c740 4 client.26878 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.476+0000 7ff874a2c740 1 -- 172.21.15.112:0/2329988228 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557745e46df0 con 0x557745ae59f0 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.476+0000 7ff867fff640 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557745ae59f0 0x557745ef4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.476+0000 7ff8477fe640 10 client.26878 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.476+0000 7ff867fff640 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557745ae59f0 0x557745ef4880 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac32ac3c34cb3b2a server_cookie=ccc3456d1cfc3b08 in_seq=0 out_seq=0 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff8477fe640 1 -- 172.21.15.112:0/2329988228 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557745e46df0 con 0x557745ae59f0 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff8477fe640 10 client.26878 handle_command_reply: tid=0 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 1 client.26878 shutdown 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 1 -- 172.21.15.112:0/2329988228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 msgr2=0x557745ee8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 0x557745ee8460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff860000f80 tx=0x7ff860007320 comp rx=0 tx=0).stop 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 1 -- 172.21.15.112:0/2329988228 shutdown_connections 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557745ae59f0 0x557745ef4880 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b27070 0x557745ee8460 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b06390 0x557745ee3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2329988228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745ee2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff8477fe640 0 client.26878 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:39:58.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 1 -- 172.21.15.112:0/2329988228 >> 172.21.15.112:0/2329988228 conn(0x557745b1b460 msgr2=0x557745ed1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 1 -- 172.21.15.112:0/2329988228 shutdown_connections 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 1 -- 172.21.15.112:0/2329988228 wait complete. 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.477+0000 7ff874a2c740 20 client.26878 trim_cache size 0 max 16384 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff874a2c740 1 Processor -- start 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff874a2c740 1 -- start start 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745edc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b06390 0x557745ee79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b27070 0x557745ecf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff874a2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681afc50 con 0x557745afe1b0 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff874a2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ad520 con 0x557745b27070 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff874a2c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ab390 con 0x557745b06390 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff867fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745edc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff867fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745edc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49494/0 (socket says 172.21.15.112:49494) 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff867fff640 1 -- 172.21.15.112:0/3090953282 learned_addr learned my addr 172.21.15.112:0/3090953282 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff86ca55640 1 --2- 172.21.15.112:0/3090953282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b27070 0x557745ecf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.491+0000 7ff85ffff640 1 --2- 172.21.15.112:0/3090953282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b06390 0x557745ee79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.492+0000 7ff86ca55640 1 -- 172.21.15.112:0/3090953282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b06390 msgr2=0x557745ee79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.492+0000 7ff86ca55640 1 --2- 172.21.15.112:0/3090953282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b06390 0x557745ee79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.492+0000 7ff86ca55640 1 -- 172.21.15.112:0/3090953282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 msgr2=0x557745edc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.492+0000 7ff86ca55640 1 --2- 172.21.15.112:0/3090953282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745edc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.492+0000 7ff86ca55640 1 -- 172.21.15.112:0/3090953282 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557745ed0990 con 0x557745b27070 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.492+0000 7ff867fff640 1 --2- 172.21.15.112:0/3090953282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745edc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.492+0000 7ff86ca55640 1 --2- 172.21.15.112:0/3090953282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b27070 0x557745ecf6d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff860004440 tx=0x7ff86000fe00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9180f02b0a558dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.493+0000 7ff85dffb640 1 -- 172.21.15.112:0/3090953282 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860026070 con 0x557745b27070 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.493+0000 7ff85dffb640 1 -- 172.21.15.112:0/3090953282 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8600183f0 con 0x557745b27070 2024-09-21T03:39:58.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.493+0000 7ff85dffb640 1 -- 172.21.15.112:0/3090953282 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860004040 con 0x557745b27070 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.494+0000 7ff874a2c740 1 -- 172.21.15.112:0/3090953282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b27070 msgr2=0x557745ecf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.494+0000 7ff874a2c740 1 --2- 172.21.15.112:0/3090953282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b27070 0x557745ecf6d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff860004440 tx=0x7ff86000fe00 comp rx=0 tx=0).stop 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.494+0000 7ff874a2c740 1 -- 172.21.15.112:0/3090953282 shutdown_connections 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.494+0000 7ff874a2c740 1 --2- 172.21.15.112:0/3090953282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745b27070 0x557745ecf6d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.494+0000 7ff874a2c740 1 --2- 172.21.15.112:0/3090953282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b06390 0x557745ee79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.494+0000 7ff874a2c740 1 --2- 172.21.15.112:0/3090953282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745afe1b0 0x557745edc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.494+0000 7ff874a2c740 1 -- 172.21.15.112:0/3090953282 >> 172.21.15.112:0/3090953282 conn(0x557745b1b460 msgr2=0x557745ee72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.494+0000 7ff874a2c740 1 -- 172.21.15.112:0/3090953282 shutdown_connections 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.494+0000 7ff874a2c740 1 -- 172.21.15.112:0/3090953282 wait complete. 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.494+0000 7ff874a2c740 1 -- start start 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff874a2c740 1 Processor -- start 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745afe1b0 0x557745eeece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b06390 0x557745eef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff874a2c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b27070 0x557745ee5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff874a2c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ab9e0 con 0x557745b27070 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff874a2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ace50 con 0x557745afe1b0 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff874a2c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ac0b0 con 0x557745b06390 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff874a2c740 10 client.0 _ll_register_callbacks cb 0x7ff8500ed420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff86ca55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b27070 0x557745ee5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff867fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745afe1b0 0x557745eeece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff86ca55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b27070 0x557745ee5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49500/0 (socket says 172.21.15.112:49500) 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff86ca55640 1 -- 172.21.15.112:0/2199441486 learned_addr learned my addr 172.21.15.112:0/2199441486 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.495+0000 7ff85ffff640 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b06390 0x557745eef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff867fff640 1 -- 172.21.15.112:0/2199441486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b06390 msgr2=0x557745eef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff867fff640 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b06390 0x557745eef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff867fff640 1 -- 172.21.15.112:0/2199441486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b27070 msgr2=0x557745ee5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:39:58.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff867fff640 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b27070 0x557745ee5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff867fff640 1 -- 172.21.15.112:0/2199441486 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557745ee58c0 con 0x557745afe1b0 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff86ca55640 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b27070 0x557745ee5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff867fff640 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745afe1b0 0x557745eeece0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff858005940 tx=0x7ff858009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e611ec1baea0ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff8477fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff8477fe640 1 -- 172.21.15.112:0/2199441486 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858043070 con 0x557745afe1b0 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff874a2c740 1 -- 172.21.15.112:0/2199441486 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8600183f0 con 0x557745afe1b0 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff8477fe640 1 -- 172.21.15.112:0/2199441486 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7ff858005510 con 0x557745afe1b0 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff8477fe640 1 -- 172.21.15.112:0/2199441486 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff858035040 con 0x557745afe1b0 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff874a2c740 10 client.26879 fetch_fsmap learned FSMap version 30 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.496+0000 7ff874a2c740 1 -- 172.21.15.112:0/2199441486 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557745eee510 con 0x557745afe1b0 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.497+0000 7ff8477fe640 1 -- 172.21.15.112:0/2199441486 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8600183f0 con 0x557745afe1b0 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.497+0000 7ff8477fe640 1 -- 172.21.15.112:0/2199441486 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff858003950 con 0x557745afe1b0 2024-09-21T03:39:58.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.497+0000 7ff874a2c740 10 client.26879 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.497+0000 7ff874a2c740 10 client.26879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.497+0000 7ff874a2c740 20 client.26879 populate_metadata read hostname 'smithi112' 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.497+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557745abada0 0x557745f612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.497+0000 7ff874a2c740 4 client.26879 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.497+0000 7ff874a2c740 1 -- 172.21.15.112:0/2199441486 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557745e428a0 con 0x557745abada0 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.497+0000 7ff85ffff640 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557745abada0 0x557745f612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.497+0000 7ff8477fe640 10 client.26879 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.497+0000 7ff85ffff640 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557745abada0 0x557745f612f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90cd0c1e487b7c64 server_cookie=364bd67080ab9e8 in_seq=0 out_seq=0 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff8477fe640 1 -- 172.21.15.112:0/2199441486 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557745e428a0 con 0x557745abada0 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff8477fe640 10 client.26879 handle_command_reply: tid=0 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 1 client.26879 shutdown 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 1 -- 172.21.15.112:0/2199441486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745afe1b0 msgr2=0x557745eeece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745afe1b0 0x557745eeece0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff858005940 tx=0x7ff858009260 comp rx=0 tx=0).stop 2024-09-21T03:39:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 1 -- 172.21.15.112:0/2199441486 shutdown_connections 2024-09-21T03:39:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557745abada0 0x557745f612f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557745b27070 0x557745ee5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff8477fe640 0 client.26879 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:39:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557745b06390 0x557745eef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 1 --2- 172.21.15.112:0/2199441486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557745afe1b0 0x557745eeece0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 1 -- 172.21.15.112:0/2199441486 >> 172.21.15.112:0/2199441486 conn(0x557745b1b460 msgr2=0x557745ee72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 1 -- 172.21.15.112:0/2199441486 shutdown_connections 2024-09-21T03:39:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 1 -- 172.21.15.112:0/2199441486 wait complete. 2024-09-21T03:39:58.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.498+0000 7ff874a2c740 20 client.26879 trim_cache size 0 max 16384 2024-09-21T03:39:58.513 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:39:58.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 -- 172.21.15.112:0/3697366016 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff8540b6680 msgr2=0x7ff8540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff8540b6680 0x7ff8540b8b90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff858009230 tx=0x7ff8580058d0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 -- 172.21.15.112:0/3697366016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8681083a0 msgr2=0x7ff868199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:39:58.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8681083a0 0x7ff868199600 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff85000e9c0 tx=0x7ff85000ee80 comp rx=0 tx=0).stop 2024-09-21T03:39:58.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 -- 172.21.15.112:0/3697366016 shutdown_connections 2024-09-21T03:39:58.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff8540b6680 0x7ff8540b8b90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff868108cf0 0x7ff868199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8681083a0 0x7ff868199600 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 --2- 172.21.15.112:0/3697366016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff868102a60 0x7ff8681990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:39:58.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 -- 172.21.15.112:0/3697366016 >> 172.21.15.112:0/3697366016 conn(0x7ff868079730 msgr2=0x7ff868101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:39:58.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 -- 172.21.15.112:0/3697366016 shutdown_connections 2024-09-21T03:39:58.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:39:58.499+0000 7ff86e4b8640 1 -- 172.21.15.112:0/3697366016 wait complete. 2024-09-21T03:39:59.244 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:39:59.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1164983 10.38 MB/sec execute 479 sec latency 39.787 ms 2024-09-21T03:39:59.523 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:39:59.928 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:39:59 smithi112 ceph-mon[31442]: pgmap v657: 97 pgs: 97 active+clean; 866 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-09-21T03:40:00.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.020+0000 7f939eab6640 1 -- 172.21.15.112:0/2003948383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93981083a0 msgr2=0x7f93981087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.020+0000 7f939eab6640 1 --2- 172.21.15.112:0/2003948383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93981083a0 0x7f93981087b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9384009900 tx=0x7f938402a410 comp rx=0 tx=0).stop 2024-09-21T03:40:00.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.020+0000 7f939eab6640 1 -- 172.21.15.112:0/2003948383 shutdown_connections 2024-09-21T03:40:00.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.020+0000 7f939eab6640 1 --2- 172.21.15.112:0/2003948383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9398108cf0 0x7f939810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.020+0000 7f939eab6640 1 --2- 172.21.15.112:0/2003948383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93981083a0 0x7f93981087b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.020+0000 7f939eab6640 1 --2- 172.21.15.112:0/2003948383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9398102a60 0x7f9398102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.020+0000 7f939eab6640 1 -- 172.21.15.112:0/2003948383 >> 172.21.15.112:0/2003948383 conn(0x7f9398079730 msgr2=0x7f9398079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:00.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.020+0000 7f939eab6640 1 -- 172.21.15.112:0/2003948383 shutdown_connections 2024-09-21T03:40:00.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.020+0000 7f939eab6640 1 -- 172.21.15.112:0/2003948383 wait complete. 2024-09-21T03:40:00.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.021+0000 7f939eab6640 1 Processor -- start 2024-09-21T03:40:00.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.021+0000 7f939eab6640 1 -- start start 2024-09-21T03:40:00.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.021+0000 7f939eab6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9398102a60 0x7f939819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.021+0000 7f939eab6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93981083a0 0x7f939819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.021+0000 7f939eab6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398108cf0 0x7f939819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.021+0000 7f939eab6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9398113a80 con 0x7f9398108cf0 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.021+0000 7f939eab6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9398113900 con 0x7f9398102a60 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.021+0000 7f939eab6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9398113c00 con 0x7f93981083a0 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f939d053640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398108cf0 0x7f939819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f939c852640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9398102a60 0x7f939819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f939d053640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398108cf0 0x7f939819fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49512/0 (socket says 172.21.15.112:49512) 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f9397fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93981083a0 0x7f939819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f939d053640 1 -- 172.21.15.112:0/2191413304 learned_addr learned my addr 172.21.15.112:0/2191413304 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f939d053640 1 -- 172.21.15.112:0/2191413304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93981083a0 msgr2=0x7f939819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f939d053640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93981083a0 0x7f939819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f939d053640 1 -- 172.21.15.112:0/2191413304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9398102a60 msgr2=0x7f939819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f939d053640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9398102a60 0x7f939819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f939d053640 1 -- 172.21.15.112:0/2191413304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93981a0500 con 0x7f9398108cf0 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f939d053640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398108cf0 0x7f939819fdd0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f939000eb10 tx=0x7f939000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f9395ffb640 1 -- 172.21.15.112:0/2191413304 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390004b20 con 0x7f9398108cf0 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f939eab6640 1 -- 172.21.15.112:0/2191413304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93981a07f0 con 0x7f9398108cf0 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f9395ffb640 1 -- 172.21.15.112:0/2191413304 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9390004cc0 con 0x7f9398108cf0 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.022+0000 7f9395ffb640 1 -- 172.21.15.112:0/2191413304 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390010670 con 0x7f9398108cf0 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.023+0000 7f939eab6640 1 -- 172.21.15.112:0/2191413304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93981a6f50 con 0x7f9398108cf0 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f93a502a740 1 Processor -- start 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f9395ffb640 1 -- 172.21.15.112:0/2191413304 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9390005020 con 0x7f9398108cf0 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f93a502a740 1 -- start start 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f93a502a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c31548a070 0x55c3158316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f93a502a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315831be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f93a502a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c315469390 0x55c315832120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f93a502a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c3157a7ea0 con 0x55c315469390 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f93a502a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c3157972d0 con 0x55c31548a070 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f93a502a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c31579b0e0 con 0x55c3154611b0 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f939c852640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315831be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f9397fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c315469390 0x55c315832120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f939c852640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315831be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52654/0 (socket says 172.21.15.112:52654) 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f939c852640 1 -- 172.21.15.112:0/614640716 learned_addr learned my addr 172.21.15.112:0/614640716 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f9397fff640 1 --2- 172.21.15.112:0/614640716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c31548a070 0x55c3158316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f9395ffb640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f93880b6680 0x7f93880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f939c852640 1 -- 172.21.15.112:0/614640716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c31548a070 msgr2=0x55c3158316a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.026+0000 7f939c852640 1 --2- 172.21.15.112:0/614640716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c31548a070 0x55c3158316a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.027+0000 7f939c852640 1 -- 172.21.15.112:0/614640716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c315469390 msgr2=0x55c315832120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:00.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.027+0000 7f939c852640 1 --2- 172.21.15.112:0/614640716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c315469390 0x55c315832120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.027+0000 7f939c852640 1 -- 172.21.15.112:0/614640716 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3154f5650 con 0x55c3154611b0 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.027+0000 7f939d053640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f93880b6680 0x7f93880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.027+0000 7f9395ffb640 1 -- 172.21.15.112:0/2191413304 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f93900e7c80 con 0x7f9398108cf0 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.027+0000 7f939c852640 1 --2- 172.21.15.112:0/614640716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315831be0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f938c009d20 tx=0x7f938c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef47a96f566ed04f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.027+0000 7f9382ffd640 1 -- 172.21.15.112:0/614640716 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c01e070 con 0x55c3154611b0 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.027+0000 7f9382ffd640 1 -- 172.21.15.112:0/614640716 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c009e10 con 0x55c3154611b0 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.027+0000 7f9382ffd640 1 -- 172.21.15.112:0/614640716 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0187a0 con 0x55c3154611b0 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.028+0000 7f93a502a740 1 -- 172.21.15.112:0/614640716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 msgr2=0x55c315831be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.028+0000 7f939d053640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f93880b6680 0x7f93880b8b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f93900ad800 tx=0x7f93900a2360 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.028+0000 7f93a502a740 1 --2- 172.21.15.112:0/614640716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315831be0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f938c009d20 tx=0x7f938c011810 comp rx=0 tx=0).stop 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.028+0000 7f93a502a740 1 -- 172.21.15.112:0/614640716 shutdown_connections 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.028+0000 7f93a502a740 1 --2- 172.21.15.112:0/614640716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c315469390 0x55c315832120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.028+0000 7f93a502a740 1 --2- 172.21.15.112:0/614640716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315831be0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.028+0000 7f93a502a740 1 --2- 172.21.15.112:0/614640716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c31548a070 0x55c3158316a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.028+0000 7f93a502a740 1 -- 172.21.15.112:0/614640716 >> 172.21.15.112:0/614640716 conn(0x55c31547e460 msgr2=0x55c31582f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.028+0000 7f93a502a740 1 -- 172.21.15.112:0/614640716 shutdown_connections 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.028+0000 7f93a502a740 1 -- 172.21.15.112:0/614640716 wait complete. 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f93a502a740 1 -- start start 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f93a502a740 1 Processor -- start 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f93a502a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3154611b0 0x55c31583cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f93a502a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c315469390 0x55c31583d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f93a502a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 0x55c3158491a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f93a502a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c3155f8700 con 0x55c31548a070 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f93a502a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c315732160 con 0x55c3154611b0 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f93a502a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c31583fe80 con 0x55c315469390 2024-09-21T03:40:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f93a502a740 10 client.0 _ll_register_callbacks cb 0x55c31574b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f9397fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 0x55c3158491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f9397fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 0x55c3158491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49538/0 (socket says 172.21.15.112:49538) 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f9397fff640 1 -- 172.21.15.112:0/3680215703 learned_addr learned my addr 172.21.15.112:0/3680215703 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f939c852640 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c315469390 0x55c31583d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.029+0000 7f9397fff640 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3154611b0 0x55c31583cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f9397fff640 1 -- 172.21.15.112:0/3680215703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c315469390 msgr2=0x55c31583d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f9397fff640 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c315469390 0x55c31583d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f9397fff640 1 -- 172.21.15.112:0/3680215703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3154611b0 msgr2=0x55c31583cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f9397fff640 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3154611b0 0x55c31583cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f9397fff640 1 -- 172.21.15.112:0/3680215703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31575fc90 con 0x55c31548a070 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f9397fff640 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 0x55c3158491a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x55c315833b40 tx=0x7f938402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c54eca205cce49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f9377fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:40:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f9377fff640 1 -- 172.21.15.112:0/3680215703 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384035040 con 0x55c31548a070 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f9377fff640 1 -- 172.21.15.112:0/3680215703 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f9384004330 con 0x55c31548a070 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f93a502a740 1 -- 172.21.15.112:0/3680215703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938c009e10 con 0x55c31548a070 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f9377fff640 1 -- 172.21.15.112:0/3680215703 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f93840035f0 con 0x55c31548a070 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f93a502a740 10 client.17337 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f93a502a740 1 -- 172.21.15.112:0/3680215703 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c31584bba0 con 0x55c31548a070 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.030+0000 7f9377fff640 1 -- 172.21.15.112:0/3680215703 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c009e10 con 0x55c31548a070 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.031+0000 7f9377fff640 1 -- 172.21.15.112:0/3680215703 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f938403d5e0 con 0x55c31548a070 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.031+0000 7f93a502a740 10 client.17337 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.031+0000 7f93a502a740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.031+0000 7f93a502a740 20 client.17337 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.031+0000 7f93a502a740 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c3154489f0 0x55c31584f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.031+0000 7f93a502a740 4 client.17337 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.031+0000 7f93a502a740 1 -- 172.21.15.112:0/3680215703 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3157a9df0 con 0x55c3154489f0 2024-09-21T03:40:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.031+0000 7f939c852640 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c3154489f0 0x55c31584f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.031+0000 7f939c852640 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c3154489f0 0x55c31584f0e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68cf0795bcbb6577 server_cookie=b312c470ddf5608a in_seq=0 out_seq=0 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.031+0000 7f9377fff640 10 client.17337 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f9377fff640 1 -- 172.21.15.112:0/3680215703 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c3157a9df0 con 0x55c3154489f0 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f9377fff640 10 client.17337 handle_command_reply: tid=0 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 1 client.17337 shutdown 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 1 -- 172.21.15.112:0/3680215703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 msgr2=0x55c3158491a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 0x55c3158491a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x55c315833b40 tx=0x7f938402a410 comp rx=0 tx=0).stop 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 1 -- 172.21.15.112:0/3680215703 shutdown_connections 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c3154489f0 0x55c31584f0e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 0x55c3158491a0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c315469390 0x55c31583d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f9377fff640 0 client.17337 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 1 --2- 172.21.15.112:0/3680215703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c3154611b0 0x55c31583cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 1 -- 172.21.15.112:0/3680215703 >> 172.21.15.112:0/3680215703 conn(0x55c31547e460 msgr2=0x55c315834830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 1 -- 172.21.15.112:0/3680215703 shutdown_connections 2024-09-21T03:40:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 1 -- 172.21.15.112:0/3680215703 wait complete. 2024-09-21T03:40:00.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.033+0000 7f93a502a740 20 client.17337 trim_cache size 0 max 16384 2024-09-21T03:40:00.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.047+0000 7f93a502a740 1 Processor -- start 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.047+0000 7f93a502a740 1 -- start start 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.047+0000 7f93a502a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315833090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f93a502a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c315469390 0x55c31583e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f93a502a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 0x55c31582ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f93a502a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93981b1fc0 con 0x55c31548a070 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f93a502a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93981af890 con 0x55c315469390 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f93a502a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f93981ad700 con 0x55c3154611b0 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f9397fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 0x55c31582ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f9397fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315833090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f9397fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315833090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52670/0 (socket says 172.21.15.112:52670) 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f9397fff640 1 -- 172.21.15.112:0/1601420964 learned_addr learned my addr 172.21.15.112:0/1601420964 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:00.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f939c852640 1 --2- 172.21.15.112:0/1601420964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c315469390 0x55c31583e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f9397fff640 1 -- 172.21.15.112:0/1601420964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c315469390 msgr2=0x55c31583e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:00.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f9397fff640 1 --2- 172.21.15.112:0/1601420964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c315469390 0x55c31583e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f9397fff640 1 -- 172.21.15.112:0/1601420964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 msgr2=0x55c31582ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:00.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f9397fff640 1 --2- 172.21.15.112:0/1601420964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 0x55c31582ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f9397fff640 1 -- 172.21.15.112:0/1601420964 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c315834500 con 0x55c3154611b0 2024-09-21T03:40:00.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.048+0000 7f9397fff640 1 --2- 172.21.15.112:0/1601420964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315833090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9384031870 tx=0x7f9384005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=573a0f66ecab88d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:00.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.049+0000 7f93827fc640 1 -- 172.21.15.112:0/1601420964 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384035070 con 0x55c3154611b0 2024-09-21T03:40:00.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.049+0000 7f93827fc640 1 -- 172.21.15.112:0/1601420964 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9384043040 con 0x55c3154611b0 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.049+0000 7f93827fc640 1 -- 172.21.15.112:0/1601420964 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384059040 con 0x55c3154611b0 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.050+0000 7f93a502a740 1 -- 172.21.15.112:0/1601420964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 msgr2=0x55c315833090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.050+0000 7f93a502a740 1 --2- 172.21.15.112:0/1601420964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315833090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9384031870 tx=0x7f9384005fd0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.050+0000 7f93a502a740 1 -- 172.21.15.112:0/1601420964 shutdown_connections 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.050+0000 7f93a502a740 1 --2- 172.21.15.112:0/1601420964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c31548a070 0x55c31582ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.050+0000 7f93a502a740 1 --2- 172.21.15.112:0/1601420964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c315469390 0x55c31583e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.050+0000 7f93a502a740 1 --2- 172.21.15.112:0/1601420964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3154611b0 0x55c315833090 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.050+0000 7f93a502a740 1 -- 172.21.15.112:0/1601420964 >> 172.21.15.112:0/1601420964 conn(0x55c31547e460 msgr2=0x55c31583e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.050+0000 7f93a502a740 1 -- 172.21.15.112:0/1601420964 shutdown_connections 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.050+0000 7f93a502a740 1 -- 172.21.15.112:0/1601420964 wait complete. 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.050+0000 7f93a502a740 1 -- start start 2024-09-21T03:40:00.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.050+0000 7f93a502a740 1 Processor -- start 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f93a502a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3154611b0 0x55c315845570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f93a502a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c315469390 0x55c315845b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f93a502a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c31548a070 0x55c315840bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f93a502a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93981add50 con 0x55c3154611b0 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f93a502a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93981af1c0 con 0x55c315469390 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f93a502a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f93981ae420 con 0x55c31548a070 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f93a502a740 10 client.0 _ll_register_callbacks cb 0x7f9384031760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f9397fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3154611b0 0x55c315845570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f939c852640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c315469390 0x55c315845b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f9397fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3154611b0 0x55c315845570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49554/0 (socket says 172.21.15.112:49554) 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f9397fff640 1 -- 172.21.15.112:0/2542327405 learned_addr learned my addr 172.21.15.112:0/2542327405 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f9397fff640 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c31548a070 0x55c315840bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f939c852640 1 -- 172.21.15.112:0/2542327405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c31548a070 msgr2=0x55c315840bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f939c852640 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c31548a070 0x55c315840bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f939c852640 1 -- 172.21.15.112:0/2542327405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3154611b0 msgr2=0x55c315845570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f939c852640 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3154611b0 0x55c315845570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f939c852640 1 -- 172.21.15.112:0/2542327405 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c315841380 con 0x55c315469390 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.051+0000 7f9397fff640 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3154611b0 0x55c315845570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.052+0000 7f939c852640 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c315469390 0x55c315845b30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f938c01c040 tx=0x7f938c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a36227aaf4aead36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.052+0000 7f9377fff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.052+0000 7f9377fff640 1 -- 172.21.15.112:0/2542327405 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c02f020 con 0x55c315469390 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.052+0000 7f9377fff640 1 -- 172.21.15.112:0/2542327405 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f938c005390 con 0x55c315469390 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.052+0000 7f93a502a740 1 -- 172.21.15.112:0/2542327405 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9384043040 con 0x55c315469390 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.052+0000 7f9377fff640 1 -- 172.21.15.112:0/2542327405 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f938c03a050 con 0x55c315469390 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.052+0000 7f93a502a740 10 client.26897 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.052+0000 7f93a502a740 1 -- 172.21.15.112:0/2542327405 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c3158b96c0 con 0x55c315469390 2024-09-21T03:40:00.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.052+0000 7f9377fff640 1 -- 172.21.15.112:0/2542327405 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9384043040 con 0x55c315469390 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.053+0000 7f9377fff640 1 -- 172.21.15.112:0/2542327405 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f938c0364a0 con 0x55c315469390 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.053+0000 7f93a502a740 10 client.26897 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.053+0000 7f93a502a740 10 client.26897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.053+0000 7f93a502a740 20 client.26897 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.053+0000 7f93a502a740 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c31541dda0 0x55c3158bbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.053+0000 7f93a502a740 4 client.26897 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.053+0000 7f93a502a740 1 -- 172.21.15.112:0/2542327405 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3157a58a0 con 0x55c31541dda0 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.053+0000 7f9397fff640 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c31541dda0 0x55c3158bbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.053+0000 7f9377fff640 10 client.26897 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f9397fff640 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c31541dda0 0x55c3158bbd40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef20a3f4d37e370 server_cookie=dd8450793d70e355 in_seq=0 out_seq=0 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f9377fff640 1 -- 172.21.15.112:0/2542327405 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3157a58a0 con 0x55c31541dda0 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f9377fff640 10 client.26897 handle_command_reply: tid=0 2024-09-21T03:40:00.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f93a502a740 1 client.26897 shutdown 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f93a502a740 1 -- 172.21.15.112:0/2542327405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c315469390 msgr2=0x55c315845b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f93a502a740 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c315469390 0x55c315845b30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f938c01c040 tx=0x7f938c005dc0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f93a502a740 1 -- 172.21.15.112:0/2542327405 shutdown_connections 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f93a502a740 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c31541dda0 0x55c3158bbd40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f93a502a740 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c31548a070 0x55c315840bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f93a502a740 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c315469390 0x55c315845b30 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f938c01c040 tx=0x7f938c005dc0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f93a502a740 1 --2- 172.21.15.112:0/2542327405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3154611b0 0x55c315845570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f9377fff640 0 client.26897 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f93a502a740 1 -- 172.21.15.112:0/2542327405 >> 172.21.15.112:0/2542327405 conn(0x55c31547e460 msgr2=0x55c315832e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.054+0000 7f93a502a740 1 -- 172.21.15.112:0/2542327405 shutdown_connections 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.055+0000 7f93a502a740 1 -- 172.21.15.112:0/2542327405 wait complete. 2024-09-21T03:40:00.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.055+0000 7f93a502a740 20 client.26897 trim_cache size 0 max 16384 2024-09-21T03:40:00.061 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:40:00.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.055+0000 7f939eab6640 1 -- 172.21.15.112:0/2191413304 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f93880b6680 msgr2=0x7f93880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.055+0000 7f939eab6640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f93880b6680 0x7f93880b8b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f93900ad800 tx=0x7f93900a2360 comp rx=0 tx=0).stop 2024-09-21T03:40:00.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.055+0000 7f939eab6640 1 -- 172.21.15.112:0/2191413304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398108cf0 msgr2=0x7f939819fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.055+0000 7f939eab6640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398108cf0 0x7f939819fdd0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f939000eb10 tx=0x7f939000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.056+0000 7f939eab6640 1 -- 172.21.15.112:0/2191413304 shutdown_connections 2024-09-21T03:40:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.056+0000 7f939eab6640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f93880b6680 0x7f93880b8b90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.056+0000 7f939eab6640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9398108cf0 0x7f939819fdd0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.056+0000 7f939eab6640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93981083a0 0x7f939819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.056+0000 7f939eab6640 1 --2- 172.21.15.112:0/2191413304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9398102a60 0x7f939819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.056+0000 7f939eab6640 1 -- 172.21.15.112:0/2191413304 >> 172.21.15.112:0/2191413304 conn(0x7f9398079730 msgr2=0x7f93981044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.056+0000 7f939eab6640 1 -- 172.21.15.112:0/2191413304 shutdown_connections 2024-09-21T03:40:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:00.056+0000 7f939eab6640 1 -- 172.21.15.112:0/2191413304 wait complete. 2024-09-21T03:40:00.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:39:59 smithi203 ceph-mon[34723]: pgmap v657: 97 pgs: 97 active+clean; 866 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-09-21T03:40:00.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:39:59 smithi187 ceph-mon[34603]: pgmap v657: 97 pgs: 97 active+clean; 866 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-09-21T03:40:00.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1165904 10.37 MB/sec execute 480 sec latency 91.614 ms 2024-09-21T03:40:01.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:00 smithi112 ceph-mon[31442]: overall HEALTH_OK 2024-09-21T03:40:01.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:00 smithi203 ceph-mon[34723]: overall HEALTH_OK 2024-09-21T03:40:01.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:00 smithi187 ceph-mon[34603]: overall HEALTH_OK 2024-09-21T03:40:01.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1167305 10.36 MB/sec execute 481 sec latency 63.802 ms 2024-09-21T03:40:01.872 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:40:01.872 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:40:02.149 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:02.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:01 smithi203 ceph-mon[34723]: pgmap v658: 97 pgs: 97 active+clean; 869 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 255 op/s 2024-09-21T03:40:02.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:01 smithi187 ceph-mon[34603]: pgmap v658: 97 pgs: 97 active+clean; 869 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 255 op/s 2024-09-21T03:40:02.272 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:01 smithi112 ceph-mon[31442]: pgmap v658: 97 pgs: 97 active+clean; 869 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 255 op/s 2024-09-21T03:40:02.272 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1169021 10.36 MB/sec execute 482 sec latency 27.645 ms 2024-09-21T03:40:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.815+0000 7fa7da2b5640 1 -- 172.21.15.112:0/2682197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d4102ae0 msgr2=0x7fa7d4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.815+0000 7fa7da2b5640 1 --2- 172.21.15.112:0/2682197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d4102ae0 0x7fa7d4102ef0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009860 tx=0x7fa7c402a340 comp rx=0 tx=0).stop 2024-09-21T03:40:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.815+0000 7fa7da2b5640 1 -- 172.21.15.112:0/2682197 shutdown_connections 2024-09-21T03:40:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.815+0000 7fa7da2b5640 1 --2- 172.21.15.112:0/2682197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d4108d70 0x7fa7d410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.815+0000 7fa7da2b5640 1 --2- 172.21.15.112:0/2682197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7d4108420 0x7fa7d4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.815+0000 7fa7da2b5640 1 --2- 172.21.15.112:0/2682197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d4102ae0 0x7fa7d4102ef0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.815+0000 7fa7da2b5640 1 -- 172.21.15.112:0/2682197 >> 172.21.15.112:0/2682197 conn(0x7fa7d4079770 msgr2=0x7fa7d4079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.815+0000 7fa7da2b5640 1 -- 172.21.15.112:0/2682197 shutdown_connections 2024-09-21T03:40:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.815+0000 7fa7da2b5640 1 -- 172.21.15.112:0/2682197 wait complete. 2024-09-21T03:40:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7da2b5640 1 Processor -- start 2024-09-21T03:40:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7da2b5640 1 -- start start 2024-09-21T03:40:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7da2b5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7d4102ae0 0x7fa7d4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7da2b5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d4108420 0x7fa7d4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7da2b5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d4108d70 0x7fa7d419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7da2b5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d4112a30 con 0x7fa7d4108420 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7da2b5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41128b0 con 0x7fa7d4102ae0 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7da2b5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d4112bb0 con 0x7fa7d4108d70 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d4108420 0x7fa7d4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d4108420 0x7fa7d4199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49566/0 (socket says 172.21.15.112:49566) 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7cbfff640 1 -- 172.21.15.112:0/1754848282 learned_addr learned my addr 172.21.15.112:0/1754848282 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7d8852640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d4108d70 0x7fa7d419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7d4102ae0 0x7fa7d4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7cbfff640 1 -- 172.21.15.112:0/1754848282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d4108d70 msgr2=0x7fa7d419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7cbfff640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d4108d70 0x7fa7d419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7cbfff640 1 -- 172.21.15.112:0/1754848282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7d4102ae0 msgr2=0x7fa7d4199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7cbfff640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7d4102ae0 0x7fa7d4199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7cbfff640 1 -- 172.21.15.112:0/1754848282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d419f3d0 con 0x7fa7d4108420 2024-09-21T03:40:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.816+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7d4102ae0 0x7fa7d4199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.817+0000 7fa7cbfff640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d4108420 0x7fa7d4199a00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00e9f0 tx=0x7fa7bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.817+0000 7fa7d1ffb640 1 -- 172.21.15.112:0/1754848282 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc00ccc0 con 0x7fa7d4108420 2024-09-21T03:40:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.817+0000 7fa7da2b5640 1 -- 172.21.15.112:0/1754848282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d419f6c0 con 0x7fa7d4108420 2024-09-21T03:40:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.817+0000 7fa7d1ffb640 1 -- 172.21.15.112:0/1754848282 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7bc0108e0 con 0x7fa7d4108420 2024-09-21T03:40:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.817+0000 7fa7da2b5640 1 -- 172.21.15.112:0/1754848282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d41a5e20 con 0x7fa7d4108420 2024-09-21T03:40:02.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.817+0000 7fa7d1ffb640 1 -- 172.21.15.112:0/1754848282 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc018860 con 0x7fa7d4108420 2024-09-21T03:40:02.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.819+0000 7fa7e0829740 1 Processor -- start 2024-09-21T03:40:02.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.819+0000 7fa7e0829740 1 -- start start 2024-09-21T03:40:02.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 0x5559f6fc9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fc9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7e0829740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5559f6f3de60 con 0x5559f6bff670 2024-09-21T03:40:02.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7e0829740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5559f6f2d290 con 0x5559f6c20350 2024-09-21T03:40:02.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7e0829740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5559f6f310a0 con 0x5559f6bf7490 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7d1ffb640 1 -- 172.21.15.112:0/1754848282 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa7bc010430 con 0x7fa7d4108420 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7d8852640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 0x5559f6fc9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7d8852640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 0x5559f6fc9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:43960/0 (socket says 172.21.15.112:43960) 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7d8852640 1 -- 172.21.15.112:0/595979281 learned_addr learned my addr 172.21.15.112:0/595979281 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7cbfff640 1 --2- 172.21.15.112:0/595979281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/595979281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fc9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7d8852640 1 -- 172.21.15.112:0/595979281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 msgr2=0x5559f6fc9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7d8852640 1 --2- 172.21.15.112:0/595979281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fc9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7d8852640 1 -- 172.21.15.112:0/595979281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 msgr2=0x5559f6fca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7d8852640 1 --2- 172.21.15.112:0/595979281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7d8852640 1 -- 172.21.15.112:0/595979281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f6c8b930 con 0x5559f6c20350 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.820+0000 7fa7d8852640 1 --2- 172.21.15.112:0/595979281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 0x5559f6fc9640 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00e450 tx=0x7fa7cc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=537aa72064fb4d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:02.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.821+0000 7fa7d1ffb640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa7c00b6680 0x7fa7c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.821+0000 7fa7ca7fc640 1 -- 172.21.15.112:0/595979281 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00d910 con 0x5559f6c20350 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.821+0000 7fa7ca7fc640 1 -- 172.21.15.112:0/595979281 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7cc01c070 con 0x5559f6c20350 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.821+0000 7fa7ca7fc640 1 -- 172.21.15.112:0/595979281 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc017e10 con 0x5559f6c20350 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.821+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa7c00b6680 0x7fa7c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.821+0000 7fa7d1ffb640 1 -- 172.21.15.112:0/1754848282 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fa7bc0e7a80 con 0x7fa7d4108420 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.822+0000 7fa7e0829740 1 -- 172.21.15.112:0/595979281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 msgr2=0x5559f6fc9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.822+0000 7fa7e0829740 1 --2- 172.21.15.112:0/595979281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 0x5559f6fc9640 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00e450 tx=0x7fa7cc0095c0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.822+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa7c00b6680 0x7fa7c00b8b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4000c00 tx=0x7fa7c4009490 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.822+0000 7fa7e0829740 1 -- 172.21.15.112:0/595979281 shutdown_connections 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.822+0000 7fa7e0829740 1 --2- 172.21.15.112:0/595979281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.822+0000 7fa7e0829740 1 --2- 172.21.15.112:0/595979281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fc9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.822+0000 7fa7e0829740 1 --2- 172.21.15.112:0/595979281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 0x5559f6fc9640 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.822+0000 7fa7e0829740 1 -- 172.21.15.112:0/595979281 >> 172.21.15.112:0/595979281 conn(0x5559f6c14740 msgr2=0x5559f6fc7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.822+0000 7fa7e0829740 1 -- 172.21.15.112:0/595979281 shutdown_connections 2024-09-21T03:40:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.822+0000 7fa7e0829740 1 -- 172.21.15.112:0/595979281 wait complete. 2024-09-21T03:40:02.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.822+0000 7fa7e0829740 1 -- start start 2024-09-21T03:40:02.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7e0829740 1 Processor -- start 2024-09-21T03:40:02.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fd3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 0x5559f6fdd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7e0829740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5559f6d8e890 con 0x5559f6bff670 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7e0829740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5559f6ec7d40 con 0x5559f6c20350 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7e0829740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5559f6fc4db0 con 0x5559f6bf7490 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7e0829740 10 client.0 _ll_register_callbacks cb 0x5559f6f2bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49582/0 (socket says 172.21.15.112:49582) 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7d8852640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 0x5559f6fdd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7cbfff640 1 -- 172.21.15.112:0/135494916 learned_addr learned my addr 172.21.15.112:0/135494916 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fd3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7d8852640 1 -- 172.21.15.112:0/135494916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 msgr2=0x5559f6fd3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7d8852640 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fd3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7d8852640 1 -- 172.21.15.112:0/135494916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 msgr2=0x5559f6fd3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:02.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7d8852640 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7d8852640 1 -- 172.21.15.112:0/135494916 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f6fdd700 con 0x5559f6c20350 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.823+0000 7fa7cbfff640 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.824+0000 7fa7d8852640 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 0x5559f6fdd090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009c20 tx=0x7fa7cc017e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad3c40ed68a70c0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.824+0000 7fa7b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.824+0000 7fa7b37fe640 1 -- 172.21.15.112:0/135494916 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc017620 con 0x5559f6c20350 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.824+0000 7fa7b37fe640 1 -- 172.21.15.112:0/135494916 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fa7cc0177c0 con 0x5559f6c20350 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.824+0000 7fa7e0829740 1 -- 172.21.15.112:0/135494916 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7cc01c070 con 0x5559f6c20350 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.824+0000 7fa7b37fe640 1 -- 172.21.15.112:0/135494916 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7cc003970 con 0x5559f6c20350 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.824+0000 7fa7e0829740 10 client.26909 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.824+0000 7fa7e0829740 1 -- 172.21.15.112:0/135494916 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559f6fe61a0 con 0x5559f6c20350 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.824+0000 7fa7b37fe640 1 -- 172.21.15.112:0/135494916 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7cc01c070 con 0x5559f6c20350 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.825+0000 7fa7b37fe640 1 -- 172.21.15.112:0/135494916 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa7cc00b040 con 0x5559f6c20350 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.825+0000 7fa7e0829740 10 client.26909 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:02.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.825+0000 7fa7e0829740 10 client.26909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.825+0000 7fa7e0829740 20 client.26909 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.825+0000 7fa7e0829740 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5559f6bdecd0 0x5559f6fe9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.825+0000 7fa7e0829740 4 client.26909 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.825+0000 7fa7e0829740 1 -- 172.21.15.112:0/135494916 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559f6f3fdb0 con 0x5559f6bdecd0 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.825+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5559f6bdecd0 0x5559f6fe9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.825+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5559f6bdecd0 0x5559f6fe9610 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d35e8044761d2b server_cookie=4d0bd3e452da832e in_seq=0 out_seq=0 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.825+0000 7fa7b37fe640 10 client.26909 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7b37fe640 1 -- 172.21.15.112:0/135494916 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5559f6f3fdb0 con 0x5559f6bdecd0 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7b37fe640 10 client.26909 handle_command_reply: tid=0 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7e0829740 1 client.26909 shutdown 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7e0829740 1 -- 172.21.15.112:0/135494916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 msgr2=0x5559f6fdd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7e0829740 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 0x5559f6fdd090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009c20 tx=0x7fa7cc017e40 comp rx=0 tx=0).stop 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7e0829740 1 -- 172.21.15.112:0/135494916 shutdown_connections 2024-09-21T03:40:02.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7e0829740 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5559f6bdecd0 0x5559f6fe9610 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7e0829740 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6c20350 0x5559f6fdd090 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7b37fe640 0 client.26909 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:02.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7e0829740 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7e0829740 1 --2- 172.21.15.112:0/135494916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fd3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7e0829740 1 -- 172.21.15.112:0/135494916 >> 172.21.15.112:0/135494916 conn(0x5559f6c14740 msgr2=0x5559f6fca670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:02.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7e0829740 1 -- 172.21.15.112:0/135494916 shutdown_connections 2024-09-21T03:40:02.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.826+0000 7fa7e0829740 1 -- 172.21.15.112:0/135494916 wait complete. 2024-09-21T03:40:02.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.827+0000 7fa7e0829740 20 client.26909 trim_cache size 0 max 16384 2024-09-21T03:40:02.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.845+0000 7fa7e0829740 1 Processor -- start 2024-09-21T03:40:02.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.845+0000 7fa7e0829740 1 -- start start 2024-09-21T03:40:02.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6bdecd0 0x5559f6fdd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fe40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7e0829740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41afe60 con 0x5559f6bff670 2024-09-21T03:40:02.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7e0829740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41ad730 con 0x5559f6bdecd0 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7e0829740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41ab5a0 con 0x5559f6bf7490 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7d8852640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7d8852640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49590/0 (socket says 172.21.15.112:49590) 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7d3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6bdecd0 0x5559f6fdd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7d8852640 1 -- 172.21.15.112:0/796140289 learned_addr learned my addr 172.21.15.112:0/796140289 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fe40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7d8852640 1 -- 172.21.15.112:0/796140289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 msgr2=0x5559f6fe40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7d8852640 1 --2- 172.21.15.112:0/796140289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fe40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7d8852640 1 -- 172.21.15.112:0/796140289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6bdecd0 msgr2=0x5559f6fdd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7d8852640 1 --2- 172.21.15.112:0/796140289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6bdecd0 0x5559f6fdd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7d8852640 1 -- 172.21.15.112:0/796140289 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f6fcb930 con 0x5559f6bff670 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/796140289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6bdecd0 0x5559f6fdd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:02.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.846+0000 7fa7d8852640 1 --2- 172.21.15.112:0/796140289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd1ae0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009600 tx=0x7fa7cc017a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb2e4110a6b531d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:02.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.847+0000 7fa7c9ffb640 1 -- 172.21.15.112:0/796140289 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc038070 con 0x5559f6bff670 2024-09-21T03:40:02.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.847+0000 7fa7c9ffb640 1 -- 172.21.15.112:0/796140289 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7cc01c070 con 0x5559f6bff670 2024-09-21T03:40:02.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.847+0000 7fa7c9ffb640 1 -- 172.21.15.112:0/796140289 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00d970 con 0x5559f6bff670 2024-09-21T03:40:02.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.848+0000 7fa7e0829740 1 -- 172.21.15.112:0/796140289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 msgr2=0x5559f6fd1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.848+0000 7fa7e0829740 1 --2- 172.21.15.112:0/796140289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd1ae0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009600 tx=0x7fa7cc017a60 comp rx=0 tx=0).stop 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.848+0000 7fa7e0829740 1 -- 172.21.15.112:0/796140289 shutdown_connections 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.848+0000 7fa7e0829740 1 --2- 172.21.15.112:0/796140289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bff670 0x5559f6fd1ae0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.848+0000 7fa7e0829740 1 --2- 172.21.15.112:0/796140289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bf7490 0x5559f6fe40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.848+0000 7fa7e0829740 1 --2- 172.21.15.112:0/796140289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6bdecd0 0x5559f6fdd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.848+0000 7fa7e0829740 1 -- 172.21.15.112:0/796140289 >> 172.21.15.112:0/796140289 conn(0x5559f6c14740 msgr2=0x5559f6fd2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.848+0000 7fa7e0829740 1 -- 172.21.15.112:0/796140289 shutdown_connections 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.848+0000 7fa7e0829740 1 -- 172.21.15.112:0/796140289 wait complete. 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.848+0000 7fa7e0829740 1 -- start start 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.848+0000 7fa7e0829740 1 Processor -- start 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bdecd0 0x5559f6fde030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6bf7490 0x5559f6fde630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7e0829740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bff670 0x5559f6fe2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7e0829740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41abbf0 con 0x5559f6bdecd0 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7e0829740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41ad060 con 0x5559f6bf7490 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7e0829740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41ac2c0 con 0x5559f6bff670 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bdecd0 0x5559f6fde030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bdecd0 0x5559f6fde030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49592/0 (socket says 172.21.15.112:49592) 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7d3fff640 1 -- 172.21.15.112:0/2203830725 learned_addr learned my addr 172.21.15.112:0/2203830725 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7d8852640 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bff670 0x5559f6fe2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7cbfff640 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6bf7490 0x5559f6fde630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7e0829740 10 client.0 _ll_register_callbacks cb 0x7fa7bc0eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7d3fff640 1 -- 172.21.15.112:0/2203830725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bff670 msgr2=0x5559f6fe2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bff670 0x5559f6fe2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7d3fff640 1 -- 172.21.15.112:0/2203830725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6bf7490 msgr2=0x5559f6fde630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6bf7490 0x5559f6fde630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7d3fff640 1 -- 172.21.15.112:0/2203830725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f6fe34f0 con 0x5559f6bdecd0 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7d3fff640 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bdecd0 0x5559f6fde030 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa7c402a860 tx=0x7fa7c4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2479d160e3d8d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.849+0000 7fa7b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7b37fe640 1 -- 172.21.15.112:0/2203830725 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4037040 con 0x5559f6bdecd0 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7b37fe640 1 -- 172.21.15.112:0/2203830725 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7fa7c4004470 con 0x5559f6bdecd0 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7e0829740 1 -- 172.21.15.112:0/2203830725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7cc01c070 con 0x5559f6bdecd0 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7b37fe640 1 -- 172.21.15.112:0/2203830725 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7c4005040 con 0x5559f6bdecd0 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7e0829740 10 client.17361 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7e0829740 1 -- 172.21.15.112:0/2203830725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559f7054020 con 0x5559f6bdecd0 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7b37fe640 1 -- 172.21.15.112:0/2203830725 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7cc01c070 con 0x5559f6bdecd0 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7b37fe640 1 -- 172.21.15.112:0/2203830725 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa7c40035f0 con 0x5559f6bdecd0 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7e0829740 10 client.17361 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7e0829740 10 client.17361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7e0829740 20 client.17361 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7e0829740 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5559f6c20350 0x5559f7056810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7e0829740 4 client.17361 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:40:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.850+0000 7fa7e0829740 1 -- 172.21.15.112:0/2203830725 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5559f6f3b860 con 0x5559f6c20350 2024-09-21T03:40:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.851+0000 7fa7cbfff640 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5559f6c20350 0x5559f7056810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.851+0000 7fa7b37fe640 10 client.17361 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:02.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.851+0000 7fa7cbfff640 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5559f6c20350 0x5559f7056810 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af5b5920ef06679a server_cookie=25626a422ad326b9 in_seq=0 out_seq=0 2024-09-21T03:40:02.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7b37fe640 1 -- 172.21.15.112:0/2203830725 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5559f6f3b860 con 0x5559f6c20350 2024-09-21T03:40:02.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7b37fe640 10 client.17361 handle_command_reply: tid=0 2024-09-21T03:40:02.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 1 client.17361 shutdown 2024-09-21T03:40:02.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 1 -- 172.21.15.112:0/2203830725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bdecd0 msgr2=0x5559f6fde030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bdecd0 0x5559f6fde030 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa7c402a860 tx=0x7fa7c4005f90 comp rx=0 tx=0).stop 2024-09-21T03:40:02.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 1 -- 172.21.15.112:0/2203830725 shutdown_connections 2024-09-21T03:40:02.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5559f6c20350 0x5559f7056810 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5559f6bff670 0x5559f6fe2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5559f6bf7490 0x5559f6fde630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 1 --2- 172.21.15.112:0/2203830725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5559f6bdecd0 0x5559f6fde030 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 1 -- 172.21.15.112:0/2203830725 >> 172.21.15.112:0/2203830725 conn(0x5559f6c14740 msgr2=0x5559f6fd46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7b37fe640 0 client.17361 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 1 -- 172.21.15.112:0/2203830725 shutdown_connections 2024-09-21T03:40:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 1 -- 172.21.15.112:0/2203830725 wait complete. 2024-09-21T03:40:02.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.853+0000 7fa7e0829740 20 client.17361 trim_cache size 0 max 16384 2024-09-21T03:40:02.859 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:40:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:40:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e", 2024-09-21T03:40:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:40:02.860 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:40:02.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 -- 172.21.15.112:0/1754848282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa7c00b6680 msgr2=0x7fa7c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa7c00b6680 0x7fa7c00b8b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4000c00 tx=0x7fa7c4009490 comp rx=0 tx=0).stop 2024-09-21T03:40:02.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 -- 172.21.15.112:0/1754848282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d4108420 msgr2=0x7fa7d4199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:02.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d4108420 0x7fa7d4199a00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00e9f0 tx=0x7fa7bc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 -- 172.21.15.112:0/1754848282 shutdown_connections 2024-09-21T03:40:02.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa7c00b6680 0x7fa7c00b8b90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d4108d70 0x7fa7d419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7d4108420 0x7fa7d4199a00 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 --2- 172.21.15.112:0/1754848282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7d4102ae0 0x7fa7d4199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:02.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 -- 172.21.15.112:0/1754848282 >> 172.21.15.112:0/1754848282 conn(0x7fa7d4079770 msgr2=0x7fa7d4106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:02.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 -- 172.21.15.112:0/1754848282 shutdown_connections 2024-09-21T03:40:02.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:02.854+0000 7fa7da2b5640 1 -- 172.21.15.112:0/1754848282 wait complete. 2024-09-21T03:40:03.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1170957 10.36 MB/sec execute 483 sec latency 58.268 ms 2024-09-21T03:40:03.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:40:03.920 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:04.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:03 smithi112 ceph-mon[31442]: pgmap v659: 97 pgs: 97 active+clean; 875 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 221 op/s 2024-09-21T03:40:04.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:03 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:40:04.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:03 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:03 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:40:04.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:03 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:40:04.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:03 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:40:04.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:03 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:03 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:03 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:03 smithi203 ceph-mon[34723]: pgmap v659: 97 pgs: 97 active+clean; 875 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 221 op/s 2024-09-21T03:40:04.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:03 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:40:04.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:03 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:03 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:40:04.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:03 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:40:04.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:03 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:40:04.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:03 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:03 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:03 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:03 smithi187 ceph-mon[34603]: pgmap v659: 97 pgs: 97 active+clean; 875 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 221 op/s 2024-09-21T03:40:04.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:03 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:40:04.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:03 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:03 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:40:04.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:03 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:40:04.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:03 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:40:04.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:03 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:03 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:03 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:04.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1172701 10.36 MB/sec execute 484 sec latency 16.214 ms 2024-09-21T03:40:04.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.431+0000 7f47b98f6640 1 -- 172.21.15.112:0/1360036045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b4109e00 msgr2=0x7f47b4110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.431+0000 7f47b98f6640 1 --2- 172.21.15.112:0/1360036045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b4109e00 0x7f47b4110630 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009ca0 tx=0x7f47ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.431+0000 7f47b98f6640 1 -- 172.21.15.112:0/1360036045 shutdown_connections 2024-09-21T03:40:04.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.431+0000 7f47b98f6640 1 --2- 172.21.15.112:0/1360036045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b4109e00 0x7f47b4110630 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.431+0000 7f47b98f6640 1 --2- 172.21.15.112:0/1360036045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47b41094b0 0x7f47b41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.431+0000 7f47b98f6640 1 --2- 172.21.15.112:0/1360036045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b4103b70 0x7f47b4103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.431+0000 7f47b98f6640 1 -- 172.21.15.112:0/1360036045 >> 172.21.15.112:0/1360036045 conn(0x7f47b40ff5e0 msgr2=0x7f47b4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.431+0000 7f47b98f6640 1 -- 172.21.15.112:0/1360036045 shutdown_connections 2024-09-21T03:40:04.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.431+0000 7f47b98f6640 1 -- 172.21.15.112:0/1360036045 wait complete. 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47b98f6640 1 Processor -- start 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47b98f6640 1 -- start start 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47b98f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b4103b70 0x7f47b419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47b98f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b41094b0 0x7f47b419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47b98f6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47b4109e00 0x7f47b419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47b98f6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4113ac0 con 0x7f47b4103b70 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47b98f6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4113940 con 0x7f47b41094b0 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47b98f6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4113c40 con 0x7f47b4109e00 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b4103b70 0x7f47b419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b4103b70 0x7f47b419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49604/0 (socket says 172.21.15.112:49604) 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47b2ffd640 1 -- 172.21.15.112:0/335468138 learned_addr learned my addr 172.21.15.112:0/335468138 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.432+0000 7f47aa7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b41094b0 0x7f47b419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47b37fe640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47b4109e00 0x7f47b419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47aa7fc640 1 -- 172.21.15.112:0/335468138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47b4109e00 msgr2=0x7f47b419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47aa7fc640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47b4109e00 0x7f47b419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47aa7fc640 1 -- 172.21.15.112:0/335468138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b4103b70 msgr2=0x7f47b419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47aa7fc640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b4103b70 0x7f47b419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47aa7fc640 1 -- 172.21.15.112:0/335468138 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b41a03e0 con 0x7f47b41094b0 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b4103b70 0x7f47b419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47aa7fc640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b41094b0 0x7f47b419aa10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f47b406bd40 tx=0x7f479c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47b88f4640 1 -- 172.21.15.112:0/335468138 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c00cc40 con 0x7f47b41094b0 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47b98f6640 1 -- 172.21.15.112:0/335468138 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b41a06d0 con 0x7f47b41094b0 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47b88f4640 1 -- 172.21.15.112:0/335468138 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f479c00cde0 con 0x7f47b41094b0 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47b88f4640 1 -- 172.21.15.112:0/335468138 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c004a00 con 0x7f47b41094b0 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.433+0000 7f47b98f6640 1 -- 172.21.15.112:0/335468138 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b41a6e30 con 0x7f47b41094b0 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47bfe6a740 1 Processor -- start 2024-09-21T03:40:04.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47bfe6a740 1 -- start start 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47b88f4640 1 -- 172.21.15.112:0/335468138 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f479c004ba0 con 0x7f47b41094b0 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562717804070 0x562717bab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627177db1b0 0x562717babb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177e3390 0x562717bac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562717b21e50 con 0x562717804070 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562717b11280 con 0x5627177e3390 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562717b15090 con 0x5627177db1b0 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627177db1b0 0x562717babb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47b37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562717804070 0x562717bab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627177db1b0 0x562717babb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52762/0 (socket says 172.21.15.112:52762) 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47b2ffd640 1 -- 172.21.15.112:0/2036218017 learned_addr learned my addr 172.21.15.112:0/2036218017 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.436+0000 7f47aa7fc640 1 --2- 172.21.15.112:0/2036218017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177e3390 0x562717bac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47b2ffd640 1 -- 172.21.15.112:0/2036218017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177e3390 msgr2=0x562717bac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/2036218017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177e3390 0x562717bac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47b2ffd640 1 -- 172.21.15.112:0/2036218017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562717804070 msgr2=0x562717bab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/2036218017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562717804070 0x562717bab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47b2ffd640 1 -- 172.21.15.112:0/2036218017 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56271786f650 con 0x5627177db1b0 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47b37fe640 1 --2- 172.21.15.112:0/2036218017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562717804070 0x562717bab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47b88f4640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f47a00b6680 0x7f47a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/2036218017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627177db1b0 0x562717babb90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f47a4011650 tx=0x7f47a4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=673ee6e18855d3d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47b37fe640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f47a00b6680 0x7f47a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47a9ffb640 1 -- 172.21.15.112:0/2036218017 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a401e070 con 0x5627177db1b0 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47b88f4640 1 -- 172.21.15.112:0/335468138 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f479c0e7670 con 0x7f47b41094b0 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47a9ffb640 1 -- 172.21.15.112:0/2036218017 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a400fb70 con 0x5627177db1b0 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.437+0000 7f47a9ffb640 1 -- 172.21.15.112:0/2036218017 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4019c60 con 0x5627177db1b0 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.438+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2036218017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627177db1b0 msgr2=0x562717babb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.438+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/2036218017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627177db1b0 0x562717babb90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f47a4011650 tx=0x7f47a4009f90 comp rx=0 tx=0).stop 2024-09-21T03:40:04.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.438+0000 7f47b37fe640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f47a00b6680 0x7f47a00b8b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f47ac02f500 tx=0x7f47ac02f490 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.438+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2036218017 shutdown_connections 2024-09-21T03:40:04.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.438+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/2036218017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177e3390 0x562717bac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.438+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/2036218017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5627177db1b0 0x562717babb90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.438+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/2036218017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562717804070 0x562717bab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.438+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2036218017 >> 172.21.15.112:0/2036218017 conn(0x5627177f8460 msgr2=0x562717ba9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.438+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2036218017 shutdown_connections 2024-09-21T03:40:04.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.438+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2036218017 wait complete. 2024-09-21T03:40:04.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47bfe6a740 1 -- start start 2024-09-21T03:40:04.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47bfe6a740 1 Processor -- start 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177db1b0 0x562717bb6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177e3390 0x562717bb7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bc3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562717972700 con 0x5627177db1b0 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562717aac110 con 0x5627177e3390 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562717bb9e30 con 0x562717804070 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177db1b0 0x562717bb6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47bfe6a740 10 client.0 _ll_register_callbacks cb 0x562717ac5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47aa7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177e3390 0x562717bb7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177db1b0 0x562717bb6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49628/0 (socket says 172.21.15.112:49628) 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47b2ffd640 1 -- 172.21.15.112:0/2756093413 learned_addr learned my addr 172.21.15.112:0/2756093413 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:04.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.439+0000 7f47b37fe640 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bc3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f47b2ffd640 1 -- 172.21.15.112:0/2756093413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 msgr2=0x562717bc3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bc3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f47b2ffd640 1 -- 172.21.15.112:0/2756093413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177e3390 msgr2=0x562717bb7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177e3390 0x562717bb7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f47b2ffd640 1 -- 172.21.15.112:0/2756093413 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562717ad9c40 con 0x5627177db1b0 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f47aa7fc640 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177e3390 0x562717bb7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177db1b0 0x562717bb6d40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f47a40197e0 tx=0x7f47a4009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b5d4081a121d95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f4792ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f4792ffd640 1 -- 172.21.15.112:0/2756093413 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a401e040 con 0x5627177db1b0 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f4792ffd640 1 -- 172.21.15.112:0/2756093413 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f47a400f6c0 con 0x5627177db1b0 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2756093413 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a400fb70 con 0x5627177db1b0 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f4792ffd640 1 -- 172.21.15.112:0/2756093413 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47a4030050 con 0x5627177db1b0 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f47bfe6a740 10 client.17373 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:04.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.440+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2756093413 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562717bc5b20 con 0x5627177db1b0 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.441+0000 7f4792ffd640 1 -- 172.21.15.112:0/2756093413 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a400fb70 con 0x5627177db1b0 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.441+0000 7f4792ffd640 1 -- 172.21.15.112:0/2756093413 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f47a4021070 con 0x5627177db1b0 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.441+0000 7f47bfe6a740 10 client.17373 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.441+0000 7f47bfe6a740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.441+0000 7f47bfe6a740 20 client.17373 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.441+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5627177c29f0 0x562717bc8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.441+0000 7f47bfe6a740 4 client.17373 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.441+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2756093413 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562717b23da0 con 0x5627177c29f0 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.441+0000 7f47aa7fc640 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5627177c29f0 0x562717bc8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.442+0000 7f47aa7fc640 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5627177c29f0 0x562717bc8ed0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd00f768a0f7438 server_cookie=1f58bacb773e906 in_seq=0 out_seq=0 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.442+0000 7f4792ffd640 10 client.17373 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f4792ffd640 1 -- 172.21.15.112:0/2756093413 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562717b23da0 con 0x5627177c29f0 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f4792ffd640 10 client.17373 handle_command_reply: tid=0 2024-09-21T03:40:04.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 1 client.17373 shutdown 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2756093413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177db1b0 msgr2=0x562717bb6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177db1b0 0x562717bb6d40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f47a40197e0 tx=0x7f47a4009ed0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2756093413 shutdown_connections 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5627177c29f0 0x562717bc8ed0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bc3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f4792ffd640 0 client.17373 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177e3390 0x562717bb7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/2756093413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177db1b0 0x562717bb6d40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2756093413 >> 172.21.15.112:0/2756093413 conn(0x5627177f8460 msgr2=0x562717bae830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2756093413 shutdown_connections 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 1 -- 172.21.15.112:0/2756093413 wait complete. 2024-09-21T03:40:04.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.443+0000 7f47bfe6a740 20 client.17373 trim_cache size 0 max 16384 2024-09-21T03:40:04.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.460+0000 7f47bfe6a740 1 Processor -- start 2024-09-21T03:40:04.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47bfe6a740 1 -- start start 2024-09-21T03:40:04.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 0x562717ba9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 0x562717bc3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b1f00 con 0x5627177e3390 2024-09-21T03:40:04.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41af7d0 con 0x5627177db1b0 2024-09-21T03:40:04.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41ad640 con 0x562717804070 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47aa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 0x562717bc3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47aa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 0x562717bc3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49640/0 (socket says 172.21.15.112:49640) 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 0x562717ba9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47aa7fc640 1 -- 172.21.15.112:0/1612758658 learned_addr learned my addr 172.21.15.112:0/1612758658 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47b37fe640 1 --2- 172.21.15.112:0/1612758658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47b2ffd640 1 -- 172.21.15.112:0/1612758658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 msgr2=0x562717bae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/1612758658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47b2ffd640 1 -- 172.21.15.112:0/1612758658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 msgr2=0x562717bc3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/1612758658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 0x562717bc3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47b2ffd640 1 -- 172.21.15.112:0/1612758658 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562717bb9480 con 0x5627177db1b0 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.461+0000 7f47aa7fc640 1 --2- 172.21.15.112:0/1612758658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 0x562717bc3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:04.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.462+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/1612758658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 0x562717ba9570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f47a4019830 tx=0x7f47a4005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a10be5a9a3ff3b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.462+0000 7f47a97fa640 1 -- 172.21.15.112:0/1612758658 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a401e070 con 0x5627177db1b0 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.462+0000 7f47a97fa640 1 -- 172.21.15.112:0/1612758658 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a4027070 con 0x5627177db1b0 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.462+0000 7f47a97fa640 1 -- 172.21.15.112:0/1612758658 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4019860 con 0x5627177db1b0 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.463+0000 7f47bfe6a740 1 -- 172.21.15.112:0/1612758658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 msgr2=0x562717ba9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.463+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/1612758658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 0x562717ba9570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f47a4019830 tx=0x7f47a4005f10 comp rx=0 tx=0).stop 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.463+0000 7f47bfe6a740 1 -- 172.21.15.112:0/1612758658 shutdown_connections 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.463+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/1612758658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.463+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/1612758658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 0x562717bc3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.463+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/1612758658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 0x562717ba9570 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.463+0000 7f47bfe6a740 1 -- 172.21.15.112:0/1612758658 >> 172.21.15.112:0/1612758658 conn(0x5627177f8460 msgr2=0x562717ba8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.463+0000 7f47bfe6a740 1 -- 172.21.15.112:0/1612758658 shutdown_connections 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.463+0000 7f47bfe6a740 1 -- 172.21.15.112:0/1612758658 wait complete. 2024-09-21T03:40:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.463+0000 7f47bfe6a740 1 -- start start 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47bfe6a740 1 Processor -- start 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 0x562717bbf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 0x562717bbab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47bfe6a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bbb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41adc90 con 0x5627177e3390 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41af100 con 0x5627177db1b0 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47bfe6a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41ae360 con 0x562717804070 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47bfe6a740 10 client.0 _ll_register_callbacks cb 0x7f47a401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47aa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 0x562717bbab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47b37fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bbb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47b37fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bbb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52770/0 (socket says 172.21.15.112:52770) 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47b37fe640 1 -- 172.21.15.112:0/191686241 learned_addr learned my addr 172.21.15.112:0/191686241 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47b2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 0x562717bbf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47b2ffd640 1 -- 172.21.15.112:0/191686241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 msgr2=0x562717bbb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bbb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47b2ffd640 1 -- 172.21.15.112:0/191686241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 msgr2=0x562717bbab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 0x562717bbab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47b2ffd640 1 -- 172.21.15.112:0/191686241 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562717bbb670 con 0x5627177db1b0 2024-09-21T03:40:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.464+0000 7f47b37fe640 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bbb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f47b2ffd640 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 0x562717bbf6c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f47a40115a0 tx=0x7f47a4009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89d97a27a261dedc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f4792ffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:40:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f4792ffd640 1 -- 172.21.15.112:0/191686241 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a401e040 con 0x5627177db1b0 2024-09-21T03:40:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f47bfe6a740 1 -- 172.21.15.112:0/191686241 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a4027070 con 0x5627177db1b0 2024-09-21T03:40:04.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f4792ffd640 1 -- 172.21.15.112:0/191686241 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8470+0+0 (secure 0 0 0) 0x7f47a400f8c0 con 0x5627177db1b0 2024-09-21T03:40:04.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f4792ffd640 1 -- 172.21.15.112:0/191686241 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47a400fac0 con 0x5627177db1b0 2024-09-21T03:40:04.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f47bfe6a740 10 client.26933 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:04.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f47bfe6a740 1 -- 172.21.15.112:0/191686241 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562717c334e0 con 0x5627177db1b0 2024-09-21T03:40:04.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f4792ffd640 1 -- 172.21.15.112:0/191686241 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a4027070 con 0x5627177db1b0 2024-09-21T03:40:04.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f4792ffd640 1 -- 172.21.15.112:0/191686241 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f47a4021070 con 0x5627177db1b0 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f47bfe6a740 10 client.26933 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f47bfe6a740 10 client.26933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f47bfe6a740 20 client.26933 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562717797da0 0x562717c35be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f47bfe6a740 4 client.26933 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.465+0000 7f47bfe6a740 1 -- 172.21.15.112:0/191686241 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562717b1f850 con 0x562717797da0 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.466+0000 7f47aa7fc640 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562717797da0 0x562717c35be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.466+0000 7f4792ffd640 10 client.26933 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.466+0000 7f47aa7fc640 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562717797da0 0x562717c35be0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e418e316ac1638 server_cookie=1d219a189d91b5c3 in_seq=0 out_seq=0 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.466+0000 7f4792ffd640 1 -- 172.21.15.112:0/191686241 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x562717b1f850 con 0x562717797da0 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.466+0000 7f4792ffd640 10 client.26933 handle_command_reply: tid=0 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.466+0000 7f47bfe6a740 1 client.26933 shutdown 2024-09-21T03:40:04.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47bfe6a740 1 -- 172.21.15.112:0/191686241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 msgr2=0x562717bbf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 0x562717bbf6c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f47a40115a0 tx=0x7f47a4009ed0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47bfe6a740 1 -- 172.21.15.112:0/191686241 shutdown_connections 2024-09-21T03:40:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x562717797da0 0x562717c35be0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562717804070 0x562717bbb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5627177e3390 0x562717bbab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f4792ffd640 0 client.26933 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47bfe6a740 1 --2- 172.21.15.112:0/191686241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5627177db1b0 0x562717bbf6c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47bfe6a740 1 -- 172.21.15.112:0/191686241 >> 172.21.15.112:0/191686241 conn(0x5627177f8460 msgr2=0x562717bae750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47bfe6a740 1 -- 172.21.15.112:0/191686241 shutdown_connections 2024-09-21T03:40:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47bfe6a740 1 -- 172.21.15.112:0/191686241 wait complete. 2024-09-21T03:40:04.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47bfe6a740 20 client.26933 trim_cache size 0 max 16384 2024-09-21T03:40:04.704 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:40:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:40:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:40:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e", 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:40:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:40:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:40:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:40:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:40:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:40:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:40:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:40:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:40:04.716 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47b98f6640 1 -- 172.21.15.112:0/335468138 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f47a00b6680 msgr2=0x7f47a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47b98f6640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f47a00b6680 0x7f47a00b8b90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f47ac02f500 tx=0x7f47ac02f490 comp rx=0 tx=0).stop 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47b98f6640 1 -- 172.21.15.112:0/335468138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b41094b0 msgr2=0x7f47b419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47b98f6640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b41094b0 0x7f47b419aa10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f47b406bd40 tx=0x7f479c00ee50 comp rx=0 tx=0).stop 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47b98f6640 1 -- 172.21.15.112:0/335468138 shutdown_connections 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47b98f6640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f47a00b6680 0x7f47a00b8b90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47b98f6640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47b4109e00 0x7f47b419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47b98f6640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47b41094b0 0x7f47b419aa10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47b98f6640 1 --2- 172.21.15.112:0/335468138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47b4103b70 0x7f47b419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.467+0000 7f47b98f6640 1 -- 172.21.15.112:0/335468138 >> 172.21.15.112:0/335468138 conn(0x7f47b40ff5e0 msgr2=0x7f47b4108b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.468+0000 7f47b98f6640 1 -- 172.21.15.112:0/335468138 shutdown_connections 2024-09-21T03:40:04.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:04.468+0000 7f47b98f6640 1 -- 172.21.15.112:0/335468138 wait complete. 2024-09-21T03:40:05.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1174717 10.35 MB/sec execute 485 sec latency 49.472 ms 2024-09-21T03:40:05.502 INFO:tasks.cephfs.filesystem:scrub status for tag:c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e - {'path': '/', 'tag': 'c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e', 'options': 'recursive,force'} 2024-09-21T03:40:06.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:05 smithi203 ceph-mon[34723]: pgmap v660: 97 pgs: 97 active+clean; 880 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 248 op/s 2024-09-21T03:40:06.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:05 smithi203 ceph-mon[34723]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:40:06.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:05 smithi187 ceph-mon[34603]: pgmap v660: 97 pgs: 97 active+clean; 880 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 248 op/s 2024-09-21T03:40:06.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:05 smithi187 ceph-mon[34603]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:40:06.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1176210 10.36 MB/sec execute 486 sec latency 142.981 ms 2024-09-21T03:40:06.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:05 smithi112 ceph-mon[31442]: pgmap v660: 97 pgs: 97 active+clean; 880 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 248 op/s 2024-09-21T03:40:06.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:05 smithi112 ceph-mon[31442]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:40:07.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1178434 10.36 MB/sec execute 487 sec latency 30.955 ms 2024-09-21T03:40:08.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:07 smithi203 ceph-mon[34723]: pgmap v662: 97 pgs: 97 active+clean; 871 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-09-21T03:40:08.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:07 smithi187 ceph-mon[34603]: pgmap v662: 97 pgs: 97 active+clean; 871 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-09-21T03:40:08.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1179942 10.35 MB/sec execute 488 sec latency 24.327 ms 2024-09-21T03:40:08.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:07 smithi112 ceph-mon[31442]: pgmap v662: 97 pgs: 97 active+clean; 871 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 271 op/s 2024-09-21T03:40:09.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1182394 10.36 MB/sec execute 489 sec latency 43.870 ms 2024-09-21T03:40:10.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:09 smithi203 ceph-mon[34723]: pgmap v663: 97 pgs: 97 active+clean; 858 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 289 op/s 2024-09-21T03:40:10.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:09 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:40:10.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:09 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:40:10.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:09 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:40:10.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:09 smithi187 ceph-mon[34603]: pgmap v663: 97 pgs: 97 active+clean; 858 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 289 op/s 2024-09-21T03:40:10.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:09 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:40:10.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:09 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:40:10.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:09 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:40:10.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1183556 10.35 MB/sec execute 490 sec latency 77.285 ms 2024-09-21T03:40:10.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:09 smithi112 ceph-mon[31442]: pgmap v663: 97 pgs: 97 active+clean; 858 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 289 op/s 2024-09-21T03:40:10.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:09 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:40:10.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:09 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:40:10.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:09 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:40:11.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1186042 10.36 MB/sec execute 491 sec latency 31.219 ms 2024-09-21T03:40:12.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:11 smithi203 ceph-mon[34723]: pgmap v664: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 335 op/s 2024-09-21T03:40:12.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:11 smithi187 ceph-mon[34603]: pgmap v664: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 335 op/s 2024-09-21T03:40:12.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1187675 10.36 MB/sec execute 492 sec latency 13.023 ms 2024-09-21T03:40:12.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:11 smithi112 ceph-mon[31442]: pgmap v664: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 335 op/s 2024-09-21T03:40:13.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1190059 10.36 MB/sec execute 493 sec latency 16.802 ms 2024-09-21T03:40:14.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:13 smithi203 ceph-mon[34723]: pgmap v665: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 297 op/s 2024-09-21T03:40:14.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:13 smithi187 ceph-mon[34603]: pgmap v665: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 297 op/s 2024-09-21T03:40:14.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1191358 10.35 MB/sec execute 494 sec latency 19.836 ms 2024-09-21T03:40:14.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:13 smithi112 ceph-mon[31442]: pgmap v665: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 297 op/s 2024-09-21T03:40:15.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1193605 10.36 MB/sec execute 495 sec latency 34.308 ms 2024-09-21T03:40:16.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:15 smithi203 ceph-mon[34723]: pgmap v666: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-09-21T03:40:16.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:15 smithi203 ceph-mon[34723]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:40:16.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:15 smithi187 ceph-mon[34603]: pgmap v666: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-09-21T03:40:16.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:15 smithi187 ceph-mon[34603]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:40:16.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1194887 10.35 MB/sec execute 496 sec latency 25.846 ms 2024-09-21T03:40:16.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:15 smithi112 ceph-mon[31442]: pgmap v666: 97 pgs: 97 active+clean; 840 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 239 op/s 2024-09-21T03:40:16.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:15 smithi112 ceph-mon[31442]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:40:17.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1197230 10.36 MB/sec execute 497 sec latency 16.264 ms 2024-09-21T03:40:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:17 smithi203 ceph-mon[34723]: pgmap v668: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 280 op/s 2024-09-21T03:40:18.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:17 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:18.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:17 smithi187 ceph-mon[34603]: pgmap v668: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 280 op/s 2024-09-21T03:40:18.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:17 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:18.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1198539 10.35 MB/sec execute 498 sec latency 97.996 ms 2024-09-21T03:40:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:17 smithi112 ceph-mon[31442]: pgmap v668: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 280 op/s 2024-09-21T03:40:18.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:17 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:40:19.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1200631 10.35 MB/sec execute 499 sec latency 98.537 ms 2024-09-21T03:40:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:19 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:19 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:19.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:19 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:20.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1201607 10.34 MB/sec execute 500 sec latency 55.552 ms 2024-09-21T03:40:20.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:20 smithi203 ceph-mon[34723]: pgmap v669: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 261 op/s 2024-09-21T03:40:20.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:20 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:20.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:20 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:20.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:20 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:20.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:20 smithi187 ceph-mon[34603]: pgmap v669: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 261 op/s 2024-09-21T03:40:20.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:20 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:20.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:20 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:20.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:20 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:20.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:20 smithi112 ceph-mon[31442]: pgmap v669: 97 pgs: 97 active+clean; 854 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 261 op/s 2024-09-21T03:40:20.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:20 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:40:20.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:20 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:40:20.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:20 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:40:21.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1204049 10.35 MB/sec execute 501 sec latency 40.263 ms 2024-09-21T03:40:22.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1205242 10.33 MB/sec execute 502 sec latency 13.922 ms 2024-09-21T03:40:22.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:22 smithi203 ceph-mon[34723]: pgmap v670: 97 pgs: 97 active+clean; 862 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 256 op/s 2024-09-21T03:40:22.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:22 smithi187 ceph-mon[34603]: pgmap v670: 97 pgs: 97 active+clean; 862 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 256 op/s 2024-09-21T03:40:22.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:22 smithi112 ceph-mon[31442]: pgmap v670: 97 pgs: 97 active+clean; 862 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 256 op/s 2024-09-21T03:40:23.273 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1207767 10.35 MB/sec execute 503 sec latency 17.155 ms 2024-09-21T03:40:24.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1209069 10.34 MB/sec execute 504 sec latency 18.814 ms 2024-09-21T03:40:24.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:24 smithi203 ceph-mon[34723]: pgmap v671: 97 pgs: 97 active+clean; 861 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 255 op/s 2024-09-21T03:40:24.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:24 smithi187 ceph-mon[34603]: pgmap v671: 97 pgs: 97 active+clean; 861 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 255 op/s 2024-09-21T03:40:24.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:24 smithi112 ceph-mon[31442]: pgmap v671: 97 pgs: 97 active+clean; 861 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 255 op/s 2024-09-21T03:40:25.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1211319 10.34 MB/sec execute 505 sec latency 31.116 ms 2024-09-21T03:40:26.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1212497 10.33 MB/sec execute 506 sec latency 31.509 ms 2024-09-21T03:40:26.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:26 smithi203 ceph-mon[34723]: pgmap v672: 97 pgs: 97 active+clean; 862 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 255 op/s 2024-09-21T03:40:26.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:26 smithi187 ceph-mon[34603]: pgmap v672: 97 pgs: 97 active+clean; 862 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 255 op/s 2024-09-21T03:40:26.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:26 smithi112 ceph-mon[31442]: pgmap v672: 97 pgs: 97 active+clean; 862 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 255 op/s 2024-09-21T03:40:27.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1214808 10.34 MB/sec execute 507 sec latency 20.127 ms 2024-09-21T03:40:28.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1216113 10.33 MB/sec execute 508 sec latency 57.060 ms 2024-09-21T03:40:28.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:28 smithi203 ceph-mon[34723]: pgmap v673: 97 pgs: 97 active+clean; 875 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 303 op/s 2024-09-21T03:40:28.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:28 smithi187 ceph-mon[34603]: pgmap v673: 97 pgs: 97 active+clean; 875 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 303 op/s 2024-09-21T03:40:28.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:28 smithi112 ceph-mon[31442]: pgmap v673: 97 pgs: 97 active+clean; 875 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 303 op/s 2024-09-21T03:40:29.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1218436 10.34 MB/sec execute 509 sec latency 45.986 ms 2024-09-21T03:40:30.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1219729 10.33 MB/sec execute 510 sec latency 43.108 ms 2024-09-21T03:40:30.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:30 smithi203 ceph-mon[34723]: pgmap v674: 97 pgs: 97 active+clean; 875 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 201 op/s 2024-09-21T03:40:30.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:30 smithi187 ceph-mon[34603]: pgmap v674: 97 pgs: 97 active+clean; 875 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 201 op/s 2024-09-21T03:40:30.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:30 smithi112 ceph-mon[31442]: pgmap v674: 97 pgs: 97 active+clean; 875 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 201 op/s 2024-09-21T03:40:31.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1221997 10.33 MB/sec execute 511 sec latency 45.694 ms 2024-09-21T03:40:32.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1223446 10.33 MB/sec execute 512 sec latency 18.528 ms 2024-09-21T03:40:32.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:32 smithi203 ceph-mon[34723]: pgmap v675: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 285 op/s 2024-09-21T03:40:32.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:32 smithi187 ceph-mon[34603]: pgmap v675: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 285 op/s 2024-09-21T03:40:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:32 smithi112 ceph-mon[31442]: pgmap v675: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 285 op/s 2024-09-21T03:40:33.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1225680 10.33 MB/sec execute 513 sec latency 14.110 ms 2024-09-21T03:40:34.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1227501 10.32 MB/sec execute 514 sec latency 20.272 ms 2024-09-21T03:40:34.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:34 smithi203 ceph-mon[34723]: pgmap v676: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 220 op/s 2024-09-21T03:40:34.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:34 smithi187 ceph-mon[34603]: pgmap v676: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 220 op/s 2024-09-21T03:40:34.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:34 smithi112 ceph-mon[31442]: pgmap v676: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 220 op/s 2024-09-21T03:40:35.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1229305 10.33 MB/sec execute 515 sec latency 44.053 ms 2024-09-21T03:40:35.503 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:40:35.769 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:36.175 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:36 smithi112 ceph-mon[31442]: pgmap v677: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 215 op/s 2024-09-21T03:40:36.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1230737 10.32 MB/sec execute 516 sec latency 31.471 ms 2024-09-21T03:40:36.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.275+0000 7ff6eca36640 1 -- 172.21.15.112:0/926339247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e81027a0 msgr2=0x7ff6e8102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.275+0000 7ff6eca36640 1 --2- 172.21.15.112:0/926339247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e81027a0 0x7ff6e8102bb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009930 tx=0x7ff6d802a440 comp rx=0 tx=0).stop 2024-09-21T03:40:36.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.275+0000 7ff6eca36640 1 -- 172.21.15.112:0/926339247 shutdown_connections 2024-09-21T03:40:36.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.275+0000 7ff6eca36640 1 --2- 172.21.15.112:0/926339247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e8107be0 0x7ff6e81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.275+0000 7ff6eca36640 1 --2- 172.21.15.112:0/926339247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e81030f0 0x7ff6e8107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.275+0000 7ff6eca36640 1 --2- 172.21.15.112:0/926339247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e81027a0 0x7ff6e8102bb0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.275+0000 7ff6eca36640 1 -- 172.21.15.112:0/926339247 >> 172.21.15.112:0/926339247 conn(0x7ff6e8079730 msgr2=0x7ff6e8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.275+0000 7ff6eca36640 1 -- 172.21.15.112:0/926339247 shutdown_connections 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.275+0000 7ff6eca36640 1 -- 172.21.15.112:0/926339247 wait complete. 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.276+0000 7ff6eca36640 1 Processor -- start 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.276+0000 7ff6eca36640 1 -- start start 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6eca36640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e81027a0 0x7ff6e81a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6eca36640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e81030f0 0x7ff6e81a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6eca36640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e8107be0 0x7ff6e81a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6eca36640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e811a090 con 0x7ff6e8107be0 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6eca36640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e8119f10 con 0x7ff6e81027a0 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6eca36640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e811a210 con 0x7ff6e81030f0 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6e6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e8107be0 0x7ff6e81a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6e659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e81027a0 0x7ff6e81a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6e659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e81027a0 0x7ff6e81a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49376/0 (socket says 172.21.15.112:49376) 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6e659c640 1 -- 172.21.15.112:0/3463601125 learned_addr learned my addr 172.21.15.112:0/3463601125 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6dffff640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e81030f0 0x7ff6e81a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6e659c640 1 -- 172.21.15.112:0/3463601125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e81030f0 msgr2=0x7ff6e81a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6e659c640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e81030f0 0x7ff6e81a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6e659c640 1 -- 172.21.15.112:0/3463601125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e8107be0 msgr2=0x7ff6e81a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6e659c640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e8107be0 0x7ff6e81a6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6e659c640 1 -- 172.21.15.112:0/3463601125 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e81a6a70 con 0x7ff6e81027a0 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.277+0000 7ff6e6d9d640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e8107be0 0x7ff6e81a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.278+0000 7ff6e659c640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e81027a0 0x7ff6e81a0b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8004770 tx=0x7ff6d8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.278+0000 7ff6df7fe640 1 -- 172.21.15.112:0/3463601125 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d803f030 con 0x7ff6e81027a0 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.278+0000 7ff6df7fe640 1 -- 172.21.15.112:0/3463601125 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d8003aa0 con 0x7ff6e81027a0 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.278+0000 7ff6df7fe640 1 -- 172.21.15.112:0/3463601125 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8035070 con 0x7ff6e81027a0 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.278+0000 7ff6eca36640 1 -- 172.21.15.112:0/3463601125 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e81a6d00 con 0x7ff6e81027a0 2024-09-21T03:40:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.278+0000 7ff6eca36640 1 -- 172.21.15.112:0/3463601125 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e81ad4c0 con 0x7ff6e81027a0 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6f2faa740 1 Processor -- start 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6f2faa740 1 -- start start 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba8b070 0x556eebe32650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6df7fe640 1 -- 172.21.15.112:0/3463601125 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff6d8042030 con 0x7ff6e81027a0 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba621b0 0x556eebe32b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe330d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556eebda8e50 con 0x556eeba6a390 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556eebd98280 con 0x556eeba621b0 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556eebd9c090 con 0x556eeba8b070 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe330d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44298/0 (socket says 172.21.15.112:44298) 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6e659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba621b0 0x556eebe32b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6dffff640 1 -- 172.21.15.112:0/939397796 learned_addr learned my addr 172.21.15.112:0/939397796 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:36.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.281+0000 7ff6e6d9d640 1 --2- 172.21.15.112:0/939397796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba8b070 0x556eebe32650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6dffff640 1 -- 172.21.15.112:0/939397796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba8b070 msgr2=0x556eebe32650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6dffff640 1 --2- 172.21.15.112:0/939397796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba8b070 0x556eebe32650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6dffff640 1 -- 172.21.15.112:0/939397796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba621b0 msgr2=0x556eebe32b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6dffff640 1 --2- 172.21.15.112:0/939397796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba621b0 0x556eebe32b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6dffff640 1 -- 172.21.15.112:0/939397796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eebaf6650 con 0x556eeba6a390 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6dffff640 1 --2- 172.21.15.112:0/939397796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe330d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000bc30 tx=0x7ff6d00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb840d18a704eff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6dcff9640 1 -- 172.21.15.112:0/939397796 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d001c480 con 0x556eeba6a390 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6df7fe640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6d40b6680 0x7ff6d40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6dcff9640 1 -- 172.21.15.112:0/939397796 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d001cad0 con 0x556eeba6a390 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6dcff9640 1 -- 172.21.15.112:0/939397796 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0011aa0 con 0x556eeba6a390 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6e6d9d640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6d40b6680 0x7ff6d40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.282+0000 7ff6df7fe640 1 -- 172.21.15.112:0/3463601125 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7ff6d8107cc0 con 0x7ff6e81027a0 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.283+0000 7ff6f2faa740 1 -- 172.21.15.112:0/939397796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 msgr2=0x556eebe330d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.283+0000 7ff6e6d9d640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6d40b6680 0x7ff6d40b8b90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009fd0 tx=0x7ff6e0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.283+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/939397796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe330d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000bc30 tx=0x7ff6d00107a0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.283+0000 7ff6f2faa740 1 -- 172.21.15.112:0/939397796 shutdown_connections 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.283+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/939397796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe330d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.283+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/939397796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba621b0 0x556eebe32b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.283+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/939397796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba8b070 0x556eebe32650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.283+0000 7ff6f2faa740 1 -- 172.21.15.112:0/939397796 >> 172.21.15.112:0/939397796 conn(0x556eeba7f460 msgr2=0x556eebe30540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.283+0000 7ff6f2faa740 1 -- 172.21.15.112:0/939397796 shutdown_connections 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.283+0000 7ff6f2faa740 1 -- 172.21.15.112:0/939397796 wait complete. 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.284+0000 7ff6f2faa740 1 -- start start 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.284+0000 7ff6f2faa740 1 Processor -- start 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.284+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe3dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.284+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe3e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.284+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba8b070 0x556eebe4a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.284+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556eebbf9700 con 0x556eeba6a390 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.284+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556eebd33110 con 0x556eeba8b070 2024-09-21T03:40:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556eebe40e30 con 0x556eeba621b0 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe3e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6e659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe3dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe3e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44314/0 (socket says 172.21.15.112:44314) 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6dffff640 1 -- 172.21.15.112:0/2745188777 learned_addr learned my addr 172.21.15.112:0/2745188777 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6f2faa740 10 client.0 _ll_register_callbacks cb 0x556eebd4c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6e6d9d640 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba8b070 0x556eebe4a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6dffff640 1 -- 172.21.15.112:0/2745188777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 msgr2=0x556eebe3dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6dffff640 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe3dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6dffff640 1 -- 172.21.15.112:0/2745188777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba8b070 msgr2=0x556eebe4a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6dffff640 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba8b070 0x556eebe4a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6dffff640 1 -- 172.21.15.112:0/2745188777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eebd60c40 con 0x556eeba6a390 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6dffff640 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe3e290 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000c00 tx=0x7ff6d0011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c283fca7c47416ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:40:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6cdffb640 1 -- 172.21.15.112:0/2745188777 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d002b060 con 0x556eeba6a390 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2745188777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d001cad0 con 0x556eeba6a390 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6cdffb640 1 -- 172.21.15.112:0/2745188777 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7ff6d001c480 con 0x556eeba6a390 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6cdffb640 1 -- 172.21.15.112:0/2745188777 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6d001c680 con 0x556eeba6a390 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6f2faa740 10 client.17391 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.285+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2745188777 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556eebe4cb80 con 0x556eeba6a390 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.286+0000 7ff6cdffb640 1 -- 172.21.15.112:0/2745188777 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d001cad0 con 0x556eeba6a390 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.286+0000 7ff6cdffb640 1 -- 172.21.15.112:0/2745188777 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff6d0011af0 con 0x556eeba6a390 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.286+0000 7ff6f2faa740 10 client.17391 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.286+0000 7ff6f2faa740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.286+0000 7ff6f2faa740 20 client.17391 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.286+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556eeba499f0 0x556eebe4ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.286+0000 7ff6f2faa740 4 client.17391 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.286+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2745188777 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eebdaada0 con 0x556eeba499f0 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.286+0000 7ff6e659c640 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556eeba499f0 0x556eebe4ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6e659c640 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556eeba499f0 0x556eebe4ff40 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bcca74e22c69ae5 server_cookie=a49f25ec066b7045 in_seq=0 out_seq=0 2024-09-21T03:40:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6cdffb640 10 client.17391 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6cdffb640 1 -- 172.21.15.112:0/2745188777 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556eebdaada0 con 0x556eeba499f0 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6cdffb640 10 client.17391 handle_command_reply: tid=0 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6f2faa740 1 client.17391 shutdown 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2745188777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 msgr2=0x556eebe3e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe3e290 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000c00 tx=0x7ff6d0011440 comp rx=0 tx=0).stop 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2745188777 shutdown_connections 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556eeba499f0 0x556eebe4ff40 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba8b070 0x556eebe4a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba6a390 0x556eebe3e290 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/2745188777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe3dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2745188777 >> 172.21.15.112:0/2745188777 conn(0x556eeba7f460 msgr2=0x556eebe357e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.287+0000 7ff6cdffb640 0 client.17391 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.288+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2745188777 shutdown_connections 2024-09-21T03:40:36.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.288+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2745188777 wait complete. 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.288+0000 7ff6f2faa740 20 client.17391 trim_cache size 0 max 16384 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6f2faa740 1 Processor -- start 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6f2faa740 1 -- start start 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe3e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba6a390 0x556eebe34040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe3f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b8540 con 0x556eeba8b070 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b5e10 con 0x556eeba6a390 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b3c80 con 0x556eeba621b0 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6e6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe3f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6e6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe3f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44322/0 (socket says 172.21.15.112:44322) 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba6a390 0x556eebe34040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.307+0000 7ff6e6d9d640 1 -- 172.21.15.112:0/2189607993 learned_addr learned my addr 172.21.15.112:0/2189607993 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.308+0000 7ff6e659c640 1 --2- 172.21.15.112:0/2189607993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe3e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.308+0000 7ff6e6d9d640 1 -- 172.21.15.112:0/2189607993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 msgr2=0x556eebe3e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.308+0000 7ff6e6d9d640 1 --2- 172.21.15.112:0/2189607993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe3e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.308+0000 7ff6e6d9d640 1 -- 172.21.15.112:0/2189607993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba6a390 msgr2=0x556eebe34040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.308+0000 7ff6e6d9d640 1 --2- 172.21.15.112:0/2189607993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba6a390 0x556eebe34040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.308+0000 7ff6e6d9d640 1 -- 172.21.15.112:0/2189607993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eebe34c10 con 0x556eeba8b070 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.308+0000 7ff6dffff640 1 --2- 172.21.15.112:0/2189607993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba6a390 0x556eebe34040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.308+0000 7ff6e6d9d640 1 --2- 172.21.15.112:0/2189607993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe3f980 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0003b30 tx=0x7ff6e0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d0f6277c60259b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.308+0000 7ff6cffff640 1 -- 172.21.15.112:0/2189607993 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000fe50 con 0x556eeba8b070 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.308+0000 7ff6cffff640 1 -- 172.21.15.112:0/2189607993 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e0004050 con 0x556eeba8b070 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.308+0000 7ff6cffff640 1 -- 172.21.15.112:0/2189607993 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00053c0 con 0x556eeba8b070 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.309+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2189607993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 msgr2=0x556eebe3f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.309+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/2189607993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe3f980 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0003b30 tx=0x7ff6e0012000 comp rx=0 tx=0).stop 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.309+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2189607993 shutdown_connections 2024-09-21T03:40:36.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.309+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/2189607993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe3f980 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.309+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/2189607993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba6a390 0x556eebe34040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.309+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/2189607993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe3e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.309+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2189607993 >> 172.21.15.112:0/2189607993 conn(0x556eeba7f460 msgr2=0x556eebe3f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.309+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2189607993 shutdown_connections 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.309+0000 7ff6f2faa740 1 -- 172.21.15.112:0/2189607993 wait complete. 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6f2faa740 1 -- start start 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6f2faa740 1 Processor -- start 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe51100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba6a390 0x556eebe51640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6f2faa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe51b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b42d0 con 0x556eeba8b070 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b5740 con 0x556eeba6a390 2024-09-21T03:40:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6f2faa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b49a0 con 0x556eeba621b0 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6f2faa740 10 client.0 _ll_register_callbacks cb 0x7ff6d810b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6e6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe51b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6e659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe51100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6e6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe51b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44336/0 (socket says 172.21.15.112:44336) 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6e6d9d640 1 -- 172.21.15.112:0/3264979203 learned_addr learned my addr 172.21.15.112:0/3264979203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.310+0000 7ff6dffff640 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba6a390 0x556eebe51640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6e6d9d640 1 -- 172.21.15.112:0/3264979203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 msgr2=0x556eebe51100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6e6d9d640 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe51100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6e6d9d640 1 -- 172.21.15.112:0/3264979203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba6a390 msgr2=0x556eebe51640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6e6d9d640 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba6a390 0x556eebe51640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6e6d9d640 1 -- 172.21.15.112:0/3264979203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eebeb9870 con 0x556eeba8b070 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6e659c640 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe51100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:36.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6e6d9d640 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe51b80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00127c0 tx=0x7ff6e00044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb5cf642baecf0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6cdffb640 1 -- 172.21.15.112:0/3264979203 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0037020 con 0x556eeba8b070 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6cdffb640 1 -- 172.21.15.112:0/3264979203 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7ff6e0029460 con 0x556eeba8b070 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6f2faa740 1 -- 172.21.15.112:0/3264979203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e0004050 con 0x556eeba8b070 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6cdffb640 1 -- 172.21.15.112:0/3264979203 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6e0042050 con 0x556eeba8b070 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6f2faa740 10 client.17397 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.311+0000 7ff6f2faa740 1 -- 172.21.15.112:0/3264979203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556eebebaae0 con 0x556eeba8b070 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.312+0000 7ff6cdffb640 1 -- 172.21.15.112:0/3264979203 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e0004050 con 0x556eeba8b070 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.312+0000 7ff6cdffb640 1 -- 172.21.15.112:0/3264979203 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff6e00334c0 con 0x556eeba8b070 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.312+0000 7ff6f2faa740 10 client.17397 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.312+0000 7ff6f2faa740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.312+0000 7ff6f2faa740 20 client.17397 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.312+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556eeba1eda0 0x556eebebd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:36.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.312+0000 7ff6f2faa740 4 client.17397 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:40:36.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.312+0000 7ff6f2faa740 1 -- 172.21.15.112:0/3264979203 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556eebda6850 con 0x556eeba1eda0 2024-09-21T03:40:36.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.312+0000 7ff6e659c640 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556eeba1eda0 0x556eebebd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:36.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6e659c640 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556eeba1eda0 0x556eebebd290 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92cc91121d608753 server_cookie=5357feb79ec8e3a1 in_seq=0 out_seq=0 2024-09-21T03:40:36.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6cdffb640 10 client.17397 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:36.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6cdffb640 1 -- 172.21.15.112:0/3264979203 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x556eebda6850 con 0x556eeba1eda0 2024-09-21T03:40:36.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6cdffb640 10 client.17397 handle_command_reply: tid=0 2024-09-21T03:40:36.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 1 client.17397 shutdown 2024-09-21T03:40:36.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 1 -- 172.21.15.112:0/3264979203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 msgr2=0x556eebe51b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe51b80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00127c0 tx=0x7ff6e00044b0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 1 -- 172.21.15.112:0/3264979203 shutdown_connections 2024-09-21T03:40:36.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556eeba1eda0 0x556eebebd290 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:40:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:40:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556eeba8b070 0x556eebe51b80 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556eeba6a390 0x556eebe51640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 1 --2- 172.21.15.112:0/3264979203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eeba621b0 0x556eebe51100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 1 -- 172.21.15.112:0/3264979203 >> 172.21.15.112:0/3264979203 conn(0x556eeba7f460 msgr2=0x556eebe33e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6cdffb640 0 client.17397 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 1 -- 172.21.15.112:0/3264979203 shutdown_connections 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 1 -- 172.21.15.112:0/3264979203 wait complete. 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.313+0000 7ff6f2faa740 20 client.17397 trim_cache size 0 max 16384 2024-09-21T03:40:36.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 -- 172.21.15.112:0/3463601125 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6d40b6680 msgr2=0x7ff6d40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6d40b6680 0x7ff6d40b8b90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009fd0 tx=0x7ff6e0009500 comp rx=0 tx=0).stop 2024-09-21T03:40:36.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 -- 172.21.15.112:0/3463601125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e81027a0 msgr2=0x7ff6e81a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:36.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e81027a0 0x7ff6e81a0b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8004770 tx=0x7ff6d8004270 comp rx=0 tx=0).stop 2024-09-21T03:40:36.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 -- 172.21.15.112:0/3463601125 shutdown_connections 2024-09-21T03:40:36.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6d40b6680 0x7ff6d40b8b90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6e8107be0 0x7ff6e81a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e81030f0 0x7ff6e81a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 --2- 172.21.15.112:0/3463601125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6e81027a0 0x7ff6e81a0b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:36.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 -- 172.21.15.112:0/3463601125 >> 172.21.15.112:0/3463601125 conn(0x7ff6e8079730 msgr2=0x7ff6e81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:36.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 -- 172.21.15.112:0/3463601125 shutdown_connections 2024-09-21T03:40:36.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:36.314+0000 7ff6eca36640 1 -- 172.21.15.112:0/3463601125 wait complete. 2024-09-21T03:40:36.429 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:36 smithi203 ceph-mon[34723]: pgmap v677: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 215 op/s 2024-09-21T03:40:36.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:36 smithi187 ceph-mon[34603]: pgmap v677: 97 pgs: 97 active+clean; 885 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 215 op/s 2024-09-21T03:40:37.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1232799 10.33 MB/sec execute 517 sec latency 16.688 ms 2024-09-21T03:40:37.379 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:40:37.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:40:37.646 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:38.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.181+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/502189956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b4103b60 msgr2=0x7fd1b4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.181+0000 7fd1b9c5d640 1 --2- 172.21.15.112:0/502189956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b4103b60 0x7fd1b4103f70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009900 tx=0x7fd1a402a410 comp rx=0 tx=0).stop 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.182+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/502189956 shutdown_connections 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.182+0000 7fd1b9c5d640 1 --2- 172.21.15.112:0/502189956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b4109df0 0x7fd1b4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.182+0000 7fd1b9c5d640 1 --2- 172.21.15.112:0/502189956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1b41094a0 0x7fd1b41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.182+0000 7fd1b9c5d640 1 --2- 172.21.15.112:0/502189956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b4103b60 0x7fd1b4103f70 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.182+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/502189956 >> 172.21.15.112:0/502189956 conn(0x7fd1b407b850 msgr2=0x7fd1b407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.182+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/502189956 shutdown_connections 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.182+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/502189956 wait complete. 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.182+0000 7fd1b9c5d640 1 Processor -- start 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b9c5d640 1 -- start start 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b9c5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b4103b60 0x7fd1b419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b9c5d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b41094a0 0x7fd1b419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b9c5d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1b4109df0 0x7fd1b419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b9c5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4113b00 con 0x7fd1b4103b60 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b9c5d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4113980 con 0x7fd1b41094a0 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b9c5d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4113c80 con 0x7fd1b4109df0 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b41094a0 0x7fd1b419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b41094a0 0x7fd1b419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49438/0 (socket says 172.21.15.112:49438) 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b2ffd640 1 -- 172.21.15.112:0/3145037007 learned_addr learned my addr 172.21.15.112:0/3145037007 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b3fff640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1b4109df0 0x7fd1b419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.183+0000 7fd1b37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b4103b60 0x7fd1b419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b2ffd640 1 -- 172.21.15.112:0/3145037007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1b4109df0 msgr2=0x7fd1b419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b2ffd640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1b4109df0 0x7fd1b419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b2ffd640 1 -- 172.21.15.112:0/3145037007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b4103b60 msgr2=0x7fd1b419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b2ffd640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b4103b60 0x7fd1b419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b2ffd640 1 -- 172.21.15.112:0/3145037007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b41a0440 con 0x7fd1b41094a0 2024-09-21T03:40:38.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b37fe640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b4103b60 0x7fd1b419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:38.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b2ffd640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b41094a0 0x7fd1b419aa70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800e9c0 tx=0x7fd1a800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:38.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b0ff9640 1 -- 172.21.15.112:0/3145037007 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800ccc0 con 0x7fd1b41094a0 2024-09-21T03:40:38.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b0ff9640 1 -- 172.21.15.112:0/3145037007 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a80054d0 con 0x7fd1b41094a0 2024-09-21T03:40:38.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b0ff9640 1 -- 172.21.15.112:0/3145037007 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8010430 con 0x7fd1b41094a0 2024-09-21T03:40:38.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/3145037007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b41a0730 con 0x7fd1b41094a0 2024-09-21T03:40:38.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.184+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/3145037007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41a6e90 con 0x7fd1b41094a0 2024-09-21T03:40:38.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.185+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/3145037007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b4103f70 con 0x7fd1b41094a0 2024-09-21T03:40:38.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.187+0000 7fd1b0ff9640 1 -- 172.21.15.112:0/3145037007 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fd1a8005020 con 0x7fd1b41094a0 2024-09-21T03:40:38.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.188+0000 7fd1b0ff9640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd19c0b6680 0x7fd19c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:38.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.188+0000 7fd1b37fe640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd19c0b6680 0x7fd19c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:38.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.189+0000 7fd1b0ff9640 1 -- 172.21.15.112:0/3145037007 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7fd1a80e7d10 con 0x7fd1b41094a0 2024-09-21T03:40:38.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.189+0000 7fd1b37fe640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd19c0b6680 0x7fd19c0b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000c00 tx=0x7fd1a4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:38.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.191+0000 7fd1b0ff9640 1 -- 172.21.15.112:0/3145037007 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd1a8056710 con 0x7fd1b41094a0 2024-09-21T03:40:38.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1234254 10.32 MB/sec execute 518 sec latency 46.382 ms 2024-09-21T03:40:38.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:38 smithi203 ceph-mon[34723]: pgmap v678: 97 pgs: 97 active+clean; 898 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 317 op/s 2024-09-21T03:40:38.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:38 smithi187 ceph-mon[34603]: pgmap v678: 97 pgs: 97 active+clean; 898 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 317 op/s 2024-09-21T03:40:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.442+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/3145037007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1b419b600 con 0x7fd1b41094a0 2024-09-21T03:40:38.444 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:38 smithi112 ceph-mon[31442]: pgmap v678: 97 pgs: 97 active+clean; 898 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 317 op/s 2024-09-21T03:40:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.443+0000 7fd1b0ff9640 1 -- 172.21.15.112:0/3145037007 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7fd1b419b600 con 0x7fd1b41094a0 2024-09-21T03:40:38.445 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:40:38.445 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"btime":"2024-09-21T03:35:32:357323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:35:32.307199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:40:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-09-21T03:40:38.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.445+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/3145037007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd19c0b6680 msgr2=0x7fd19c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.445+0000 7fd1b9c5d640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd19c0b6680 0x7fd19c0b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000c00 tx=0x7fd1a4031400 comp rx=0 tx=0).stop 2024-09-21T03:40:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.445+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/3145037007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b41094a0 msgr2=0x7fd1b419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.445+0000 7fd1b9c5d640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b41094a0 0x7fd1b419aa70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800e9c0 tx=0x7fd1a800ee80 comp rx=0 tx=0).stop 2024-09-21T03:40:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.445+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/3145037007 shutdown_connections 2024-09-21T03:40:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.445+0000 7fd1b9c5d640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd19c0b6680 0x7fd19c0b8b90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.445+0000 7fd1b9c5d640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1b4109df0 0x7fd1b419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.445+0000 7fd1b9c5d640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b41094a0 0x7fd1b419aa70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.445+0000 7fd1b9c5d640 1 --2- 172.21.15.112:0/3145037007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1b4103b60 0x7fd1b419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.445+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/3145037007 >> 172.21.15.112:0/3145037007 conn(0x7fd1b407b850 msgr2=0x7fd1b4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.445+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/3145037007 shutdown_connections 2024-09-21T03:40:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:38.446+0000 7fd1b9c5d640 1 -- 172.21.15.112:0/3145037007 wait complete. 2024-09-21T03:40:39.180 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:40:39.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1236230 10.32 MB/sec execute 519 sec latency 40.718 ms 2024-09-21T03:40:39.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:39 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3145037007' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:39.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:39 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3145037007' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:39.453 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:39.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:39 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3145037007' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:40:39.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.954+0000 7f72a3d30640 1 -- 172.21.15.112:0/603635232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729c1084b0 msgr2=0x7f729c10ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:39.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.954+0000 7f72a3d30640 1 --2- 172.21.15.112:0/603635232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729c1084b0 0x7f729c10ece0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7294009ca0 tx=0x7f729402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.954+0000 7f72a3d30640 1 -- 172.21.15.112:0/603635232 shutdown_connections 2024-09-21T03:40:39.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.954+0000 7f72a3d30640 1 --2- 172.21.15.112:0/603635232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729c1084b0 0x7f729c10ece0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.954+0000 7f72a3d30640 1 --2- 172.21.15.112:0/603635232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c107b60 0x7f729c107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.954+0000 7f72a3d30640 1 --2- 172.21.15.112:0/603635232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f729c102270 0x7f729c102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.954+0000 7f72a3d30640 1 -- 172.21.15.112:0/603635232 >> 172.21.15.112:0/603635232 conn(0x7f729c077cc0 msgr2=0x7f729c078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.954+0000 7f72a3d30640 1 -- 172.21.15.112:0/603635232 shutdown_connections 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.954+0000 7f72a3d30640 1 -- 172.21.15.112:0/603635232 wait complete. 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a3d30640 1 Processor -- start 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a3d30640 1 -- start start 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a3d30640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729c102270 0x7f729c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a3d30640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f729c107b60 0x7f729c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a3d30640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c1084b0 0x7f729c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a3d30640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f729c112a00 con 0x7f729c102270 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a3d30640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f729c112880 con 0x7f729c107b60 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a3d30640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f729c112b80 con 0x7f729c1084b0 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a1acc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729c102270 0x7f729c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a1acc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729c102270 0x7f729c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44380/0 (socket says 172.21.15.112:44380) 2024-09-21T03:40:39.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a1acc640 1 -- 172.21.15.112:0/3119054627 learned_addr learned my addr 172.21.15.112:0/3119054627 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a22cd640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c1084b0 0x7f729c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.955+0000 7f72a12cb640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f729c107b60 0x7f729c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.956+0000 7f72a1acc640 1 -- 172.21.15.112:0/3119054627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c1084b0 msgr2=0x7f729c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.956+0000 7f72a1acc640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c1084b0 0x7f729c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.956+0000 7f72a1acc640 1 -- 172.21.15.112:0/3119054627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f729c107b60 msgr2=0x7f729c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.956+0000 7f72a1acc640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f729c107b60 0x7f729c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.956+0000 7f72a1acc640 1 -- 172.21.15.112:0/3119054627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c1a0430 con 0x7f729c102270 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.956+0000 7f72a1acc640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729c102270 0x7f729c19a4c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f729800e9f0 tx=0x7f729800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.956+0000 7f7292ffd640 1 -- 172.21.15.112:0/3119054627 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729800cd80 con 0x7f729c102270 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.956+0000 7f7292ffd640 1 -- 172.21.15.112:0/3119054627 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72980108e0 con 0x7f729c102270 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.956+0000 7f7292ffd640 1 -- 172.21.15.112:0/3119054627 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298018980 con 0x7f729c102270 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.956+0000 7f72a3d30640 1 -- 172.21.15.112:0/3119054627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729c1a0720 con 0x7f729c102270 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.956+0000 7f72a3d30640 1 -- 172.21.15.112:0/3119054627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f729c1a6e80 con 0x7f729c102270 2024-09-21T03:40:39.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.959+0000 7f7292ffd640 1 -- 172.21.15.112:0/3119054627 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7298010430 con 0x7f729c102270 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.959+0000 7f72aa2a4740 1 Processor -- start 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.959+0000 7f72aa2a4740 1 -- start start 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f33b070 0x55612f6e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f6e2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f31a390 0x55612f6e2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55612f658ea0 con 0x55612f31a390 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55612f6482d0 con 0x55612f33b070 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55612f64c0e0 con 0x55612f3121b0 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72a1acc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f6e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72a12cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f31a390 0x55612f6e2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72a1acc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f6e2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:33134/0 (socket says 172.21.15.112:33134) 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72a1acc640 1 -- 172.21.15.112:0/409598029 learned_addr learned my addr 172.21.15.112:0/409598029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f7292ffd640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72880b6700 0x7f72880b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:39.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72a22cd640 1 --2- 172.21.15.112:0/409598029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f33b070 0x55612f6e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72a22cd640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72880b6700 0x7f72880b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72a12cb640 1 -- 172.21.15.112:0/409598029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 msgr2=0x55612f6e2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72a12cb640 1 --2- 172.21.15.112:0/409598029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f6e2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72a12cb640 1 -- 172.21.15.112:0/409598029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f33b070 msgr2=0x55612f6e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72a12cb640 1 --2- 172.21.15.112:0/409598029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f33b070 0x55612f6e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f72a12cb640 1 -- 172.21.15.112:0/409598029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612f3a6650 con 0x55612f31a390 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.960+0000 7f7292ffd640 1 -- 172.21.15.112:0/3119054627 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f72980e7990 con 0x7f729c102270 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.961+0000 7f72a12cb640 1 --2- 172.21.15.112:0/409598029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f31a390 0x55612f6e2d40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f729c075840 tx=0x7f728c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446d7aed4d8e1c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.961+0000 7f72a22cd640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72880b6700 0x7f72880b8c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7294006010 tx=0x7f729402a8c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.961+0000 7f7287fff640 1 -- 172.21.15.112:0/409598029 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c004250 con 0x55612f31a390 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.961+0000 7f7287fff640 1 -- 172.21.15.112:0/409598029 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f728c009e10 con 0x55612f31a390 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.961+0000 7f7287fff640 1 -- 172.21.15.112:0/409598029 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c004c70 con 0x55612f31a390 2024-09-21T03:40:39.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.962+0000 7f72aa2a4740 1 -- 172.21.15.112:0/409598029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f31a390 msgr2=0x55612f6e2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.962+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/409598029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f31a390 0x55612f6e2d40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f729c075840 tx=0x7f728c011810 comp rx=0 tx=0).stop 2024-09-21T03:40:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.962+0000 7f72aa2a4740 1 -- 172.21.15.112:0/409598029 shutdown_connections 2024-09-21T03:40:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.962+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/409598029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f31a390 0x55612f6e2d40 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.962+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/409598029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f6e2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.962+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/409598029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f33b070 0x55612f6e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.962+0000 7f72aa2a4740 1 -- 172.21.15.112:0/409598029 >> 172.21.15.112:0/409598029 conn(0x55612f32f460 msgr2=0x55612f6e01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.962+0000 7f72aa2a4740 1 -- 172.21.15.112:0/409598029 shutdown_connections 2024-09-21T03:40:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.962+0000 7f72aa2a4740 1 -- 172.21.15.112:0/409598029 wait complete. 2024-09-21T03:40:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.962+0000 7f72aa2a4740 1 -- start start 2024-09-21T03:40:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72aa2a4740 1 Processor -- start 2024-09-21T03:40:40.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f6ed8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f31a390 0x55612f6ede00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f33b070 0x55612f6ee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55612f4a9700 con 0x55612f33b070 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55612f5e3160 con 0x55612f31a390 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55612f6f0aa0 con 0x55612f3121b0 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72aa2a4740 10 client.0 _ll_register_callbacks cb 0x55612f5fc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72a22cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f33b070 0x55612f6ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72a1acc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f6ed8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72a1acc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f6ed8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:33148/0 (socket says 172.21.15.112:33148) 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72a22cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f33b070 0x55612f6ee340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44400/0 (socket says 172.21.15.112:44400) 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72a1acc640 1 -- 172.21.15.112:0/3360585224 learned_addr learned my addr 172.21.15.112:0/3360585224 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.963+0000 7f72a12cb640 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f31a390 0x55612f6ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72a1acc640 1 -- 172.21.15.112:0/3360585224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f31a390 msgr2=0x55612f6ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:40.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72a1acc640 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f31a390 0x55612f6ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72a1acc640 1 -- 172.21.15.112:0/3360585224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f33b070 msgr2=0x55612f6ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72a1acc640 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f33b070 0x55612f6ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72a1acc640 1 -- 172.21.15.112:0/3360585224 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612f610c90 con 0x55612f3121b0 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72a1acc640 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f6ed8c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f72980e5710 tx=0x7f7298010cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9e6228f4038dbfb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72857fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72857fa640 1 -- 172.21.15.112:0/3360585224 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980ad690 con 0x55612f3121b0 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3360585224 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f728c009e10 con 0x55612f3121b0 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72857fa640 1 -- 172.21.15.112:0/3360585224 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f7298018d20 con 0x55612f3121b0 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72857fa640 1 -- 172.21.15.112:0/3360585224 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72980ad030 con 0x55612f3121b0 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72aa2a4740 10 client.26974 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3360585224 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55612f6fc740 con 0x55612f3121b0 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.964+0000 7f72857fa640 1 -- 172.21.15.112:0/3360585224 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f728c009e10 con 0x55612f3121b0 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.965+0000 7f72857fa640 1 -- 172.21.15.112:0/3360585224 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f72980ac590 con 0x55612f3121b0 2024-09-21T03:40:40.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.965+0000 7f72aa2a4740 10 client.26974 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.965+0000 7f72aa2a4740 10 client.26974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.965+0000 7f72aa2a4740 20 client.26974 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.965+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55612f2f99f0 0x55612f6ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.965+0000 7f72aa2a4740 4 client.26974 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.965+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3360585224 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55612f65adf0 con 0x55612f2f99f0 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.965+0000 7f72a12cb640 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55612f2f99f0 0x55612f6ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.965+0000 7f72857fa640 10 client.26974 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.965+0000 7f72a12cb640 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55612f2f99f0 0x55612f6ffba0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4eb2cb883e113c8 server_cookie=536335ff5f50b67d in_seq=0 out_seq=0 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72857fa640 1 -- 172.21.15.112:0/3360585224 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55612f65adf0 con 0x55612f2f99f0 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72857fa640 10 client.26974 handle_command_reply: tid=0 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 1 client.26974 shutdown 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3360585224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 msgr2=0x55612f6ed8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f6ed8c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f72980e5710 tx=0x7f7298010cc0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3360585224 shutdown_connections 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55612f2f99f0 0x55612f6ffba0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72857fa640 0 client.26974 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f33b070 0x55612f6ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f31a390 0x55612f6ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3360585224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f6ed8c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3360585224 >> 172.21.15.112:0/3360585224 conn(0x55612f32f460 msgr2=0x55612f6e54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3360585224 shutdown_connections 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3360585224 wait complete. 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.967+0000 7f72aa2a4740 20 client.26974 trim_cache size 0 max 16384 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72aa2a4740 1 Processor -- start 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72aa2a4740 1 -- start start 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f705750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f31a390 0x55612f705c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1b1fb0 con 0x55612f31a390 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1af880 con 0x55612f2f99f0 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1ad6f0 con 0x55612f3121b0 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72a12cb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f705750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72a1acc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72a22cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f31a390 0x55612f705c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72a12cb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f705750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:33164/0 (socket says 172.21.15.112:33164) 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72a12cb640 1 -- 172.21.15.112:0/4031093201 learned_addr learned my addr 172.21.15.112:0/4031093201 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.983+0000 7f72a1acc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6e3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49490/0 (socket says 172.21.15.112:49490) 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.984+0000 7f72a1acc640 1 -- 172.21.15.112:0/4031093201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 msgr2=0x55612f705750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.984+0000 7f72a1acc640 1 --2- 172.21.15.112:0/4031093201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f705750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.984+0000 7f72a1acc640 1 -- 172.21.15.112:0/4031093201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f31a390 msgr2=0x55612f705c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.984+0000 7f72a1acc640 1 --2- 172.21.15.112:0/4031093201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f31a390 0x55612f705c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.984+0000 7f72a1acc640 1 -- 172.21.15.112:0/4031093201 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612f6e45b0 con 0x55612f2f99f0 2024-09-21T03:40:40.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.984+0000 7f72a1acc640 1 --2- 172.21.15.112:0/4031093201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6e3a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f72980ac9a0 tx=0x7f7298018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f26e7bc79d4aa8fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:40.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.984+0000 7f72877fe640 1 -- 172.21.15.112:0/4031093201 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980ad210 con 0x55612f2f99f0 2024-09-21T03:40:40.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.984+0000 7f72877fe640 1 -- 172.21.15.112:0/4031093201 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7298010720 con 0x55612f2f99f0 2024-09-21T03:40:40.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.984+0000 7f72877fe640 1 -- 172.21.15.112:0/4031093201 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980b37f0 con 0x55612f2f99f0 2024-09-21T03:40:40.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.985+0000 7f72aa2a4740 1 -- 172.21.15.112:0/4031093201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 msgr2=0x55612f6e3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:40.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.985+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/4031093201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6e3a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f72980ac9a0 tx=0x7f7298018420 comp rx=0 tx=0).stop 2024-09-21T03:40:40.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.985+0000 7f72aa2a4740 1 -- 172.21.15.112:0/4031093201 shutdown_connections 2024-09-21T03:40:40.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.985+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/4031093201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f31a390 0x55612f705c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.985+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/4031093201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f3121b0 0x55612f705750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.985+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/4031093201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6e3a00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.985+0000 7f72aa2a4740 1 -- 172.21.15.112:0/4031093201 >> 172.21.15.112:0/4031093201 conn(0x55612f32f460 msgr2=0x55612f6e32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.985+0000 7f72aa2a4740 1 -- 172.21.15.112:0/4031093201 shutdown_connections 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.985+0000 7f72aa2a4740 1 -- 172.21.15.112:0/4031093201 wait complete. 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72aa2a4740 1 -- start start 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72aa2a4740 1 Processor -- start 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f3121b0 0x55612f6f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72aa2a4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f31a390 0x55612f7676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1add40 con 0x55612f3121b0 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1af1b0 con 0x55612f2f99f0 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72aa2a4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1ae410 con 0x55612f31a390 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72aa2a4740 10 client.0 _ll_register_callbacks cb 0x7f72980a9850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72a12cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f3121b0 0x55612f6f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72a1acc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72a1acc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49504/0 (socket says 172.21.15.112:49504) 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72a12cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f3121b0 0x55612f6f2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44430/0 (socket says 172.21.15.112:44430) 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72a1acc640 1 -- 172.21.15.112:0/3448285376 learned_addr learned my addr 172.21.15.112:0/3448285376 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.986+0000 7f72a22cd640 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f31a390 0x55612f7676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72a1acc640 1 -- 172.21.15.112:0/3448285376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f31a390 msgr2=0x55612f7676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72a1acc640 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f31a390 0x55612f7676a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72a1acc640 1 -- 172.21.15.112:0/3448285376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f3121b0 msgr2=0x55612f6f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72a1acc640 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f3121b0 0x55612f6f2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72a1acc640 1 -- 172.21.15.112:0/3448285376 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612f767d10 con 0x55612f2f99f0 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72857fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72a1acc640 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6f27e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f72980b34e0 tx=0x7f72980b37c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acee1225d01d6f8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72857fa640 1 -- 172.21.15.112:0/3448285376 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980ad210 con 0x55612f2f99f0 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72857fa640 1 -- 172.21.15.112:0/3448285376 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f7298018b20 con 0x55612f2f99f0 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3448285376 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55612f6568a0 con 0x55612f2f99f0 2024-09-21T03:40:40.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72857fa640 1 -- 172.21.15.112:0/3448285376 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72980b7230 con 0x55612f2f99f0 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72aa2a4740 10 client.26975 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3448285376 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55612f76f100 con 0x55612f2f99f0 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72857fa640 1 -- 172.21.15.112:0/3448285376 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55612f6568a0 con 0x55612f2f99f0 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.987+0000 7f72857fa640 1 -- 172.21.15.112:0/3448285376 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f72980b6770 con 0x55612f2f99f0 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72aa2a4740 10 client.26975 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72aa2a4740 10 client.26975 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72aa2a4740 20 client.26975 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55612f33b070 0x55612f7715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72aa2a4740 4 client.26975 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3448285376 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55612f653cd0 con 0x55612f33b070 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72a12cb640 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55612f33b070 0x55612f7715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72857fa640 10 client.26975 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72a12cb640 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55612f33b070 0x55612f7715f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be9e84c237c4b871 server_cookie=1765a387c6140fc in_seq=0 out_seq=0 2024-09-21T03:40:40.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72857fa640 1 -- 172.21.15.112:0/3448285376 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55612f653cd0 con 0x55612f33b070 2024-09-21T03:40:40.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72857fa640 10 client.26975 handle_command_reply: tid=0 2024-09-21T03:40:40.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72aa2a4740 1 client.26975 shutdown 2024-09-21T03:40:40.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3448285376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 msgr2=0x55612f6f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:40.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.988+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6f27e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f72980b34e0 tx=0x7f72980b37c0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.010 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:40:40.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3448285376 shutdown_connections 2024-09-21T03:40:40.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55612f33b070 0x55612f7715f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55612f31a390 0x55612f7676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612f3121b0 0x55612f6f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72857fa640 0 client.26975 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72aa2a4740 1 --2- 172.21.15.112:0/3448285376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612f2f99f0 0x55612f6f27e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3448285376 >> 172.21.15.112:0/3448285376 conn(0x55612f32f460 msgr2=0x55612f6ec510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3448285376 shutdown_connections 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72aa2a4740 1 -- 172.21.15.112:0/3448285376 wait complete. 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72aa2a4740 20 client.26975 trim_cache size 0 max 16384 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72a3d30640 1 -- 172.21.15.112:0/3119054627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72880b6700 msgr2=0x7f72880b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72a3d30640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72880b6700 0x7f72880b8c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7294006010 tx=0x7f729402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72a3d30640 1 -- 172.21.15.112:0/3119054627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729c102270 msgr2=0x7f729c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72a3d30640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729c102270 0x7f729c19a4c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f729800e9f0 tx=0x7f729800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72a3d30640 1 -- 172.21.15.112:0/3119054627 shutdown_connections 2024-09-21T03:40:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72a3d30640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f72880b6700 0x7f72880b8c10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72a3d30640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f729c1084b0 0x7f729c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72a3d30640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f729c107b60 0x7f729c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72a3d30640 1 --2- 172.21.15.112:0/3119054627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f729c102270 0x7f729c19a4c0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72a3d30640 1 -- 172.21.15.112:0/3119054627 >> 172.21.15.112:0/3119054627 conn(0x7f729c077cc0 msgr2=0x7f729c10f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.989+0000 7f72a3d30640 1 -- 172.21.15.112:0/3119054627 shutdown_connections 2024-09-21T03:40:40.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:39.990+0000 7f72a3d30640 1 -- 172.21.15.112:0/3119054627 wait complete. 2024-09-21T03:40:40.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1237453 10.31 MB/sec execute 520 sec latency 39.857 ms 2024-09-21T03:40:40.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:40 smithi112 ceph-mon[31442]: pgmap v679: 97 pgs: 97 active+clean; 898 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 221 op/s 2024-09-21T03:40:40.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:40 smithi203 ceph-mon[34723]: pgmap v679: 97 pgs: 97 active+clean; 898 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 221 op/s 2024-09-21T03:40:40.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:40 smithi187 ceph-mon[34603]: pgmap v679: 97 pgs: 97 active+clean; 898 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 221 op/s 2024-09-21T03:40:40.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:40:41.154 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:41.276 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1239708 10.32 MB/sec execute 521 sec latency 70.747 ms 2024-09-21T03:40:41.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.687+0000 7fa8a2486640 1 -- 172.21.15.112:0/1017339265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa89c108d00 msgr2=0x7fa89c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.687+0000 7fa8a2486640 1 --2- 172.21.15.112:0/1017339265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa89c108d00 0x7fa89c10f530 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa894009ca0 tx=0x7fa89402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.687+0000 7fa8a2486640 1 -- 172.21.15.112:0/1017339265 shutdown_connections 2024-09-21T03:40:41.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.687+0000 7fa8a2486640 1 --2- 172.21.15.112:0/1017339265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa89c108d00 0x7fa89c10f530 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.687+0000 7fa8a2486640 1 --2- 172.21.15.112:0/1017339265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa89c1083b0 0x7fa89c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.687+0000 7fa8a2486640 1 --2- 172.21.15.112:0/1017339265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa89c102a70 0x7fa89c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.687+0000 7fa8a2486640 1 -- 172.21.15.112:0/1017339265 >> 172.21.15.112:0/1017339265 conn(0x7fa89c079640 msgr2=0x7fa89c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.687+0000 7fa8a2486640 1 -- 172.21.15.112:0/1017339265 shutdown_connections 2024-09-21T03:40:41.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.687+0000 7fa8a2486640 1 -- 172.21.15.112:0/1017339265 wait complete. 2024-09-21T03:40:41.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.688+0000 7fa8a2486640 1 Processor -- start 2024-09-21T03:40:41.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.688+0000 7fa8a2486640 1 -- start start 2024-09-21T03:40:41.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.688+0000 7fa8a2486640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa89c102a70 0x7fa89c0ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.688+0000 7fa8a2486640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa89c1083b0 0x7fa89c0ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.688+0000 7fa8a2486640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa89c108d00 0x7fa89c100120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.688+0000 7fa8a2486640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1129c0 con 0x7fa89c1083b0 2024-09-21T03:40:41.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa8a2486640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c112840 con 0x7fa89c102a70 2024-09-21T03:40:41.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa8a2486640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c112b40 con 0x7fa89c108d00 2024-09-21T03:40:41.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa89b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa89c1083b0 0x7fa89c0ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa89b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa89c1083b0 0x7fa89c0ffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44442/0 (socket says 172.21.15.112:44442) 2024-09-21T03:40:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa89b7fe640 1 -- 172.21.15.112:0/3341334388 learned_addr learned my addr 172.21.15.112:0/3341334388 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa8a0a23640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa89c108d00 0x7fa89c100120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa89bfff640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa89c102a70 0x7fa89c0ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa89b7fe640 1 -- 172.21.15.112:0/3341334388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa89c108d00 msgr2=0x7fa89c100120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa89b7fe640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa89c108d00 0x7fa89c100120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa89b7fe640 1 -- 172.21.15.112:0/3341334388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa89c102a70 msgr2=0x7fa89c0ff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa89b7fe640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa89c102a70 0x7fa89c0ff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa89b7fe640 1 -- 172.21.15.112:0/3341334388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c1a8690 con 0x7fa89c1083b0 2024-09-21T03:40:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa89b7fe640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa89c1083b0 0x7fa89c0ffbe0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa89000e9c0 tx=0x7fa89000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa8997fa640 1 -- 172.21.15.112:0/3341334388 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89000ccc0 con 0x7fa89c1083b0 2024-09-21T03:40:41.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa8997fa640 1 -- 172.21.15.112:0/3341334388 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8900054d0 con 0x7fa89c1083b0 2024-09-21T03:40:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa8a2486640 1 -- 172.21.15.112:0/3341334388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89c1a8980 con 0x7fa89c1083b0 2024-09-21T03:40:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.689+0000 7fa8997fa640 1 -- 172.21.15.112:0/3341334388 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890010430 con 0x7fa89c1083b0 2024-09-21T03:40:41.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.690+0000 7fa8a2486640 1 -- 172.21.15.112:0/3341334388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89c1a8e70 con 0x7fa89c1083b0 2024-09-21T03:40:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.692+0000 7fa8a89fa740 1 Processor -- start 2024-09-21T03:40:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.692+0000 7fa8a89fa740 1 -- start start 2024-09-21T03:40:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.692+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x5575800336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.692+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc631b0 0x557580033be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.692+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc6b390 0x557580034120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.692+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55757ffa9ea0 con 0x55757fc8c070 2024-09-21T03:40:41.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.692+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55757ff992d0 con 0x55757fc631b0 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.692+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55757ff9d0e0 con 0x55757fc6b390 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8997fa640 1 -- 172.21.15.112:0/3341334388 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa8900105d0 con 0x7fa89c1083b0 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8a0a23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x5575800336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8a0a23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x5575800336a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44444/0 (socket says 172.21.15.112:44444) 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa89b7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc6b390 0x557580034120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8a0a23640 1 -- 172.21.15.112:0/2817120497 learned_addr learned my addr 172.21.15.112:0/2817120497 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa89bfff640 1 --2- 172.21.15.112:0/2817120497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc631b0 0x557580033be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8a0a23640 1 -- 172.21.15.112:0/2817120497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc6b390 msgr2=0x557580034120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8a0a23640 1 --2- 172.21.15.112:0/2817120497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc6b390 0x557580034120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8a0a23640 1 -- 172.21.15.112:0/2817120497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc631b0 msgr2=0x557580033be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8a0a23640 1 --2- 172.21.15.112:0/2817120497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc631b0 0x557580033be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8a0a23640 1 -- 172.21.15.112:0/2817120497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757fcf7650 con 0x55757fc8c070 2024-09-21T03:40:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8a0a23640 1 --2- 172.21.15.112:0/2817120497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x5575800336a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa8940098b0 tx=0x7fa8940098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fbfabce2263d58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8827fc640 1 -- 172.21.15.112:0/2817120497 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894031e30 con 0x55757fc8c070 2024-09-21T03:40:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8997fa640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa8840b6700 0x7fa8840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8827fc640 1 -- 172.21.15.112:0/2817120497 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa894036030 con 0x55757fc8c070 2024-09-21T03:40:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.693+0000 7fa8827fc640 1 -- 172.21.15.112:0/2817120497 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894004260 con 0x55757fc8c070 2024-09-21T03:40:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.694+0000 7fa89bfff640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa8840b6700 0x7fa8840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.694+0000 7fa8997fa640 1 -- 172.21.15.112:0/3341334388 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7fa8900e7ba0 con 0x7fa89c1083b0 2024-09-21T03:40:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.694+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2817120497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 msgr2=0x5575800336a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.694+0000 7fa89bfff640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa8840b6700 0x7fa8840b8c10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x557580032320 tx=0x7fa88c00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.694+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/2817120497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x5575800336a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa8940098b0 tx=0x7fa8940098e0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.694+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2817120497 shutdown_connections 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.694+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/2817120497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc6b390 0x557580034120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.694+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/2817120497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc631b0 0x557580033be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.694+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/2817120497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x5575800336a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.694+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2817120497 >> 172.21.15.112:0/2817120497 conn(0x55757fc80460 msgr2=0x557580031590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.694+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2817120497 shutdown_connections 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.695+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2817120497 wait complete. 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.695+0000 7fa8a89fa740 1 -- start start 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.695+0000 7fa8a89fa740 1 Processor -- start 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.695+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc631b0 0x55758003ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.695+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc6b390 0x55758003f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.695+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x55758004af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.695+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55757fdfa700 con 0x55757fc8c070 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.695+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55757ff34160 con 0x55757fc631b0 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.695+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557580041ca0 con 0x55757fc6b390 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa8a89fa740 10 client.0 _ll_register_callbacks cb 0x55757ff4d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa8a0a23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x55758004af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa8a0a23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x55758004af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44452/0 (socket says 172.21.15.112:44452) 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa8a0a23640 1 -- 172.21.15.112:0/1140531081 learned_addr learned my addr 172.21.15.112:0/1140531081 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa89b7fe640 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc6b390 0x55758003f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa89bfff640 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc631b0 0x55758003ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa8a0a23640 1 -- 172.21.15.112:0/1140531081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc6b390 msgr2=0x55758003f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa8a0a23640 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc6b390 0x55758003f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa8a0a23640 1 -- 172.21.15.112:0/1140531081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc631b0 msgr2=0x55758003ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa8a0a23640 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc631b0 0x55758003ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa8a0a23640 1 -- 172.21.15.112:0/1140531081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757ff61c90 con 0x55757fc8c070 2024-09-21T03:40:41.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa89b7fe640 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc6b390 0x55758003f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa8a0a23640 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x55758004af90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa894003ca0 tx=0x7fa894003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf3ee60bb815b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.696+0000 7fa87b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:40:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa87b7fe640 1 -- 172.21.15.112:0/1140531081 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89404a040 con 0x55757fc8c070 2024-09-21T03:40:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa87b7fe640 1 -- 172.21.15.112:0/1140531081 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7fa894031650 con 0x55757fc8c070 2024-09-21T03:40:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa8a89fa740 1 -- 172.21.15.112:0/1140531081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa894036030 con 0x55757fc8c070 2024-09-21T03:40:41.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa87b7fe640 1 -- 172.21.15.112:0/1140531081 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa894031850 con 0x55757fc8c070 2024-09-21T03:40:41.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa8a89fa740 10 client.17433 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:41.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa8a89fa740 1 -- 172.21.15.112:0/1140531081 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55758004d9e0 con 0x55757fc8c070 2024-09-21T03:40:41.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa87b7fe640 1 -- 172.21.15.112:0/1140531081 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa894036030 con 0x55757fc8c070 2024-09-21T03:40:41.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa87b7fe640 1 -- 172.21.15.112:0/1140531081 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa89404d030 con 0x55757fc8c070 2024-09-21T03:40:41.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa8a89fa740 10 client.17433 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:41.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa8a89fa740 10 client.17433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:40:41.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa8a89fa740 20 client.17433 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:41.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55757fc4a9f0 0x557580050d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa8a89fa740 4 client.17433 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:41.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.697+0000 7fa8a89fa740 1 -- 172.21.15.112:0/1140531081 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55757ffabdf0 con 0x55757fc4a9f0 2024-09-21T03:40:41.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.698+0000 7fa89bfff640 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55757fc4a9f0 0x557580050d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.698+0000 7fa89bfff640 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55757fc4a9f0 0x557580050d40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e11117454ddcc1f1 server_cookie=47330362af7bb19a in_seq=0 out_seq=0 2024-09-21T03:40:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.698+0000 7fa87b7fe640 10 client.17433 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:40:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa87b7fe640 1 -- 172.21.15.112:0/1140531081 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55757ffabdf0 con 0x55757fc4a9f0 2024-09-21T03:40:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa87b7fe640 10 client.17433 handle_command_reply: tid=0 2024-09-21T03:40:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 1 client.17433 shutdown 2024-09-21T03:40:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 1 -- 172.21.15.112:0/1140531081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 msgr2=0x55758004af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x55758004af90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa894003ca0 tx=0x7fa894003cd0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 1 -- 172.21.15.112:0/1140531081 shutdown_connections 2024-09-21T03:40:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55757fc4a9f0 0x557580050d40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x55758004af90 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc6b390 0x55758003f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa87b7fe640 0 client.17433 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:40:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/1140531081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc631b0 0x55758003ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 1 -- 172.21.15.112:0/1140531081 >> 172.21.15.112:0/1140531081 conn(0x55757fc80460 msgr2=0x557580036880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 1 -- 172.21.15.112:0/1140531081 shutdown_connections 2024-09-21T03:40:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 1 -- 172.21.15.112:0/1140531081 wait complete. 2024-09-21T03:40:41.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.699+0000 7fa8a89fa740 20 client.17433 trim_cache size 0 max 16384 2024-09-21T03:40:41.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa8a89fa740 1 Processor -- start 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa8a89fa740 1 -- start start 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc631b0 0x55758003f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 0x55758004ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc8c070 0x55758003bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1b3ff0 con 0x55757fc631b0 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1b18c0 con 0x55757fc6b390 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1af730 con 0x55757fc8c070 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa89bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc631b0 0x55758003f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa89bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc631b0 0x55758003f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44466/0 (socket says 172.21.15.112:44466) 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa89bfff640 1 -- 172.21.15.112:0/3760490633 learned_addr learned my addr 172.21.15.112:0/3760490633 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.713+0000 7fa8a0a23640 1 --2- 172.21.15.112:0/3760490633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc8c070 0x55758003bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.714+0000 7fa89b7fe640 1 --2- 172.21.15.112:0/3760490633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 0x55758004ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.714+0000 7fa89bfff640 1 -- 172.21.15.112:0/3760490633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc8c070 msgr2=0x55758003bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.714+0000 7fa89bfff640 1 --2- 172.21.15.112:0/3760490633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc8c070 0x55758003bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.714+0000 7fa89bfff640 1 -- 172.21.15.112:0/3760490633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 msgr2=0x55758004ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.714+0000 7fa89bfff640 1 --2- 172.21.15.112:0/3760490633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 0x55758004ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.714+0000 7fa89bfff640 1 -- 172.21.15.112:0/3760490633 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557580035990 con 0x55757fc631b0 2024-09-21T03:40:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.714+0000 7fa89b7fe640 1 --2- 172.21.15.112:0/3760490633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 0x55758004ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.714+0000 7fa89bfff640 1 --2- 172.21.15.112:0/3760490633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc631b0 0x55758003f2a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00ab10 tx=0x7fa88c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1619748669fcdab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.714+0000 7fa881ffb640 1 -- 172.21.15.112:0/3760490633 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c033070 con 0x55757fc631b0 2024-09-21T03:40:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.714+0000 7fa881ffb640 1 -- 172.21.15.112:0/3760490633 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa88c01bd90 con 0x55757fc631b0 2024-09-21T03:40:41.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.714+0000 7fa881ffb640 1 -- 172.21.15.112:0/3760490633 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c028750 con 0x55757fc631b0 2024-09-21T03:40:41.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.715+0000 7fa8a89fa740 1 -- 172.21.15.112:0/3760490633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc631b0 msgr2=0x55758003f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.715+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/3760490633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc631b0 0x55758003f2a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00ab10 tx=0x7fa88c019000 comp rx=0 tx=0).stop 2024-09-21T03:40:41.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.715+0000 7fa8a89fa740 1 -- 172.21.15.112:0/3760490633 shutdown_connections 2024-09-21T03:40:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.715+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/3760490633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc8c070 0x55758003bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.715+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/3760490633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 0x55758004ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.715+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/3760490633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc631b0 0x55758003f2a0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.715+0000 7fa8a89fa740 1 -- 172.21.15.112:0/3760490633 >> 172.21.15.112:0/3760490633 conn(0x55757fc80460 msgr2=0x55758003eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.715+0000 7fa8a89fa740 1 -- 172.21.15.112:0/3760490633 shutdown_connections 2024-09-21T03:40:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.715+0000 7fa8a89fa740 1 -- 172.21.15.112:0/3760490633 wait complete. 2024-09-21T03:40:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa8a89fa740 1 -- start start 2024-09-21T03:40:41.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa8a89fa740 1 Processor -- start 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc631b0 0x557580043050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 0x557580043590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa8a89fa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x557580043ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1afd80 con 0x55757fc8c070 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1b11f0 con 0x55757fc6b390 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa8a89fa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1b0450 con 0x55757fc631b0 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa8a89fa740 10 client.0 _ll_register_callbacks cb 0x7fa88c00a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa89bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc631b0 0x557580043050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa89b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 0x557580043590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa8a0a23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x557580043ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa89bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc631b0 0x557580043050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:33208/0 (socket says 172.21.15.112:33208) 2024-09-21T03:40:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa89bfff640 1 -- 172.21.15.112:0/2871074324 learned_addr learned my addr 172.21.15.112:0/2871074324 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.716+0000 7fa89b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 0x557580043590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49546/0 (socket says 172.21.15.112:49546) 2024-09-21T03:40:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa89b7fe640 1 -- 172.21.15.112:0/2871074324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc631b0 msgr2=0x557580043050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa89b7fe640 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc631b0 0x557580043050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa89b7fe640 1 -- 172.21.15.112:0/2871074324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 msgr2=0x557580043ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa89b7fe640 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x557580043ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa89b7fe640 1 -- 172.21.15.112:0/2871074324 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575800c2ef0 con 0x55757fc6b390 2024-09-21T03:40:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa8a0a23640 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x557580043ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa89bfff640 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc631b0 0x557580043050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:41.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa89b7fe640 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 0x557580043590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa8900e52c0 tx=0x7fa8900b71d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b8d111c4688f610 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:41.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa87b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:40:41.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa87b7fe640 1 -- 172.21.15.112:0/2871074324 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8900a9860 con 0x55757fc6b390 2024-09-21T03:40:41.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa87b7fe640 1 -- 172.21.15.112:0/2871074324 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7fa8900ac690 con 0x55757fc6b390 2024-09-21T03:40:41.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2871074324 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa88c01bd90 con 0x55757fc6b390 2024-09-21T03:40:41.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa87b7fe640 1 -- 172.21.15.112:0/2871074324 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8900a9130 con 0x55757fc6b390 2024-09-21T03:40:41.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa8a89fa740 10 client.26990 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:41.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2871074324 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5575800c41d0 con 0x55757fc6b390 2024-09-21T03:40:41.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.717+0000 7fa87b7fe640 1 -- 172.21.15.112:0/2871074324 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa890097030 con 0x55757fc6b390 2024-09-21T03:40:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.718+0000 7fa87b7fe640 1 -- 172.21.15.112:0/2871074324 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa8900ac890 con 0x55757fc6b390 2024-09-21T03:40:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.718+0000 7fa8a89fa740 10 client.26990 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.718+0000 7fa8a89fa740 10 client.26990 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:40:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.718+0000 7fa8a89fa740 20 client.26990 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.718+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55757fc1fda0 0x5575800c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.718+0000 7fa8a89fa740 4 client.26990 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:40:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.718+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2871074324 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55757ffa78a0 con 0x55757fc1fda0 2024-09-21T03:40:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.718+0000 7fa89bfff640 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55757fc1fda0 0x5575800c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa87b7fe640 10 client.26990 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:40:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa89bfff640 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55757fc1fda0 0x5575800c69e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b51ebfe79719b664 server_cookie=9d7fd1ec6f6120f6 in_seq=0 out_seq=0 2024-09-21T03:40:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa87b7fe640 1 -- 172.21.15.112:0/2871074324 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55757ffa78a0 con 0x55757fc1fda0 2024-09-21T03:40:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa87b7fe640 10 client.26990 handle_command_reply: tid=0 2024-09-21T03:40:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 1 client.26990 shutdown 2024-09-21T03:40:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2871074324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 msgr2=0x557580043590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 0x557580043590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa8900e52c0 tx=0x7fa8900b71d0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2871074324 shutdown_connections 2024-09-21T03:40:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x55757fc1fda0 0x5575800c69e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55757fc8c070 0x557580043ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55757fc6b390 0x557580043590 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa87b7fe640 0 client.26990 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:40:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 1 --2- 172.21.15.112:0/2871074324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55757fc631b0 0x557580043050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2871074324 >> 172.21.15.112:0/2871074324 conn(0x55757fc80460 msgr2=0x5575800346b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2871074324 shutdown_connections 2024-09-21T03:40:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 1 -- 172.21.15.112:0/2871074324 wait complete. 2024-09-21T03:40:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.719+0000 7fa8a89fa740 20 client.26990 trim_cache size 0 max 16384 2024-09-21T03:40:41.729 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:40:41.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 -- 172.21.15.112:0/3341334388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa8840b6700 msgr2=0x7fa8840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa8840b6700 0x7fa8840b8c10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x557580032320 tx=0x7fa88c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 -- 172.21.15.112:0/3341334388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa89c1083b0 msgr2=0x7fa89c0ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:41.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa89c1083b0 0x7fa89c0ffbe0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa89000e9c0 tx=0x7fa89000edc0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 -- 172.21.15.112:0/3341334388 shutdown_connections 2024-09-21T03:40:41.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa8840b6700 0x7fa8840b8c10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa89c108d00 0x7fa89c100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa89c1083b0 0x7fa89c0ffbe0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 --2- 172.21.15.112:0/3341334388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa89c102a70 0x7fa89c0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:41.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 -- 172.21.15.112:0/3341334388 >> 172.21.15.112:0/3341334388 conn(0x7fa89c079640 msgr2=0x7fa89c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:41.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 -- 172.21.15.112:0/3341334388 shutdown_connections 2024-09-21T03:40:41.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:41.720+0000 7fa8a2486640 1 -- 172.21.15.112:0/3341334388 wait complete. 2024-09-21T03:40:42.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1240976 10.31 MB/sec execute 522 sec latency 30.480 ms 2024-09-21T03:40:42.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:42 smithi203 ceph-mon[34723]: pgmap v680: 97 pgs: 97 active+clean; 903 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 268 op/s 2024-09-21T03:40:42.422 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:42 smithi112 ceph-mon[31442]: pgmap v680: 97 pgs: 97 active+clean; 903 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 268 op/s 2024-09-21T03:40:42.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:42 smithi187 ceph-mon[34603]: pgmap v680: 97 pgs: 97 active+clean; 903 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 268 op/s 2024-09-21T03:40:42.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:40:42.718 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:43.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.218+0000 7f4e87fb1640 1 -- 172.21.15.112:0/3882420930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80108450 msgr2=0x7f4e80108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.219+0000 7f4e87fb1640 1 --2- 172.21.15.112:0/3882420930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80108450 0x7f4e80108860 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009900 tx=0x7f4e7002a410 comp rx=0 tx=0).stop 2024-09-21T03:40:43.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.219+0000 7f4e87fb1640 1 -- 172.21.15.112:0/3882420930 shutdown_connections 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.219+0000 7f4e87fb1640 1 --2- 172.21.15.112:0/3882420930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e80108da0 0x7f4e8010f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.219+0000 7f4e87fb1640 1 --2- 172.21.15.112:0/3882420930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80108450 0x7f4e80108860 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.219+0000 7f4e87fb1640 1 --2- 172.21.15.112:0/3882420930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e80102b10 0x7f4e80102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.219+0000 7f4e87fb1640 1 -- 172.21.15.112:0/3882420930 >> 172.21.15.112:0/3882420930 conn(0x7f4e800796d0 msgr2=0x7f4e80079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.219+0000 7f4e87fb1640 1 -- 172.21.15.112:0/3882420930 shutdown_connections 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.219+0000 7f4e87fb1640 1 -- 172.21.15.112:0/3882420930 wait complete. 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.219+0000 7f4e87fb1640 1 Processor -- start 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.219+0000 7f4e87fb1640 1 -- start start 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e87fb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80102b10 0x7f4e8019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e87fb1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e80108450 0x7f4e8019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e87fb1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e80108da0 0x7f4e8019fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e87fb1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80112a60 con 0x7f4e80102b10 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e87fb1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801128e0 con 0x7f4e80108450 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e87fb1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80112be0 con 0x7f4e80108da0 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e85d4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80102b10 0x7f4e8019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e85d4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80102b10 0x7f4e8019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44488/0 (socket says 172.21.15.112:44488) 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e85d4d640 1 -- 172.21.15.112:0/2953457264 learned_addr learned my addr 172.21.15.112:0/2953457264 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e8654e640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e80108da0 0x7f4e8019fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e8554c640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e80108450 0x7f4e8019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e85d4d640 1 -- 172.21.15.112:0/2953457264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e80108da0 msgr2=0x7f4e8019fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e80108da0 0x7f4e8019fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e85d4d640 1 -- 172.21.15.112:0/2953457264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e80108450 msgr2=0x7f4e8019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e80108450 0x7f4e8019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e85d4d640 1 -- 172.21.15.112:0/2953457264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e801a05d0 con 0x7f4e80102b10 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e8554c640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e80108450 0x7f4e8019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80102b10 0x7f4e8019a660 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00e9f0 tx=0x7f4e7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.220+0000 7f4e76ffd640 1 -- 172.21.15.112:0/2953457264 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00cd80 con 0x7f4e80102b10 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.221+0000 7f4e76ffd640 1 -- 172.21.15.112:0/2953457264 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e7c0108e0 con 0x7f4e80102b10 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.221+0000 7f4e87fb1640 1 -- 172.21.15.112:0/2953457264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e801a08c0 con 0x7f4e80102b10 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.221+0000 7f4e76ffd640 1 -- 172.21.15.112:0/2953457264 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c018980 con 0x7f4e80102b10 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.221+0000 7f4e87fb1640 1 -- 172.21.15.112:0/2953457264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e801a7020 con 0x7f4e80102b10 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.223+0000 7f4e76ffd640 1 -- 172.21.15.112:0/2953457264 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f4e7c010430 con 0x7f4e80102b10 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.223+0000 7f4e8e525740 1 Processor -- start 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8e525740 1 -- start start 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda6a070 0x55ddede116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede11be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda49390 0x55ddede12120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8e525740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ddedd87ea0 con 0x55ddeda49390 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8e525740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ddedd772d0 con 0x55ddeda411b0 2024-09-21T03:40:43.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8e525740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ddedd7b0e0 con 0x55ddeda6a070 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8554c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda49390 0x55ddede12120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8554c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda49390 0x55ddede12120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44498/0 (socket says 172.21.15.112:44498) 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8554c640 1 -- 172.21.15.112:0/1016920717 learned_addr learned my addr 172.21.15.112:0/1016920717 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/1016920717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede11be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8654e640 1 --2- 172.21.15.112:0/1016920717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda6a070 0x55ddede116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8554c640 1 -- 172.21.15.112:0/1016920717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda6a070 msgr2=0x55ddede116a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8554c640 1 --2- 172.21.15.112:0/1016920717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda6a070 0x55ddede116a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8554c640 1 -- 172.21.15.112:0/1016920717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 msgr2=0x55ddede11be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8554c640 1 --2- 172.21.15.112:0/1016920717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede11be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e8554c640 1 -- 172.21.15.112:0/1016920717 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddedad5650 con 0x55ddeda49390 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.224+0000 7f4e76ffd640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e6c0b6680 0x7f4e6c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.225+0000 7f4e8654e640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e6c0b6680 0x7f4e6c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.225+0000 7f4e8554c640 1 --2- 172.21.15.112:0/1016920717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda49390 0x55ddede12120 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4e70031440 tx=0x7f4e70032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c09f5969392f35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.225+0000 7f4e76ffd640 1 -- 172.21.15.112:0/2953457264 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f4e7c0e7ac0 con 0x7f4e80102b10 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.225+0000 7f4e6bfff640 1 -- 172.21.15.112:0/1016920717 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7003f070 con 0x55ddeda49390 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.225+0000 7f4e6bfff640 1 -- 172.21.15.112:0/1016920717 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e700316a0 con 0x55ddeda49390 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.225+0000 7f4e6bfff640 1 -- 172.21.15.112:0/1016920717 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70035070 con 0x55ddeda49390 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8e525740 1 -- 172.21.15.112:0/1016920717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda49390 msgr2=0x55ddede12120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8654e640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e6c0b6680 0x7f4e6c0b8b90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4e78005fd0 tx=0x7f4e78009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8e525740 1 --2- 172.21.15.112:0/1016920717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda49390 0x55ddede12120 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4e70031440 tx=0x7f4e70032c50 comp rx=0 tx=0).stop 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8e525740 1 -- 172.21.15.112:0/1016920717 shutdown_connections 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8e525740 1 --2- 172.21.15.112:0/1016920717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda49390 0x55ddede12120 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8e525740 1 --2- 172.21.15.112:0/1016920717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede11be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8e525740 1 --2- 172.21.15.112:0/1016920717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda6a070 0x55ddede116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8e525740 1 -- 172.21.15.112:0/1016920717 >> 172.21.15.112:0/1016920717 conn(0x55ddeda5e460 msgr2=0x55ddede0f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8e525740 1 -- 172.21.15.112:0/1016920717 shutdown_connections 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8e525740 1 -- 172.21.15.112:0/1016920717 wait complete. 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8e525740 1 -- start start 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.226+0000 7f4e8e525740 1 Processor -- start 2024-09-21T03:40:43.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede1ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede1d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8e525740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ddedbd8700 con 0x55ddeda6a070 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8e525740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ddedd12160 con 0x55ddeda411b0 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8e525740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ddede1fe20 con 0x55ddeda49390 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8e525740 10 client.0 _ll_register_callbacks cb 0x55ddedd2b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8654e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8654e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede1d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44500/0 (socket says 172.21.15.112:44500) 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8654e640 1 -- 172.21.15.112:0/510161783 learned_addr learned my addr 172.21.15.112:0/510161783 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8554c640 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede1d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede1ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8554c640 1 -- 172.21.15.112:0/510161783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 msgr2=0x55ddede1ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8554c640 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede1ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8554c640 1 -- 172.21.15.112:0/510161783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 msgr2=0x55ddede1d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8554c640 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede1d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8554c640 1 -- 172.21.15.112:0/510161783 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddedd3fc90 con 0x55ddeda49390 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.227+0000 7f4e8654e640 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede1d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e8554c640 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede1d200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4e7002a450 tx=0x7f4e70032f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5f62ad0111397f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e697fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e697fa640 1 -- 172.21.15.112:0/510161783 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7003f070 con 0x55ddeda49390 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e697fa640 1 -- 172.21.15.112:0/510161783 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f4e70004440 con 0x55ddeda49390 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e8e525740 1 -- 172.21.15.112:0/510161783 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e700316a0 con 0x55ddeda49390 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e697fa640 1 -- 172.21.15.112:0/510161783 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e70054050 con 0x55ddeda49390 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e8e525740 10 client.27010 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e8e525740 1 -- 172.21.15.112:0/510161783 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ddede2baa0 con 0x55ddeda49390 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e697fa640 1 -- 172.21.15.112:0/510161783 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e700316a0 con 0x55ddeda49390 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e697fa640 1 -- 172.21.15.112:0/510161783 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4e70004860 con 0x55ddeda49390 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e8e525740 10 client.27010 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e8e525740 10 client.27010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:43.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e8e525740 20 client.27010 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.228+0000 7f4e8e525740 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ddeda289f0 0x55ddede2ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.229+0000 7f4e8e525740 4 client.27010 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.229+0000 7f4e8e525740 1 -- 172.21.15.112:0/510161783 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddedd89df0 con 0x55ddeda289f0 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.229+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ddeda289f0 0x55ddede2ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.229+0000 7f4e697fa640 10 client.27010 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.229+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ddeda289f0 0x55ddede2ee00 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02b20691e106ef5 server_cookie=6dae932e89c90fb1 in_seq=0 out_seq=0 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e697fa640 1 -- 172.21.15.112:0/510161783 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ddedd89df0 con 0x55ddeda289f0 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e697fa640 10 client.27010 handle_command_reply: tid=0 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 1 client.27010 shutdown 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 1 -- 172.21.15.112:0/510161783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 msgr2=0x55ddede1d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede1d200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4e7002a450 tx=0x7f4e70032f80 comp rx=0 tx=0).stop 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 1 -- 172.21.15.112:0/510161783 shutdown_connections 2024-09-21T03:40:43.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55ddeda289f0 0x55ddede2ee00 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede1d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede1d200 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e697fa640 0 client.27010 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:43.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 1 --2- 172.21.15.112:0/510161783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede1ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 1 -- 172.21.15.112:0/510161783 >> 172.21.15.112:0/510161783 conn(0x55ddeda5e460 msgr2=0x55ddede14830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 1 -- 172.21.15.112:0/510161783 shutdown_connections 2024-09-21T03:40:43.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 1 -- 172.21.15.112:0/510161783 wait complete. 2024-09-21T03:40:43.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.230+0000 7f4e8e525740 20 client.27010 trim_cache size 0 max 16384 2024-09-21T03:40:43.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8e525740 1 Processor -- start 2024-09-21T03:40:43.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8e525740 1 -- start start 2024-09-21T03:40:43.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede13090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede1e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede19eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8e525740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801b2050 con 0x55ddeda6a070 2024-09-21T03:40:43.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8e525740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801af920 con 0x55ddeda411b0 2024-09-21T03:40:43.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8e525740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801ad790 con 0x55ddeda49390 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8654e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede19eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8554c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede1e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8654e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede19eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44508/0 (socket says 172.21.15.112:44508) 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8654e640 1 -- 172.21.15.112:0/3564183562 learned_addr learned my addr 172.21.15.112:0/3564183562 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/3564183562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede13090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e85d4d640 1 -- 172.21.15.112:0/3564183562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 msgr2=0x55ddede1e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/3564183562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede1e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e85d4d640 1 -- 172.21.15.112:0/3564183562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 msgr2=0x55ddede19eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/3564183562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede19eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e85d4d640 1 -- 172.21.15.112:0/3564183562 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddede0f680 con 0x55ddeda411b0 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.244+0000 7f4e8654e640 1 --2- 172.21.15.112:0/3564183562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede19eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.245+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/3564183562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede13090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0e5240 tx=0x7f4e7c0a9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67c8e1144b3ffe7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.245+0000 7f4e6b7fe640 1 -- 172.21.15.112:0/3564183562 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0ad710 con 0x55ddeda411b0 2024-09-21T03:40:43.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.245+0000 7f4e6b7fe640 1 -- 172.21.15.112:0/3564183562 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e7c010720 con 0x55ddeda411b0 2024-09-21T03:40:43.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.245+0000 7f4e6b7fe640 1 -- 172.21.15.112:0/3564183562 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0a90b0 con 0x55ddeda411b0 2024-09-21T03:40:43.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.246+0000 7f4e8e525740 1 -- 172.21.15.112:0/3564183562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 msgr2=0x55ddede13090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.246+0000 7f4e8e525740 1 --2- 172.21.15.112:0/3564183562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede13090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0e5240 tx=0x7f4e7c0a9040 comp rx=0 tx=0).stop 2024-09-21T03:40:43.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.246+0000 7f4e8e525740 1 -- 172.21.15.112:0/3564183562 shutdown_connections 2024-09-21T03:40:43.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.246+0000 7f4e8e525740 1 --2- 172.21.15.112:0/3564183562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede19eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.246+0000 7f4e8e525740 1 --2- 172.21.15.112:0/3564183562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede1e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.246+0000 7f4e8e525740 1 --2- 172.21.15.112:0/3564183562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede13090 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.246+0000 7f4e8e525740 1 -- 172.21.15.112:0/3564183562 >> 172.21.15.112:0/3564183562 conn(0x55ddeda5e460 msgr2=0x55ddede1e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.246+0000 7f4e8e525740 1 -- 172.21.15.112:0/3564183562 shutdown_connections 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.246+0000 7f4e8e525740 1 -- 172.21.15.112:0/3564183562 wait complete. 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.246+0000 7f4e8e525740 1 -- start start 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.246+0000 7f4e8e525740 1 Processor -- start 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede32600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede2fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8e525740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede30010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8e525740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801adde0 con 0x55ddeda6a070 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8e525740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801af250 con 0x55ddeda411b0 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8e525740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801ae4b0 con 0x55ddeda49390 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8e525740 10 client.0 _ll_register_callbacks cb 0x7f4e7c0eb4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:43.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e85d4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede32600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8654e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede30010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8654e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede30010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44510/0 (socket says 172.21.15.112:44510) 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e85d4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede32600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49606/0 (socket says 172.21.15.112:49606) 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8654e640 1 -- 172.21.15.112:0/1465461342 learned_addr learned my addr 172.21.15.112:0/1465461342 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8554c640 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede2fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8654e640 1 -- 172.21.15.112:0/1465461342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 msgr2=0x55ddede2fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8654e640 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede2fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8654e640 1 -- 172.21.15.112:0/1465461342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 msgr2=0x55ddede32600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8654e640 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede32600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8654e640 1 -- 172.21.15.112:0/1465461342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddede30780 con 0x55ddeda6a070 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede32600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e8654e640 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede30010 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f4e780099e0 tx=0x7f4e78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1addc316beef86dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:43.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.247+0000 7f4e697fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e697fa640 1 -- 172.21.15.112:0/1465461342 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7800fcd0 con 0x55ddeda6a070 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e697fa640 1 -- 172.21.15.112:0/1465461342 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f4e78004b20 con 0x55ddeda6a070 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e8e525740 1 -- 172.21.15.112:0/1465461342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ddedd858a0 con 0x55ddeda6a070 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e697fa640 1 -- 172.21.15.112:0/1465461342 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e78004d60 con 0x55ddeda6a070 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e8e525740 10 client.17457 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e8e525740 1 -- 172.21.15.112:0/1465461342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ddede99500 con 0x55ddeda6a070 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e697fa640 1 -- 172.21.15.112:0/1465461342 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ddedd858a0 con 0x55ddeda6a070 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e697fa640 1 -- 172.21.15.112:0/1465461342 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4e7801e430 con 0x55ddeda6a070 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e8e525740 10 client.17457 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e8e525740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e8e525740 20 client.17457 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e8e525740 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55dded9fdda0 0x55ddede9bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e8e525740 4 client.17457 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e8e525740 1 -- 172.21.15.112:0/1465461342 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddedd82cd0 con 0x55dded9fdda0 2024-09-21T03:40:43.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.248+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55dded9fdda0 0x55ddede9bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e697fa640 10 client.17457 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e85d4d640 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55dded9fdda0 0x55ddede9bd70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77dd829c7fdf99ef server_cookie=28af1e265f0d17a9 in_seq=0 out_seq=0 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e697fa640 1 -- 172.21.15.112:0/1465461342 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddedd82cd0 con 0x55dded9fdda0 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e697fa640 10 client.17457 handle_command_reply: tid=0 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e8e525740 1 client.17457 shutdown 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e8e525740 1 -- 172.21.15.112:0/1465461342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 msgr2=0x55ddede30010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e8e525740 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede30010 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f4e780099e0 tx=0x7f4e78004a40 comp rx=0 tx=0).stop 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e8e525740 1 -- 172.21.15.112:0/1465461342 shutdown_connections 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e8e525740 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55dded9fdda0 0x55ddede9bd70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e8e525740 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ddeda6a070 0x55ddede30010 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e8e525740 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ddeda49390 0x55ddede2fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e8e525740 1 --2- 172.21.15.112:0/1465461342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ddeda411b0 0x55ddede32600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e697fa640 0 client.17457 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.249+0000 7f4e8e525740 1 -- 172.21.15.112:0/1465461342 >> 172.21.15.112:0/1465461342 conn(0x55ddeda5e460 msgr2=0x55ddede28a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e8e525740 1 -- 172.21.15.112:0/1465461342 shutdown_connections 2024-09-21T03:40:43.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e8e525740 1 -- 172.21.15.112:0/1465461342 wait complete. 2024-09-21T03:40:43.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e8e525740 20 client.17457 trim_cache size 0 max 16384 2024-09-21T03:40:43.257 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:40:43.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 -- 172.21.15.112:0/2953457264 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e6c0b6680 msgr2=0x7f4e6c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e6c0b6680 0x7f4e6c0b8b90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4e78005fd0 tx=0x7f4e78009500 comp rx=0 tx=0).stop 2024-09-21T03:40:43.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 -- 172.21.15.112:0/2953457264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80102b10 msgr2=0x7f4e8019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:43.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80102b10 0x7f4e8019a660 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00e9f0 tx=0x7f4e7c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 -- 172.21.15.112:0/2953457264 shutdown_connections 2024-09-21T03:40:43.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4e6c0b6680 0x7f4e6c0b8b90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e80108da0 0x7f4e8019fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e80108450 0x7f4e8019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 --2- 172.21.15.112:0/2953457264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80102b10 0x7f4e8019a660 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:43.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 -- 172.21.15.112:0/2953457264 >> 172.21.15.112:0/2953457264 conn(0x7f4e800796d0 msgr2=0x7f4e80101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:43.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 -- 172.21.15.112:0/2953457264 shutdown_connections 2024-09-21T03:40:43.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:43.250+0000 7f4e87fb1640 1 -- 172.21.15.112:0/2953457264 wait complete. 2024-09-21T03:40:43.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1243284 10.32 MB/sec execute 523 sec latency 19.395 ms 2024-09-21T03:40:44.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1244015 10.30 MB/sec execute 524 sec latency 257.986 ms 2024-09-21T03:40:44.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:44 smithi203 ceph-mon[34723]: pgmap v681: 97 pgs: 97 active+clean; 908 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 215 op/s 2024-09-21T03:40:44.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:44 smithi187 ceph-mon[34603]: pgmap v681: 97 pgs: 97 active+clean; 908 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 215 op/s 2024-09-21T03:40:44.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:44 smithi112 ceph-mon[31442]: pgmap v681: 97 pgs: 97 active+clean; 908 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 215 op/s 2024-09-21T03:40:45.131 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:40:45.131 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:40:45.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1246553 10.31 MB/sec execute 525 sec latency 21.432 ms 2024-09-21T03:40:45.392 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:45.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.921+0000 7f3731307640 1 -- 172.21.15.112:0/3161372461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c1049b0 msgr2=0x7f372c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.921+0000 7f3731307640 1 --2- 172.21.15.112:0/3161372461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c1049b0 0x7f372c104dc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f371c009900 tx=0x7f371c02a410 comp rx=0 tx=0).stop 2024-09-21T03:40:45.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.921+0000 7f3731307640 1 -- 172.21.15.112:0/3161372461 shutdown_connections 2024-09-21T03:40:45.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.921+0000 7f3731307640 1 --2- 172.21.15.112:0/3161372461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f372c10de10 0x7f372c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.921+0000 7f3731307640 1 --2- 172.21.15.112:0/3161372461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f372c105300 0x7f372c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.921+0000 7f3731307640 1 --2- 172.21.15.112:0/3161372461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c1049b0 0x7f372c104dc0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.921+0000 7f3731307640 1 -- 172.21.15.112:0/3161372461 >> 172.21.15.112:0/3161372461 conn(0x7f372c079730 msgr2=0x7f372c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.921+0000 7f3731307640 1 -- 172.21.15.112:0/3161372461 shutdown_connections 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.921+0000 7f3731307640 1 -- 172.21.15.112:0/3161372461 wait complete. 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f3731307640 1 Processor -- start 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f3731307640 1 -- start start 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f3731307640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c1049b0 0x7f372c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f3731307640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f372c105300 0x7f372c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f3731307640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f372c10de10 0x7f372c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f3731307640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f372c113ae0 con 0x7f372c1049b0 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f3731307640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f372c113960 con 0x7f372c105300 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f3731307640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f372c113c60 con 0x7f372c10de10 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f372affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c1049b0 0x7f372c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f372affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c1049b0 0x7f372c19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38178/0 (socket says 172.21.15.112:38178) 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f372affd640 1 -- 172.21.15.112:0/1418953197 learned_addr learned my addr 172.21.15.112:0/1418953197 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.922+0000 7f37227fc640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f372c105300 0x7f372c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f372b7fe640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f372c10de10 0x7f372c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f372affd640 1 -- 172.21.15.112:0/1418953197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f372c10de10 msgr2=0x7f372c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f372affd640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f372c10de10 0x7f372c19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f372affd640 1 -- 172.21.15.112:0/1418953197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f372c105300 msgr2=0x7f372c19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f372affd640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f372c105300 0x7f372c19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f372affd640 1 -- 172.21.15.112:0/1418953197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c1a0460 con 0x7f372c1049b0 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f37227fc640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f372c105300 0x7f372c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f372affd640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c1049b0 0x7f372c19a4f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f371c005ab0 tx=0x7f371c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f3728ff9640 1 -- 172.21.15.112:0/1418953197 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c03f070 con 0x7f372c1049b0 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f3728ff9640 1 -- 172.21.15.112:0/1418953197 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f371c031b50 con 0x7f372c1049b0 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f3731307640 1 -- 172.21.15.112:0/1418953197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c1a06f0 con 0x7f372c1049b0 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f3728ff9640 1 -- 172.21.15.112:0/1418953197 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c035030 con 0x7f372c1049b0 2024-09-21T03:40:45.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.923+0000 7f3731307640 1 -- 172.21.15.112:0/1418953197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c1a6eb0 con 0x7f372c1049b0 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.926+0000 7f3728ff9640 1 -- 172.21.15.112:0/1418953197 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f371c042070 con 0x7f372c1049b0 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.926+0000 7f373787b740 1 Processor -- start 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.926+0000 7f373787b740 1 -- start start 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.926+0000 7f373787b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ac5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.926+0000 7f373787b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86f5490 0x55b5f8ac5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.926+0000 7f373787b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8ac5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.926+0000 7f373787b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b5f8a3be60 con 0x55b5f871e350 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.926+0000 7f373787b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b5f8a2b290 con 0x55b5f86fd670 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.926+0000 7f373787b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b5f8a2f0a0 con 0x55b5f86f5490 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ac5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ac5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38190/0 (socket says 172.21.15.112:38190) 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f372b7fe640 1 -- 172.21.15.112:0/2115634362 learned_addr learned my addr 172.21.15.112:0/2115634362 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:45.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f372affd640 1 --2- 172.21.15.112:0/2115634362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86f5490 0x55b5f8ac5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f37227fc640 1 --2- 172.21.15.112:0/2115634362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8ac5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f3728ff9640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37180b6680 0x7f37180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f372b7fe640 1 -- 172.21.15.112:0/2115634362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86f5490 msgr2=0x55b5f8ac5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f372b7fe640 1 --2- 172.21.15.112:0/2115634362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86f5490 0x55b5f8ac5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f372b7fe640 1 -- 172.21.15.112:0/2115634362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 msgr2=0x55b5f8ac5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f372b7fe640 1 --2- 172.21.15.112:0/2115634362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8ac5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f372b7fe640 1 -- 172.21.15.112:0/2115634362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5f8789930 con 0x55b5f871e350 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f3728ff9640 1 -- 172.21.15.112:0/1418953197 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f371c107cb0 con 0x7f372c1049b0 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f372b7fe640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37180b6680 0x7f37180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.927+0000 7f372b7fe640 1 --2- 172.21.15.112:0/2115634362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ac5430 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3724011590 tx=0x7f3724011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b7ea6d382295cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.928+0000 7f37217fa640 1 -- 172.21.15.112:0/2115634362 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724017050 con 0x55b5f871e350 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.928+0000 7f37217fa640 1 -- 172.21.15.112:0/2115634362 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372400fcf0 con 0x55b5f871e350 2024-09-21T03:40:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.928+0000 7f372b7fe640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37180b6680 0x7f37180b8b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3724022000 tx=0x7f3724023000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.928+0000 7f37217fa640 1 -- 172.21.15.112:0/2115634362 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372401ba10 con 0x55b5f871e350 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.929+0000 7f373787b740 1 -- 172.21.15.112:0/2115634362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 msgr2=0x55b5f8ac5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.929+0000 7f373787b740 1 --2- 172.21.15.112:0/2115634362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ac5430 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3724011590 tx=0x7f3724011a50 comp rx=0 tx=0).stop 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.929+0000 7f373787b740 1 -- 172.21.15.112:0/2115634362 shutdown_connections 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.929+0000 7f373787b740 1 --2- 172.21.15.112:0/2115634362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8ac5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.929+0000 7f373787b740 1 --2- 172.21.15.112:0/2115634362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86f5490 0x55b5f8ac5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.929+0000 7f373787b740 1 --2- 172.21.15.112:0/2115634362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ac5430 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.929+0000 7f373787b740 1 -- 172.21.15.112:0/2115634362 >> 172.21.15.112:0/2115634362 conn(0x55b5f8712740 msgr2=0x55b5f8ac3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.929+0000 7f373787b740 1 -- 172.21.15.112:0/2115634362 shutdown_connections 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.929+0000 7f373787b740 1 -- 172.21.15.112:0/2115634362 wait complete. 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.929+0000 7f373787b740 1 -- start start 2024-09-21T03:40:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.929+0000 7f373787b740 1 Processor -- start 2024-09-21T03:40:45.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f373787b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8ad1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f373787b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86fd670 0x55b5f8ad15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f373787b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f871e350 0x55b5f8adcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f373787b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b5f888c890 con 0x55b5f86f5490 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f373787b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b5f89c5d40 con 0x55b5f871e350 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f373787b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b5f8ad3c10 con 0x55b5f86fd670 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f373787b740 10 client.0 _ll_register_callbacks cb 0x55b5f8a29c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f372affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8ad1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f372affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8ad1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38196/0 (socket says 172.21.15.112:38196) 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f372affd640 1 -- 172.21.15.112:0/1476666666 learned_addr learned my addr 172.21.15.112:0/1476666666 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f37227fc640 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86fd670 0x55b5f8ad15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f372b7fe640 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f871e350 0x55b5f8adcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f372affd640 1 -- 172.21.15.112:0/1476666666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86fd670 msgr2=0x55b5f8ad15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f372affd640 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86fd670 0x55b5f8ad15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f372affd640 1 -- 172.21.15.112:0/1476666666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f871e350 msgr2=0x55b5f8adcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f372affd640 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f871e350 0x55b5f8adcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f372affd640 1 -- 172.21.15.112:0/1476666666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5f8add540 con 0x55b5f86f5490 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f37227fc640 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86fd670 0x55b5f8ad15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f372b7fe640 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f871e350 0x55b5f8adcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f372affd640 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8ad1050 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f371c105490 tx=0x7f371c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734aae1b8ab024f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.930+0000 7f37127fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f37127fc640 1 -- 172.21.15.112:0/1476666666 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c049070 con 0x55b5f86f5490 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f37127fc640 1 -- 172.21.15.112:0/1476666666 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f371c0c9e50 con 0x55b5f86f5490 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f373787b740 1 -- 172.21.15.112:0/1476666666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f372400fcf0 con 0x55b5f86f5490 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f37127fc640 1 -- 172.21.15.112:0/1476666666 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f371c0cee60 con 0x55b5f86f5490 2024-09-21T03:40:45.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f373787b740 10 client.17469 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f373787b740 1 -- 172.21.15.112:0/1476666666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b5f8adfe70 con 0x55b5f86f5490 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f37127fc640 1 -- 172.21.15.112:0/1476666666 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372400fcf0 con 0x55b5f86f5490 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f37127fc640 1 -- 172.21.15.112:0/1476666666 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f371c0d70c0 con 0x55b5f86f5490 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f373787b740 10 client.17469 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f373787b740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f373787b740 20 client.17469 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f373787b740 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b5f86dccd0 0x55b5f8ae3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f373787b740 4 client.17469 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.931+0000 7f373787b740 1 -- 172.21.15.112:0/1476666666 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5f8a3ddb0 con 0x55b5f86dccd0 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.932+0000 7f37227fc640 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b5f86dccd0 0x55b5f8ae3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.932+0000 7f37127fc640 10 client.17469 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.932+0000 7f37227fc640 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b5f86dccd0 0x55b5f8ae3460 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a67545c420a7772 server_cookie=e1ad515fecc11dba in_seq=0 out_seq=0 2024-09-21T03:40:45.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f37127fc640 1 -- 172.21.15.112:0/1476666666 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b5f8a3ddb0 con 0x55b5f86dccd0 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f37127fc640 10 client.17469 handle_command_reply: tid=0 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 1 client.17469 shutdown 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 1 -- 172.21.15.112:0/1476666666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 msgr2=0x55b5f8ad1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8ad1050 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f371c105490 tx=0x7f371c005c90 comp rx=0 tx=0).stop 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 1 -- 172.21.15.112:0/1476666666 shutdown_connections 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b5f86dccd0 0x55b5f8ae3460 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f871e350 0x55b5f8adcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86fd670 0x55b5f8ad15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 1 --2- 172.21.15.112:0/1476666666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8ad1050 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 1 -- 172.21.15.112:0/1476666666 >> 172.21.15.112:0/1476666666 conn(0x55b5f8712740 msgr2=0x55b5f8ac8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f37127fc640 0 client.17469 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 1 -- 172.21.15.112:0/1476666666 shutdown_connections 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 1 -- 172.21.15.112:0/1476666666 wait complete. 2024-09-21T03:40:45.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.933+0000 7f373787b740 20 client.17469 trim_cache size 0 max 16384 2024-09-21T03:40:45.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.951+0000 7f373787b740 1 Processor -- start 2024-09-21T03:40:45.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.951+0000 7f373787b740 1 -- start start 2024-09-21T03:40:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f373787b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86f5490 0x55b5f8adce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f373787b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8acb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f373787b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ace040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f373787b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1afd20 con 0x55b5f871e350 2024-09-21T03:40:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f373787b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1ad5f0 con 0x55b5f86fd670 2024-09-21T03:40:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f373787b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1ab460 con 0x55b5f86f5490 2024-09-21T03:40:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ace040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ace040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38198/0 (socket says 172.21.15.112:38198) 2024-09-21T03:40:45.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f372b7fe640 1 -- 172.21.15.112:0/3014988846 learned_addr learned my addr 172.21.15.112:0/3014988846 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f372affd640 1 --2- 172.21.15.112:0/3014988846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86f5490 0x55b5f8adce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f37227fc640 1 --2- 172.21.15.112:0/3014988846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8acb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f372b7fe640 1 -- 172.21.15.112:0/3014988846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86f5490 msgr2=0x55b5f8adce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f372b7fe640 1 --2- 172.21.15.112:0/3014988846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86f5490 0x55b5f8adce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f372b7fe640 1 -- 172.21.15.112:0/3014988846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 msgr2=0x55b5f8acb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f372b7fe640 1 --2- 172.21.15.112:0/3014988846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8acb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f372b7fe640 1 -- 172.21.15.112:0/3014988846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5f8ac7720 con 0x55b5f871e350 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.952+0000 7f37227fc640 1 --2- 172.21.15.112:0/3014988846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8acb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.953+0000 7f372b7fe640 1 --2- 172.21.15.112:0/3014988846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ace040 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f372401b620 tx=0x7f372401b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b64eb3fefa2502 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.953+0000 7f3720ff9640 1 -- 172.21.15.112:0/3014988846 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724036070 con 0x55b5f871e350 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.953+0000 7f3720ff9640 1 -- 172.21.15.112:0/3014988846 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3724005b70 con 0x55b5f871e350 2024-09-21T03:40:45.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.953+0000 7f3720ff9640 1 -- 172.21.15.112:0/3014988846 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372400f8a0 con 0x55b5f871e350 2024-09-21T03:40:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.954+0000 7f373787b740 1 -- 172.21.15.112:0/3014988846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 msgr2=0x55b5f8ace040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.954+0000 7f373787b740 1 --2- 172.21.15.112:0/3014988846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ace040 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f372401b620 tx=0x7f372401b6f0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.954+0000 7f373787b740 1 -- 172.21.15.112:0/3014988846 shutdown_connections 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.954+0000 7f373787b740 1 --2- 172.21.15.112:0/3014988846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f871e350 0x55b5f8ace040 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.954+0000 7f373787b740 1 --2- 172.21.15.112:0/3014988846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8acb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.954+0000 7f373787b740 1 --2- 172.21.15.112:0/3014988846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f86f5490 0x55b5f8adce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.954+0000 7f373787b740 1 -- 172.21.15.112:0/3014988846 >> 172.21.15.112:0/3014988846 conn(0x55b5f8712740 msgr2=0x55b5f8acf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.954+0000 7f373787b740 1 -- 172.21.15.112:0/3014988846 shutdown_connections 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.954+0000 7f373787b740 1 -- 172.21.15.112:0/3014988846 wait complete. 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.954+0000 7f373787b740 1 -- start start 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.954+0000 7f373787b740 1 Processor -- start 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f373787b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8b47130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f373787b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8b476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f373787b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f871e350 0x55b5f8ad5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f373787b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1abab0 con 0x55b5f86f5490 2024-09-21T03:40:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f373787b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1acf20 con 0x55b5f86fd670 2024-09-21T03:40:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f373787b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1ac180 con 0x55b5f871e350 2024-09-21T03:40:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f373787b740 10 client.0 _ll_register_callbacks cb 0x7f371c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f372affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8b47130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f372affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8b47130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38210/0 (socket says 172.21.15.112:38210) 2024-09-21T03:40:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f372affd640 1 -- 172.21.15.112:0/106915068 learned_addr learned my addr 172.21.15.112:0/106915068 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f37227fc640 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8b476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f372b7fe640 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f871e350 0x55b5f8ad5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f372affd640 1 -- 172.21.15.112:0/106915068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f871e350 msgr2=0x55b5f8ad5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f372affd640 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f871e350 0x55b5f8ad5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f372affd640 1 -- 172.21.15.112:0/106915068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 msgr2=0x55b5f8b476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f372affd640 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8b476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f372affd640 1 -- 172.21.15.112:0/106915068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5f8ad5ca0 con 0x55b5f86f5490 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f37227fc640 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8b476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f372affd640 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8b47130 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f371c006010 tx=0x7f371c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8422bdfbf9f5e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f37127fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f37127fc640 1 -- 172.21.15.112:0/106915068 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c049070 con 0x55b5f86f5490 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f37127fc640 1 -- 172.21.15.112:0/106915068 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f371c0d21c0 con 0x55b5f86f5490 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f373787b740 1 -- 172.21.15.112:0/106915068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3724005b70 con 0x55b5f86f5490 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.955+0000 7f37127fc640 1 -- 172.21.15.112:0/106915068 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f371c0cd530 con 0x55b5f86f5490 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.956+0000 7f373787b740 10 client.17475 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:45.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.956+0000 7f373787b740 1 -- 172.21.15.112:0/106915068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b5f8b4e8c0 con 0x55b5f86f5490 2024-09-21T03:40:45.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.956+0000 7f37127fc640 1 -- 172.21.15.112:0/106915068 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3724005b70 con 0x55b5f86f5490 2024-09-21T03:40:45.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.956+0000 7f37127fc640 1 -- 172.21.15.112:0/106915068 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f371c03bb90 con 0x55b5f86f5490 2024-09-21T03:40:45.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.956+0000 7f373787b740 10 client.17475 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:45.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.956+0000 7f373787b740 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:45.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.956+0000 7f373787b740 20 client.17475 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:45.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.956+0000 7f373787b740 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b5f86b2080 0x55b5f8b50ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:45.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.956+0000 7f373787b740 4 client.17475 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:40:45.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.956+0000 7f373787b740 1 -- 172.21.15.112:0/106915068 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b5f8a39860 con 0x55b5f86b2080 2024-09-21T03:40:45.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.956+0000 7f37227fc640 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b5f86b2080 0x55b5f8b50ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:45.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.957+0000 7f37127fc640 10 client.17475 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:45.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.957+0000 7f37227fc640 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b5f86b2080 0x55b5f8b50ef0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f8c4326e4eacd79 server_cookie=f852ae2c970aef37 in_seq=0 out_seq=0 2024-09-21T03:40:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f37127fc640 1 -- 172.21.15.112:0/106915068 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b5f8a39860 con 0x55b5f86b2080 2024-09-21T03:40:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f37127fc640 10 client.17475 handle_command_reply: tid=0 2024-09-21T03:40:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 1 client.17475 shutdown 2024-09-21T03:40:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 1 -- 172.21.15.112:0/106915068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 msgr2=0x55b5f8b47130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8b47130 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f371c006010 tx=0x7f371c009680 comp rx=0 tx=0).stop 2024-09-21T03:40:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 1 -- 172.21.15.112:0/106915068 shutdown_connections 2024-09-21T03:40:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55b5f86b2080 0x55b5f8b50ef0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5f871e350 0x55b5f8ad5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5f86fd670 0x55b5f8b476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 1 --2- 172.21.15.112:0/106915068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b5f86f5490 0x55b5f8b47130 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f37127fc640 0 client.17475 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 1 -- 172.21.15.112:0/106915068 >> 172.21.15.112:0/106915068 conn(0x55b5f8712740 msgr2=0x55b5f8ac6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 1 -- 172.21.15.112:0/106915068 shutdown_connections 2024-09-21T03:40:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 1 -- 172.21.15.112:0/106915068 wait complete. 2024-09-21T03:40:45.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.959+0000 7f373787b740 20 client.17475 trim_cache size 0 max 16384 2024-09-21T03:40:45.966 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:40:45.966 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:40:45.966 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c", 2024-09-21T03:40:45.966 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:40:45.966 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:40:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 -- 172.21.15.112:0/1418953197 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37180b6680 msgr2=0x7f37180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37180b6680 0x7f37180b8b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3724022000 tx=0x7f3724023000 comp rx=0 tx=0).stop 2024-09-21T03:40:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 -- 172.21.15.112:0/1418953197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c1049b0 msgr2=0x7f372c19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c1049b0 0x7f372c19a4f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f371c005ab0 tx=0x7f371c003b90 comp rx=0 tx=0).stop 2024-09-21T03:40:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 -- 172.21.15.112:0/1418953197 shutdown_connections 2024-09-21T03:40:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f37180b6680 0x7f37180b8b90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f372c10de10 0x7f372c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f372c105300 0x7f372c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 --2- 172.21.15.112:0/1418953197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f372c1049b0 0x7f372c19a4f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:45.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 -- 172.21.15.112:0/1418953197 >> 172.21.15.112:0/1418953197 conn(0x7f372c079730 msgr2=0x7f372c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:45.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 -- 172.21.15.112:0/1418953197 shutdown_connections 2024-09-21T03:40:45.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:45.960+0000 7f3731307640 1 -- 172.21.15.112:0/1418953197 wait complete. 2024-09-21T03:40:46.274 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1247381 10.30 MB/sec execute 526 sec latency 85.054 ms 2024-09-21T03:40:46.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:46 smithi203 ceph-mon[34723]: pgmap v682: 97 pgs: 97 active+clean; 900 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 207 op/s 2024-09-21T03:40:46.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:46 smithi187 ceph-mon[34603]: pgmap v682: 97 pgs: 97 active+clean; 900 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 207 op/s 2024-09-21T03:40:46.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:46 smithi112 ceph-mon[31442]: pgmap v682: 97 pgs: 97 active+clean; 900 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 207 op/s 2024-09-21T03:40:46.739 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:40:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:40:47.279 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1249867 10.30 MB/sec execute 527 sec latency 23.303 ms 2024-09-21T03:40:47.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:47 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:40:47.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:47 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:47 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:40:47.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:47 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:40:47.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:47 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:40:47.399 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:47 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.399 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:47 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.399 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:47 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:47 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:40:47.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:47 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:47 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:40:47.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:47 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:40:47.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:47 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:40:47.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:47 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:47 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:47 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:47 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:40:47.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:47 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:47 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:40:47.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:47 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:40:47.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:47 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:40:47.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:47 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:47 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:47 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:40:47.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.494+0000 7f31d0ca1640 1 -- 172.21.15.112:0/7901820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31cc103b80 msgr2=0x7f31cc103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.494+0000 7f31d0ca1640 1 --2- 172.21.15.112:0/7901820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31cc103b80 0x7f31cc103f90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009ca0 tx=0x7f31bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.494+0000 7f31d0ca1640 1 -- 172.21.15.112:0/7901820 shutdown_connections 2024-09-21T03:40:47.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.494+0000 7f31d0ca1640 1 --2- 172.21.15.112:0/7901820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31cc109e10 0x7f31cc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.494+0000 7f31d0ca1640 1 --2- 172.21.15.112:0/7901820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31cc1094c0 0x7f31cc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.494+0000 7f31d0ca1640 1 --2- 172.21.15.112:0/7901820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31cc103b80 0x7f31cc103f90 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.494+0000 7f31d0ca1640 1 -- 172.21.15.112:0/7901820 >> 172.21.15.112:0/7901820 conn(0x7f31cc0ff590 msgr2=0x7f31cc1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.494+0000 7f31d0ca1640 1 -- 172.21.15.112:0/7901820 shutdown_connections 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.494+0000 7f31d0ca1640 1 -- 172.21.15.112:0/7901820 wait complete. 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31d0ca1640 1 Processor -- start 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31d0ca1640 1 -- start start 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31d0ca1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31cc103b80 0x7f31cc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31d0ca1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31cc1094c0 0x7f31cc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31d0ca1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31cc109e10 0x7f31cc1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31d0ca1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc113ad0 con 0x7f31cc1094c0 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31d0ca1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc113950 con 0x7f31cc109e10 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31d0ca1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc113c50 con 0x7f31cc103b80 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31c9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31cc1094c0 0x7f31cc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31c9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31cc1094c0 0x7f31cc19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38228/0 (socket says 172.21.15.112:38228) 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31ca59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31cc103b80 0x7f31cc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31c9d9b640 1 -- 172.21.15.112:0/36556813 learned_addr learned my addr 172.21.15.112:0/36556813 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.495+0000 7f31cad9d640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31cc109e10 0x7f31cc1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.496+0000 7f31c9d9b640 1 -- 172.21.15.112:0/36556813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31cc103b80 msgr2=0x7f31cc19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.496+0000 7f31c9d9b640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31cc103b80 0x7f31cc19c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.496+0000 7f31c9d9b640 1 -- 172.21.15.112:0/36556813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31cc109e10 msgr2=0x7f31cc1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.496+0000 7f31c9d9b640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31cc109e10 0x7f31cc1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.496+0000 7f31c9d9b640 1 -- 172.21.15.112:0/36556813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31cc1a2590 con 0x7f31cc1094c0 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.496+0000 7f31c9d9b640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31cc1094c0 0x7f31cc19cc80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f31b400c8c0 tx=0x7f31b400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.496+0000 7f31c37fe640 1 -- 172.21.15.112:0/36556813 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4007bb0 con 0x7f31cc1094c0 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.496+0000 7f31d0ca1640 1 -- 172.21.15.112:0/36556813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31cc1a2880 con 0x7f31cc1094c0 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.496+0000 7f31c37fe640 1 -- 172.21.15.112:0/36556813 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31b4007d50 con 0x7f31cc1094c0 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.496+0000 7f31c37fe640 1 -- 172.21.15.112:0/36556813 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b40187c0 con 0x7f31cc1094c0 2024-09-21T03:40:47.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.496+0000 7f31d0ca1640 1 -- 172.21.15.112:0/36556813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31cc1a90a0 con 0x7f31cc1094c0 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.499+0000 7f31d7215740 1 Processor -- start 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.499+0000 7f31d7215740 1 -- start start 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.499+0000 7f31c37fe640 1 -- 172.21.15.112:0/36556813 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f31b4018960 con 0x7f31cc1094c0 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.499+0000 7f31d7215740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c6860070 0x5574c6c07650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.499+0000 7f31d7215740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c68371b0 0x5574c6c07b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.499+0000 7f31d7215740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 0x5574c6c080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.499+0000 7f31d7215740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5574c6b7de50 con 0x5574c683f390 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.499+0000 7f31d7215740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5574c6b6d280 con 0x5574c68371b0 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.499+0000 7f31d7215740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5574c6b71090 con 0x5574c6860070 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31cad9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c6860070 0x5574c6c07650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31c9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 0x5574c6c080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31cad9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c6860070 0x5574c6c07650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52672/0 (socket says 172.21.15.112:52672) 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31ca59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c68371b0 0x5574c6c07b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31cad9d640 1 -- 172.21.15.112:0/3270680652 learned_addr learned my addr 172.21.15.112:0/3270680652 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31cad9d640 1 -- 172.21.15.112:0/3270680652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c68371b0 msgr2=0x5574c6c07b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31cad9d640 1 --2- 172.21.15.112:0/3270680652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c68371b0 0x5574c6c07b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31cad9d640 1 -- 172.21.15.112:0/3270680652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 msgr2=0x5574c6c080d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31cad9d640 1 --2- 172.21.15.112:0/3270680652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 0x5574c6c080d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31cad9d640 1 -- 172.21.15.112:0/3270680652 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c68cb650 con 0x5574c6860070 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31cad9d640 1 --2- 172.21.15.112:0/3270680652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c6860070 0x5574c6c07650 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f31c400b6b0 tx=0x7f31c400bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9375a4afd37d991 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31c37fe640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31b80b66b0 0x7f31b80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31ca59c640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31b80b66b0 0x7f31b80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31c0ff9640 1 -- 172.21.15.112:0/3270680652 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4003bd0 con 0x5574c6860070 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31c37fe640 1 -- 172.21.15.112:0/36556813 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f31b40e7760 con 0x7f31cc1094c0 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31c0ff9640 1 -- 172.21.15.112:0/3270680652 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c4015070 con 0x5574c6860070 2024-09-21T03:40:47.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.500+0000 7f31c0ff9640 1 -- 172.21.15.112:0/3270680652 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c40116a0 con 0x5574c6860070 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.501+0000 7f31d7215740 1 -- 172.21.15.112:0/3270680652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c6860070 msgr2=0x5574c6c07650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.501+0000 7f31ca59c640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31b80b66b0 0x7f31b80b8bc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5574c6c062d0 tx=0x7f31bc02f310 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.501+0000 7f31d7215740 1 --2- 172.21.15.112:0/3270680652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c6860070 0x5574c6c07650 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f31c400b6b0 tx=0x7f31c400bb70 comp rx=0 tx=0).stop 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.501+0000 7f31d7215740 1 -- 172.21.15.112:0/3270680652 shutdown_connections 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.501+0000 7f31d7215740 1 --2- 172.21.15.112:0/3270680652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 0x5574c6c080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.501+0000 7f31d7215740 1 --2- 172.21.15.112:0/3270680652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c68371b0 0x5574c6c07b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.501+0000 7f31d7215740 1 --2- 172.21.15.112:0/3270680652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c6860070 0x5574c6c07650 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.501+0000 7f31d7215740 1 -- 172.21.15.112:0/3270680652 >> 172.21.15.112:0/3270680652 conn(0x5574c6854460 msgr2=0x5574c6c05540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.502+0000 7f31d7215740 1 -- 172.21.15.112:0/3270680652 shutdown_connections 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.502+0000 7f31d7215740 1 -- 172.21.15.112:0/3270680652 wait complete. 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.502+0000 7f31d7215740 1 -- start start 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.502+0000 7f31d7215740 1 Processor -- start 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31d7215740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c12d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31d7215740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 0x5574c6c13270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31d7215740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c6860070 0x5574c6c1f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31d7215740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5574c69ce700 con 0x5574c683f390 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31d7215740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5574c6b08110 con 0x5574c6860070 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31d7215740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5574c6c15e30 con 0x5574c68371b0 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31d7215740 10 client.0 _ll_register_callbacks cb 0x5574c6b21990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31c9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 0x5574c6c13270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31c9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 0x5574c6c13270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38246/0 (socket says 172.21.15.112:38246) 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31c9d9b640 1 -- 172.21.15.112:0/682662330 learned_addr learned my addr 172.21.15.112:0/682662330 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31ca59c640 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c12d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31cad9d640 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c6860070 0x5574c6c1f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31c9d9b640 1 -- 172.21.15.112:0/682662330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 msgr2=0x5574c6c12d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31c9d9b640 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c12d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31c9d9b640 1 -- 172.21.15.112:0/682662330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c6860070 msgr2=0x5574c6c1f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31c9d9b640 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c6860070 0x5574c6c1f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31c9d9b640 1 -- 172.21.15.112:0/682662330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c6b35c40 con 0x5574c683f390 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31c9d9b640 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 0x5574c6c13270 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f31b4014000 tx=0x7f31b40e7a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce683e01020cf5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31b1ffb640 1 -- 172.21.15.112:0/682662330 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b40a9480 con 0x5574c683f390 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31b1ffb640 1 -- 172.21.15.112:0/682662330 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f31b40b6240 con 0x5574c683f390 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.503+0000 7f31d7215740 1 -- 172.21.15.112:0/682662330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c4015070 con 0x5574c683f390 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31b1ffb640 1 -- 172.21.15.112:0/682662330 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f31b4018e60 con 0x5574c683f390 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31d7215740 10 client.17487 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31d7215740 1 -- 172.21.15.112:0/682662330 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574c6c21c20 con 0x5574c683f390 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31b1ffb640 1 -- 172.21.15.112:0/682662330 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c4015070 con 0x5574c683f390 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31b1ffb640 1 -- 172.21.15.112:0/682662330 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f31b40a7ce0 con 0x5574c683f390 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31d7215740 10 client.17487 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31d7215740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31d7215740 20 client.17487 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:47.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31d7215740 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5574c681e9f0 0x5574c6c250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31d7215740 4 client.17487 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31d7215740 1 -- 172.21.15.112:0/682662330 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c6b7fda0 con 0x5574c681e9f0 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.504+0000 7f31ca59c640 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5574c681e9f0 0x5574c6c250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.505+0000 7f31b1ffb640 10 client.17487 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.505+0000 7f31ca59c640 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5574c681e9f0 0x5574c6c250f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53bc53ef93a0ee5 server_cookie=8c7f988d67f4dd10 in_seq=0 out_seq=0 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.505+0000 7f31b1ffb640 1 -- 172.21.15.112:0/682662330 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5574c6b7fda0 con 0x5574c681e9f0 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.505+0000 7f31b1ffb640 10 client.17487 handle_command_reply: tid=0 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 1 client.17487 shutdown 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 1 -- 172.21.15.112:0/682662330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 msgr2=0x5574c6c13270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 0x5574c6c13270 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f31b4014000 tx=0x7f31b40e7a60 comp rx=0 tx=0).stop 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 1 -- 172.21.15.112:0/682662330 shutdown_connections 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5574c681e9f0 0x5574c6c250f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c6860070 0x5574c6c1f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31b1ffb640 0 client.17487 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:47.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c683f390 0x5574c6c13270 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 1 --2- 172.21.15.112:0/682662330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c12d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 1 -- 172.21.15.112:0/682662330 >> 172.21.15.112:0/682662330 conn(0x5574c6854460 msgr2=0x5574c6c0a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 1 -- 172.21.15.112:0/682662330 shutdown_connections 2024-09-21T03:40:47.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 1 -- 172.21.15.112:0/682662330 wait complete. 2024-09-21T03:40:47.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.506+0000 7f31d7215740 20 client.17487 trim_cache size 0 max 16384 2024-09-21T03:40:47.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.524+0000 7f31d7215740 1 Processor -- start 2024-09-21T03:40:47.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.524+0000 7f31d7215740 1 -- start start 2024-09-21T03:40:47.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31d7215740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c05570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31d7215740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c683f390 0x5574c6c1f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31d7215740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 0x5574c6c0ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31d7215740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc1b4080 con 0x5574c6860070 2024-09-21T03:40:47.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31d7215740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc1b1950 con 0x5574c683f390 2024-09-21T03:40:47.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31d7215740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc1af7c0 con 0x5574c68371b0 2024-09-21T03:40:47.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31cad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 0x5574c6c0ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31cad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 0x5574c6c0ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38254/0 (socket says 172.21.15.112:38254) 2024-09-21T03:40:47.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31cad9d640 1 -- 172.21.15.112:0/2119980630 learned_addr learned my addr 172.21.15.112:0/2119980630 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31ca59c640 1 --2- 172.21.15.112:0/2119980630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c05570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31c9d9b640 1 --2- 172.21.15.112:0/2119980630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c683f390 0x5574c6c1f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31cad9d640 1 -- 172.21.15.112:0/2119980630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 msgr2=0x5574c6c05570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31cad9d640 1 --2- 172.21.15.112:0/2119980630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c05570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31cad9d640 1 -- 172.21.15.112:0/2119980630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c683f390 msgr2=0x5574c6c1f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31cad9d640 1 --2- 172.21.15.112:0/2119980630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c683f390 0x5574c6c1f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31cad9d640 1 -- 172.21.15.112:0/2119980630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c6c06300 con 0x5574c6860070 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31c9d9b640 1 --2- 172.21.15.112:0/2119980630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c683f390 0x5574c6c1f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.525+0000 7f31cad9d640 1 --2- 172.21.15.112:0/2119980630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 0x5574c6c0ff20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f31c40045c0 tx=0x7f31c4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c941a8c5465af6df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.526+0000 7f31b3fff640 1 -- 172.21.15.112:0/2119980630 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c402a070 con 0x5574c6860070 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.526+0000 7f31b3fff640 1 -- 172.21.15.112:0/2119980630 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c4015040 con 0x5574c6860070 2024-09-21T03:40:47.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.526+0000 7f31b3fff640 1 -- 172.21.15.112:0/2119980630 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c40116a0 con 0x5574c6860070 2024-09-21T03:40:47.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.526+0000 7f31d7215740 1 -- 172.21.15.112:0/2119980630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 msgr2=0x5574c6c0ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.526+0000 7f31d7215740 1 --2- 172.21.15.112:0/2119980630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 0x5574c6c0ff20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f31c40045c0 tx=0x7f31c4013040 comp rx=0 tx=0).stop 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 -- 172.21.15.112:0/2119980630 shutdown_connections 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 --2- 172.21.15.112:0/2119980630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 0x5574c6c0ff20 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 --2- 172.21.15.112:0/2119980630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c683f390 0x5574c6c1f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 --2- 172.21.15.112:0/2119980630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c05570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 -- 172.21.15.112:0/2119980630 >> 172.21.15.112:0/2119980630 conn(0x5574c6854460 msgr2=0x5574c6c04e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 -- 172.21.15.112:0/2119980630 shutdown_connections 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 -- 172.21.15.112:0/2119980630 wait complete. 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 -- start start 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 Processor -- start 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c683f390 0x5574c6c17a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 0x5574c6c8e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc1afe10 con 0x5574c6860070 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc1b1280 con 0x5574c683f390 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc1b04e0 con 0x5574c68371b0 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.527+0000 7f31d7215740 10 client.0 _ll_register_callbacks cb 0x7f31b400c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31ca59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31cad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 0x5574c6c8e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31ca59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c174b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:52700/0 (socket says 172.21.15.112:52700) 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31ca59c640 1 -- 172.21.15.112:0/3150721190 learned_addr learned my addr 172.21.15.112:0/3150721190 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31c9d9b640 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c683f390 0x5574c6c17a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31cad9d640 1 -- 172.21.15.112:0/3150721190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 msgr2=0x5574c6c174b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31cad9d640 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c174b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31cad9d640 1 -- 172.21.15.112:0/3150721190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c683f390 msgr2=0x5574c6c17a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31cad9d640 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c683f390 0x5574c6c17a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31cad9d640 1 -- 172.21.15.112:0/3150721190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c6c8ef10 con 0x5574c6860070 2024-09-21T03:40:47.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31ca59c640 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31cad9d640 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 0x5574c6c8e9d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f31c4000c00 tx=0x7f31c4003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd505cda18c77cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31b1ffb640 1 -- 172.21.15.112:0/3150721190 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c402a040 con 0x5574c6860070 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31b1ffb640 1 -- 172.21.15.112:0/3150721190 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8470+0+0 (secure 0 0 0) 0x7f31c401a470 con 0x5574c6860070 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.528+0000 7f31d7215740 1 -- 172.21.15.112:0/3150721190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c4015040 con 0x5574c6860070 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31b1ffb640 1 -- 172.21.15.112:0/3150721190 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f31c401a690 con 0x5574c6860070 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31d7215740 10 client.17493 fetch_fsmap learned FSMap version 30 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31d7215740 1 -- 172.21.15.112:0/3150721190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574c6c900b0 con 0x5574c6860070 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31b1ffb640 1 -- 172.21.15.112:0/3150721190 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c4015040 con 0x5574c6860070 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31b1ffb640 1 -- 172.21.15.112:0/3150721190 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f31c4011c00 con 0x5574c6860070 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31d7215740 10 client.17493 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31d7215740 10 client.17493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31d7215740 20 client.17493 populate_metadata read hostname 'smithi112' 2024-09-21T03:40:47.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31d7215740 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5574c67f3da0 0x5574c6c92590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31d7215740 4 client.17493 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31d7215740 1 -- 172.21.15.112:0/3150721190 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574c6b7b850 con 0x5574c67f3da0 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.529+0000 7f31ca59c640 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5574c67f3da0 0x5574c6c92590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31b1ffb640 10 client.17493 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31ca59c640 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5574c67f3da0 0x5574c6c92590 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33902c55ae8508f server_cookie=6c7a0d78d02e877e in_seq=0 out_seq=0 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31b1ffb640 1 -- 172.21.15.112:0/3150721190 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5682 (crc 0 0 0) 0x5574c6b7b850 con 0x5574c67f3da0 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31b1ffb640 10 client.17493 handle_command_reply: tid=0 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31d7215740 1 client.17493 shutdown 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31d7215740 1 -- 172.21.15.112:0/3150721190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 msgr2=0x5574c6c8e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31d7215740 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 0x5574c6c8e9d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f31c4000c00 tx=0x7f31c4003e80 comp rx=0 tx=0).stop 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31d7215740 1 -- 172.21.15.112:0/3150721190 shutdown_connections 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31d7215740 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5574c67f3da0 0x5574c6c92590 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31d7215740 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574c6860070 0x5574c6c8e9d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31d7215740 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5574c683f390 0x5574c6c17a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31d7215740 1 --2- 172.21.15.112:0/3150721190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5574c68371b0 0x5574c6c174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.530+0000 7f31d7215740 1 -- 172.21.15.112:0/3150721190 >> 172.21.15.112:0/3150721190 conn(0x5574c6854460 msgr2=0x5574c6c08680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31b1ffb640 0 client.17493 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:40:47.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d7215740 1 -- 172.21.15.112:0/3150721190 shutdown_connections 2024-09-21T03:40:47.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d7215740 1 -- 172.21.15.112:0/3150721190 wait complete. 2024-09-21T03:40:47.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d7215740 20 client.17493 trim_cache size 0 max 16384 2024-09-21T03:40:47.539 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:40:47.539 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:40:47.539 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:40:47.539 INFO:teuthology.orchestra.run.smithi112.stdout: "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c": { 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c", 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c": { 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:45.957537+0000", 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:40:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:40:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:40:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:40:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:40:47.551 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:40:47.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 -- 172.21.15.112:0/36556813 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31b80b66b0 msgr2=0x7f31b80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31b80b66b0 0x7f31b80b8bc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5574c6c062d0 tx=0x7f31bc02f310 comp rx=0 tx=0).stop 2024-09-21T03:40:47.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 -- 172.21.15.112:0/36556813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31cc1094c0 msgr2=0x7f31cc19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:40:47.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31cc1094c0 0x7f31cc19cc80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f31b400c8c0 tx=0x7f31b400cd80 comp rx=0 tx=0).stop 2024-09-21T03:40:47.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 -- 172.21.15.112:0/36556813 shutdown_connections 2024-09-21T03:40:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f31b80b66b0 0x7f31b80b8bc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31cc109e10 0x7f31cc1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31cc1094c0 0x7f31cc19cc80 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 --2- 172.21.15.112:0/36556813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31cc103b80 0x7f31cc19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:40:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 -- 172.21.15.112:0/36556813 >> 172.21.15.112:0/36556813 conn(0x7f31cc0ff590 msgr2=0x7f31cc107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:40:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 -- 172.21.15.112:0/36556813 shutdown_connections 2024-09-21T03:40:47.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:40:47.531+0000 7f31d0ca1640 1 -- 172.21.15.112:0/36556813 wait complete. 2024-09-21T03:40:48.246 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:48 smithi112 ceph-mon[31442]: pgmap v683: 97 pgs: 97 active+clean; 857 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 296 op/s 2024-09-21T03:40:48.275 INFO:tasks.cephfs.filesystem:scrub status for tag:d0d08a40-4f31-4299-9cb9-9dfae9d0f21c - {'path': '/', 'tag': 'd0d08a40-4f31-4299-9cb9-9dfae9d0f21c', 'options': 'recursive,force'} 2024-09-21T03:40:48.279 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1251060 10.29 MB/sec execute 528 sec latency 22.450 ms 2024-09-21T03:40:48.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:48 smithi203 ceph-mon[34723]: pgmap v683: 97 pgs: 97 active+clean; 857 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 296 op/s 2024-09-21T03:40:48.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:48 smithi187 ceph-mon[34603]: pgmap v683: 97 pgs: 97 active+clean; 857 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 296 op/s 2024-09-21T03:40:49.279 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1253723 10.31 MB/sec execute 529 sec latency 18.347 ms 2024-09-21T03:40:50.279 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1254757 10.29 MB/sec execute 530 sec latency 38.889 ms 2024-09-21T03:40:50.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:50 smithi203 ceph-mon[34723]: pgmap v684: 97 pgs: 97 active+clean; 857 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 192 op/s 2024-09-21T03:40:50.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:50 smithi187 ceph-mon[34603]: pgmap v684: 97 pgs: 97 active+clean; 857 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 192 op/s 2024-09-21T03:40:50.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:50 smithi112 ceph-mon[31442]: pgmap v684: 97 pgs: 97 active+clean; 857 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 192 op/s 2024-09-21T03:40:51.279 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1257286 10.30 MB/sec execute 531 sec latency 54.527 ms 2024-09-21T03:40:52.279 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1258444 10.29 MB/sec execute 532 sec latency 37.755 ms 2024-09-21T03:40:52.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:52 smithi203 ceph-mon[34723]: pgmap v685: 97 pgs: 97 active+clean; 865 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 256 op/s 2024-09-21T03:40:52.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:52 smithi187 ceph-mon[34603]: pgmap v685: 97 pgs: 97 active+clean; 865 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 256 op/s 2024-09-21T03:40:52.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:52 smithi112 ceph-mon[31442]: pgmap v685: 97 pgs: 97 active+clean; 865 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 256 op/s 2024-09-21T03:40:53.279 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1260916 10.30 MB/sec execute 533 sec latency 19.850 ms 2024-09-21T03:40:54.279 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1262140 10.29 MB/sec execute 534 sec latency 33.528 ms 2024-09-21T03:40:54.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:54 smithi203 ceph-mon[34723]: pgmap v686: 97 pgs: 97 active+clean; 865 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 235 op/s 2024-09-21T03:40:54.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:54 smithi187 ceph-mon[34603]: pgmap v686: 97 pgs: 97 active+clean; 865 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 235 op/s 2024-09-21T03:40:54.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:54 smithi112 ceph-mon[31442]: pgmap v686: 97 pgs: 97 active+clean; 865 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 235 op/s 2024-09-21T03:40:55.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1264438 10.30 MB/sec execute 535 sec latency 17.130 ms 2024-09-21T03:40:55.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:55 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:40:55.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:55 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:40:55.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:55 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:40:55.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:55 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:40:55.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:55 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:40:55.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:55 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:40:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:55 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:40:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:55 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:40:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:55 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:40:56.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1265277 10.28 MB/sec execute 536 sec latency 46.686 ms 2024-09-21T03:40:56.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:56 smithi203 ceph-mon[34723]: pgmap v687: 97 pgs: 97 active+clean; 868 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2024-09-21T03:40:56.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:56 smithi187 ceph-mon[34603]: pgmap v687: 97 pgs: 97 active+clean; 868 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2024-09-21T03:40:56.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:56 smithi112 ceph-mon[31442]: pgmap v687: 97 pgs: 97 active+clean; 868 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2024-09-21T03:40:57.279 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1267634 10.29 MB/sec execute 537 sec latency 36.407 ms 2024-09-21T03:40:58.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1268720 10.28 MB/sec execute 538 sec latency 16.527 ms 2024-09-21T03:40:58.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:40:58 smithi203 ceph-mon[34723]: pgmap v688: 97 pgs: 97 active+clean; 878 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 307 op/s 2024-09-21T03:40:58.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:40:58 smithi187 ceph-mon[34603]: pgmap v688: 97 pgs: 97 active+clean; 878 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 307 op/s 2024-09-21T03:40:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:40:58 smithi112 ceph-mon[31442]: pgmap v688: 97 pgs: 97 active+clean; 878 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 307 op/s 2024-09-21T03:40:59.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1271130 10.29 MB/sec execute 539 sec latency 17.677 ms 2024-09-21T03:41:00.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1272291 10.28 MB/sec execute 540 sec latency 13.594 ms 2024-09-21T03:41:00.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:00 smithi203 ceph-mon[34723]: pgmap v689: 97 pgs: 97 active+clean; 877 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-09-21T03:41:00.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:00 smithi187 ceph-mon[34603]: pgmap v689: 97 pgs: 97 active+clean; 877 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-09-21T03:41:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:00 smithi112 ceph-mon[31442]: pgmap v689: 97 pgs: 97 active+clean; 877 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-09-21T03:41:01.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1273678 10.27 MB/sec execute 541 sec latency 59.901 ms 2024-09-21T03:41:02.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1275236 10.27 MB/sec execute 542 sec latency 28.617 ms 2024-09-21T03:41:02.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:02 smithi203 ceph-mon[34723]: pgmap v690: 97 pgs: 97 active+clean; 884 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 275 op/s 2024-09-21T03:41:02.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:02 smithi187 ceph-mon[34603]: pgmap v690: 97 pgs: 97 active+clean; 884 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 275 op/s 2024-09-21T03:41:02.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:02 smithi112 ceph-mon[31442]: pgmap v690: 97 pgs: 97 active+clean; 884 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.5 MiB/s wr, 275 op/s 2024-09-21T03:41:03.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1276641 10.26 MB/sec execute 543 sec latency 15.282 ms 2024-09-21T03:41:04.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1278775 10.27 MB/sec execute 544 sec latency 35.247 ms 2024-09-21T03:41:04.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:04 smithi203 ceph-mon[34723]: pgmap v691: 97 pgs: 97 active+clean; 885 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 232 op/s 2024-09-21T03:41:04.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:04 smithi187 ceph-mon[34603]: pgmap v691: 97 pgs: 97 active+clean; 885 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 232 op/s 2024-09-21T03:41:04.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:04 smithi112 ceph-mon[31442]: pgmap v691: 97 pgs: 97 active+clean; 885 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 232 op/s 2024-09-21T03:41:05.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1280147 10.26 MB/sec execute 545 sec latency 18.956 ms 2024-09-21T03:41:05.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:05 smithi203 ceph-mon[34723]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:41:05.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:05 smithi187 ceph-mon[34603]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:41:05.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:05 smithi112 ceph-mon[31442]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:41:06.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1282368 10.27 MB/sec execute 546 sec latency 51.605 ms 2024-09-21T03:41:06.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:06 smithi203 ceph-mon[34723]: pgmap v693: 97 pgs: 97 active+clean; 890 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 266 op/s 2024-09-21T03:41:06.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:06 smithi187 ceph-mon[34603]: pgmap v693: 97 pgs: 97 active+clean; 890 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 266 op/s 2024-09-21T03:41:06.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:06 smithi112 ceph-mon[31442]: pgmap v693: 97 pgs: 97 active+clean; 890 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 266 op/s 2024-09-21T03:41:07.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1283854 10.26 MB/sec execute 547 sec latency 24.791 ms 2024-09-21T03:41:08.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1285973 10.27 MB/sec execute 548 sec latency 18.072 ms 2024-09-21T03:41:08.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:08 smithi203 ceph-mon[34723]: pgmap v694: 97 pgs: 97 active+clean; 893 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-09-21T03:41:08.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:08 smithi187 ceph-mon[34603]: pgmap v694: 97 pgs: 97 active+clean; 893 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-09-21T03:41:08.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:08 smithi112 ceph-mon[31442]: pgmap v694: 97 pgs: 97 active+clean; 893 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-09-21T03:41:09.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1287663 10.26 MB/sec execute 549 sec latency 27.404 ms 2024-09-21T03:41:10.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1289559 10.26 MB/sec execute 550 sec latency 15.305 ms 2024-09-21T03:41:10.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:10 smithi203 ceph-mon[34723]: pgmap v695: 97 pgs: 97 active+clean; 893 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 277 op/s 2024-09-21T03:41:10.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:10 smithi203 ceph-mon[34723]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:41:10.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:10 smithi187 ceph-mon[34603]: pgmap v695: 97 pgs: 97 active+clean; 893 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 277 op/s 2024-09-21T03:41:10.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:10 smithi187 ceph-mon[34603]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:41:10.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:10 smithi112 ceph-mon[31442]: pgmap v695: 97 pgs: 97 active+clean; 893 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 277 op/s 2024-09-21T03:41:10.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:10 smithi112 ceph-mon[31442]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:41:11.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1291466 10.26 MB/sec execute 551 sec latency 44.564 ms 2024-09-21T03:41:12.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1293217 10.26 MB/sec execute 552 sec latency 20.128 ms 2024-09-21T03:41:12.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:12 smithi203 ceph-mon[34723]: pgmap v697: 97 pgs: 97 active+clean; 904 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 350 op/s 2024-09-21T03:41:12.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:12 smithi187 ceph-mon[34603]: pgmap v697: 97 pgs: 97 active+clean; 904 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 350 op/s 2024-09-21T03:41:12.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:12 smithi112 ceph-mon[31442]: pgmap v697: 97 pgs: 97 active+clean; 904 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 350 op/s 2024-09-21T03:41:13.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1295303 10.26 MB/sec execute 553 sec latency 17.698 ms 2024-09-21T03:41:14.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1296934 10.26 MB/sec execute 554 sec latency 18.563 ms 2024-09-21T03:41:14.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:14 smithi203 ceph-mon[34723]: pgmap v698: 97 pgs: 97 active+clean; 906 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 325 op/s 2024-09-21T03:41:14.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:14 smithi187 ceph-mon[34603]: pgmap v698: 97 pgs: 97 active+clean; 906 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 325 op/s 2024-09-21T03:41:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:14 smithi112 ceph-mon[31442]: pgmap v698: 97 pgs: 97 active+clean; 906 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 325 op/s 2024-09-21T03:41:15.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1299457 10.26 MB/sec execute 555 sec latency 18.878 ms 2024-09-21T03:41:16.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1300618 10.26 MB/sec execute 556 sec latency 58.064 ms 2024-09-21T03:41:16.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:16 smithi203 ceph-mon[34723]: pgmap v699: 97 pgs: 97 active+clean; 907 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-09-21T03:41:16.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:16 smithi187 ceph-mon[34603]: pgmap v699: 97 pgs: 97 active+clean; 907 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-09-21T03:41:16.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:16 smithi112 ceph-mon[31442]: pgmap v699: 97 pgs: 97 active+clean; 907 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 263 op/s 2024-09-21T03:41:17.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1302826 10.26 MB/sec execute 557 sec latency 20.422 ms 2024-09-21T03:41:18.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:41:18.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1304075 10.25 MB/sec execute 558 sec latency 14.827 ms 2024-09-21T03:41:18.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:18 smithi203 ceph-mon[34723]: pgmap v700: 97 pgs: 97 active+clean; 917 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 263 op/s 2024-09-21T03:41:18.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:18 smithi187 ceph-mon[34603]: pgmap v700: 97 pgs: 97 active+clean; 917 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 263 op/s 2024-09-21T03:41:18.534 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:18.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:18 smithi112 ceph-mon[31442]: pgmap v700: 97 pgs: 97 active+clean; 917 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 263 op/s 2024-09-21T03:41:19.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.092+0000 7fca5afbf640 1 -- 172.21.15.112:0/1747593110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca541083d0 msgr2=0x7fca541087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.092+0000 7fca5afbf640 1 --2- 172.21.15.112:0/1747593110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca541083d0 0x7fca541087e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fca40009900 tx=0x7fca4002a410 comp rx=0 tx=0).stop 2024-09-21T03:41:19.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.092+0000 7fca5afbf640 1 -- 172.21.15.112:0/1747593110 shutdown_connections 2024-09-21T03:41:19.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.092+0000 7fca5afbf640 1 --2- 172.21.15.112:0/1747593110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca54108d20 0x7fca5410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.092+0000 7fca5afbf640 1 --2- 172.21.15.112:0/1747593110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca541083d0 0x7fca541087e0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.092+0000 7fca5afbf640 1 --2- 172.21.15.112:0/1747593110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca54102ae0 0x7fca54102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.092+0000 7fca5afbf640 1 -- 172.21.15.112:0/1747593110 >> 172.21.15.112:0/1747593110 conn(0x7fca540797b0 msgr2=0x7fca54079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:19.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.092+0000 7fca5afbf640 1 -- 172.21.15.112:0/1747593110 shutdown_connections 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.092+0000 7fca5afbf640 1 -- 172.21.15.112:0/1747593110 wait complete. 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.093+0000 7fca5afbf640 1 Processor -- start 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.093+0000 7fca5afbf640 1 -- start start 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.093+0000 7fca5afbf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca54102ae0 0x7fca54199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.093+0000 7fca5afbf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca541083d0 0x7fca54199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.093+0000 7fca5afbf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca54108d20 0x7fca5419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.093+0000 7fca5afbf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca541129e0 con 0x7fca541083d0 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.093+0000 7fca5afbf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fca54112860 con 0x7fca54108d20 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.093+0000 7fca5afbf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fca54112b60 con 0x7fca54102ae0 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca53fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca541083d0 0x7fca54199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca58d5b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca54102ae0 0x7fca54199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca53fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca541083d0 0x7fca54199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50608/0 (socket says 172.21.15.112:50608) 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca53fff640 1 -- 172.21.15.112:0/3608962621 learned_addr learned my addr 172.21.15.112:0/3608962621 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:19.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca5955c640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca54108d20 0x7fca5419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca58d5b640 1 -- 172.21.15.112:0/3608962621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca54108d20 msgr2=0x7fca5419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca58d5b640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca54108d20 0x7fca5419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca58d5b640 1 -- 172.21.15.112:0/3608962621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca541083d0 msgr2=0x7fca54199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca58d5b640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca541083d0 0x7fca54199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca58d5b640 1 -- 172.21.15.112:0/3608962621 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5419f2e0 con 0x7fca54102ae0 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca53fff640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca541083d0 0x7fca54199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca58d5b640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca54102ae0 0x7fca54199370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fca4800e9f0 tx=0x7fca4800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca51ffb640 1 -- 172.21.15.112:0/3608962621 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4800ccc0 con 0x7fca54102ae0 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca5afbf640 1 -- 172.21.15.112:0/3608962621 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5419f5d0 con 0x7fca54102ae0 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca51ffb640 1 -- 172.21.15.112:0/3608962621 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca480108e0 con 0x7fca54102ae0 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.094+0000 7fca51ffb640 1 -- 172.21.15.112:0/3608962621 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48018880 con 0x7fca54102ae0 2024-09-21T03:41:19.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.095+0000 7fca5afbf640 1 -- 172.21.15.112:0/3608962621 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5406b820 con 0x7fca54102ae0 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca61533740 1 Processor -- start 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca61533740 1 -- start start 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca61533740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c6ad070 0x55f85ca54270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca61533740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c6841b0 0x55f85ca547b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca61533740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c68c390 0x55f85ca54cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca61533740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f85c9cae50 con 0x55f85c68c390 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca61533740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f85c9ba280 con 0x55f85c6841b0 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca61533740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f85c9be090 con 0x55f85c6ad070 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca53fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c68c390 0x55f85ca54cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca5955c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c6ad070 0x55f85ca54270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca53fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c68c390 0x55f85ca54cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50612/0 (socket says 172.21.15.112:50612) 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca5955c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c6ad070 0x55f85ca54270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60034/0 (socket says 172.21.15.112:60034) 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.097+0000 7fca53fff640 1 -- 172.21.15.112:0/2773639618 learned_addr learned my addr 172.21.15.112:0/2773639618 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:19.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca51ffb640 1 -- 172.21.15.112:0/3608962621 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fca48010430 con 0x7fca54102ae0 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca5955c640 1 -- 172.21.15.112:0/2773639618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c6841b0 msgr2=0x55f85ca547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca5955c640 1 --2- 172.21.15.112:0/2773639618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c6841b0 0x55f85ca547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca5955c640 1 -- 172.21.15.112:0/2773639618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c68c390 msgr2=0x55f85ca54cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca5955c640 1 --2- 172.21.15.112:0/2773639618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c68c390 0x55f85ca54cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca5955c640 1 -- 172.21.15.112:0/2773639618 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85c718650 con 0x55f85c6ad070 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca53fff640 1 --2- 172.21.15.112:0/2773639618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c68c390 0x55f85ca54cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca5955c640 1 --2- 172.21.15.112:0/2773639618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c6ad070 0x55f85ca54270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009f30 tx=0x7fca4c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c39790b518dfcc70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca3effd640 1 -- 172.21.15.112:0/2773639618 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c01b020 con 0x55f85c6ad070 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca3effd640 1 -- 172.21.15.112:0/2773639618 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca4c003e20 con 0x55f85c6ad070 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca3effd640 1 -- 172.21.15.112:0/2773639618 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c0047d0 con 0x55f85c6ad070 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.098+0000 7fca51ffb640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca440b6680 0x7fca440b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca58d5b640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca440b6680 0x7fca440b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca51ffb640 1 -- 172.21.15.112:0/3608962621 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7fca480e7a20 con 0x7fca54102ae0 2024-09-21T03:41:19.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca61533740 1 -- 172.21.15.112:0/2773639618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c6ad070 msgr2=0x55f85ca54270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca58d5b640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca440b6680 0x7fca440b8b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fca480aabc0 tx=0x7fca4807a600 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca61533740 1 --2- 172.21.15.112:0/2773639618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c6ad070 0x55f85ca54270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009f30 tx=0x7fca4c009500 comp rx=0 tx=0).stop 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca61533740 1 -- 172.21.15.112:0/2773639618 shutdown_connections 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca61533740 1 --2- 172.21.15.112:0/2773639618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c68c390 0x55f85ca54cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca61533740 1 --2- 172.21.15.112:0/2773639618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c6841b0 0x55f85ca547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca61533740 1 --2- 172.21.15.112:0/2773639618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c6ad070 0x55f85ca54270 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca61533740 1 -- 172.21.15.112:0/2773639618 >> 172.21.15.112:0/2773639618 conn(0x55f85c6a1460 msgr2=0x55f85ca52160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca61533740 1 -- 172.21.15.112:0/2773639618 shutdown_connections 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.099+0000 7fca61533740 1 -- 172.21.15.112:0/2773639618 wait complete. 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca61533740 1 -- start start 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca61533740 1 Processor -- start 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca61533740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c6841b0 0x55f85ca63d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca61533740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 0x55f85ca64250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca61533740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6ad070 0x55f85ca647b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca61533740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f85c81b700 con 0x55f85c6ad070 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca61533740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f85c955110 con 0x55f85c6841b0 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca61533740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f85ca66d80 con 0x55f85c68c390 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca61533740 10 client.0 _ll_register_callbacks cb 0x55f85c96e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca5955c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6ad070 0x55f85ca647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca53fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 0x55f85ca64250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca5955c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6ad070 0x55f85ca647b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50628/0 (socket says 172.21.15.112:50628) 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca5955c640 1 -- 172.21.15.112:0/1456352435 learned_addr learned my addr 172.21.15.112:0/1456352435 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca53fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 0x55f85ca64250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60042/0 (socket says 172.21.15.112:60042) 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.100+0000 7fca5955c640 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c6841b0 0x55f85ca63d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca53fff640 1 -- 172.21.15.112:0/1456352435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c6841b0 msgr2=0x55f85ca63d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca53fff640 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c6841b0 0x55f85ca63d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca53fff640 1 -- 172.21.15.112:0/1456352435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6ad070 msgr2=0x55f85ca647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca53fff640 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6ad070 0x55f85ca647b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca53fff640 1 -- 172.21.15.112:0/1456352435 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85c982c40 con 0x55f85c68c390 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca5955c640 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6ad070 0x55f85ca647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca53fff640 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 0x55f85ca64250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fca40009230 tx=0x7fca400316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c175cf213515411 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca33fff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca33fff640 1 -- 172.21.15.112:0/1456352435 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40035040 con 0x55f85c68c390 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca61533740 1 -- 172.21.15.112:0/1456352435 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca4c003e20 con 0x55f85c68c390 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca33fff640 1 -- 172.21.15.112:0/1456352435 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7fca40005240 con 0x55f85c68c390 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca33fff640 1 -- 172.21.15.112:0/1456352435 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca40005440 con 0x55f85c68c390 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca61533740 10 client.27064 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.101+0000 7fca61533740 1 -- 172.21.15.112:0/1456352435 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f85ca72a40 con 0x55f85c68c390 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.102+0000 7fca33fff640 1 -- 172.21.15.112:0/1456352435 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca4c003e20 con 0x55f85c68c390 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.102+0000 7fca33fff640 1 -- 172.21.15.112:0/1456352435 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fca40048750 con 0x55f85c68c390 2024-09-21T03:41:19.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.102+0000 7fca61533740 10 client.27064 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.102+0000 7fca61533740 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.102+0000 7fca61533740 20 client.27064 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.102+0000 7fca61533740 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f85c66b9f0 0x55f85ca75eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.102+0000 7fca61533740 4 client.27064 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.102+0000 7fca61533740 1 -- 172.21.15.112:0/1456352435 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f85c9ccda0 con 0x55f85c66b9f0 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.102+0000 7fca5955c640 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f85c66b9f0 0x55f85ca75eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.102+0000 7fca5955c640 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f85c66b9f0 0x55f85ca75eb0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a875db97041b10 server_cookie=79c41ac48c640f72 in_seq=0 out_seq=0 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.102+0000 7fca33fff640 10 client.27064 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca33fff640 1 -- 172.21.15.112:0/1456352435 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f85c9ccda0 con 0x55f85c66b9f0 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca33fff640 10 client.27064 handle_command_reply: tid=0 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 1 client.27064 shutdown 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 1 -- 172.21.15.112:0/1456352435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 msgr2=0x55f85ca64250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 0x55f85ca64250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fca40009230 tx=0x7fca400316d0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 1 -- 172.21.15.112:0/1456352435 shutdown_connections 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f85c66b9f0 0x55f85ca75eb0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6ad070 0x55f85ca647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 0x55f85ca64250 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca33fff640 0 client.27064 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 1 --2- 172.21.15.112:0/1456352435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c6841b0 0x55f85ca63d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 1 -- 172.21.15.112:0/1456352435 >> 172.21.15.112:0/1456352435 conn(0x55f85c6a1460 msgr2=0x55f85ca57400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 1 -- 172.21.15.112:0/1456352435 shutdown_connections 2024-09-21T03:41:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 1 -- 172.21.15.112:0/1456352435 wait complete. 2024-09-21T03:41:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.103+0000 7fca61533740 20 client.27064 trim_cache size 0 max 16384 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.121+0000 7fca61533740 1 Processor -- start 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.121+0000 7fca61533740 1 -- start start 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.121+0000 7fca61533740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c66b9f0 0x55f85ca559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.121+0000 7fca61533740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca60f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.121+0000 7fca61533740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 0x55f85ca61440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.121+0000 7fca61533740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca541afd80 con 0x55f85c6841b0 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.121+0000 7fca61533740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fca541ad650 con 0x55f85c66b9f0 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.121+0000 7fca61533740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fca541ab4c0 con 0x55f85c68c390 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.121+0000 7fca53fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca60f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca53fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca60f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50636/0 (socket says 172.21.15.112:50636) 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca53fff640 1 -- 172.21.15.112:0/1311855283 learned_addr learned my addr 172.21.15.112:0/1311855283 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca5955c640 1 --2- 172.21.15.112:0/1311855283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 0x55f85ca61440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca5955c640 1 --2- 172.21.15.112:0/1311855283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c66b9f0 0x55f85ca559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca53fff640 1 -- 172.21.15.112:0/1311855283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 msgr2=0x55f85ca61440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca53fff640 1 --2- 172.21.15.112:0/1311855283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 0x55f85ca61440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca53fff640 1 -- 172.21.15.112:0/1311855283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c66b9f0 msgr2=0x55f85ca559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca53fff640 1 --2- 172.21.15.112:0/1311855283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c66b9f0 0x55f85ca559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca53fff640 1 -- 172.21.15.112:0/1311855283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f85ca4c3f0 con 0x55f85c6841b0 2024-09-21T03:41:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca53fff640 1 --2- 172.21.15.112:0/1311855283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca60f00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fca40031440 tx=0x7fca40031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4029445dac6a32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca3e7fc640 1 -- 172.21.15.112:0/1311855283 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40035070 con 0x55f85c6841b0 2024-09-21T03:41:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca3e7fc640 1 -- 172.21.15.112:0/1311855283 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca40048680 con 0x55f85c6841b0 2024-09-21T03:41:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.122+0000 7fca3e7fc640 1 -- 172.21.15.112:0/1311855283 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4003d450 con 0x55f85c6841b0 2024-09-21T03:41:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.123+0000 7fca61533740 1 -- 172.21.15.112:0/1311855283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 msgr2=0x55f85ca60f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.123+0000 7fca61533740 1 --2- 172.21.15.112:0/1311855283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca60f00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fca40031440 tx=0x7fca40031a50 comp rx=0 tx=0).stop 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.123+0000 7fca61533740 1 -- 172.21.15.112:0/1311855283 shutdown_connections 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.123+0000 7fca61533740 1 --2- 172.21.15.112:0/1311855283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c68c390 0x55f85ca61440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.123+0000 7fca61533740 1 --2- 172.21.15.112:0/1311855283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca60f00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.123+0000 7fca61533740 1 --2- 172.21.15.112:0/1311855283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c66b9f0 0x55f85ca559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.123+0000 7fca61533740 1 -- 172.21.15.112:0/1311855283 >> 172.21.15.112:0/1311855283 conn(0x55f85c6a1460 msgr2=0x55f85ca552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.123+0000 7fca61533740 1 -- 172.21.15.112:0/1311855283 shutdown_connections 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.123+0000 7fca61533740 1 -- 172.21.15.112:0/1311855283 wait complete. 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.124+0000 7fca61533740 1 -- start start 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.124+0000 7fca61533740 1 Processor -- start 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.124+0000 7fca61533740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c66b9f0 0x55f85ca67f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.124+0000 7fca61533740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca68480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.124+0000 7fca61533740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c68c390 0x55f85ca689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.124+0000 7fca61533740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca541abb10 con 0x55f85c6841b0 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.124+0000 7fca61533740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fca541acf80 con 0x55f85c68c390 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.124+0000 7fca61533740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fca541ac1e0 con 0x55f85c66b9f0 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.124+0000 7fca61533740 10 client.0 _ll_register_callbacks cb 0x7fca40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca53fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca68480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca5955c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c66b9f0 0x55f85ca67f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca53fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca68480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50646/0 (socket says 172.21.15.112:50646) 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca53fff640 1 -- 172.21.15.112:0/1111656789 learned_addr learned my addr 172.21.15.112:0/1111656789 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca5955c640 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c68c390 0x55f85ca689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca53fff640 1 -- 172.21.15.112:0/1111656789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c66b9f0 msgr2=0x55f85ca67f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca53fff640 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c66b9f0 0x55f85ca67f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca53fff640 1 -- 172.21.15.112:0/1111656789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c68c390 msgr2=0x55f85ca689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca53fff640 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c68c390 0x55f85ca689c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca53fff640 1 -- 172.21.15.112:0/1111656789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f85cadf850 con 0x55f85c6841b0 2024-09-21T03:41:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca53fff640 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca68480 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fca40048cc0 tx=0x7fca400316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d62079d684dfe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca33fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:41:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca33fff640 1 -- 172.21.15.112:0/1111656789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40069060 con 0x55f85c6841b0 2024-09-21T03:41:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca33fff640 1 -- 172.21.15.112:0/1111656789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7fca40051b80 con 0x55f85c6841b0 2024-09-21T03:41:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca61533740 1 -- 172.21.15.112:0/1111656789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca40048680 con 0x55f85c6841b0 2024-09-21T03:41:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca33fff640 1 -- 172.21.15.112:0/1111656789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca40035040 con 0x55f85c6841b0 2024-09-21T03:41:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca61533740 10 client.17511 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.125+0000 7fca61533740 1 -- 172.21.15.112:0/1111656789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f85cae0a40 con 0x55f85c6841b0 2024-09-21T03:41:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.126+0000 7fca33fff640 1 -- 172.21.15.112:0/1111656789 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca40048680 con 0x55f85c6841b0 2024-09-21T03:41:19.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.126+0000 7fca33fff640 1 -- 172.21.15.112:0/1111656789 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fca4003d750 con 0x55f85c6841b0 2024-09-21T03:41:19.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.126+0000 7fca61533740 10 client.17511 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:19.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.126+0000 7fca61533740 10 client.17511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:41:19.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.126+0000 7fca61533740 20 client.17511 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:19.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.126+0000 7fca61533740 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f85c6ad070 0x55f85cae3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:19.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.126+0000 7fca61533740 4 client.17511 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:41:19.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.126+0000 7fca61533740 1 -- 172.21.15.112:0/1111656789 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f85c9c8850 con 0x55f85c6ad070 2024-09-21T03:41:19.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.126+0000 7fca5955c640 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f85c6ad070 0x55f85cae3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:19.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.126+0000 7fca33fff640 10 client.17511 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.126+0000 7fca5955c640 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f85c6ad070 0x55f85cae3060 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7469cc754a36fa1f server_cookie=d97b358a6a06c91b in_seq=0 out_seq=0 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca33fff640 1 -- 172.21.15.112:0/1111656789 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x55f85c9c8850 con 0x55f85c6ad070 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca33fff640 10 client.17511 handle_command_reply: tid=0 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 1 client.17511 shutdown 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 1 -- 172.21.15.112:0/1111656789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 msgr2=0x55f85ca68480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca68480 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fca40048cc0 tx=0x7fca400316b0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 1 -- 172.21.15.112:0/1111656789 shutdown_connections 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f85c6ad070 0x55f85cae3060 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca33fff640 0 client.17511 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f85c68c390 0x55f85ca689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f85c6841b0 0x55f85ca68480 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 1 --2- 172.21.15.112:0/1111656789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f85c66b9f0 0x55f85ca67f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 1 -- 172.21.15.112:0/1111656789 >> 172.21.15.112:0/1111656789 conn(0x55f85c6a1460 msgr2=0x55f85ca65aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 1 -- 172.21.15.112:0/1111656789 shutdown_connections 2024-09-21T03:41:19.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 1 -- 172.21.15.112:0/1111656789 wait complete. 2024-09-21T03:41:19.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.127+0000 7fca61533740 20 client.17511 trim_cache size 0 max 16384 2024-09-21T03:41:19.134 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:41:19.134 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:41:19.134 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:41:19.134 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:41:19.134 INFO:teuthology.orchestra.run.smithi112.stdout: "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c": { 2024-09-21T03:41:19.134 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:45.957537+0000", 2024-09-21T03:41:19.134 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.134 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.134 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.134 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.135 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:41:19.136 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:41:19.137 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.138 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.139 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.142 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.143 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:41:19.144 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:41:19.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 -- 172.21.15.112:0/3608962621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca440b6680 msgr2=0x7fca440b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca440b6680 0x7fca440b8b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fca480aabc0 tx=0x7fca4807a600 comp rx=0 tx=0).stop 2024-09-21T03:41:19.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 -- 172.21.15.112:0/3608962621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca54102ae0 msgr2=0x7fca54199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:19.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca54102ae0 0x7fca54199370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fca4800e9f0 tx=0x7fca4800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 -- 172.21.15.112:0/3608962621 shutdown_connections 2024-09-21T03:41:19.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fca440b6680 0x7fca440b8b90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca54108d20 0x7fca5419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca541083d0 0x7fca54199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 --2- 172.21.15.112:0/3608962621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca54102ae0 0x7fca54199370 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:19.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 -- 172.21.15.112:0/3608962621 >> 172.21.15.112:0/3608962621 conn(0x7fca540797b0 msgr2=0x7fca54106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:19.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 -- 172.21.15.112:0/3608962621 shutdown_connections 2024-09-21T03:41:19.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:19.128+0000 7fca5afbf640 1 -- 172.21.15.112:0/3608962621 wait complete. 2024-09-21T03:41:19.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1305831 10.25 MB/sec execute 559 sec latency 51.251 ms 2024-09-21T03:41:20.103 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:41:20.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:41:20.280 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1307454 10.25 MB/sec execute 560 sec latency 11.754 ms 2024-09-21T03:41:20.408 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:20.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:20 smithi203 ceph-mon[34723]: pgmap v701: 97 pgs: 97 active+clean; 918 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 273 op/s 2024-09-21T03:41:20.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:20 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:20.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:20 smithi187 ceph-mon[34603]: pgmap v701: 97 pgs: 97 active+clean; 918 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 273 op/s 2024-09-21T03:41:20.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:20 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:20.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:20 smithi112 ceph-mon[31442]: pgmap v701: 97 pgs: 97 active+clean; 918 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 273 op/s 2024-09-21T03:41:20.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:20 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:41:20.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.931+0000 7f139c9ea640 1 -- 172.21.15.112:0/3787605632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f139810de10 msgr2=0x7f1398110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.931+0000 7f139c9ea640 1 --2- 172.21.15.112:0/3787605632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f139810de10 0x7f1398110300 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1390009ca0 tx=0x7f139002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.931+0000 7f139c9ea640 1 -- 172.21.15.112:0/3787605632 shutdown_connections 2024-09-21T03:41:20.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.931+0000 7f139c9ea640 1 --2- 172.21.15.112:0/3787605632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f139810de10 0x7f1398110300 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.931+0000 7f139c9ea640 1 --2- 172.21.15.112:0/3787605632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398105300 0x7f139810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.931+0000 7f139c9ea640 1 --2- 172.21.15.112:0/3787605632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13981049b0 0x7f1398104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.931+0000 7f139c9ea640 1 -- 172.21.15.112:0/3787605632 >> 172.21.15.112:0/3787605632 conn(0x7f1398079730 msgr2=0x7f1398079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.931+0000 7f139c9ea640 1 -- 172.21.15.112:0/3787605632 shutdown_connections 2024-09-21T03:41:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.931+0000 7f139c9ea640 1 -- 172.21.15.112:0/3787605632 wait complete. 2024-09-21T03:41:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.932+0000 7f139c9ea640 1 Processor -- start 2024-09-21T03:41:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.932+0000 7f139c9ea640 1 -- start start 2024-09-21T03:41:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.932+0000 7f139c9ea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13981049b0 0x7f139810aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.932+0000 7f139c9ea640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398105300 0x7f139810b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.932+0000 7f139c9ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f139810de10 0x7f13981ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f139c9ea640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1398104230 con 0x7f139810de10 2024-09-21T03:41:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f139c9ea640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f13981040b0 con 0x7f13981049b0 2024-09-21T03:41:20.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f139c9ea640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f13981043b0 con 0x7f1398105300 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f1396d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f139810de10 0x7f13981ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f1396d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f139810de10 0x7f13981ac7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50672/0 (socket says 172.21.15.112:50672) 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f1396d9d640 1 -- 172.21.15.112:0/1868825918 learned_addr learned my addr 172.21.15.112:0/1868825918 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f138ffff640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398105300 0x7f139810b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f139659c640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13981049b0 0x7f139810aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f1396d9d640 1 -- 172.21.15.112:0/1868825918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398105300 msgr2=0x7f139810b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f1396d9d640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398105300 0x7f139810b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f1396d9d640 1 -- 172.21.15.112:0/1868825918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13981049b0 msgr2=0x7f139810aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f1396d9d640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13981049b0 0x7f139810aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f1396d9d640 1 -- 172.21.15.112:0/1868825918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139810bcf0 con 0x7f139810de10 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f138ffff640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398105300 0x7f139810b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f1396d9d640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f139810de10 0x7f13981ac7a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1390005cc0 tx=0x7f1390031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.933+0000 7f138f7fe640 1 -- 172.21.15.112:0/1868825918 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390003b70 con 0x7f139810de10 2024-09-21T03:41:20.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.934+0000 7f139c9ea640 1 -- 172.21.15.112:0/1868825918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13981acda0 con 0x7f139810de10 2024-09-21T03:41:20.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.934+0000 7f139c9ea640 1 -- 172.21.15.112:0/1868825918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13981ad2b0 con 0x7f139810de10 2024-09-21T03:41:20.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.934+0000 7f138f7fe640 1 -- 172.21.15.112:0/1868825918 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13900318f0 con 0x7f139810de10 2024-09-21T03:41:20.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.934+0000 7f138f7fe640 1 -- 172.21.15.112:0/1868825918 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139002f950 con 0x7f139810de10 2024-09-21T03:41:20.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.935+0000 7f139c9ea640 1 -- 172.21.15.112:0/1868825918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1378005290 con 0x7f139810de10 2024-09-21T03:41:20.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.936+0000 7f138f7fe640 1 -- 172.21.15.112:0/1868825918 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f139002faf0 con 0x7f139810de10 2024-09-21T03:41:20.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.937+0000 7f138f7fe640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f13840b6680 0x7f13840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:20.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.937+0000 7f139659c640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f13840b6680 0x7f13840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:20.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.937+0000 7f138f7fe640 1 -- 172.21.15.112:0/1868825918 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f13901087c0 con 0x7f139810de10 2024-09-21T03:41:20.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.938+0000 7f139659c640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f13840b6680 0x7f13840b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f139810d940 tx=0x7f1388005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:20.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:20.941+0000 7f138f7fe640 1 -- 172.21.15.112:0/1868825918 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13900cedd0 con 0x7f139810de10 2024-09-21T03:41:21.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.195+0000 7f139c9ea640 1 -- 172.21.15.112:0/1868825918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1378005580 con 0x7f139810de10 2024-09-21T03:41:21.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.195+0000 7f138f7fe640 1 -- 172.21.15.112:0/1868825918 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f13900d31b0 con 0x7f139810de10 2024-09-21T03:41:21.197 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:41:21.198 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":30,"btime":"2024-09-21T03:35:32:357323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:35:32.307199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:41:21.198 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 30 2024-09-21T03:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 -- 172.21.15.112:0/1868825918 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f13840b6680 msgr2=0x7f13840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f13840b6680 0x7f13840b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f139810d940 tx=0x7f1388005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 -- 172.21.15.112:0/1868825918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f139810de10 msgr2=0x7f13981ac7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f139810de10 0x7f13981ac7a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1390005cc0 tx=0x7f1390031700 comp rx=0 tx=0).stop 2024-09-21T03:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 -- 172.21.15.112:0/1868825918 shutdown_connections 2024-09-21T03:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f13840b6680 0x7f13840b8b90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f139810de10 0x7f13981ac7a0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398105300 0x7f139810b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:21.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 --2- 172.21.15.112:0/1868825918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13981049b0 0x7f139810aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 -- 172.21.15.112:0/1868825918 >> 172.21.15.112:0/1868825918 conn(0x7f1398079730 msgr2=0x7f1398115200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 -- 172.21.15.112:0/1868825918 shutdown_connections 2024-09-21T03:41:21.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:21.198+0000 7f139c9ea640 1 -- 172.21.15.112:0/1868825918 wait complete. 2024-09-21T03:41:21.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1309337 10.25 MB/sec execute 561 sec latency 41.740 ms 2024-09-21T03:41:21.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:21.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:21.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:41:21.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:21.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:21.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:41:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:41:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:41:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:41:21.968 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:41:22.228 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:22.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1310918 10.25 MB/sec execute 562 sec latency 44.031 ms 2024-09-21T03:41:22.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:22 smithi112 ceph-mon[31442]: pgmap v702: 97 pgs: 97 active+clean; 926 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-09-21T03:41:22.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:22 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1868825918' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:22.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:22 smithi203 ceph-mon[34723]: pgmap v702: 97 pgs: 97 active+clean; 926 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-09-21T03:41:22.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:22 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1868825918' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:22.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:22 smithi187 ceph-mon[34603]: pgmap v702: 97 pgs: 97 active+clean; 926 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-09-21T03:41:22.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:22 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1868825918' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:41:22.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.765+0000 7fec722a9640 1 -- 172.21.15.112:0/3981436734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c108cf0 msgr2=0x7fec6c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:22.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.765+0000 7fec722a9640 1 --2- 172.21.15.112:0/3981436734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c108cf0 0x7fec6c10f520 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fec64009ca0 tx=0x7fec6402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.765+0000 7fec722a9640 1 -- 172.21.15.112:0/3981436734 shutdown_connections 2024-09-21T03:41:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.765+0000 7fec722a9640 1 --2- 172.21.15.112:0/3981436734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c108cf0 0x7fec6c10f520 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.765+0000 7fec722a9640 1 --2- 172.21.15.112:0/3981436734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec6c1083a0 0x7fec6c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.765+0000 7fec722a9640 1 --2- 172.21.15.112:0/3981436734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec6c102a60 0x7fec6c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.765+0000 7fec722a9640 1 -- 172.21.15.112:0/3981436734 >> 172.21.15.112:0/3981436734 conn(0x7fec6c079730 msgr2=0x7fec6c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.765+0000 7fec722a9640 1 -- 172.21.15.112:0/3981436734 shutdown_connections 2024-09-21T03:41:22.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.765+0000 7fec722a9640 1 -- 172.21.15.112:0/3981436734 wait complete. 2024-09-21T03:41:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.766+0000 7fec722a9640 1 Processor -- start 2024-09-21T03:41:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.766+0000 7fec722a9640 1 -- start start 2024-09-21T03:41:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.766+0000 7fec722a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec6c102a60 0x7fec6c19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.766+0000 7fec722a9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec6c1083a0 0x7fec6c19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.766+0000 7fec722a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c108cf0 0x7fec6c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.766+0000 7fec722a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c112960 con 0x7fec6c108cf0 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.766+0000 7fec722a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1127e0 con 0x7fec6c102a60 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.766+0000 7fec722a9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c112ae0 con 0x7fec6c1083a0 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec70846640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c108cf0 0x7fec6c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec70846640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c108cf0 0x7fec6c19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50682/0 (socket says 172.21.15.112:50682) 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec63fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec6c1083a0 0x7fec6c19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec70846640 1 -- 172.21.15.112:0/4088752070 learned_addr learned my addr 172.21.15.112:0/4088752070 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec6bfff640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec6c102a60 0x7fec6c19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec70846640 1 -- 172.21.15.112:0/4088752070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec6c1083a0 msgr2=0x7fec6c19b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec70846640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec6c1083a0 0x7fec6c19b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec70846640 1 -- 172.21.15.112:0/4088752070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec6c102a60 msgr2=0x7fec6c19b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec70846640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec6c102a60 0x7fec6c19b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec70846640 1 -- 172.21.15.112:0/4088752070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c1a6520 con 0x7fec6c108cf0 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec70846640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c108cf0 0x7fec6c19bd10 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fec64005cc0 tx=0x7fec64031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:22.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec69ffb640 1 -- 172.21.15.112:0/4088752070 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64003b70 con 0x7fec6c108cf0 2024-09-21T03:41:22.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec69ffb640 1 -- 172.21.15.112:0/4088752070 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec640318f0 con 0x7fec6c108cf0 2024-09-21T03:41:22.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec722a9640 1 -- 172.21.15.112:0/4088752070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6c1a67b0 con 0x7fec6c108cf0 2024-09-21T03:41:22.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec69ffb640 1 -- 172.21.15.112:0/4088752070 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6402f950 con 0x7fec6c108cf0 2024-09-21T03:41:22.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.767+0000 7fec722a9640 1 -- 172.21.15.112:0/4088752070 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec6c1a6d40 con 0x7fec6c108cf0 2024-09-21T03:41:22.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec69ffb640 1 -- 172.21.15.112:0/4088752070 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fec6402fb10 con 0x7fec6c108cf0 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec7881d740 1 Processor -- start 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec7881d740 1 -- start start 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec7881d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c5019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec7881d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c5019c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec7881d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdd4390 0x560c5019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec7881d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560c50112ea0 con 0x560c4fdcc1b0 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec7881d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560c501022d0 con 0x560c4fdf5070 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec7881d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560c501060e0 con 0x560c4fdd4390 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec6bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c5019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec63fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c5019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec6bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c5019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50688/0 (socket says 172.21.15.112:50688) 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.770+0000 7fec6bfff640 1 -- 172.21.15.112:0/3556899151 learned_addr learned my addr 172.21.15.112:0/3556899151 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:22.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec63fff640 1 --2- 172.21.15.112:0/3556899151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdd4390 0x560c5019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec6bfff640 1 -- 172.21.15.112:0/3556899151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdd4390 msgr2=0x560c5019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec6bfff640 1 --2- 172.21.15.112:0/3556899151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdd4390 0x560c5019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec6bfff640 1 -- 172.21.15.112:0/3556899151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 msgr2=0x560c5019c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec6bfff640 1 --2- 172.21.15.112:0/3556899151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c5019c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec6bfff640 1 -- 172.21.15.112:0/3556899151 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c4fe60650 con 0x560c4fdcc1b0 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec63fff640 1 --2- 172.21.15.112:0/3556899151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c5019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec69ffb640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec580b6730 0x7fec580b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec6bfff640 1 --2- 172.21.15.112:0/3556899151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c5019c800 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009ca0 tx=0x7fec5c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcd4309d2a54620 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec63fff640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec580b6730 0x7fec580b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec627fc640 1 -- 172.21.15.112:0/3556899151 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c005220 con 0x560c4fdcc1b0 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec69ffb640 1 -- 172.21.15.112:0/4088752070 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7fec64108720 con 0x7fec6c108cf0 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec627fc640 1 -- 172.21.15.112:0/3556899151 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec5c004c50 con 0x560c4fdcc1b0 2024-09-21T03:41:22.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.771+0000 7fec627fc640 1 -- 172.21.15.112:0/3556899151 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0196b0 con 0x560c4fdcc1b0 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.772+0000 7fec7881d740 1 -- 172.21.15.112:0/3556899151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 msgr2=0x560c5019c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.772+0000 7fec63fff640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec580b6730 0x7fec580b8c40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fec6c105f80 tx=0x7fec54004730 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.772+0000 7fec7881d740 1 --2- 172.21.15.112:0/3556899151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c5019c800 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009ca0 tx=0x7fec5c011810 comp rx=0 tx=0).stop 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.772+0000 7fec7881d740 1 -- 172.21.15.112:0/3556899151 shutdown_connections 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.772+0000 7fec7881d740 1 --2- 172.21.15.112:0/3556899151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdd4390 0x560c5019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.772+0000 7fec7881d740 1 --2- 172.21.15.112:0/3556899151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c5019c800 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.772+0000 7fec7881d740 1 --2- 172.21.15.112:0/3556899151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c5019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.772+0000 7fec7881d740 1 -- 172.21.15.112:0/3556899151 >> 172.21.15.112:0/3556899151 conn(0x560c4fde9460 msgr2=0x560c5019a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.772+0000 7fec7881d740 1 -- 172.21.15.112:0/3556899151 shutdown_connections 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.772+0000 7fec7881d740 1 -- 172.21.15.112:0/3556899151 wait complete. 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec7881d740 1 -- start start 2024-09-21T03:41:22.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec7881d740 1 Processor -- start 2024-09-21T03:41:22.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec7881d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 0x560c501a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec7881d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 0x560c501a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec7881d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c501a8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec7881d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560c4ff63700 con 0x560c4fdd4390 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec7881d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560c5009d160 con 0x560c4fdf5070 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec7881d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560c501aaaa0 con 0x560c4fdcc1b0 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec63fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 0x560c501a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec7881d740 10 client.0 _ll_register_callbacks cb 0x560c500b69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec63fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 0x560c501a7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50700/0 (socket says 172.21.15.112:50700) 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec63fff640 1 -- 172.21.15.112:0/1588663697 learned_addr learned my addr 172.21.15.112:0/1588663697 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.773+0000 7fec6bfff640 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 0x560c501a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec70846640 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c501a8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec63fff640 1 -- 172.21.15.112:0/1588663697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 msgr2=0x560c501a7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:22.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec63fff640 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 0x560c501a7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec63fff640 1 -- 172.21.15.112:0/1588663697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 msgr2=0x560c501a8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec63fff640 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c501a8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec63fff640 1 -- 172.21.15.112:0/1588663697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c500cac90 con 0x560c4fdd4390 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec70846640 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c501a8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec6bfff640 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 0x560c501a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec63fff640 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 0x560c501a7e50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fec54020940 tx=0x7fec54005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cb349f1eab3c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec4b7fe640 1 -- 172.21.15.112:0/1588663697 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5402d060 con 0x560c4fdd4390 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec7881d740 1 -- 172.21.15.112:0/1588663697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec5c004c50 con 0x560c4fdd4390 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec4b7fe640 1 -- 172.21.15.112:0/1588663697 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7fec5401a070 con 0x560c4fdd4390 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec4b7fe640 1 -- 172.21.15.112:0/1588663697 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec5401e620 con 0x560c4fdd4390 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec7881d740 10 client.17529 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:22.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.774+0000 7fec7881d740 1 -- 172.21.15.112:0/1588663697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560c501b67a0 con 0x560c4fdd4390 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.775+0000 7fec4b7fe640 1 -- 172.21.15.112:0/1588663697 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec5c004c50 con 0x560c4fdd4390 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.775+0000 7fec4b7fe640 1 -- 172.21.15.112:0/1588663697 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fec54027030 con 0x560c4fdd4390 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.775+0000 7fec7881d740 10 client.17529 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.775+0000 7fec7881d740 10 client.17529 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.775+0000 7fec7881d740 20 client.17529 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.775+0000 7fec7881d740 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x560c4fdb39f0 0x560c501b9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.775+0000 7fec7881d740 4 client.17529 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.775+0000 7fec7881d740 1 -- 172.21.15.112:0/1588663697 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c50114df0 con 0x560c4fdb39f0 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.775+0000 7fec6bfff640 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x560c4fdb39f0 0x560c501b9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.776+0000 7fec6bfff640 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x560c4fdb39f0 0x560c501b9c70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=edb8ea8a577c719 server_cookie=e1caa3237611cadb in_seq=0 out_seq=0 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.776+0000 7fec4b7fe640 10 client.17529 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.776+0000 7fec4b7fe640 1 -- 172.21.15.112:0/1588663697 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560c50114df0 con 0x560c4fdb39f0 2024-09-21T03:41:22.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.776+0000 7fec4b7fe640 10 client.17529 handle_command_reply: tid=0 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.776+0000 7fec7881d740 1 client.17529 shutdown 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.776+0000 7fec7881d740 1 -- 172.21.15.112:0/1588663697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 msgr2=0x560c501a7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.776+0000 7fec7881d740 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 0x560c501a7e50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fec54020940 tx=0x7fec54005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.777+0000 7fec7881d740 1 -- 172.21.15.112:0/1588663697 shutdown_connections 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.777+0000 7fec7881d740 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x560c4fdb39f0 0x560c501b9c70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.777+0000 7fec7881d740 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c501a8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.777+0000 7fec7881d740 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 0x560c501a7e50 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.777+0000 7fec4b7fe640 0 client.17529 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.777+0000 7fec7881d740 1 --2- 172.21.15.112:0/1588663697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 0x560c501a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.777+0000 7fec7881d740 1 -- 172.21.15.112:0/1588663697 >> 172.21.15.112:0/1588663697 conn(0x560c4fde9460 msgr2=0x560c5019f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.777+0000 7fec7881d740 1 -- 172.21.15.112:0/1588663697 shutdown_connections 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.777+0000 7fec7881d740 1 -- 172.21.15.112:0/1588663697 wait complete. 2024-09-21T03:41:22.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.777+0000 7fec7881d740 20 client.17529 trim_cache size 0 max 16384 2024-09-21T03:41:22.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec7881d740 1 Processor -- start 2024-09-21T03:41:22.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec7881d740 1 -- start start 2024-09-21T03:41:22.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec7881d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c501a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec7881d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdd4390 0x560c5019da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec7881d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c50199760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec7881d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1b1f70 con 0x560c4fdcc1b0 2024-09-21T03:41:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec7881d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1af840 con 0x560c4fdf5070 2024-09-21T03:41:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec7881d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1ad6b0 con 0x560c4fdd4390 2024-09-21T03:41:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec6bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c501a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec6bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c501a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50710/0 (socket says 172.21.15.112:50710) 2024-09-21T03:41:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec6bfff640 1 -- 172.21.15.112:0/638141540 learned_addr learned my addr 172.21.15.112:0/638141540 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec63fff640 1 --2- 172.21.15.112:0/638141540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdd4390 0x560c5019da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec70846640 1 --2- 172.21.15.112:0/638141540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c50199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec6bfff640 1 -- 172.21.15.112:0/638141540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdd4390 msgr2=0x560c5019da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec6bfff640 1 --2- 172.21.15.112:0/638141540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdd4390 0x560c5019da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec6bfff640 1 -- 172.21.15.112:0/638141540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 msgr2=0x560c50199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:22.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec6bfff640 1 --2- 172.21.15.112:0/638141540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c50199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.790+0000 7fec6bfff640 1 -- 172.21.15.112:0/638141540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5019e5b0 con 0x560c4fdcc1b0 2024-09-21T03:41:22.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.791+0000 7fec6bfff640 1 --2- 172.21.15.112:0/638141540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c501a97f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x560c5019af40 tx=0x7fec5c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631e967acdd9dcd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:22.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.791+0000 7fec61ffb640 1 -- 172.21.15.112:0/638141540 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c031070 con 0x560c4fdcc1b0 2024-09-21T03:41:22.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.791+0000 7fec61ffb640 1 -- 172.21.15.112:0/638141540 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec5c0044f0 con 0x560c4fdcc1b0 2024-09-21T03:41:22.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.791+0000 7fec61ffb640 1 -- 172.21.15.112:0/638141540 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0035f0 con 0x560c4fdcc1b0 2024-09-21T03:41:22.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.792+0000 7fec7881d740 1 -- 172.21.15.112:0/638141540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 msgr2=0x560c501a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:22.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.792+0000 7fec7881d740 1 --2- 172.21.15.112:0/638141540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c501a97f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x560c5019af40 tx=0x7fec5c005850 comp rx=0 tx=0).stop 2024-09-21T03:41:22.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.792+0000 7fec7881d740 1 -- 172.21.15.112:0/638141540 shutdown_connections 2024-09-21T03:41:22.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.792+0000 7fec7881d740 1 --2- 172.21.15.112:0/638141540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c50199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.792+0000 7fec7881d740 1 --2- 172.21.15.112:0/638141540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdd4390 0x560c5019da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.792+0000 7fec7881d740 1 --2- 172.21.15.112:0/638141540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdcc1b0 0x560c501a97f0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.792+0000 7fec7881d740 1 -- 172.21.15.112:0/638141540 >> 172.21.15.112:0/638141540 conn(0x560c4fde9460 msgr2=0x560c5019d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.792+0000 7fec7881d740 1 -- 172.21.15.112:0/638141540 shutdown_connections 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.792+0000 7fec7881d740 1 -- 172.21.15.112:0/638141540 wait complete. 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.792+0000 7fec7881d740 1 -- start start 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.792+0000 7fec7881d740 1 Processor -- start 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec7881d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 0x560c501abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec7881d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 0x560c501ac380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec7881d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c501ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec7881d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1add00 con 0x560c4fdd4390 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec7881d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1af170 con 0x560c4fdf5070 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec7881d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c1ae3d0 con 0x560c4fdcc1b0 2024-09-21T03:41:22.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec7881d740 10 client.0 _ll_register_callbacks cb 0x7fec54009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec63fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 0x560c501ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec6bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 0x560c501abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec70846640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c501ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec6bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 0x560c501abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60136/0 (socket says 172.21.15.112:60136) 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec63fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 0x560c501ac380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50726/0 (socket says 172.21.15.112:50726) 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec6bfff640 1 -- 172.21.15.112:0/342970476 learned_addr learned my addr 172.21.15.112:0/342970476 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec70846640 1 -- 172.21.15.112:0/342970476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 msgr2=0x560c501abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec70846640 1 --2- 172.21.15.112:0/342970476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 0x560c501abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec70846640 1 -- 172.21.15.112:0/342970476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 msgr2=0x560c501ac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec70846640 1 --2- 172.21.15.112:0/342970476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 0x560c501ac380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec70846640 1 -- 172.21.15.112:0/342970476 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c502237f0 con 0x560c4fdf5070 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec70846640 1 --2- 172.21.15.112:0/342970476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c501ac8c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec64105ea0 tx=0x7fec640ce4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2281efe47751ec3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:41:22.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.793+0000 7fec4b7fe640 1 -- 172.21.15.112:0/342970476 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec640f5070 con 0x560c4fdf5070 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec7881d740 1 -- 172.21.15.112:0/342970476 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec5c0044f0 con 0x560c4fdf5070 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec4b7fe640 1 -- 172.21.15.112:0/342970476 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7fec640d7b30 con 0x560c4fdf5070 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec4b7fe640 1 -- 172.21.15.112:0/342970476 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec640ceca0 con 0x560c4fdf5070 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec7881d740 10 client.27080 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec7881d740 1 -- 172.21.15.112:0/342970476 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560c50224b20 con 0x560c4fdf5070 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec4b7fe640 1 -- 172.21.15.112:0/342970476 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec5c0044f0 con 0x560c4fdf5070 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec4b7fe640 1 -- 172.21.15.112:0/342970476 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fec640d70d0 con 0x560c4fdf5070 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec7881d740 10 client.27080 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec7881d740 10 client.27080 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec7881d740 20 client.27080 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec7881d740 1 --2- 172.21.15.112:0/342970476 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x560c4fd88da0 0x560c50227290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec7881d740 4 client.27080 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec7881d740 1 -- 172.21.15.112:0/342970476 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c501108a0 con 0x560c4fd88da0 2024-09-21T03:41:22.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.794+0000 7fec6bfff640 1 --2- 172.21.15.112:0/342970476 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x560c4fd88da0 0x560c50227290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec4b7fe640 10 client.27080 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec6bfff640 1 --2- 172.21.15.112:0/342970476 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x560c4fd88da0 0x560c50227290 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4ae0367660e0b01 server_cookie=6f2fe505a60bbe7 in_seq=0 out_seq=0 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec4b7fe640 1 -- 172.21.15.112:0/342970476 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c501108a0 con 0x560c4fd88da0 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec4b7fe640 10 client.27080 handle_command_reply: tid=0 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 1 client.27080 shutdown 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 1 -- 172.21.15.112:0/342970476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 msgr2=0x560c501ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 1 --2- 172.21.15.112:0/342970476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c501ac8c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec64105ea0 tx=0x7fec640ce4a0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 1 -- 172.21.15.112:0/342970476 shutdown_connections 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 1 --2- 172.21.15.112:0/342970476 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x560c4fd88da0 0x560c50227290 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 1 --2- 172.21.15.112:0/342970476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c4fdf5070 0x560c501ac8c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 1 --2- 172.21.15.112:0/342970476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c4fdd4390 0x560c501ac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 1 --2- 172.21.15.112:0/342970476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c4fdcc1b0 0x560c501abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec4b7fe640 0 client.27080 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 1 -- 172.21.15.112:0/342970476 >> 172.21.15.112:0/342970476 conn(0x560c4fde9460 msgr2=0x560c5019dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:22.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 1 -- 172.21.15.112:0/342970476 shutdown_connections 2024-09-21T03:41:22.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 1 -- 172.21.15.112:0/342970476 wait complete. 2024-09-21T03:41:22.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.795+0000 7fec7881d740 20 client.27080 trim_cache size 0 max 16384 2024-09-21T03:41:22.803 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:41:22.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 -- 172.21.15.112:0/4088752070 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec580b6730 msgr2=0x7fec580b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:22.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec580b6730 0x7fec580b8c40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fec6c105f80 tx=0x7fec54004730 comp rx=0 tx=0).stop 2024-09-21T03:41:22.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 -- 172.21.15.112:0/4088752070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c108cf0 msgr2=0x7fec6c19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:22.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c108cf0 0x7fec6c19bd10 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fec64005cc0 tx=0x7fec64031700 comp rx=0 tx=0).stop 2024-09-21T03:41:22.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 -- 172.21.15.112:0/4088752070 shutdown_connections 2024-09-21T03:41:22.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fec580b6730 0x7fec580b8c40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c108cf0 0x7fec6c19bd10 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec6c1083a0 0x7fec6c19b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 --2- 172.21.15.112:0/4088752070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec6c102a60 0x7fec6c19b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:22.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 -- 172.21.15.112:0/4088752070 >> 172.21.15.112:0/4088752070 conn(0x7fec6c079730 msgr2=0x7fec6c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:22.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 -- 172.21.15.112:0/4088752070 shutdown_connections 2024-09-21T03:41:22.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:22.796+0000 7fec722a9640 1 -- 172.21.15.112:0/4088752070 wait complete. 2024-09-21T03:41:23.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1312783 10.24 MB/sec execute 563 sec latency 38.403 ms 2024-09-21T03:41:23.870 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:41:24.129 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:24.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1314451 10.25 MB/sec execute 564 sec latency 14.520 ms 2024-09-21T03:41:24.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:24 smithi203 ceph-mon[34723]: pgmap v703: 97 pgs: 97 active+clean; 927 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 213 op/s 2024-09-21T03:41:24.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:24 smithi187 ceph-mon[34603]: pgmap v703: 97 pgs: 97 active+clean; 927 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 213 op/s 2024-09-21T03:41:24.446 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:24 smithi112 ceph-mon[31442]: pgmap v703: 97 pgs: 97 active+clean; 927 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 213 op/s 2024-09-21T03:41:24.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.675+0000 7f498035a640 1 -- 172.21.15.112:0/2278218353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4978109e40 msgr2=0x7f4978110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:24.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.675+0000 7f498035a640 1 --2- 172.21.15.112:0/2278218353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4978109e40 0x7f4978110670 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4970009ca0 tx=0x7f497002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.676+0000 7f498035a640 1 -- 172.21.15.112:0/2278218353 shutdown_connections 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.676+0000 7f498035a640 1 --2- 172.21.15.112:0/2278218353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4978109e40 0x7f4978110670 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.676+0000 7f498035a640 1 --2- 172.21.15.112:0/2278218353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49781094f0 0x7f4978109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.676+0000 7f498035a640 1 --2- 172.21.15.112:0/2278218353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4978103bb0 0x7f4978103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.676+0000 7f498035a640 1 -- 172.21.15.112:0/2278218353 >> 172.21.15.112:0/2278218353 conn(0x7f49780ff600 msgr2=0x7f4978101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.676+0000 7f498035a640 1 -- 172.21.15.112:0/2278218353 shutdown_connections 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.676+0000 7f498035a640 1 -- 172.21.15.112:0/2278218353 wait complete. 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.676+0000 7f498035a640 1 Processor -- start 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f498035a640 1 -- start start 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f498035a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4978103bb0 0x7f497819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f498035a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49781094f0 0x7f497819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f498035a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4978109e40 0x7f497819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f498035a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4978113b00 con 0x7f4978109e40 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f498035a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4978113980 con 0x7f4978103bb0 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f498035a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4978113c80 con 0x7f49781094f0 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f497e8f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4978109e40 0x7f497819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f497d8f5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49781094f0 0x7f497819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f497e0f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4978103bb0 0x7f497819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f497d8f5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49781094f0 0x7f497819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60152/0 (socket says 172.21.15.112:60152) 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f497d8f5640 1 -- 172.21.15.112:0/248186869 learned_addr learned my addr 172.21.15.112:0/248186869 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f497e8f7640 1 -- 172.21.15.112:0/248186869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49781094f0 msgr2=0x7f497819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f497e8f7640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49781094f0 0x7f497819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f497e8f7640 1 -- 172.21.15.112:0/248186869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4978103bb0 msgr2=0x7f497819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f497e8f7640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4978103bb0 0x7f497819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.677+0000 7f497e8f7640 1 -- 172.21.15.112:0/248186869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49781a03e0 con 0x7f4978109e40 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.678+0000 7f497e8f7640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4978109e40 0x7f497819fcb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f497002ade0 tx=0x7f49700316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:24.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.678+0000 7f496f7fe640 1 -- 172.21.15.112:0/248186869 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970003b70 con 0x7f4978109e40 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.678+0000 7f496f7fe640 1 -- 172.21.15.112:0/248186869 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4970003d10 con 0x7f4978109e40 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.678+0000 7f496f7fe640 1 -- 172.21.15.112:0/248186869 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497002f9e0 con 0x7f4978109e40 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.678+0000 7f498035a640 1 -- 172.21.15.112:0/248186869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49781a0670 con 0x7f4978109e40 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.678+0000 7f498035a640 1 -- 172.21.15.112:0/248186869 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49781a6e30 con 0x7f4978109e40 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f496f7fe640 1 -- 172.21.15.112:0/248186869 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f497002fb80 con 0x7f4978109e40 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f49868ce740 1 Processor -- start 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f49868ce740 1 -- start start 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f49868ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0843070 0x5561e0bea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f49868ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e081a1b0 0x5561e0bea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f49868ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e0822390 0x5561e0bead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f49868ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5561e0b60ea0 con 0x5561e0822390 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f49868ce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5561e0b502d0 con 0x5561e0843070 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f49868ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5561e0b540e0 con 0x5561e081a1b0 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f497e0f6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e081a1b0 0x5561e0bea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f497d8f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e0822390 0x5561e0bead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f497e0f6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e081a1b0 0x5561e0bea7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:60168/0 (socket says 172.21.15.112:60168) 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.681+0000 7f497e0f6640 1 -- 172.21.15.112:0/3549256297 learned_addr learned my addr 172.21.15.112:0/3549256297 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f497d8f5640 1 --2- 172.21.15.112:0/3549256297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0843070 0x5561e0bea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f497e0f6640 1 -- 172.21.15.112:0/3549256297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0843070 msgr2=0x5561e0bea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f497e0f6640 1 --2- 172.21.15.112:0/3549256297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0843070 0x5561e0bea290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f496f7fe640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49640b6680 0x7f49640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f497e0f6640 1 -- 172.21.15.112:0/3549256297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e0822390 msgr2=0x5561e0bead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f497e0f6640 1 --2- 172.21.15.112:0/3549256297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e0822390 0x5561e0bead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f497e0f6640 1 -- 172.21.15.112:0/3549256297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e08ae650 con 0x5561e081a1b0 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f497d8f5640 1 --2- 172.21.15.112:0/3549256297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0843070 0x5561e0bea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f497e8f7640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49640b6680 0x7f49640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f497e0f6640 1 --2- 172.21.15.112:0/3549256297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e081a1b0 0x5561e0bea7d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4974009cf0 tx=0x7f4974011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b3013ff36234cf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f496f7fe640 1 -- 172.21.15.112:0/248186869 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f4970108790 con 0x7f4978109e40 2024-09-21T03:41:24.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f496cd9a640 1 -- 172.21.15.112:0/3549256297 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497401e070 con 0x5561e081a1b0 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f496cd9a640 1 -- 172.21.15.112:0/3549256297 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4974009de0 con 0x5561e081a1b0 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.682+0000 7f496cd9a640 1 -- 172.21.15.112:0/3549256297 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974019450 con 0x5561e081a1b0 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.683+0000 7f49868ce740 1 -- 172.21.15.112:0/3549256297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e081a1b0 msgr2=0x5561e0bea7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.683+0000 7f497e8f7640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49640b6680 0x7f49640b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4970105fc0 tx=0x7f497009ba00 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.683+0000 7f49868ce740 1 --2- 172.21.15.112:0/3549256297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e081a1b0 0x5561e0bea7d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4974009cf0 tx=0x7f4974011810 comp rx=0 tx=0).stop 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.683+0000 7f49868ce740 1 -- 172.21.15.112:0/3549256297 shutdown_connections 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.683+0000 7f49868ce740 1 --2- 172.21.15.112:0/3549256297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e0822390 0x5561e0bead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.683+0000 7f49868ce740 1 --2- 172.21.15.112:0/3549256297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e081a1b0 0x5561e0bea7d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.683+0000 7f49868ce740 1 --2- 172.21.15.112:0/3549256297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0843070 0x5561e0bea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.683+0000 7f49868ce740 1 -- 172.21.15.112:0/3549256297 >> 172.21.15.112:0/3549256297 conn(0x5561e0837460 msgr2=0x5561e0be8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.683+0000 7f49868ce740 1 -- 172.21.15.112:0/3549256297 shutdown_connections 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.683+0000 7f49868ce740 1 -- 172.21.15.112:0/3549256297 wait complete. 2024-09-21T03:41:24.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f49868ce740 1 -- start start 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f49868ce740 1 Processor -- start 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f49868ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f49868ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0bfa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f49868ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bfa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f49868ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5561e09b1700 con 0x5561e081a1b0 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f49868ce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5561e0aeb160 con 0x5561e0822390 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f49868ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5561e0bfcdd0 con 0x5561e0843070 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f49868ce740 10 client.0 _ll_register_callbacks cb 0x5561e0b049e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f497d8f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f497d8f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50764/0 (socket says 172.21.15.112:50764) 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f497e0f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0bfa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f497d8f5640 1 -- 172.21.15.112:0/3346205345 learned_addr learned my addr 172.21.15.112:0/3346205345 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f497d8f5640 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bfa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f497d8f5640 1 -- 172.21.15.112:0/3346205345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 msgr2=0x5561e0bfa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f497d8f5640 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bfa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f497d8f5640 1 -- 172.21.15.112:0/3346205345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 msgr2=0x5561e0bfa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f497d8f5640 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0bfa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f497d8f5640 1 -- 172.21.15.112:0/3346205345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561e0b18c90 con 0x5561e081a1b0 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.684+0000 7f497e0f6640 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0bfa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.685+0000 7f497d8f5640 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9cf0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f49680044c0 tx=0x7f496800bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d231570e01f7264 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.685+0000 7f4961ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.685+0000 7f4961ffb640 1 -- 172.21.15.112:0/3346205345 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968012930 con 0x5561e081a1b0 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.685+0000 7f4961ffb640 1 -- 172.21.15.112:0/3346205345 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f4968012ad0 con 0x5561e081a1b0 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.685+0000 7f49868ce740 1 -- 172.21.15.112:0/3346205345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4974009de0 con 0x5561e081a1b0 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.685+0000 7f4961ffb640 1 -- 172.21.15.112:0/3346205345 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4968012cd0 con 0x5561e081a1b0 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.685+0000 7f49868ce740 10 client.17547 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.685+0000 7f49868ce740 1 -- 172.21.15.112:0/3346205345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5561e0c08b50 con 0x5561e081a1b0 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.685+0000 7f4961ffb640 1 -- 172.21.15.112:0/3346205345 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4974009de0 con 0x5561e081a1b0 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.686+0000 7f4961ffb640 1 -- 172.21.15.112:0/3346205345 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f49680283a0 con 0x5561e081a1b0 2024-09-21T03:41:24.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.686+0000 7f49868ce740 10 client.17547 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:24.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.686+0000 7f49868ce740 10 client.17547 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:41:24.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.686+0000 7f49868ce740 20 client.17547 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:24.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.686+0000 7f49868ce740 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5561e08019f0 0x5561e0c0c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.686+0000 7f49868ce740 4 client.17547 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:24.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.686+0000 7f49868ce740 1 -- 172.21.15.112:0/3346205345 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561e0b62df0 con 0x5561e08019f0 2024-09-21T03:41:24.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.686+0000 7f497e0f6640 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5561e08019f0 0x5561e0c0c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.686+0000 7f4961ffb640 10 client.17547 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:41:24.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.686+0000 7f497e0f6640 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5561e08019f0 0x5561e0c0c0a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=399e72be3c8564f8 server_cookie=6bf8134ea798772f in_seq=0 out_seq=0 2024-09-21T03:41:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.687+0000 7f4961ffb640 1 -- 172.21.15.112:0/3346205345 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5561e0b62df0 con 0x5561e08019f0 2024-09-21T03:41:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.687+0000 7f4961ffb640 10 client.17547 handle_command_reply: tid=0 2024-09-21T03:41:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.687+0000 7f49868ce740 1 client.17547 shutdown 2024-09-21T03:41:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.687+0000 7f49868ce740 1 -- 172.21.15.112:0/3346205345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 msgr2=0x5561e0bf9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.687+0000 7f49868ce740 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9cf0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f49680044c0 tx=0x7f496800bbe0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.687+0000 7f49868ce740 1 -- 172.21.15.112:0/3346205345 shutdown_connections 2024-09-21T03:41:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.687+0000 7f49868ce740 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5561e08019f0 0x5561e0c0c0a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.688+0000 7f49868ce740 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bfa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.688+0000 7f49868ce740 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0bfa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.688+0000 7f49868ce740 1 --2- 172.21.15.112:0/3346205345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9cf0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.688+0000 7f49868ce740 1 -- 172.21.15.112:0/3346205345 >> 172.21.15.112:0/3346205345 conn(0x5561e0837460 msgr2=0x5561e0bebc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.688+0000 7f4961ffb640 0 client.17547 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.688+0000 7f49868ce740 1 -- 172.21.15.112:0/3346205345 shutdown_connections 2024-09-21T03:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.688+0000 7f49868ce740 1 -- 172.21.15.112:0/3346205345 wait complete. 2024-09-21T03:41:24.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.688+0000 7f49868ce740 20 client.17547 trim_cache size 0 max 16384 2024-09-21T03:41:24.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.702+0000 7f49868ce740 1 Processor -- start 2024-09-21T03:41:24.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.702+0000 7f49868ce740 1 -- start start 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f49868ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f49868ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0bf9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f49868ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bf6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f49868ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49781afd60 con 0x5561e081a1b0 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f49868ce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49781ad630 con 0x5561e0822390 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f49868ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f49781ab4a0 con 0x5561e0843070 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497d8f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497d8f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50766/0 (socket says 172.21.15.112:50766) 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497d8f5640 1 -- 172.21.15.112:0/3014959967 learned_addr learned my addr 172.21.15.112:0/3014959967 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497e0f6640 1 --2- 172.21.15.112:0/3014959967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0bf9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497d8f5640 1 --2- 172.21.15.112:0/3014959967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bf6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497d8f5640 1 -- 172.21.15.112:0/3014959967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 msgr2=0x5561e0bf6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497d8f5640 1 --2- 172.21.15.112:0/3014959967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bf6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497d8f5640 1 -- 172.21.15.112:0/3014959967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 msgr2=0x5561e0bf9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497d8f5640 1 --2- 172.21.15.112:0/3014959967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0bf9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497d8f5640 1 -- 172.21.15.112:0/3014959967 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e0bed050 con 0x5561e081a1b0 2024-09-21T03:41:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497e0f6640 1 --2- 172.21.15.112:0/3014959967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0bf9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.703+0000 7f497d8f5640 1 --2- 172.21.15.112:0/3014959967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9aa0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f49680287c0 tx=0x7f4968005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d10eefdf132652 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.704+0000 7f4963fff640 1 -- 172.21.15.112:0/3014959967 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968012120 con 0x5561e081a1b0 2024-09-21T03:41:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.704+0000 7f4963fff640 1 -- 172.21.15.112:0/3014959967 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f496800b440 con 0x5561e081a1b0 2024-09-21T03:41:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.704+0000 7f4963fff640 1 -- 172.21.15.112:0/3014959967 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968029450 con 0x5561e081a1b0 2024-09-21T03:41:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.705+0000 7f49868ce740 1 -- 172.21.15.112:0/3014959967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 msgr2=0x5561e0bf9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.705+0000 7f49868ce740 1 --2- 172.21.15.112:0/3014959967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9aa0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f49680287c0 tx=0x7f4968005fd0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.705+0000 7f49868ce740 1 -- 172.21.15.112:0/3014959967 shutdown_connections 2024-09-21T03:41:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.705+0000 7f49868ce740 1 --2- 172.21.15.112:0/3014959967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bf6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.705+0000 7f49868ce740 1 --2- 172.21.15.112:0/3014959967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0bf9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.705+0000 7f49868ce740 1 --2- 172.21.15.112:0/3014959967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0bf9aa0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.705+0000 7f49868ce740 1 -- 172.21.15.112:0/3014959967 >> 172.21.15.112:0/3014959967 conn(0x5561e0837460 msgr2=0x5561e0bf8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.705+0000 7f49868ce740 1 -- 172.21.15.112:0/3014959967 shutdown_connections 2024-09-21T03:41:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.705+0000 7f49868ce740 1 -- 172.21.15.112:0/3014959967 wait complete. 2024-09-21T03:41:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.705+0000 7f49868ce740 1 -- start start 2024-09-21T03:41:24.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.705+0000 7f49868ce740 1 Processor -- start 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f49868ce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0c02360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f49868ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0c028c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f49868ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bf1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f49868ce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49781abaf0 con 0x5561e081a1b0 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f49868ce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49781acf60 con 0x5561e0822390 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f49868ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f49781ac1c0 con 0x5561e0843070 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f49868ce740 10 client.0 _ll_register_callbacks cb 0x7f496800bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497d8f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0c02360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497d8f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0c02360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50772/0 (socket says 172.21.15.112:50772) 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497d8f5640 1 -- 172.21.15.112:0/990720983 learned_addr learned my addr 172.21.15.112:0/990720983 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497e0f6640 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0c028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497d8f5640 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bf1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497e0f6640 1 -- 172.21.15.112:0/990720983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 msgr2=0x5561e0bf1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497e0f6640 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bf1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497e0f6640 1 -- 172.21.15.112:0/990720983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 msgr2=0x5561e0c02360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497e0f6640 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0c02360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497e0f6640 1 -- 172.21.15.112:0/990720983 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561e0bf1d60 con 0x5561e0822390 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497d8f5640 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0c02360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f497e0f6640 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0c028c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f497401c040 tx=0x7f4974005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=512ad436e0fad6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.706+0000 7f4961ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f4961ffb640 1 -- 172.21.15.112:0/990720983 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974031060 con 0x5561e0822390 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f49868ce740 1 -- 172.21.15.112:0/990720983 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f496800b440 con 0x5561e0822390 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f4961ffb640 1 -- 172.21.15.112:0/990720983 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f4974009de0 con 0x5561e0822390 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f4961ffb640 1 -- 172.21.15.112:0/990720983 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f497401e040 con 0x5561e0822390 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f49868ce740 10 client.27098 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f49868ce740 1 -- 172.21.15.112:0/990720983 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5561e0bfec50 con 0x5561e0822390 2024-09-21T03:41:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f4961ffb640 1 -- 172.21.15.112:0/990720983 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f496800b440 con 0x5561e0822390 2024-09-21T03:41:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f4961ffb640 1 -- 172.21.15.112:0/990720983 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4974036820 con 0x5561e0822390 2024-09-21T03:41:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f49868ce740 10 client.27098 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f49868ce740 10 client.27098 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:41:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f49868ce740 20 client.27098 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f49868ce740 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5561e07d6da0 0x5561e0c7d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f49868ce740 4 client.27098 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:41:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.707+0000 7f49868ce740 1 -- 172.21.15.112:0/990720983 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561e0b5e8a0 con 0x5561e07d6da0 2024-09-21T03:41:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.708+0000 7f497d8f5640 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5561e07d6da0 0x5561e0c7d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.708+0000 7f4961ffb640 10 client.27098 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:41:24.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.708+0000 7f497d8f5640 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5561e07d6da0 0x5561e0c7d370 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b58a957d6491362f server_cookie=b2bec898cbcf0bc4 in_seq=0 out_seq=0 2024-09-21T03:41:24.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.708+0000 7f4961ffb640 1 -- 172.21.15.112:0/990720983 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561e0b5e8a0 con 0x5561e07d6da0 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.708+0000 7f4961ffb640 10 client.27098 handle_command_reply: tid=0 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.708+0000 7f49868ce740 1 client.27098 shutdown 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.708+0000 7f49868ce740 1 -- 172.21.15.112:0/990720983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 msgr2=0x5561e0c028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.708+0000 7f49868ce740 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0c028c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f497401c040 tx=0x7f4974005dc0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.708+0000 7f49868ce740 1 -- 172.21.15.112:0/990720983 shutdown_connections 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.708+0000 7f49868ce740 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5561e07d6da0 0x5561e0c7d370 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.708+0000 7f49868ce740 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561e0843070 0x5561e0bf1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f49868ce740 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5561e0822390 0x5561e0c028c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f49868ce740 1 --2- 172.21.15.112:0/990720983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561e081a1b0 0x5561e0c02360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f4961ffb640 0 client.27098 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f49868ce740 1 -- 172.21.15.112:0/990720983 >> 172.21.15.112:0/990720983 conn(0x5561e0837460 msgr2=0x5561e0bf9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f49868ce740 1 -- 172.21.15.112:0/990720983 shutdown_connections 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f49868ce740 1 -- 172.21.15.112:0/990720983 wait complete. 2024-09-21T03:41:24.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f49868ce740 20 client.27098 trim_cache size 0 max 16384 2024-09-21T03:41:24.717 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:41:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f498035a640 1 -- 172.21.15.112:0/248186869 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49640b6680 msgr2=0x7f49640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:24.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f498035a640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49640b6680 0x7f49640b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4970105fc0 tx=0x7f497009ba00 comp rx=0 tx=0).stop 2024-09-21T03:41:24.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f498035a640 1 -- 172.21.15.112:0/248186869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4978109e40 msgr2=0x7f497819fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:24.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f498035a640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4978109e40 0x7f497819fcb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f497002ade0 tx=0x7f49700316f0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.709+0000 7f498035a640 1 -- 172.21.15.112:0/248186869 shutdown_connections 2024-09-21T03:41:24.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.710+0000 7f498035a640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f49640b6680 0x7f49640b8b90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.710+0000 7f498035a640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4978109e40 0x7f497819fcb0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.710+0000 7f498035a640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49781094f0 0x7f497819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.710+0000 7f498035a640 1 --2- 172.21.15.112:0/248186869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4978103bb0 0x7f497819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:24.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.710+0000 7f498035a640 1 -- 172.21.15.112:0/248186869 >> 172.21.15.112:0/248186869 conn(0x7f49780ff600 msgr2=0x7f4978101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:24.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.710+0000 7f498035a640 1 -- 172.21.15.112:0/248186869 shutdown_connections 2024-09-21T03:41:24.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:24.710+0000 7f498035a640 1 -- 172.21.15.112:0/248186869 wait complete. 2024-09-21T03:41:25.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1316182 10.24 MB/sec execute 565 sec latency 38.990 ms 2024-09-21T03:41:25.414 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:41:25.688 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.174+0000 7f2b126da640 1 -- 172.21.15.112:0/3881576177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c102a70 msgr2=0x7f2b0c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.174+0000 7f2b126da640 1 --2- 172.21.15.112:0/3881576177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c102a70 0x7f2b0c102e80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009900 tx=0x7f2afc02a410 comp rx=0 tx=0).stop 2024-09-21T03:41:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.175+0000 7f2b126da640 1 -- 172.21.15.112:0/3881576177 shutdown_connections 2024-09-21T03:41:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.175+0000 7f2b126da640 1 --2- 172.21.15.112:0/3881576177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b0c108d00 0x7f2b0c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.175+0000 7f2b126da640 1 --2- 172.21.15.112:0/3881576177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b0c1083b0 0x7f2b0c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.175+0000 7f2b126da640 1 --2- 172.21.15.112:0/3881576177 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c102a70 0x7f2b0c102e80 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.175+0000 7f2b126da640 1 -- 172.21.15.112:0/3881576177 >> 172.21.15.112:0/3881576177 conn(0x7f2b0c079730 msgr2=0x7f2b0c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.175+0000 7f2b126da640 1 -- 172.21.15.112:0/3881576177 shutdown_connections 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.175+0000 7f2b126da640 1 -- 172.21.15.112:0/3881576177 wait complete. 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.175+0000 7f2b126da640 1 Processor -- start 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.175+0000 7f2b126da640 1 -- start start 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b126da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b0c102a70 0x7f2b0c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b126da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b0c1083b0 0x7f2b0c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b126da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c108d00 0x7f2b0c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b126da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c113a90 con 0x7f2b0c108d00 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b126da640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c113910 con 0x7f2b0c102a70 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b126da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c113c10 con 0x7f2b0c1083b0 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b10c77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c108d00 0x7f2b0c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b0bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b0c102a70 0x7f2b0c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b0c1083b0 0x7f2b0c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b10c77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c108d00 0x7f2b0c19fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51292/0 (socket says 172.21.15.112:51292) 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b0bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b0c102a70 0x7f2b0c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:46754/0 (socket says 172.21.15.112:46754) 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b10c77640 1 -- 172.21.15.112:0/575903179 learned_addr learned my addr 172.21.15.112:0/575903179 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b10c77640 1 -- 172.21.15.112:0/575903179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b0c1083b0 msgr2=0x7f2b0c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b10c77640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b0c1083b0 0x7f2b0c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b10c77640 1 -- 172.21.15.112:0/575903179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b0c102a70 msgr2=0x7f2b0c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b10c77640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b0c102a70 0x7f2b0c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b10c77640 1 -- 172.21.15.112:0/575903179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c1a0390 con 0x7f2b0c108d00 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b03fff640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b0c1083b0 0x7f2b0c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b10c77640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c108d00 0x7f2b0c19fc60 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400eb10 tx=0x7f2b0400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.176+0000 7f2b09ffb640 1 -- 172.21.15.112:0/575903179 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04004b20 con 0x7f2b0c108d00 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.177+0000 7f2b126da640 1 -- 172.21.15.112:0/575903179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0c1a0680 con 0x7f2b0c108d00 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.177+0000 7f2b09ffb640 1 -- 172.21.15.112:0/575903179 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b04004cc0 con 0x7f2b0c108d00 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.177+0000 7f2b09ffb640 1 -- 172.21.15.112:0/575903179 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04010670 con 0x7f2b0c108d00 2024-09-21T03:41:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.177+0000 7f2b126da640 1 -- 172.21.15.112:0/575903179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0c1a6de0 con 0x7f2b0c108d00 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.179+0000 7f2b18c4e740 1 Processor -- start 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.179+0000 7f2b09ffb640 1 -- 172.21.15.112:0/575903179 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2b04005020 con 0x7f2b0c108d00 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.179+0000 7f2b18c4e740 1 -- start start 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85ad070 0x5597e89546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e8954be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e858c390 0x5597e8955120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5597e88caea0 con 0x5597e85841b0 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5597e88ba2d0 con 0x5597e858c390 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5597e88be0e0 con 0x5597e85ad070 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85ad070 0x5597e89546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b0bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e8954be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b0bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e8954be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51308/0 (socket says 172.21.15.112:51308) 2024-09-21T03:41:26.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b0bfff640 1 -- 172.21.15.112:0/1818871768 learned_addr learned my addr 172.21.15.112:0/1818871768 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85ad070 0x5597e89546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:54314/0 (socket says 172.21.15.112:54314) 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b03fff640 1 --2- 172.21.15.112:0/1818871768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e858c390 0x5597e8955120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b09ffb640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2af80b6680 0x7f2af80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b03fff640 1 -- 172.21.15.112:0/1818871768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85ad070 msgr2=0x5597e89546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b03fff640 1 --2- 172.21.15.112:0/1818871768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85ad070 0x5597e89546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b03fff640 1 -- 172.21.15.112:0/1818871768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 msgr2=0x5597e8954be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b03fff640 1 --2- 172.21.15.112:0/1818871768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e8954be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b03fff640 1 -- 172.21.15.112:0/1818871768 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e8618650 con 0x5597e858c390 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b0bfff640 1 --2- 172.21.15.112:0/1818871768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e8954be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.180+0000 7f2b10c77640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2af80b6680 0x7f2af80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.181+0000 7f2b03fff640 1 --2- 172.21.15.112:0/1818871768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e858c390 0x5597e8955120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c079e30 tx=0x7f2af4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4803bd5cbb39acb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.181+0000 7f2b09ffb640 1 -- 172.21.15.112:0/575903179 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f2b040e7cc0 con 0x7f2b0c108d00 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.181+0000 7f2b027fc640 1 -- 172.21.15.112:0/1818871768 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af400f910 con 0x5597e858c390 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.181+0000 7f2b027fc640 1 -- 172.21.15.112:0/1818871768 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2af40117e0 con 0x5597e858c390 2024-09-21T03:41:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.181+0000 7f2b027fc640 1 -- 172.21.15.112:0/1818871768 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4004a00 con 0x5597e858c390 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1818871768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e858c390 msgr2=0x5597e8955120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b10c77640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2af80b6680 0x7f2af80b8b90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2b040aadb0 tx=0x7f2b0406c920 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/1818871768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e858c390 0x5597e8955120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c079e30 tx=0x7f2af4005f90 comp rx=0 tx=0).stop 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1818871768 shutdown_connections 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/1818871768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e858c390 0x5597e8955120 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/1818871768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e8954be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/1818871768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85ad070 0x5597e89546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1818871768 >> 172.21.15.112:0/1818871768 conn(0x5597e85a1460 msgr2=0x5597e8952590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1818871768 shutdown_connections 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1818871768 wait complete. 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b18c4e740 1 -- start start 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.182+0000 7f2b18c4e740 1 Processor -- start 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e895fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e858c390 0x5597e8960380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85ad070 0x5597e896c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5597e871b700 con 0x5597e85841b0 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5597e8855160 con 0x5597e858c390 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5597e8962e80 con 0x5597e85ad070 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b18c4e740 10 client.0 _ll_register_callbacks cb 0x5597e886e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e895fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e895fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51322/0 (socket says 172.21.15.112:51322) 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b03fff640 1 -- 172.21.15.112:0/4202646751 learned_addr learned my addr 172.21.15.112:0/4202646751 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b0bfff640 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e858c390 0x5597e8960380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b03fff640 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85ad070 0x5597e896c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b03fff640 1 -- 172.21.15.112:0/4202646751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85ad070 msgr2=0x5597e896c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:26.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b03fff640 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85ad070 0x5597e896c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b03fff640 1 -- 172.21.15.112:0/4202646751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e858c390 msgr2=0x5597e8960380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b03fff640 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e858c390 0x5597e8960380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b03fff640 1 -- 172.21.15.112:0/4202646751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e8882c90 con 0x5597e85841b0 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2b03fff640 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e895fe40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2af4005bc0 tx=0x7f2af40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfe24e22e5752d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.183+0000 7f2aeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/4202646751 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af400fe10 con 0x5597e85841b0 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/4202646751 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f2af40048d0 con 0x5597e85841b0 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2b18c4e740 1 -- 172.21.15.112:0/4202646751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2af40117e0 con 0x5597e85841b0 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/4202646751 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2af4004ad0 con 0x5597e85841b0 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2b18c4e740 10 client.17565 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2b18c4e740 1 -- 172.21.15.112:0/4202646751 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5597e896ec70 con 0x5597e85841b0 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/4202646751 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2af40117e0 con 0x5597e85841b0 2024-09-21T03:41:26.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/4202646751 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2af401fb00 con 0x5597e85841b0 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2b18c4e740 10 client.17565 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2b18c4e740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2b18c4e740 20 client.17565 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597e856b9f0 0x5597e8971fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2b18c4e740 4 client.17565 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.184+0000 7f2b18c4e740 1 -- 172.21.15.112:0/4202646751 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597e88ccdf0 con 0x5597e856b9f0 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.185+0000 7f2b0bfff640 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597e856b9f0 0x5597e8971fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.185+0000 7f2aeb7fe640 10 client.17565 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.185+0000 7f2b0bfff640 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597e856b9f0 0x5597e8971fd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=765266845e49316a server_cookie=1f55cca47685b6d8 in_seq=0 out_seq=0 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/4202646751 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5597e88ccdf0 con 0x5597e856b9f0 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2aeb7fe640 10 client.17565 handle_command_reply: tid=0 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 1 client.17565 shutdown 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 1 -- 172.21.15.112:0/4202646751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 msgr2=0x5597e895fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e895fe40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2af4005bc0 tx=0x7f2af40056a0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 1 -- 172.21.15.112:0/4202646751 shutdown_connections 2024-09-21T03:41:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597e856b9f0 0x5597e8971fd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85ad070 0x5597e896c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e858c390 0x5597e8960380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2aeb7fe640 0 client.17565 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/4202646751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e895fe40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 1 -- 172.21.15.112:0/4202646751 >> 172.21.15.112:0/4202646751 conn(0x5597e85a1460 msgr2=0x5597e8957830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 1 -- 172.21.15.112:0/4202646751 shutdown_connections 2024-09-21T03:41:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 1 -- 172.21.15.112:0/4202646751 wait complete. 2024-09-21T03:41:26.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.186+0000 7f2b18c4e740 20 client.17565 trim_cache size 0 max 16384 2024-09-21T03:41:26.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.199+0000 7f2b18c4e740 1 Processor -- start 2024-09-21T03:41:26.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.199+0000 7f2b18c4e740 1 -- start start 2024-09-21T03:41:26.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.199+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e856b9f0 0x5597e8960300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.199+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e896c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.199+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e858c390 0x5597e895d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.199+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1b1ea0 con 0x5597e85841b0 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.199+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1af770 con 0x5597e856b9f0 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.199+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1ad5e0 con 0x5597e858c390 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b0bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e896c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b0bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e896c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51328/0 (socket says 172.21.15.112:51328) 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b0bfff640 1 -- 172.21.15.112:0/154677517 learned_addr learned my addr 172.21.15.112:0/154677517 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b03fff640 1 --2- 172.21.15.112:0/154677517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e858c390 0x5597e895d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b03fff640 1 --2- 172.21.15.112:0/154677517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e856b9f0 0x5597e8960300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b0bfff640 1 -- 172.21.15.112:0/154677517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e858c390 msgr2=0x5597e895d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b0bfff640 1 --2- 172.21.15.112:0/154677517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e858c390 0x5597e895d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b0bfff640 1 -- 172.21.15.112:0/154677517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e856b9f0 msgr2=0x5597e8960300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b0bfff640 1 --2- 172.21.15.112:0/154677517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e856b9f0 0x5597e8960300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b0bfff640 1 -- 172.21.15.112:0/154677517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e895ed30 con 0x5597e85841b0 2024-09-21T03:41:26.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b03fff640 1 --2- 172.21.15.112:0/154677517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e858c390 0x5597e895d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:26.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b0bfff640 1 --2- 172.21.15.112:0/154677517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e896c0d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009900 tx=0x7f2afc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e772b7689b23f5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b01ffb640 1 -- 172.21.15.112:0/154677517 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc035040 con 0x5597e85841b0 2024-09-21T03:41:26.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b01ffb640 1 -- 172.21.15.112:0/154677517 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2afc0054f0 con 0x5597e85841b0 2024-09-21T03:41:26.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.200+0000 7f2b01ffb640 1 -- 172.21.15.112:0/154677517 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc004e50 con 0x5597e85841b0 2024-09-21T03:41:26.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.201+0000 7f2b18c4e740 1 -- 172.21.15.112:0/154677517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 msgr2=0x5597e896c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.201+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/154677517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e896c0d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009900 tx=0x7f2afc000bc0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.201+0000 7f2b18c4e740 1 -- 172.21.15.112:0/154677517 shutdown_connections 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.201+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/154677517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e858c390 0x5597e895d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.201+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/154677517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e85841b0 0x5597e896c0d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.201+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/154677517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e856b9f0 0x5597e8960300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.201+0000 7f2b18c4e740 1 -- 172.21.15.112:0/154677517 >> 172.21.15.112:0/154677517 conn(0x5597e85a1460 msgr2=0x5597e895fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.201+0000 7f2b18c4e740 1 -- 172.21.15.112:0/154677517 shutdown_connections 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.201+0000 7f2b18c4e740 1 -- 172.21.15.112:0/154677517 wait complete. 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.202+0000 7f2b18c4e740 1 -- start start 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.202+0000 7f2b18c4e740 1 Processor -- start 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.202+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e856b9f0 0x5597e8968750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.202+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85841b0 0x5597e8963c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.202+0000 7f2b18c4e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e858c390 0x5597e8964140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.202+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1adc30 con 0x5597e858c390 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.202+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1af0a0 con 0x5597e856b9f0 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.202+0000 7f2b18c4e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c1ae300 con 0x5597e85841b0 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.202+0000 7f2b18c4e740 10 client.0 _ll_register_callbacks cb 0x7f2af4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b0bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85841b0 0x5597e8963c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e858c390 0x5597e8964140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b0bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85841b0 0x5597e8963c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:54332/0 (socket says 172.21.15.112:54332) 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b0bfff640 1 -- 172.21.15.112:0/1053637554 learned_addr learned my addr 172.21.15.112:0/1053637554 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b03fff640 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e856b9f0 0x5597e8968750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b0bfff640 1 -- 172.21.15.112:0/1053637554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e856b9f0 msgr2=0x5597e8968750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b0bfff640 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e856b9f0 0x5597e8968750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b0bfff640 1 -- 172.21.15.112:0/1053637554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e858c390 msgr2=0x5597e8964140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:26.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b0bfff640 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e858c390 0x5597e8964140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b0bfff640 1 -- 172.21.15.112:0/1053637554 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e8964700 con 0x5597e85841b0 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b03fff640 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e858c390 0x5597e8964140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b0bfff640 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85841b0 0x5597e8963c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2afc02ab90 tx=0x7f2afc02abc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a9452bc35439fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2aeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/1053637554 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc054070 con 0x5597e85841b0 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/1053637554 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f2afc004750 con 0x5597e85841b0 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.203+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1053637554 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2afc0054f0 con 0x5597e85841b0 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/1053637554 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2afc035040 con 0x5597e85841b0 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2b18c4e740 10 client.27130 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1053637554 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5597e89dc540 con 0x5597e85841b0 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/1053637554 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2afc0054f0 con 0x5597e85841b0 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/1053637554 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2afc032520 con 0x5597e85841b0 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2b18c4e740 10 client.27130 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2b18c4e740 10 client.27130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:41:26.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2b18c4e740 20 client.27130 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597e85ad070 0x5597e89dec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2b18c4e740 4 client.27130 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1053637554 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597e88c88a0 con 0x5597e85ad070 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.204+0000 7f2b03fff640 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597e85ad070 0x5597e89dec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2aeb7fe640 10 client.27130 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b03fff640 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597e85ad070 0x5597e89dec40 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af937304347d0c71 server_cookie=76d9a4503a0072ed in_seq=0 out_seq=0 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2aeb7fe640 1 -- 172.21.15.112:0/1053637554 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597e88c88a0 con 0x5597e85ad070 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2aeb7fe640 10 client.27130 handle_command_reply: tid=0 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 1 client.27130 shutdown 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1053637554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85841b0 msgr2=0x5597e8963c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85841b0 0x5597e8963c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2afc02ab90 tx=0x7f2afc02abc0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1053637554 shutdown_connections 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5597e85ad070 0x5597e89dec40 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5597e858c390 0x5597e8964140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597e85841b0 0x5597e8963c00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2aeb7fe640 0 client.27130 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:26.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 1 --2- 172.21.15.112:0/1053637554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5597e856b9f0 0x5597e8968750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1053637554 >> 172.21.15.112:0/1053637554 conn(0x5597e85a1460 msgr2=0x5597e89555f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:26.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1053637554 shutdown_connections 2024-09-21T03:41:26.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 1 -- 172.21.15.112:0/1053637554 wait complete. 2024-09-21T03:41:26.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.205+0000 7f2b18c4e740 20 client.27130 trim_cache size 0 max 16384 2024-09-21T03:41:26.213 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:41:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 -- 172.21.15.112:0/575903179 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2af80b6680 msgr2=0x7f2af80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2af80b6680 0x7f2af80b8b90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2b040aadb0 tx=0x7f2b0406c920 comp rx=0 tx=0).stop 2024-09-21T03:41:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 -- 172.21.15.112:0/575903179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c108d00 msgr2=0x7f2b0c19fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c108d00 0x7f2b0c19fc60 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400eb10 tx=0x7f2b0400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 -- 172.21.15.112:0/575903179 shutdown_connections 2024-09-21T03:41:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2af80b6680 0x7f2af80b8b90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c108d00 0x7f2b0c19fc60 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b0c1083b0 0x7f2b0c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 --2- 172.21.15.112:0/575903179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b0c102a70 0x7f2b0c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 -- 172.21.15.112:0/575903179 >> 172.21.15.112:0/575903179 conn(0x7f2b0c079730 msgr2=0x7f2b0c1014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:26.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 -- 172.21.15.112:0/575903179 shutdown_connections 2024-09-21T03:41:26.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:26.206+0000 7f2b126da640 1 -- 172.21.15.112:0/575903179 wait complete. 2024-09-21T03:41:26.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1317906 10.24 MB/sec execute 566 sec latency 63.352 ms 2024-09-21T03:41:26.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:26 smithi203 ceph-mon[34723]: pgmap v704: 97 pgs: 97 active+clean; 929 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 217 op/s 2024-09-21T03:41:26.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:26 smithi187 ceph-mon[34603]: pgmap v704: 97 pgs: 97 active+clean; 929 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 217 op/s 2024-09-21T03:41:26.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:26 smithi112 ceph-mon[31442]: pgmap v704: 97 pgs: 97 active+clean; 929 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 217 op/s 2024-09-21T03:41:27.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1319196 10.23 MB/sec execute 567 sec latency 49.747 ms 2024-09-21T03:41:28.059 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:41:28.059 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:41:28.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1321392 10.24 MB/sec execute 568 sec latency 42.482 ms 2024-09-21T03:41:28.323 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:28.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:28 smithi203 ceph-mon[34723]: pgmap v705: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 292 op/s 2024-09-21T03:41:28.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:28 smithi187 ceph-mon[34603]: pgmap v705: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 292 op/s 2024-09-21T03:41:28.444 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:28 smithi112 ceph-mon[31442]: pgmap v705: 97 pgs: 97 active+clean; 938 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 292 op/s 2024-09-21T03:41:28.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.837+0000 7f6f0ff48640 1 -- 172.21.15.112:0/4080936880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08109560 msgr2=0x7f6f08109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.837+0000 7f6f0ff48640 1 --2- 172.21.15.112:0/4080936880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08109560 0x7f6f08109970 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009900 tx=0x7f6efc02a410 comp rx=0 tx=0).stop 2024-09-21T03:41:28.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.837+0000 7f6f0ff48640 1 -- 172.21.15.112:0/4080936880 shutdown_connections 2024-09-21T03:41:28.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.837+0000 7f6f0ff48640 1 --2- 172.21.15.112:0/4080936880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f08109eb0 0x7f6f081106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.837+0000 7f6f0ff48640 1 --2- 172.21.15.112:0/4080936880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08109560 0x7f6f08109970 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.837+0000 7f6f0ff48640 1 --2- 172.21.15.112:0/4080936880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f08102080 0x7f6f08102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.837+0000 7f6f0ff48640 1 -- 172.21.15.112:0/4080936880 >> 172.21.15.112:0/4080936880 conn(0x7f6f08079ec0 msgr2=0x7f6f0807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.837+0000 7f6f0ff48640 1 -- 172.21.15.112:0/4080936880 shutdown_connections 2024-09-21T03:41:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.837+0000 7f6f0ff48640 1 -- 172.21.15.112:0/4080936880 wait complete. 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0ff48640 1 Processor -- start 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0ff48640 1 -- start start 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0ff48640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f08102080 0x7f6f0819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0ff48640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f08109560 0x7f6f0819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0ff48640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08109eb0 0x7f6f0819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0ff48640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08113b70 con 0x7f6f08109eb0 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0ff48640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081139f0 con 0x7f6f08102080 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0ff48640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08113cf0 con 0x7f6f08109560 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0e4e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08109eb0 0x7f6f0819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0e4e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08109eb0 0x7f6f0819fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51356/0 (socket says 172.21.15.112:51356) 2024-09-21T03:41:28.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0e4e5640 1 -- 172.21.15.112:0/1676661596 learned_addr learned my addr 172.21.15.112:0/1676661596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f08102080 0x7f6f0819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f08109560 0x7f6f0819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0e4e5640 1 -- 172.21.15.112:0/1676661596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f08109560 msgr2=0x7f6f0819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0e4e5640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f08109560 0x7f6f0819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0e4e5640 1 -- 172.21.15.112:0/1676661596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f08102080 msgr2=0x7f6f0819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0e4e5640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f08102080 0x7f6f0819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0e4e5640 1 -- 172.21.15.112:0/1676661596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f081a04d0 con 0x7f6f08109eb0 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.838+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f08102080 0x7f6f0819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.839+0000 7f6f0e4e5640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08109eb0 0x7f6f0819fda0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000e970 tx=0x7f6f0000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.839+0000 7f6f06ffd640 1 -- 172.21.15.112:0/1676661596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000ccc0 con 0x7f6f08109eb0 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.839+0000 7f6f0ff48640 1 -- 172.21.15.112:0/1676661596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f081a07c0 con 0x7f6f08109eb0 2024-09-21T03:41:28.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.839+0000 7f6f06ffd640 1 -- 172.21.15.112:0/1676661596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0000ce60 con 0x7f6f08109eb0 2024-09-21T03:41:28.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.839+0000 7f6f06ffd640 1 -- 172.21.15.112:0/1676661596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00010430 con 0x7f6f08109eb0 2024-09-21T03:41:28.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.839+0000 7f6f0ff48640 1 -- 172.21.15.112:0/1676661596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f081a6f20 con 0x7f6f08109eb0 2024-09-21T03:41:28.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.842+0000 7f6f164bc740 1 Processor -- start 2024-09-21T03:41:28.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.842+0000 7f6f06ffd640 1 -- 172.21.15.112:0/1676661596 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f6f00004ff0 con 0x7f6f08109eb0 2024-09-21T03:41:28.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f164bc740 1 -- start start 2024-09-21T03:41:28.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f164bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1d38430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f164bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 0x556cb1d38970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f164bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d38eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f164bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556cb1caee60 con 0x556cb1968490 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f164bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556cb1c9e290 con 0x556cb1991350 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f164bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556cb1ca20a0 con 0x556cb1970670 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f0dce4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 0x556cb1d38970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f0d4e3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1d38430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f0dce4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 0x556cb1d38970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51358/0 (socket says 172.21.15.112:51358) 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f0dce4640 1 -- 172.21.15.112:0/1357396477 learned_addr learned my addr 172.21.15.112:0/1357396477 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.843+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/1357396477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d38eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6f0d4e3640 1 -- 172.21.15.112:0/1357396477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 msgr2=0x556cb1d38eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/1357396477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d38eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6f0d4e3640 1 -- 172.21.15.112:0/1357396477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 msgr2=0x556cb1d38970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/1357396477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 0x556cb1d38970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6f0d4e3640 1 -- 172.21.15.112:0/1357396477 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb19fc930 con 0x556cb1991350 2024-09-21T03:41:28.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6f06ffd640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6ef00b6680 0x7f6ef00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/1357396477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1d38430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6efc02a930 tx=0x7f6efc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ed05752a4ea2f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6ef00b6680 0x7f6ef00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6f06ffd640 1 -- 172.21.15.112:0/1676661596 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f6f000e7ae0 con 0x7f6f08109eb0 2024-09-21T03:41:28.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6eeffff640 1 -- 172.21.15.112:0/1357396477 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc035030 con 0x556cb1991350 2024-09-21T03:41:28.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6eeffff640 1 -- 172.21.15.112:0/1357396477 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6efc0318a0 con 0x556cb1991350 2024-09-21T03:41:28.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.844+0000 7f6eeffff640 1 -- 172.21.15.112:0/1357396477 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc005280 con 0x556cb1991350 2024-09-21T03:41:28.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.845+0000 7f6f164bc740 1 -- 172.21.15.112:0/1357396477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 msgr2=0x556cb1d38430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.845+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6ef00b6680 0x7f6ef00b8b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x556cb1d370b0 tx=0x7f6ef8009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.845+0000 7f6f164bc740 1 --2- 172.21.15.112:0/1357396477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1d38430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6efc02a930 tx=0x7f6efc000bc0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.845+0000 7f6f164bc740 1 -- 172.21.15.112:0/1357396477 shutdown_connections 2024-09-21T03:41:28.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.845+0000 7f6f164bc740 1 --2- 172.21.15.112:0/1357396477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d38eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.845+0000 7f6f164bc740 1 --2- 172.21.15.112:0/1357396477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 0x556cb1d38970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.845+0000 7f6f164bc740 1 --2- 172.21.15.112:0/1357396477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1d38430 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.845+0000 7f6f164bc740 1 -- 172.21.15.112:0/1357396477 >> 172.21.15.112:0/1357396477 conn(0x556cb1985740 msgr2=0x556cb1d36320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.845+0000 7f6f164bc740 1 -- 172.21.15.112:0/1357396477 shutdown_connections 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.846+0000 7f6f164bc740 1 -- 172.21.15.112:0/1357396477 wait complete. 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.846+0000 7f6f164bc740 1 -- start start 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.846+0000 7f6f164bc740 1 Processor -- start 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f164bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1968490 0x556cb1d48410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f164bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d48950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f164bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1991350 0x556cb1d54180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f164bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556cb1aff890 con 0x556cb1991350 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f164bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556cb1c38d40 con 0x556cb1968490 2024-09-21T03:41:28.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f164bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556cb1d4ae50 con 0x556cb1970670 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0d4e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d48950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f164bc740 10 client.0 _ll_register_callbacks cb 0x556cb1c9cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0e4e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1991350 0x556cb1d54180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0d4e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d48950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:54384/0 (socket says 172.21.15.112:54384) 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0d4e3640 1 -- 172.21.15.112:0/3423910756 learned_addr learned my addr 172.21.15.112:0/3423910756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1968490 0x556cb1d48410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0d4e3640 1 -- 172.21.15.112:0/3423910756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1968490 msgr2=0x556cb1d48410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1968490 0x556cb1d48410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0d4e3640 1 -- 172.21.15.112:0/3423910756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1991350 msgr2=0x556cb1d54180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1991350 0x556cb1d54180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0d4e3640 1 -- 172.21.15.112:0/3423910756 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb1d54740 con 0x556cb1970670 2024-09-21T03:41:28.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0e4e5640 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1991350 0x556cb1d54180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:28.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d48950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6efc031df0 tx=0x7f6efc031e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e0be76506da5523 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.847+0000 7f6eed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:41:28.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6eed7fa640 1 -- 172.21.15.112:0/3423910756 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc035070 con 0x556cb1970670 2024-09-21T03:41:28.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6f164bc740 1 -- 172.21.15.112:0/3423910756 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6efc0318a0 con 0x556cb1970670 2024-09-21T03:41:28.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6eed7fa640 1 -- 172.21.15.112:0/3423910756 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f6efc005230 con 0x556cb1970670 2024-09-21T03:41:28.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6eed7fa640 1 -- 172.21.15.112:0/3423910756 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6efc005430 con 0x556cb1970670 2024-09-21T03:41:28.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6f164bc740 10 client.27142 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:28.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6f164bc740 1 -- 172.21.15.112:0/3423910756 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556cb1d57070 con 0x556cb1970670 2024-09-21T03:41:28.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6eed7fa640 1 -- 172.21.15.112:0/3423910756 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6efc0318a0 con 0x556cb1970670 2024-09-21T03:41:28.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6eed7fa640 1 -- 172.21.15.112:0/3423910756 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6efc03d680 con 0x556cb1970670 2024-09-21T03:41:28.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6f164bc740 10 client.27142 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6f164bc740 10 client.27142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6f164bc740 20 client.27142 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6f164bc740 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556cb194fcd0 0x556cb1d5a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6f164bc740 4 client.27142 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.848+0000 7f6f164bc740 1 -- 172.21.15.112:0/3423910756 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cb1cb0db0 con 0x556cb194fcd0 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.849+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556cb194fcd0 0x556cb1d5a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.849+0000 7f6eed7fa640 10 client.27142 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.849+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556cb194fcd0 0x556cb1d5a3d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f3c9691a890ae4f server_cookie=6a06d53da519d147 in_seq=0 out_seq=0 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6eed7fa640 1 -- 172.21.15.112:0/3423910756 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556cb1cb0db0 con 0x556cb194fcd0 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6eed7fa640 10 client.27142 handle_command_reply: tid=0 2024-09-21T03:41:28.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6f164bc740 1 client.27142 shutdown 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6f164bc740 1 -- 172.21.15.112:0/3423910756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 msgr2=0x556cb1d48950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6f164bc740 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d48950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6efc031df0 tx=0x7f6efc031e20 comp rx=0 tx=0).stop 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6f164bc740 1 -- 172.21.15.112:0/3423910756 shutdown_connections 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6f164bc740 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556cb194fcd0 0x556cb1d5a3d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6f164bc740 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1991350 0x556cb1d54180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6f164bc740 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d48950 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6f164bc740 1 --2- 172.21.15.112:0/3423910756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1968490 0x556cb1d48410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6eed7fa640 0 client.27142 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.850+0000 7f6f164bc740 1 -- 172.21.15.112:0/3423910756 >> 172.21.15.112:0/3423910756 conn(0x556cb1985740 msgr2=0x556cb1d34850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.851+0000 7f6f164bc740 1 -- 172.21.15.112:0/3423910756 shutdown_connections 2024-09-21T03:41:28.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.851+0000 7f6f164bc740 1 -- 172.21.15.112:0/3423910756 wait complete. 2024-09-21T03:41:28.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.851+0000 7f6f164bc740 20 client.27142 trim_cache size 0 max 16384 2024-09-21T03:41:28.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f164bc740 1 Processor -- start 2024-09-21T03:41:28.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f164bc740 1 -- start start 2024-09-21T03:41:28.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f164bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1968490 0x556cb1d439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f164bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1970670 0x556cb1d47970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f164bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1d49e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f164bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081afd00 con 0x556cb1970670 2024-09-21T03:41:28.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f164bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081ad5d0 con 0x556cb1991350 2024-09-21T03:41:28.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f164bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081ab440 con 0x556cb1968490 2024-09-21T03:41:28.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f0d4e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1970670 0x556cb1d47970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f0d4e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1970670 0x556cb1d47970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51382/0 (socket says 172.21.15.112:51382) 2024-09-21T03:41:28.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f0d4e3640 1 -- 172.21.15.112:0/4100785854 learned_addr learned my addr 172.21.15.112:0/4100785854 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/4100785854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1968490 0x556cb1d439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f0e4e5640 1 --2- 172.21.15.112:0/4100785854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1d49e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f0dce4640 1 -- 172.21.15.112:0/4100785854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 msgr2=0x556cb1d49e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/4100785854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1d49e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f0dce4640 1 -- 172.21.15.112:0/4100785854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1970670 msgr2=0x556cb1d47970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/4100785854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1970670 0x556cb1d47970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f0dce4640 1 -- 172.21.15.112:0/4100785854 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb1d4abf0 con 0x556cb1968490 2024-09-21T03:41:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.873+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/4100785854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1970670 0x556cb1d47970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:28.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.874+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/4100785854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1968490 0x556cb1d439c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x556cb1caeef0 tx=0x7f6ef801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=478647186333668c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.874+0000 7f6eef7fe640 1 -- 172.21.15.112:0/4100785854 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef802b070 con 0x556cb1968490 2024-09-21T03:41:28.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.874+0000 7f6eef7fe640 1 -- 172.21.15.112:0/4100785854 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef801ca40 con 0x556cb1968490 2024-09-21T03:41:28.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.874+0000 7f6eef7fe640 1 -- 172.21.15.112:0/4100785854 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8029bb0 con 0x556cb1968490 2024-09-21T03:41:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.875+0000 7f6f164bc740 1 -- 172.21.15.112:0/4100785854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1968490 msgr2=0x556cb1d439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.875+0000 7f6f164bc740 1 --2- 172.21.15.112:0/4100785854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1968490 0x556cb1d439c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x556cb1caeef0 tx=0x7f6ef801a000 comp rx=0 tx=0).stop 2024-09-21T03:41:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.875+0000 7f6f164bc740 1 -- 172.21.15.112:0/4100785854 shutdown_connections 2024-09-21T03:41:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.875+0000 7f6f164bc740 1 --2- 172.21.15.112:0/4100785854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1d49e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.875+0000 7f6f164bc740 1 --2- 172.21.15.112:0/4100785854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1970670 0x556cb1d47970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.875+0000 7f6f164bc740 1 --2- 172.21.15.112:0/4100785854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1968490 0x556cb1d439c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.875+0000 7f6f164bc740 1 -- 172.21.15.112:0/4100785854 >> 172.21.15.112:0/4100785854 conn(0x556cb1985740 msgr2=0x556cb1d49730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.876+0000 7f6f164bc740 1 -- 172.21.15.112:0/4100785854 shutdown_connections 2024-09-21T03:41:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.876+0000 7f6f164bc740 1 -- 172.21.15.112:0/4100785854 wait complete. 2024-09-21T03:41:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.876+0000 7f6f164bc740 1 -- start start 2024-09-21T03:41:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.876+0000 7f6f164bc740 1 Processor -- start 2024-09-21T03:41:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.877+0000 7f6f164bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 0x556cb1d5bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.877+0000 7f6f164bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d5c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.877+0000 7f6f0dce4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 0x556cb1d5bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.877+0000 7f6f0dce4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 0x556cb1d5bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51398/0 (socket says 172.21.15.112:51398) 2024-09-21T03:41:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.877+0000 7f6f0dce4640 1 -- 172.21.15.112:0/949366454 learned_addr learned my addr 172.21.15.112:0/949366454 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.877+0000 7f6f164bc740 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1dc3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.877+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d5c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.877+0000 7f6f164bc740 1 -- 172.21.15.112:0/949366454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081aba90 con 0x556cb1968490 2024-09-21T03:41:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.877+0000 7f6f0e4e5640 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1dc3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.877+0000 7f6f164bc740 1 -- 172.21.15.112:0/949366454 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081acf00 con 0x556cb1991350 2024-09-21T03:41:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.877+0000 7f6f164bc740 1 -- 172.21.15.112:0/949366454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081ac160 con 0x556cb1970670 2024-09-21T03:41:28.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.878+0000 7f6f164bc740 10 client.0 _ll_register_callbacks cb 0x7f6ef8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:28.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.878+0000 7f6f0d4e3640 1 -- 172.21.15.112:0/949366454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 msgr2=0x556cb1dc3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.878+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1dc3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.878+0000 7f6f0d4e3640 1 -- 172.21.15.112:0/949366454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 msgr2=0x556cb1d5bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.878+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 0x556cb1d5bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.878+0000 7f6f0d4e3640 1 -- 172.21.15.112:0/949366454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb1dc4190 con 0x556cb1970670 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.878+0000 7f6f0e4e5640 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1dc3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.878+0000 7f6f164bc740 1 -- 172.21.15.112:0/949366454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ef801ca40 con 0x556cb1970670 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.878+0000 7f6f0d4e3640 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d5c1d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6efc03daf0 tx=0x7f6efc005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0912aa6f2ccdda4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.878+0000 7f6eed7fa640 10 client.27148 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.879+0000 7f6eed7fa640 1 -- 172.21.15.112:0/949366454 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc035040 con 0x556cb1970670 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.879+0000 7f6eed7fa640 1 -- 172.21.15.112:0/949366454 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f6efc03d650 con 0x556cb1970670 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.879+0000 7f6eed7fa640 1 -- 172.21.15.112:0/949366454 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6efc004020 con 0x556cb1970670 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.879+0000 7f6f164bc740 10 client.27148 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:28.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.879+0000 7f6f164bc740 1 -- 172.21.15.112:0/949366454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({fsmap=30}) -- 0x556cb1dc5370 con 0x556cb1970670 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.879+0000 7f6eed7fa640 1 -- 172.21.15.112:0/949366454 <== mon.1 v2:172.21.15.203:3300/0 4 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6efc0041c0 con 0x556cb1970670 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.879+0000 7f6f164bc740 10 client.27148 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.879+0000 7f6f164bc740 10 client.27148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.879+0000 7f6f164bc740 20 client.27148 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.879+0000 7f6f164bc740 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556cb1925080 0x556cb1dc7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.880+0000 7f6f164bc740 4 client.27148 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.880+0000 7f6f164bc740 1 -- 172.21.15.112:0/949366454 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556cb1cac860 con 0x556cb1925080 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.880+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556cb1925080 0x556cb1dc7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.880+0000 7f6eed7fa640 10 client.27148 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.880+0000 7f6f0dce4640 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556cb1925080 0x556cb1dc7800 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9294eb55937b291d server_cookie=f7ef081afc41f942 in_seq=0 out_seq=0 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6eed7fa640 1 -- 172.21.15.112:0/949366454 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556cb1cac860 con 0x556cb1925080 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6eed7fa640 10 client.27148 handle_command_reply: tid=0 2024-09-21T03:41:28.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 1 client.27148 shutdown 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 1 -- 172.21.15.112:0/949366454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 msgr2=0x556cb1d5c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d5c1d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6efc03daf0 tx=0x7f6efc005eb0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 1 -- 172.21.15.112:0/949366454 shutdown_connections 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556cb1925080 0x556cb1dc7800 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cb1991350 0x556cb1dc3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556cb1970670 0x556cb1d5c1d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6eed7fa640 0 client.27148 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 1 --2- 172.21.15.112:0/949366454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556cb1968490 0x556cb1d5bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 1 -- 172.21.15.112:0/949366454 >> 172.21.15.112:0/949366454 conn(0x556cb1985740 msgr2=0x556cb1d49680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 1 -- 172.21.15.112:0/949366454 shutdown_connections 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 1 -- 172.21.15.112:0/949366454 wait complete. 2024-09-21T03:41:28.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.882+0000 7f6f164bc740 20 client.27148 trim_cache size 0 max 16384 2024-09-21T03:41:28.889 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:41:28.889 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:41:28.889 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "3d0e4442-3ee3-44bd-99e2-f0a81392c8c4", 2024-09-21T03:41:28.889 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:41:28.889 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:41:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.883+0000 7f6f0ff48640 1 -- 172.21.15.112:0/1676661596 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6ef00b6680 msgr2=0x7f6ef00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.883+0000 7f6f0ff48640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6ef00b6680 0x7f6ef00b8b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x556cb1d370b0 tx=0x7f6ef8009500 comp rx=0 tx=0).stop 2024-09-21T03:41:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.883+0000 7f6f0ff48640 1 -- 172.21.15.112:0/1676661596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08109eb0 msgr2=0x7f6f0819fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.883+0000 7f6f0ff48640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08109eb0 0x7f6f0819fda0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000e970 tx=0x7f6f0000ed70 comp rx=0 tx=0).stop 2024-09-21T03:41:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.883+0000 7f6f0ff48640 1 -- 172.21.15.112:0/1676661596 shutdown_connections 2024-09-21T03:41:28.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.883+0000 7f6f0ff48640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6ef00b6680 0x7f6ef00b8b90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.883+0000 7f6f0ff48640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08109eb0 0x7f6f0819fda0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.883+0000 7f6f0ff48640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f08109560 0x7f6f0819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.883+0000 7f6f0ff48640 1 --2- 172.21.15.112:0/1676661596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f08102080 0x7f6f0819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.883+0000 7f6f0ff48640 1 -- 172.21.15.112:0/1676661596 >> 172.21.15.112:0/1676661596 conn(0x7f6f08079ec0 msgr2=0x7f6f081055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.883+0000 7f6f0ff48640 1 -- 172.21.15.112:0/1676661596 shutdown_connections 2024-09-21T03:41:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:28.884+0000 7f6f0ff48640 1 -- 172.21.15.112:0/1676661596 wait complete. 2024-09-21T03:41:29.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1322536 10.23 MB/sec execute 569 sec latency 176.757 ms 2024-09-21T03:41:29.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:29 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:41:29.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:29 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:29.476 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:29 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:41:29.476 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:29 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:29 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:41:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:29 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:29.803 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:41:30.067 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:41:30.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1324897 10.24 MB/sec execute 570 sec latency 20.485 ms 2024-09-21T03:41:30.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:30 smithi187 ceph-mon[34603]: pgmap v706: 97 pgs: 97 active+clean; 940 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-09-21T03:41:30.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:30 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:41:30.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:30 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:30.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:30 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:41:30.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:30 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:41:30.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:30 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:30.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:30 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:30.488 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:30 smithi112 ceph-mon[31442]: pgmap v706: 97 pgs: 97 active+clean; 940 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-09-21T03:41:30.488 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:30 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:41:30.488 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:30 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:30.488 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:30 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:41:30.488 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:30 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:41:30.488 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:30 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:30.488 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:30 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:30.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.583+0000 7f4f4544c640 1 -- 172.21.15.112:0/3363903594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f401083e0 msgr2=0x7f4f401087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.583+0000 7f4f4544c640 1 --2- 172.21.15.112:0/3363903594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f401083e0 0x7f4f401087f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009900 tx=0x7f4f2802a410 comp rx=0 tx=0).stop 2024-09-21T03:41:30.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.583+0000 7f4f4544c640 1 -- 172.21.15.112:0/3363903594 shutdown_connections 2024-09-21T03:41:30.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.583+0000 7f4f4544c640 1 --2- 172.21.15.112:0/3363903594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f40108d30 0x7f4f4010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.583+0000 7f4f4544c640 1 --2- 172.21.15.112:0/3363903594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f401083e0 0x7f4f401087f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.583+0000 7f4f4544c640 1 --2- 172.21.15.112:0/3363903594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f40102aa0 0x7f4f40102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.583+0000 7f4f4544c640 1 -- 172.21.15.112:0/3363903594 >> 172.21.15.112:0/3363903594 conn(0x7f4f400797e0 msgr2=0x7f4f40079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.583+0000 7f4f4544c640 1 -- 172.21.15.112:0/3363903594 shutdown_connections 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.583+0000 7f4f4544c640 1 -- 172.21.15.112:0/3363903594 wait complete. 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f4544c640 1 Processor -- start 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f4544c640 1 -- start start 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f4544c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f40102aa0 0x7f4f4019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f4544c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f401083e0 0x7f4f4019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f4544c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f40108d30 0x7f4f4019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f4544c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f401129a0 con 0x7f4f401083e0 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f4544c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40112820 con 0x7f4f40108d30 2024-09-21T03:41:30.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f4544c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40112b20 con 0x7f4f40102aa0 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f401083e0 0x7f4f4019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f40102aa0 0x7f4f4019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f401083e0 0x7f4f4019b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51426/0 (socket says 172.21.15.112:51426) 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f37fff640 1 -- 172.21.15.112:0/3529003715 learned_addr learned my addr 172.21.15.112:0/3529003715 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.584+0000 7f4f3f7fe640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f40108d30 0x7f4f4019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f3f7fe640 1 -- 172.21.15.112:0/3529003715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f40102aa0 msgr2=0x7f4f4019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f3f7fe640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f40102aa0 0x7f4f4019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f3f7fe640 1 -- 172.21.15.112:0/3529003715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f401083e0 msgr2=0x7f4f4019b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f3f7fe640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f401083e0 0x7f4f4019b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f3f7fe640 1 -- 172.21.15.112:0/3529003715 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f401a65c0 con 0x7f4f40108d30 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f3effd640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f40102aa0 0x7f4f4019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f3f7fe640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f40108d30 0x7f4f4019bdb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009fd0 tx=0x7f4f3800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:30.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f3cff9640 1 -- 172.21.15.112:0/3529003715 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800ce20 con 0x7f4f40108d30 2024-09-21T03:41:30.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f4544c640 1 -- 172.21.15.112:0/3529003715 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f401a68b0 con 0x7f4f40108d30 2024-09-21T03:41:30.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f3cff9640 1 -- 172.21.15.112:0/3529003715 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f38005020 con 0x7f4f40108d30 2024-09-21T03:41:30.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f3cff9640 1 -- 172.21.15.112:0/3529003715 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38010690 con 0x7f4f40108d30 2024-09-21T03:41:30.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.585+0000 7f4f4544c640 1 -- 172.21.15.112:0/3529003715 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f401a6df0 con 0x7f4f40108d30 2024-09-21T03:41:30.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.589+0000 7f4f4b9c0740 1 Processor -- start 2024-09-21T03:41:30.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.589+0000 7f4f3cff9640 1 -- 172.21.15.112:0/3529003715 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f4f380054d0 con 0x7f4f40108d30 2024-09-21T03:41:30.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.589+0000 7f4f4b9c0740 1 -- start start 2024-09-21T03:41:30.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141a2070 0x556914549650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141791b0 0x556914549b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556914181390 0x55691454a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5569144bfe50 con 0x5569141a2070 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5569144af280 con 0x5569141791b0 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5569144b3090 con 0x556914181390 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141a2070 0x556914549650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141a2070 0x556914549650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51440/0 (socket says 172.21.15.112:51440) 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141791b0 0x556914549b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f37fff640 1 -- 172.21.15.112:0/227077308 learned_addr learned my addr 172.21.15.112:0/227077308 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f37fff640 1 --2- 172.21.15.112:0/227077308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556914181390 0x55691454a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f3effd640 1 -- 172.21.15.112:0/227077308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556914181390 msgr2=0x55691454a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f3effd640 1 --2- 172.21.15.112:0/227077308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556914181390 0x55691454a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f3effd640 1 -- 172.21.15.112:0/227077308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141a2070 msgr2=0x556914549650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f3effd640 1 --2- 172.21.15.112:0/227077308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141a2070 0x556914549650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f3effd640 1 -- 172.21.15.112:0/227077308 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55691420d650 con 0x5569141791b0 2024-09-21T03:41:30.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f3cff9640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f2c0b66b0 0x7f4f2c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.590+0000 7f4f3effd640 1 --2- 172.21.15.112:0/227077308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141791b0 0x556914549b90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009fa0 tx=0x7f4f300118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b84d135ab74e28e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.591+0000 7f4f37fff640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f2c0b66b0 0x7f4f2c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.591+0000 7f4f357fa640 1 -- 172.21.15.112:0/227077308 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30004030 con 0x5569141791b0 2024-09-21T03:41:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.591+0000 7f4f3cff9640 1 -- 172.21.15.112:0/3529003715 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f4f380e7d30 con 0x7f4f40108d30 2024-09-21T03:41:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.591+0000 7f4f357fa640 1 -- 172.21.15.112:0/227077308 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f300054d0 con 0x5569141791b0 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.591+0000 7f4f357fa640 1 -- 172.21.15.112:0/227077308 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30004c20 con 0x5569141791b0 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/227077308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141791b0 msgr2=0x556914549b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/227077308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141791b0 0x556914549b90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009fa0 tx=0x7f4f300118d0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f37fff640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f2c0b66b0 0x7f4f2c0b8bc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4f280096c0 tx=0x7f4f28041040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/227077308 shutdown_connections 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/227077308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556914181390 0x55691454a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/227077308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141791b0 0x556914549b90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/227077308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141a2070 0x556914549650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/227077308 >> 172.21.15.112:0/227077308 conn(0x556914196460 msgr2=0x556914547540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/227077308 shutdown_connections 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/227077308 wait complete. 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f4b9c0740 1 -- start start 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.592+0000 7f4f4b9c0740 1 Processor -- start 2024-09-21T03:41:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x5569145550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556914181390 0x556914555630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 0x55691455a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556914310700 con 0x5569141791b0 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55691444a110 con 0x5569141a2070 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55691454fb00 con 0x556914181390 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x5569145550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x5569145550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51454/0 (socket says 172.21.15.112:51454) 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f3effd640 1 -- 172.21.15.112:0/3443298803 learned_addr learned my addr 172.21.15.112:0/3443298803 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f37fff640 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556914181390 0x556914555630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f3f7fe640 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 0x55691455a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f4b9c0740 10 client.0 _ll_register_callbacks cb 0x556914463990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f3effd640 1 -- 172.21.15.112:0/3443298803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556914181390 msgr2=0x556914555630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f3effd640 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556914181390 0x556914555630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f3effd640 1 -- 172.21.15.112:0/3443298803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 msgr2=0x55691455a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f3effd640 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 0x55691455a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f3effd640 1 -- 172.21.15.112:0/3443298803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556914477c40 con 0x5569141791b0 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f37fff640 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556914181390 0x556914555630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f3effd640 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x5569145550f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000c00 tx=0x7f4f3000fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f875460fd4dc1af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.593+0000 7f4f267fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f267fc640 1 -- 172.21.15.112:0/3443298803 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30028020 con 0x5569141791b0 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f267fc640 1 -- 172.21.15.112:0/3443298803 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f4f300049c0 con 0x5569141791b0 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/3443298803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f300054d0 con 0x5569141791b0 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f267fc640 1 -- 172.21.15.112:0/3443298803 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f30033050 con 0x5569141791b0 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f4b9c0740 10 client.17601 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/3443298803 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556914563950 con 0x5569141791b0 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f267fc640 1 -- 172.21.15.112:0/3443298803 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f300054d0 con 0x5569141791b0 2024-09-21T03:41:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f267fc640 1 -- 172.21.15.112:0/3443298803 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4f30023470 con 0x5569141791b0 2024-09-21T03:41:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f4b9c0740 10 client.17601 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f4b9c0740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:41:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f4b9c0740 20 client.17601 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5569141609f0 0x556914566dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f4b9c0740 4 client.17601 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:41:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.594+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/3443298803 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569144c1da0 con 0x5569141609f0 2024-09-21T03:41:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.595+0000 7f4f37fff640 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5569141609f0 0x556914566dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.595+0000 7f4f267fc640 10 client.17601 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.595+0000 7f4f37fff640 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5569141609f0 0x556914566dc0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d3b214835345a6 server_cookie=3da6462fe7fb9c69 in_seq=0 out_seq=0 2024-09-21T03:41:30.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f267fc640 1 -- 172.21.15.112:0/3443298803 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5569144c1da0 con 0x5569141609f0 2024-09-21T03:41:30.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f267fc640 10 client.17601 handle_command_reply: tid=0 2024-09-21T03:41:30.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 1 client.17601 shutdown 2024-09-21T03:41:30.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/3443298803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 msgr2=0x5569145550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x5569145550f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000c00 tx=0x7f4f3000fc70 comp rx=0 tx=0).stop 2024-09-21T03:41:30.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/3443298803 shutdown_connections 2024-09-21T03:41:30.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5569141609f0 0x556914566dc0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 0x55691455a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556914181390 0x556914555630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f267fc640 0 client.17601 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:30.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/3443298803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x5569145550f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/3443298803 >> 172.21.15.112:0/3443298803 conn(0x556914196460 msgr2=0x556914545b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/3443298803 shutdown_connections 2024-09-21T03:41:30.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/3443298803 wait complete. 2024-09-21T03:41:30.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.596+0000 7f4f4b9c0740 20 client.17601 trim_cache size 0 max 16384 2024-09-21T03:41:30.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.615+0000 7f4f4b9c0740 1 Processor -- start 2024-09-21T03:41:30.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f4b9c0740 1 -- start start 2024-09-21T03:41:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569141791b0 0x5569145475b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556914181390 0x5569145522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 0x556914552820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f401b8640 con 0x556914181390 2024-09-21T03:41:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f401b5f10 con 0x5569141a2070 2024-09-21T03:41:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f401b3d80 con 0x5569141791b0 2024-09-21T03:41:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f3f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 0x556914552820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569141791b0 0x5569145475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556914181390 0x5569145522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f3f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 0x556914552820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:46918/0 (socket says 172.21.15.112:46918) 2024-09-21T03:41:30.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569141791b0 0x5569145475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:54436/0 (socket says 172.21.15.112:54436) 2024-09-21T03:41:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.616+0000 7f4f3f7fe640 1 -- 172.21.15.112:0/1140886605 learned_addr learned my addr 172.21.15.112:0/1140886605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.617+0000 7f4f3f7fe640 1 -- 172.21.15.112:0/1140886605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569141791b0 msgr2=0x5569145475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.617+0000 7f4f3f7fe640 1 --2- 172.21.15.112:0/1140886605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569141791b0 0x5569145475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.617+0000 7f4f3f7fe640 1 -- 172.21.15.112:0/1140886605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556914181390 msgr2=0x5569145522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.617+0000 7f4f3f7fe640 1 --2- 172.21.15.112:0/1140886605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556914181390 0x5569145522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.617+0000 7f4f3f7fe640 1 -- 172.21.15.112:0/1140886605 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55691454c480 con 0x5569141a2070 2024-09-21T03:41:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.617+0000 7f4f3f7fe640 1 --2- 172.21.15.112:0/1140886605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 0x556914552820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f380ad8b0 tx=0x7f4f380ad3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae0bf67eee9afd6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.617+0000 7f4f34ff9640 1 -- 172.21.15.112:0/1140886605 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380ae2b0 con 0x5569141a2070 2024-09-21T03:41:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.617+0000 7f4f34ff9640 1 -- 172.21.15.112:0/1140886605 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f380b2690 con 0x5569141a2070 2024-09-21T03:41:30.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.617+0000 7f4f34ff9640 1 -- 172.21.15.112:0/1140886605 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380cfbc0 con 0x5569141a2070 2024-09-21T03:41:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.618+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/1140886605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 msgr2=0x556914552820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.618+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/1140886605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 0x556914552820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f380ad8b0 tx=0x7f4f380ad3e0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.618+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/1140886605 shutdown_connections 2024-09-21T03:41:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.618+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/1140886605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5569141a2070 0x556914552820 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.618+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/1140886605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556914181390 0x5569145522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.618+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/1140886605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569141791b0 0x5569145475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.618+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/1140886605 >> 172.21.15.112:0/1140886605 conn(0x556914196460 msgr2=0x556914546ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.618+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/1140886605 shutdown_connections 2024-09-21T03:41:30.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.618+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/1140886605 wait complete. 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f4b9c0740 1 -- start start 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f4b9c0740 1 Processor -- start 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x556914561590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556914181390 0x55691455eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f4b9c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569141a2070 0x55691455eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f401b43d0 con 0x5569141791b0 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f401b5840 con 0x556914181390 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f4b9c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f401b4aa0 con 0x5569141a2070 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f4b9c0740 10 client.0 _ll_register_callbacks cb 0x7f4f380093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x556914561590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x556914561590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51468/0 (socket says 172.21.15.112:51468) 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f3effd640 1 -- 172.21.15.112:0/435873436 learned_addr learned my addr 172.21.15.112:0/435873436 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f3f7fe640 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569141a2070 0x55691455eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.619+0000 7f4f37fff640 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556914181390 0x55691455eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f3effd640 1 -- 172.21.15.112:0/435873436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569141a2070 msgr2=0x55691455eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f3effd640 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569141a2070 0x55691455eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f3effd640 1 -- 172.21.15.112:0/435873436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556914181390 msgr2=0x55691455eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f3effd640 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556914181390 0x55691455eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f3effd640 1 -- 172.21.15.112:0/435873436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691455f530 con 0x5569141791b0 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f3effd640 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x556914561590 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000f9f0 tx=0x7f4f30023f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8947ad955c8e0613 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f267fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f267fc640 1 -- 172.21.15.112:0/435873436 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30028040 con 0x5569141791b0 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f267fc640 1 -- 172.21.15.112:0/435873436 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f4f30023ab0 con 0x5569141791b0 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/435873436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f380b2690 con 0x5569141791b0 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f267fc640 1 -- 172.21.15.112:0/435873436 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f3001d040 con 0x5569141791b0 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f4b9c0740 10 client.17607 fetch_fsmap learned FSMap version 30 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/435873436 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5569145d1400 con 0x5569141791b0 2024-09-21T03:41:30.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.620+0000 7f4f267fc640 1 -- 172.21.15.112:0/435873436 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f380b2690 con 0x5569141791b0 2024-09-21T03:41:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.621+0000 7f4f267fc640 1 -- 172.21.15.112:0/435873436 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4f30003660 con 0x5569141791b0 2024-09-21T03:41:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.621+0000 7f4f4b9c0740 10 client.17607 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:41:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.621+0000 7f4f4b9c0740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:41:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.621+0000 7f4f4b9c0740 20 client.17607 populate_metadata read hostname 'smithi112' 2024-09-21T03:41:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.621+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556914135da0 0x5569145d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:41:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.621+0000 7f4f4b9c0740 4 client.17607 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:41:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.621+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/435873436 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5569144bd850 con 0x556914135da0 2024-09-21T03:41:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.621+0000 7f4f37fff640 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556914135da0 0x5569145d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:41:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.621+0000 7f4f267fc640 10 client.17607 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:30.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.621+0000 7f4f37fff640 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556914135da0 0x5569145d3c10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d52aa644739ace server_cookie=4715dfeb3de94a71 in_seq=0 out_seq=0 2024-09-21T03:41:30.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f267fc640 1 -- 172.21.15.112:0/435873436 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5966 (crc 0 0 0) 0x5569144bd850 con 0x556914135da0 2024-09-21T03:41:30.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f267fc640 10 client.17607 handle_command_reply: tid=0 2024-09-21T03:41:30.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 1 client.17607 shutdown 2024-09-21T03:41:30.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/435873436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 msgr2=0x556914561590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x556914561590 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000f9f0 tx=0x7f4f30023f30 comp rx=0 tx=0).stop 2024-09-21T03:41:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/435873436 shutdown_connections 2024-09-21T03:41:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x556914135da0 0x5569145d3c10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569141a2070 0x55691455eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556914181390 0x55691455eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 1 --2- 172.21.15.112:0/435873436 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5569141791b0 0x556914561590 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/435873436 >> 172.21.15.112:0/435873436 conn(0x556914196460 msgr2=0x55691454a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f267fc640 0 client.17607 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:41:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/435873436 shutdown_connections 2024-09-21T03:41:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 1 -- 172.21.15.112:0/435873436 wait complete. 2024-09-21T03:41:30.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.622+0000 7f4f4b9c0740 20 client.17607 trim_cache size 0 max 16384 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: "3d0e4442-3ee3-44bd-99e2-f0a81392c8c4": { 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "3d0e4442-3ee3-44bd-99e2-f0a81392c8c4", 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c": { 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:45.957537+0000", 2024-09-21T03:41:30.629 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "3d0e4442-3ee3-44bd-99e2-f0a81392c8c4": { 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:41:28.881491+0000", 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.630 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.631 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.632 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.633 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:41:30.634 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.635 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.637 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.638 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.639 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:41:30.640 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 -- 172.21.15.112:0/3529003715 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f2c0b66b0 msgr2=0x7f4f2c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f2c0b66b0 0x7f4f2c0b8bc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4f280096c0 tx=0x7f4f28041040 comp rx=0 tx=0).stop 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 -- 172.21.15.112:0/3529003715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f40108d30 msgr2=0x7f4f4019bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f40108d30 0x7f4f4019bdb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009fd0 tx=0x7f4f3800ef10 comp rx=0 tx=0).stop 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 -- 172.21.15.112:0/3529003715 shutdown_connections 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f4f2c0b66b0 0x7f4f2c0b8bc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f40108d30 0x7f4f4019bdb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f401083e0 0x7f4f4019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 --2- 172.21.15.112:0/3529003715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f40102aa0 0x7f4f4019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 -- 172.21.15.112:0/3529003715 >> 172.21.15.112:0/3529003715 conn(0x7f4f400797e0 msgr2=0x7f4f40104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 -- 172.21.15.112:0/3529003715 shutdown_connections 2024-09-21T03:41:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:41:30.623+0000 7f4f4544c640 1 -- 172.21.15.112:0/3529003715 wait complete. 2024-09-21T03:41:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:30 smithi203 ceph-mon[34723]: pgmap v706: 97 pgs: 97 active+clean; 940 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-09-21T03:41:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:30 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:41:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:30 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:30 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:41:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:30 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:41:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:30 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:30 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:41:31.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1326305 10.23 MB/sec execute 571 sec latency 30.437 ms 2024-09-21T03:41:31.358 INFO:tasks.cephfs.filesystem:scrub status for tag:3d0e4442-3ee3-44bd-99e2-f0a81392c8c4 - {'path': '/', 'tag': '3d0e4442-3ee3-44bd-99e2-f0a81392c8c4', 'options': 'recursive,force'} 2024-09-21T03:41:32.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1328506 10.24 MB/sec execute 572 sec latency 55.934 ms 2024-09-21T03:41:32.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:32 smithi187 ceph-mon[34603]: pgmap v707: 97 pgs: 97 active+clean; 909 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 275 op/s 2024-09-21T03:41:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:32 smithi112 ceph-mon[31442]: pgmap v707: 97 pgs: 97 active+clean; 909 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 275 op/s 2024-09-21T03:41:32.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:32 smithi203 ceph-mon[34723]: pgmap v707: 97 pgs: 97 active+clean; 909 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 275 op/s 2024-09-21T03:41:33.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1329984 10.23 MB/sec execute 573 sec latency 40.866 ms 2024-09-21T03:41:34.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1332136 10.23 MB/sec execute 574 sec latency 14.397 ms 2024-09-21T03:41:34.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:34 smithi187 ceph-mon[34603]: pgmap v708: 97 pgs: 97 active+clean; 892 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-09-21T03:41:34.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:34 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:41:34.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:34 smithi112 ceph-mon[31442]: pgmap v708: 97 pgs: 97 active+clean; 892 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-09-21T03:41:34.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:34 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:41:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:34 smithi203 ceph-mon[34723]: pgmap v708: 97 pgs: 97 active+clean; 892 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-09-21T03:41:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:34 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:41:35.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1333988 10.23 MB/sec execute 575 sec latency 21.725 ms 2024-09-21T03:41:35.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:35 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:41:35.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:35 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:41:35.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:35 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:41:35.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:35 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:41:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:35 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:41:35.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:35 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:41:36.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1335824 10.23 MB/sec execute 576 sec latency 11.693 ms 2024-09-21T03:41:36.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:36 smithi187 ceph-mon[34603]: pgmap v709: 97 pgs: 97 active+clean; 888 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2024-09-21T03:41:36.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:36 smithi112 ceph-mon[31442]: pgmap v709: 97 pgs: 97 active+clean; 888 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2024-09-21T03:41:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:36 smithi203 ceph-mon[34723]: pgmap v709: 97 pgs: 97 active+clean; 888 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 252 op/s 2024-09-21T03:41:37.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1337719 10.23 MB/sec execute 577 sec latency 60.329 ms 2024-09-21T03:41:38.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1339293 10.23 MB/sec execute 578 sec latency 19.102 ms 2024-09-21T03:41:38.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:38 smithi112 ceph-mon[31442]: pgmap v710: 97 pgs: 97 active+clean; 893 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 299 op/s 2024-09-21T03:41:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:38 smithi203 ceph-mon[34723]: pgmap v710: 97 pgs: 97 active+clean; 893 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 299 op/s 2024-09-21T03:41:38.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:38 smithi187 ceph-mon[34603]: pgmap v710: 97 pgs: 97 active+clean; 893 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 299 op/s 2024-09-21T03:41:39.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1341262 10.22 MB/sec execute 579 sec latency 18.379 ms 2024-09-21T03:41:40.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1343083 10.23 MB/sec execute 580 sec latency 18.390 ms 2024-09-21T03:41:40.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:40 smithi112 ceph-mon[31442]: pgmap v711: 97 pgs: 97 active+clean; 893 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 220 op/s 2024-09-21T03:41:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:40 smithi203 ceph-mon[34723]: pgmap v711: 97 pgs: 97 active+clean; 893 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 220 op/s 2024-09-21T03:41:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:40 smithi187 ceph-mon[34603]: pgmap v711: 97 pgs: 97 active+clean; 893 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 220 op/s 2024-09-21T03:41:41.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1345569 10.23 MB/sec execute 581 sec latency 17.058 ms 2024-09-21T03:41:42.281 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1346691 10.23 MB/sec execute 582 sec latency 64.876 ms 2024-09-21T03:41:42.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:42 smithi112 ceph-mon[31442]: pgmap v712: 97 pgs: 97 active+clean; 900 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 266 op/s 2024-09-21T03:41:42.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:42 smithi203 ceph-mon[34723]: pgmap v712: 97 pgs: 97 active+clean; 900 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 266 op/s 2024-09-21T03:41:42.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:42 smithi187 ceph-mon[34603]: pgmap v712: 97 pgs: 97 active+clean; 900 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 266 op/s 2024-09-21T03:41:43.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1348517 10.22 MB/sec execute 583 sec latency 25.899 ms 2024-09-21T03:41:44.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1350047 10.22 MB/sec execute 584 sec latency 20.329 ms 2024-09-21T03:41:44.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:44 smithi112 ceph-mon[31442]: pgmap v713: 97 pgs: 97 active+clean; 906 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 229 op/s 2024-09-21T03:41:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:44 smithi203 ceph-mon[34723]: pgmap v713: 97 pgs: 97 active+clean; 906 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 229 op/s 2024-09-21T03:41:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:44 smithi187 ceph-mon[34603]: pgmap v713: 97 pgs: 97 active+clean; 906 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 229 op/s 2024-09-21T03:41:45.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1352219 10.22 MB/sec execute 585 sec latency 21.941 ms 2024-09-21T03:41:46.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1353609 10.22 MB/sec execute 586 sec latency 19.850 ms 2024-09-21T03:41:46.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:46 smithi112 ceph-mon[31442]: pgmap v714: 97 pgs: 97 active+clean; 906 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2024-09-21T03:41:46.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:46 smithi203 ceph-mon[34723]: pgmap v714: 97 pgs: 97 active+clean; 906 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2024-09-21T03:41:46.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:46 smithi187 ceph-mon[34603]: pgmap v714: 97 pgs: 97 active+clean; 906 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 229 op/s 2024-09-21T03:41:47.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1355560 10.22 MB/sec execute 587 sec latency 68.966 ms 2024-09-21T03:41:48.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1357104 10.22 MB/sec execute 588 sec latency 30.632 ms 2024-09-21T03:41:48.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:48 smithi112 ceph-mon[31442]: pgmap v715: 97 pgs: 97 active+clean; 916 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 285 op/s 2024-09-21T03:41:48.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:48 smithi203 ceph-mon[34723]: pgmap v715: 97 pgs: 97 active+clean; 916 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 285 op/s 2024-09-21T03:41:48.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:48 smithi187 ceph-mon[34603]: pgmap v715: 97 pgs: 97 active+clean; 916 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 285 op/s 2024-09-21T03:41:49.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1358945 10.21 MB/sec execute 589 sec latency 32.436 ms 2024-09-21T03:41:50.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1360499 10.22 MB/sec execute 590 sec latency 13.565 ms 2024-09-21T03:41:50.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:50 smithi112 ceph-mon[31442]: pgmap v716: 97 pgs: 97 active+clean; 917 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 225 op/s 2024-09-21T03:41:50.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:50 smithi203 ceph-mon[34723]: pgmap v716: 97 pgs: 97 active+clean; 917 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 225 op/s 2024-09-21T03:41:50.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:50 smithi187 ceph-mon[34603]: pgmap v716: 97 pgs: 97 active+clean; 917 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 225 op/s 2024-09-21T03:41:51.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1361954 10.21 MB/sec execute 591 sec latency 25.156 ms 2024-09-21T03:41:52.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1363902 10.21 MB/sec execute 592 sec latency 42.027 ms 2024-09-21T03:41:52.463 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:52 smithi203 ceph-mon[34723]: pgmap v717: 97 pgs: 97 active+clean; 925 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 287 op/s 2024-09-21T03:41:52.481 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:52 smithi187 ceph-mon[34603]: pgmap v717: 97 pgs: 97 active+clean; 925 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 287 op/s 2024-09-21T03:41:52.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:52 smithi112 ceph-mon[31442]: pgmap v717: 97 pgs: 97 active+clean; 925 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 287 op/s 2024-09-21T03:41:53.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1365164 10.20 MB/sec execute 593 sec latency 31.992 ms 2024-09-21T03:41:54.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1367459 10.21 MB/sec execute 594 sec latency 16.051 ms 2024-09-21T03:41:54.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:54 smithi112 ceph-mon[31442]: pgmap v718: 97 pgs: 97 active+clean; 926 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 234 op/s 2024-09-21T03:41:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:54 smithi203 ceph-mon[34723]: pgmap v718: 97 pgs: 97 active+clean; 926 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 234 op/s 2024-09-21T03:41:54.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:54 smithi187 ceph-mon[34603]: pgmap v718: 97 pgs: 97 active+clean; 926 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 234 op/s 2024-09-21T03:41:55.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1368832 10.20 MB/sec execute 595 sec latency 21.554 ms 2024-09-21T03:41:56.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1371032 10.21 MB/sec execute 596 sec latency 19.553 ms 2024-09-21T03:41:56.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:56 smithi112 ceph-mon[31442]: pgmap v719: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2024-09-21T03:41:56.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:56 smithi203 ceph-mon[34723]: pgmap v719: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2024-09-21T03:41:56.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:56 smithi187 ceph-mon[34603]: pgmap v719: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 228 op/s 2024-09-21T03:41:57.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1372394 10.20 MB/sec execute 597 sec latency 43.633 ms 2024-09-21T03:41:58.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1374574 10.21 MB/sec execute 598 sec latency 20.114 ms 2024-09-21T03:41:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:41:58 smithi112 ceph-mon[31442]: pgmap v720: 97 pgs: 97 active+clean; 937 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 287 op/s 2024-09-21T03:41:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:41:58 smithi203 ceph-mon[34723]: pgmap v720: 97 pgs: 97 active+clean; 937 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 287 op/s 2024-09-21T03:41:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:41:58 smithi187 ceph-mon[34603]: pgmap v720: 97 pgs: 97 active+clean; 937 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 287 op/s 2024-09-21T03:41:59.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1375421 10.21 MB/sec execute 599 sec latency 48.165 ms 2024-09-21T03:42:00.282 INFO:tasks.workunit.client.0.smithi112.stdout: 1 cleanup 600 sec 2024-09-21T03:42:00.439 INFO:tasks.workunit.client.0.smithi112.stdout: 0 cleanup 600 sec 2024-09-21T03:42:00.439 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-09-21T03:42:00.439 INFO:tasks.workunit.client.0.smithi112.stdout: Operation Count AvgLat MaxLat 2024-09-21T03:42:00.439 INFO:tasks.workunit.client.0.smithi112.stdout: ---------------------------------------- 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: NTCreateX 194984 1.301 1656.415 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: Close 143237 0.002 0.108 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: Rename 8256 4.999 74.629 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: Unlink 39379 1.647 4838.178 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: Deltree 6 77.486 105.524 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: Mkdir 3 0.005 0.006 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: Qpathinfo 176688 0.949 161.961 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: Qfileinfo 30997 0.001 0.033 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: Qfsinfo 32400 0.221 38.987 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: Sfileinfo 15896 0.015 3.598 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: Find 68323 0.173 176.747 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: WriteX 97403 0.054 102.040 2024-09-21T03:42:00.440 INFO:tasks.workunit.client.0.smithi112.stdout: ReadX 305475 0.004 2.366 2024-09-21T03:42:00.441 INFO:tasks.workunit.client.0.smithi112.stdout: LockX 634 0.767 5.018 2024-09-21T03:42:00.441 INFO:tasks.workunit.client.0.smithi112.stdout: UnlockX 634 0.661 2.269 2024-09-21T03:42:00.441 INFO:tasks.workunit.client.0.smithi112.stdout: Flush 13676 3.133 61.865 2024-09-21T03:42:00.441 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-09-21T03:42:00.441 INFO:tasks.workunit.client.0.smithi112.stdout:Throughput 10.2133 MB/sec 1 clients 1 procs max_latency=4838.188 ms 2024-09-21T03:42:00.441 INFO:tasks.workunit.client.0.smithi112.stderr:+ dbench 10 2024-09-21T03:42:00.441 INFO:tasks.workunit.client.0.smithi112.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-21T03:42:00.442 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-09-21T03:42:00.442 INFO:tasks.workunit.client.0.smithi112.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-21T03:42:00.442 INFO:tasks.workunit.client.0.smithi112.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-21T03:42:00.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:00 smithi112 ceph-mon[31442]: pgmap v721: 97 pgs: 97 active+clean; 939 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-09-21T03:42:00.541 INFO:tasks.workunit.client.0.smithi112.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-21T03:42:00.541 INFO:tasks.workunit.client.0.smithi112.stdout:releasing clients 2024-09-21T03:42:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:00 smithi203 ceph-mon[34723]: pgmap v721: 97 pgs: 97 active+clean; 939 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-09-21T03:42:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:00 smithi187 ceph-mon[34603]: pgmap v721: 97 pgs: 97 active+clean; 939 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-09-21T03:42:01.360 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:42:01.541 INFO:tasks.workunit.client.0.smithi112.stdout: 10 380 135.65 MB/sec warmup 1 sec latency 54.544 ms 2024-09-21T03:42:01.623 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:02.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.121+0000 7f7d1b35e640 1 -- 172.21.15.112:0/3837113800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d141051c0 msgr2=0x7f7d141055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.121+0000 7f7d1b35e640 1 --2- 172.21.15.112:0/3837113800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d141051c0 0x7f7d141055d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009900 tx=0x7f7d0402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.122+0000 7f7d1b35e640 1 -- 172.21.15.112:0/3837113800 shutdown_connections 2024-09-21T03:42:02.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.122+0000 7f7d1b35e640 1 --2- 172.21.15.112:0/3837113800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d14105b10 0x7f7d1410f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.122+0000 7f7d1b35e640 1 --2- 172.21.15.112:0/3837113800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d141051c0 0x7f7d141055d0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.122+0000 7f7d1b35e640 1 --2- 172.21.15.112:0/3837113800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d141031f0 0x7f7d14103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.122+0000 7f7d1b35e640 1 -- 172.21.15.112:0/3837113800 >> 172.21.15.112:0/3837113800 conn(0x7f7d14079f90 msgr2=0x7f7d1407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.122+0000 7f7d1b35e640 1 -- 172.21.15.112:0/3837113800 shutdown_connections 2024-09-21T03:42:02.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.122+0000 7f7d1b35e640 1 -- 172.21.15.112:0/3837113800 wait complete. 2024-09-21T03:42:02.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.122+0000 7f7d1b35e640 1 Processor -- start 2024-09-21T03:42:02.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.122+0000 7f7d1b35e640 1 -- start start 2024-09-21T03:42:02.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d1b35e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d141031f0 0x7f7d1419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d1b35e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d141051c0 0x7f7d1419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d1b35e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d14105b10 0x7f7d1419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d1b35e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14112a60 con 0x7f7d14105b10 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d1b35e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141128e0 con 0x7f7d141051c0 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d1b35e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14112be0 con 0x7f7d141031f0 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d198fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d14105b10 0x7f7d1419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d198fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d14105b10 0x7f7d1419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46312/0 (socket says 172.21.15.112:46312) 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d190fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d141031f0 0x7f7d1419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d198fb640 1 -- 172.21.15.112:0/1569900439 learned_addr learned my addr 172.21.15.112:0/1569900439 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d188f9640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d141051c0 0x7f7d1419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d198fb640 1 -- 172.21.15.112:0/1569900439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d141031f0 msgr2=0x7f7d1419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d198fb640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d141031f0 0x7f7d1419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d198fb640 1 -- 172.21.15.112:0/1569900439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d141051c0 msgr2=0x7f7d1419b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d198fb640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d141051c0 0x7f7d1419b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d198fb640 1 -- 172.21.15.112:0/1569900439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d141a65c0 con 0x7f7d14105b10 2024-09-21T03:42:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d198fb640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d14105b10 0x7f7d1419bdb0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009fd0 tx=0x7f7d0c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.123+0000 7f7d0a7fc640 1 -- 172.21.15.112:0/1569900439 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c017070 con 0x7f7d14105b10 2024-09-21T03:42:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.124+0000 7f7d1b35e640 1 -- 172.21.15.112:0/1569900439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d141a68b0 con 0x7f7d14105b10 2024-09-21T03:42:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.124+0000 7f7d0a7fc640 1 -- 172.21.15.112:0/1569900439 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d0c003e20 con 0x7f7d14105b10 2024-09-21T03:42:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.124+0000 7f7d0a7fc640 1 -- 172.21.15.112:0/1569900439 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c010460 con 0x7f7d14105b10 2024-09-21T03:42:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.124+0000 7f7d1b35e640 1 -- 172.21.15.112:0/1569900439 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d141a6d70 con 0x7f7d14105b10 2024-09-21T03:42:02.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.126+0000 7f7d218d2740 1 Processor -- start 2024-09-21T03:42:02.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d0a7fc640 1 -- 172.21.15.112:0/1569900439 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7d0c003970 con 0x7f7d14105b10 2024-09-21T03:42:02.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d218d2740 1 -- start start 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631c8070 0x56156356f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56156319f1b0 0x56156356f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615631a7390 0x56156356fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d218d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5615634e5e50 con 0x56156319f1b0 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d218d2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5615634d5280 con 0x5615631c8070 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d218d2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5615634d9090 con 0x5615631a7390 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d190fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56156319f1b0 0x56156356f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d188f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631c8070 0x56156356f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d190fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56156319f1b0 0x56156356f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46328/0 (socket says 172.21.15.112:46328) 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d190fa640 1 -- 172.21.15.112:0/1354368832 learned_addr learned my addr 172.21.15.112:0/1354368832 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.127+0000 7f7d188f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631c8070 0x56156356f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:57622/0 (socket says 172.21.15.112:57622) 2024-09-21T03:42:02.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7d188f9640 1 --2- 172.21.15.112:0/1354368832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615631a7390 0x56156356fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7d190fa640 1 -- 172.21.15.112:0/1354368832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615631a7390 msgr2=0x56156356fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7d190fa640 1 --2- 172.21.15.112:0/1354368832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615631a7390 0x56156356fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7d190fa640 1 -- 172.21.15.112:0/1354368832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631c8070 msgr2=0x56156356f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7d190fa640 1 --2- 172.21.15.112:0/1354368832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631c8070 0x56156356f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7d190fa640 1 -- 172.21.15.112:0/1354368832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561563233650 con 0x56156319f1b0 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7d0a7fc640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7d000b66b0 0x7f7d000b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7d190fa640 1 --2- 172.21.15.112:0/1354368832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56156319f1b0 0x56156356f7b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009c90 tx=0x7f7d10011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c532e6f90ceb89be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7d188f9640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7d000b66b0 0x7f7d000b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7d0a7fc640 1 -- 172.21.15.112:0/1569900439 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f7d0c0e73c0 con 0x7f7d14105b10 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7cff7fe640 1 -- 172.21.15.112:0/1354368832 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10009e40 con 0x56156319f1b0 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7cff7fe640 1 -- 172.21.15.112:0/1354368832 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d100054d0 con 0x56156319f1b0 2024-09-21T03:42:02.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.128+0000 7f7cff7fe640 1 -- 172.21.15.112:0/1354368832 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10019670 con 0x56156319f1b0 2024-09-21T03:42:02.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.129+0000 7f7d218d2740 1 -- 172.21.15.112:0/1354368832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56156319f1b0 msgr2=0x56156356f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.129+0000 7f7d188f9640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7d000b66b0 0x7f7d000b8bc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7d0402a9e0 tx=0x7f7d04005780 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.129+0000 7f7d218d2740 1 --2- 172.21.15.112:0/1354368832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56156319f1b0 0x56156356f7b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009c90 tx=0x7f7d10011810 comp rx=0 tx=0).stop 2024-09-21T03:42:02.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.129+0000 7f7d218d2740 1 -- 172.21.15.112:0/1354368832 shutdown_connections 2024-09-21T03:42:02.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.129+0000 7f7d218d2740 1 --2- 172.21.15.112:0/1354368832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615631a7390 0x56156356fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.129+0000 7f7d218d2740 1 --2- 172.21.15.112:0/1354368832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56156319f1b0 0x56156356f7b0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.129+0000 7f7d218d2740 1 --2- 172.21.15.112:0/1354368832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631c8070 0x56156356f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.129+0000 7f7d218d2740 1 -- 172.21.15.112:0/1354368832 >> 172.21.15.112:0/1354368832 conn(0x5615631bc460 msgr2=0x56156356d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.129+0000 7f7d218d2740 1 -- 172.21.15.112:0/1354368832 shutdown_connections 2024-09-21T03:42:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.129+0000 7f7d218d2740 1 -- 172.21.15.112:0/1354368832 wait complete. 2024-09-21T03:42:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d218d2740 1 -- start start 2024-09-21T03:42:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d218d2740 1 Processor -- start 2024-09-21T03:42:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x56156357ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x56156357f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x56156358b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d218d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561563336700 con 0x5615631c8070 2024-09-21T03:42:02.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d218d2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561563470110 con 0x5615631a7390 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d218d2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561563581db0 con 0x56156319f1b0 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d218d2740 10 client.0 _ll_register_callbacks cb 0x561563489990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d190fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x56156357ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d198fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x56156358b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.130+0000 7f7d190fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x56156357ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:33038/0 (socket says 172.21.15.112:33038) 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7d198fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x56156358b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46334/0 (socket says 172.21.15.112:46334) 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7d190fa640 1 -- 172.21.15.112:0/2974336962 learned_addr learned my addr 172.21.15.112:0/2974336962 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7d188f9640 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x56156357f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7d190fa640 1 -- 172.21.15.112:0/2974336962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 msgr2=0x56156357f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7d190fa640 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x56156357f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7d190fa640 1 -- 172.21.15.112:0/2974336962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 msgr2=0x56156358b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7d190fa640 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x56156358b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7d190fa640 1 -- 172.21.15.112:0/2974336962 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56156349dc40 con 0x56156319f1b0 2024-09-21T03:42:02.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7d190fa640 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x56156357ed70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d10005890 tx=0x7f7d10004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60f574fca1d2b230 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7cfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:42:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7cfcff9640 1 -- 172.21.15.112:0/2974336962 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10028070 con 0x56156319f1b0 2024-09-21T03:42:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7d218d2740 1 -- 172.21.15.112:0/2974336962 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d100054d0 con 0x56156319f1b0 2024-09-21T03:42:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.131+0000 7f7cfcff9640 1 -- 172.21.15.112:0/2974336962 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f7d10009e40 con 0x56156319f1b0 2024-09-21T03:42:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7cfcff9640 1 -- 172.21.15.112:0/2974336962 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d10034050 con 0x56156319f1b0 2024-09-21T03:42:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7d218d2740 10 client.27178 fetch_fsmap learned FSMap version 30 2024-09-21T03:42:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7d218d2740 1 -- 172.21.15.112:0/2974336962 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56156358db00 con 0x56156319f1b0 2024-09-21T03:42:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7cfcff9640 1 -- 172.21.15.112:0/2974336962 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d100054d0 con 0x56156319f1b0 2024-09-21T03:42:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7cfcff9640 1 -- 172.21.15.112:0/2974336962 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7d10024c70 con 0x56156319f1b0 2024-09-21T03:42:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7d218d2740 10 client.27178 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:42:02.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7d218d2740 10 client.27178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:02.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7d218d2740 20 client.27178 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:02.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7d218d2740 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5615631869f0 0x561563590f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7d218d2740 4 client.27178 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:02.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7d218d2740 1 -- 172.21.15.112:0/2974336962 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615634e7da0 con 0x5615631869f0 2024-09-21T03:42:02.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.132+0000 7f7d188f9640 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5615631869f0 0x561563590f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.133+0000 7f7cfcff9640 10 client.27178 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:02.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.133+0000 7f7d188f9640 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5615631869f0 0x561563590f20 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31addca154292171 server_cookie=dc1f75143a04c7b5 in_seq=0 out_seq=0 2024-09-21T03:42:02.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7cfcff9640 1 -- 172.21.15.112:0/2974336962 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615634e7da0 con 0x5615631869f0 2024-09-21T03:42:02.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7cfcff9640 10 client.27178 handle_command_reply: tid=0 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 1 client.27178 shutdown 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 1 -- 172.21.15.112:0/2974336962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 msgr2=0x56156357ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x56156357ed70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d10005890 tx=0x7f7d10004fb0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 1 -- 172.21.15.112:0/2974336962 shutdown_connections 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5615631869f0 0x561563590f20 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x56156358b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7cfcff9640 0 client.27178 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x56156357f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 1 --2- 172.21.15.112:0/2974336962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x56156357ed70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 1 -- 172.21.15.112:0/2974336962 >> 172.21.15.112:0/2974336962 conn(0x5615631bc460 msgr2=0x561563570c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 1 -- 172.21.15.112:0/2974336962 shutdown_connections 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 1 -- 172.21.15.112:0/2974336962 wait complete. 2024-09-21T03:42:02.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.136+0000 7f7d218d2740 20 client.27178 trim_cache size 0 max 16384 2024-09-21T03:42:02.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.154+0000 7f7d218d2740 1 Processor -- start 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.154+0000 7f7d218d2740 1 -- start start 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.154+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x56156357a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.154+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x5615635809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.154+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x56156357bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.154+0000 7f7d218d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141afd70 con 0x5615631c8070 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.154+0000 7f7d218d2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141ad640 con 0x5615631a7390 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.154+0000 7f7d218d2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141ab4b0 con 0x56156319f1b0 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7d198fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x56156357bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7d198fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x56156357bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46338/0 (socket says 172.21.15.112:46338) 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7d198fb640 1 -- 172.21.15.112:0/4101976798 learned_addr learned my addr 172.21.15.112:0/4101976798 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:02.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7d190fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x56156357a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7d188f9640 1 --2- 172.21.15.112:0/4101976798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x5615635809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7d198fb640 1 -- 172.21.15.112:0/4101976798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 msgr2=0x56156357a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7d198fb640 1 --2- 172.21.15.112:0/4101976798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x56156357a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7d198fb640 1 -- 172.21.15.112:0/4101976798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 msgr2=0x5615635809a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7d198fb640 1 --2- 172.21.15.112:0/4101976798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x5615635809a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7d198fb640 1 -- 172.21.15.112:0/4101976798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615635753a0 con 0x5615631c8070 2024-09-21T03:42:02.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7d198fb640 1 --2- 172.21.15.112:0/4101976798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x56156357bf60 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0a9ab0 tx=0x7f7d0c0acd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2e2771f7876f9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7cfeffd640 1 -- 172.21.15.112:0/4101976798 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c020040 con 0x5615631c8070 2024-09-21T03:42:02.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.155+0000 7f7cfeffd640 1 -- 172.21.15.112:0/4101976798 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d0c0ace00 con 0x5615631c8070 2024-09-21T03:42:02.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.156+0000 7f7cfeffd640 1 -- 172.21.15.112:0/4101976798 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0ac220 con 0x5615631c8070 2024-09-21T03:42:02.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.156+0000 7f7d218d2740 1 -- 172.21.15.112:0/4101976798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 msgr2=0x56156357bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.156+0000 7f7d218d2740 1 --2- 172.21.15.112:0/4101976798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x56156357bf60 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0a9ab0 tx=0x7f7d0c0acd60 comp rx=0 tx=0).stop 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.156+0000 7f7d218d2740 1 -- 172.21.15.112:0/4101976798 shutdown_connections 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.156+0000 7f7d218d2740 1 --2- 172.21.15.112:0/4101976798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x56156357bf60 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.156+0000 7f7d218d2740 1 --2- 172.21.15.112:0/4101976798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x5615635809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.156+0000 7f7d218d2740 1 --2- 172.21.15.112:0/4101976798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x56156357a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.156+0000 7f7d218d2740 1 -- 172.21.15.112:0/4101976798 >> 172.21.15.112:0/4101976798 conn(0x5615631bc460 msgr2=0x561563580290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.156+0000 7f7d218d2740 1 -- 172.21.15.112:0/4101976798 shutdown_connections 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.157+0000 7f7d218d2740 1 -- 172.21.15.112:0/4101976798 wait complete. 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.157+0000 7f7d218d2740 1 -- start start 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.157+0000 7f7d218d2740 1 Processor -- start 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.157+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x561563587520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.157+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x5615635849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.157+0000 7f7d218d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x561563584f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.157+0000 7f7d218d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141abb00 con 0x5615631c8070 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.157+0000 7f7d218d2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141acf70 con 0x5615631a7390 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.157+0000 7f7d218d2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141ac1d0 con 0x56156319f1b0 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.157+0000 7f7d218d2740 10 client.0 _ll_register_callbacks cb 0x7f7d04009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d198fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x561563584f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d198fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x561563584f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46342/0 (socket says 172.21.15.112:46342) 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d198fb640 1 -- 172.21.15.112:0/4294227648 learned_addr learned my addr 172.21.15.112:0/4294227648 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d188f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x5615635849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d190fa640 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x561563587520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d198fb640 1 -- 172.21.15.112:0/4294227648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 msgr2=0x561563587520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d198fb640 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x561563587520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d198fb640 1 -- 172.21.15.112:0/4294227648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 msgr2=0x5615635849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d198fb640 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x5615635849f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d198fb640 1 -- 172.21.15.112:0/4294227648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561563585470 con 0x5615631c8070 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d188f9640 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x5615635849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d198fb640 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x561563584f30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b760 tx=0x7f7d0c0e5070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac75518c62a73634 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7cfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7cfcff9640 1 -- 172.21.15.112:0/4294227648 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0d4070 con 0x5615631c8070 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7cfcff9640 1 -- 172.21.15.112:0/4294227648 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f7d0c0d0dc0 con 0x5615631c8070 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d218d2740 1 -- 172.21.15.112:0/4294227648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d0c0ace00 con 0x5615631c8070 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7cfcff9640 1 -- 172.21.15.112:0/4294227648 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d0c020070 con 0x5615631c8070 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d218d2740 10 client.17625 fetch_fsmap learned FSMap version 30 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.158+0000 7f7d218d2740 1 -- 172.21.15.112:0/4294227648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5615635fb4b0 con 0x5615631c8070 2024-09-21T03:42:02.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.159+0000 7f7cfcff9640 1 -- 172.21.15.112:0/4294227648 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d0c0ace00 con 0x5615631c8070 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.159+0000 7f7cfcff9640 1 -- 172.21.15.112:0/4294227648 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7d0c0ac1c0 con 0x5615631c8070 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.159+0000 7f7d218d2740 10 client.17625 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.159+0000 7f7d218d2740 10 client.17625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.159+0000 7f7d218d2740 20 client.17625 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.159+0000 7f7d218d2740 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56156315bda0 0x5615635fdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.159+0000 7f7d218d2740 4 client.17625 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.159+0000 7f7d218d2740 1 -- 172.21.15.112:0/4294227648 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615634e3850 con 0x56156315bda0 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.159+0000 7f7d190fa640 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56156315bda0 0x5615635fdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:02.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.160+0000 7f7cfcff9640 10 client.17625 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.160+0000 7f7d190fa640 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56156315bda0 0x5615635fdbb0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa2901a82bef9f9 server_cookie=e4e55aa387b19ab0 in_seq=0 out_seq=0 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7cfcff9640 1 -- 172.21.15.112:0/4294227648 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5770 (crc 0 0 0) 0x5615634e3850 con 0x56156315bda0 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7cfcff9640 10 client.17625 handle_command_reply: tid=0 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 1 client.17625 shutdown 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 1 -- 172.21.15.112:0/4294227648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 msgr2=0x561563584f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x561563584f30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b760 tx=0x7f7d0c0e5070 comp rx=0 tx=0).stop 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 1 -- 172.21.15.112:0/4294227648 shutdown_connections 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x56156315bda0 0x5615635fdbb0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5615631c8070 0x561563584f30 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615631a7390 0x5615635849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 1 --2- 172.21.15.112:0/4294227648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56156319f1b0 0x561563587520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 1 -- 172.21.15.112:0/4294227648 >> 172.21.15.112:0/4294227648 conn(0x5615631bc460 msgr2=0x5615635801b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7cfcff9640 0 client.17625 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 1 -- 172.21.15.112:0/4294227648 shutdown_connections 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 1 -- 172.21.15.112:0/4294227648 wait complete. 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.161+0000 7f7d218d2740 20 client.17625 trim_cache size 0 max 16384 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:42:02.170 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c": { 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:45.957537+0000", 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "3d0e4442-3ee3-44bd-99e2-f0a81392c8c4": { 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:41:28.881491+0000", 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.171 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.172 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:42:02.173 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.174 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.176 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:42:02.177 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.178 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.179 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.180 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:02.181 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:02.182 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:02.182 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:02.182 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:42:02.182 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:42:02.182 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:42:02.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 -- 172.21.15.112:0/1569900439 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7d000b66b0 msgr2=0x7f7d000b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7d000b66b0 0x7f7d000b8bc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7d0402a9e0 tx=0x7f7d04005780 comp rx=0 tx=0).stop 2024-09-21T03:42:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 -- 172.21.15.112:0/1569900439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d14105b10 msgr2=0x7f7d1419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d14105b10 0x7f7d1419bdb0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009fd0 tx=0x7f7d0c00b260 comp rx=0 tx=0).stop 2024-09-21T03:42:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 -- 172.21.15.112:0/1569900439 shutdown_connections 2024-09-21T03:42:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7d000b66b0 0x7f7d000b8bc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d14105b10 0x7f7d1419bdb0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d141051c0 0x7f7d1419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 --2- 172.21.15.112:0/1569900439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d141031f0 0x7f7d1419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 -- 172.21.15.112:0/1569900439 >> 172.21.15.112:0/1569900439 conn(0x7f7d14079f90 msgr2=0x7f7d141064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 -- 172.21.15.112:0/1569900439 shutdown_connections 2024-09-21T03:42:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:02.162+0000 7f7d1b35e640 1 -- 172.21.15.112:0/1569900439 wait complete. 2024-09-21T03:42:02.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:02 smithi112 ceph-mon[31442]: pgmap v722: 97 pgs: 97 active+clean; 947 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 267 op/s 2024-09-21T03:42:02.541 INFO:tasks.workunit.client.0.smithi112.stdout: 10 762 104.64 MB/sec warmup 2 sec latency 127.143 ms 2024-09-21T03:42:02.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:02 smithi203 ceph-mon[34723]: pgmap v722: 97 pgs: 97 active+clean; 947 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 267 op/s 2024-09-21T03:42:02.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:02 smithi187 ceph-mon[34603]: pgmap v722: 97 pgs: 97 active+clean; 947 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 267 op/s 2024-09-21T03:42:03.094 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:42:03.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:03.360 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:03.541 INFO:tasks.workunit.client.0.smithi112.stdout: 10 1325 83.34 MB/sec warmup 3 sec latency 101.308 ms 2024-09-21T03:42:03.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.864+0000 7f6b9228a640 1 -- 172.21.15.112:0/510493316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c1083c0 msgr2=0x7f6b8c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.864+0000 7f6b9228a640 1 --2- 172.21.15.112:0/510493316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c1083c0 0x7f6b8c1087d0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009900 tx=0x7f6b8002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.864+0000 7f6b9228a640 1 -- 172.21.15.112:0/510493316 shutdown_connections 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.864+0000 7f6b9228a640 1 --2- 172.21.15.112:0/510493316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b8c108d10 0x7f6b8c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.864+0000 7f6b9228a640 1 --2- 172.21.15.112:0/510493316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c1083c0 0x7f6b8c1087d0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.864+0000 7f6b9228a640 1 --2- 172.21.15.112:0/510493316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c102a80 0x7f6b8c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.864+0000 7f6b9228a640 1 -- 172.21.15.112:0/510493316 >> 172.21.15.112:0/510493316 conn(0x7f6b8c079740 msgr2=0x7f6b8c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.864+0000 7f6b9228a640 1 -- 172.21.15.112:0/510493316 shutdown_connections 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.864+0000 7f6b9228a640 1 -- 172.21.15.112:0/510493316 wait complete. 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.865+0000 7f6b9228a640 1 Processor -- start 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.865+0000 7f6b9228a640 1 -- start start 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.865+0000 7f6b9228a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c102a80 0x7f6b8c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b9228a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b8c1083c0 0x7f6b8c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b9228a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108d10 0x7f6b8c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b9228a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c112980 con 0x7f6b8c102a80 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b9228a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c112800 con 0x7f6b8c1083c0 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b9228a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c112b00 con 0x7f6b8c108d10 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c102a80 0x7f6b8c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c102a80 0x7f6b8c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46364/0 (socket says 172.21.15.112:46364) 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b8bfff640 1 -- 172.21.15.112:0/3472831215 learned_addr learned my addr 172.21.15.112:0/3472831215 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b90827640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108d10 0x7f6b8c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b8b7fe640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b8c1083c0 0x7f6b8c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b90827640 1 -- 172.21.15.112:0/3472831215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b8c1083c0 msgr2=0x7f6b8c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b90827640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b8c1083c0 0x7f6b8c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b90827640 1 -- 172.21.15.112:0/3472831215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c102a80 msgr2=0x7f6b8c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b90827640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c102a80 0x7f6b8c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.866+0000 7f6b90827640 1 -- 172.21.15.112:0/3472831215 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c1a0430 con 0x7f6b8c108d10 2024-09-21T03:42:03.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.867+0000 7f6b90827640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108d10 0x7f6b8c19fd00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400e940 tx=0x7f6b8400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.867+0000 7f6b897fa640 1 -- 172.21.15.112:0/3472831215 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84004d20 con 0x7f6b8c108d10 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.867+0000 7f6b897fa640 1 -- 172.21.15.112:0/3472831215 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b84005020 con 0x7f6b8c108d10 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.867+0000 7f6b897fa640 1 -- 172.21.15.112:0/3472831215 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840116d0 con 0x7f6b8c108d10 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.867+0000 7f6b9228a640 1 -- 172.21.15.112:0/3472831215 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c1a0720 con 0x7f6b8c108d10 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.867+0000 7f6b9228a640 1 -- 172.21.15.112:0/3472831215 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c1a6e80 con 0x7f6b8c108d10 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.868+0000 7f6b9228a640 1 -- 172.21.15.112:0/3472831215 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b8c103d00 con 0x7f6b8c108d10 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.870+0000 7f6b897fa640 1 -- 172.21.15.112:0/3472831215 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f6b84005460 con 0x7f6b8c108d10 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.871+0000 7f6b897fa640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6b740b67d0 0x7f6b740b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.871+0000 7f6b8bfff640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6b740b67d0 0x7f6b740b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.871+0000 7f6b897fa640 1 -- 172.21.15.112:0/3472831215 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8470+0+0 (secure 0 0 0) 0x7f6b840e78d0 con 0x7f6b8c108d10 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.872+0000 7f6b8bfff640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6b740b67d0 0x7f6b740b8ce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009c90 tx=0x7f6b7c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:03.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:03.878+0000 7f6b897fa640 1 -- 172.21.15.112:0/3472831215 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6b84016030 con 0x7f6b8c108d10 2024-09-21T03:42:04.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.146+0000 7f6b9228a640 1 -- 172.21.15.112:0/3472831215 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b8c102e90 con 0x7f6b8c108d10 2024-09-21T03:42:04.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.147+0000 7f6b897fa640 1 -- 172.21.15.112:0/3472831215 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11930 (secure 0 0 0) 0x7f6b840b2330 con 0x7f6b8c108d10 2024-09-21T03:42:04.150 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:42:04.150 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":31,"btime":"2024-09-21T03:42:03:204501+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:42:02.316138+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:42:04.150 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 31 2024-09-21T03:42:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 -- 172.21.15.112:0/3472831215 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6b740b67d0 msgr2=0x7f6b740b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6b740b67d0 0x7f6b740b8ce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009c90 tx=0x7f6b7c005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 -- 172.21.15.112:0/3472831215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108d10 msgr2=0x7f6b8c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108d10 0x7f6b8c19fd00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400e940 tx=0x7f6b8400ee00 comp rx=0 tx=0).stop 2024-09-21T03:42:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 -- 172.21.15.112:0/3472831215 shutdown_connections 2024-09-21T03:42:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f6b740b67d0 0x7f6b740b8ce0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b8c108d10 0x7f6b8c19fd00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b8c1083c0 0x7f6b8c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 --2- 172.21.15.112:0/3472831215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b8c102a80 0x7f6b8c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 -- 172.21.15.112:0/3472831215 >> 172.21.15.112:0/3472831215 conn(0x7f6b8c079740 msgr2=0x7f6b8c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 -- 172.21.15.112:0/3472831215 shutdown_connections 2024-09-21T03:42:04.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:04.149+0000 7f6b9228a640 1 -- 172.21.15.112:0/3472831215 wait complete. 2024-09-21T03:42:04.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:04 smithi112 ceph-mon[31442]: pgmap v723: 97 pgs: 97 active+clean; 947 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-09-21T03:42:04.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:04 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:04.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:04 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3472831215' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:04.541 INFO:tasks.workunit.client.0.smithi112.stdout: 10 2223 68.67 MB/sec warmup 4 sec latency 89.516 ms 2024-09-21T03:42:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:04 smithi203 ceph-mon[34723]: pgmap v723: 97 pgs: 97 active+clean; 947 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-09-21T03:42:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:04 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:04 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3472831215' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:04 smithi187 ceph-mon[34603]: pgmap v723: 97 pgs: 97 active+clean; 947 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 211 op/s 2024-09-21T03:42:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:04 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:04 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3472831215' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:04.983 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:42:05.265 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:05.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:05 smithi112 ceph-mon[31442]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T03:42:05.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:05 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:05.541 INFO:tasks.workunit.client.0.smithi112.stdout: 10 2765 65.08 MB/sec warmup 5 sec latency 1052.734 ms 2024-09-21T03:42:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:05 smithi203 ceph-mon[34723]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T03:42:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:05 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:05 smithi187 ceph-mon[34603]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T03:42:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:05 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:05.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.779+0000 7f77eff77640 1 -- 172.21.15.112:0/1833102791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e8108cf0 msgr2=0x7f77e810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.779+0000 7f77eff77640 1 --2- 172.21.15.112:0/1833102791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e8108cf0 0x7f77e810f520 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009ca0 tx=0x7f77e002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.779+0000 7f77eff77640 1 -- 172.21.15.112:0/1833102791 shutdown_connections 2024-09-21T03:42:05.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.779+0000 7f77eff77640 1 --2- 172.21.15.112:0/1833102791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e8108cf0 0x7f77e810f520 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.779+0000 7f77eff77640 1 --2- 172.21.15.112:0/1833102791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e81083a0 0x7f77e81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.779+0000 7f77eff77640 1 --2- 172.21.15.112:0/1833102791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77e8102a60 0x7f77e8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.779+0000 7f77eff77640 1 -- 172.21.15.112:0/1833102791 >> 172.21.15.112:0/1833102791 conn(0x7f77e8079730 msgr2=0x7f77e8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.779+0000 7f77eff77640 1 -- 172.21.15.112:0/1833102791 shutdown_connections 2024-09-21T03:42:05.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.779+0000 7f77eff77640 1 -- 172.21.15.112:0/1833102791 wait complete. 2024-09-21T03:42:05.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.780+0000 7f77eff77640 1 Processor -- start 2024-09-21T03:42:05.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.780+0000 7f77eff77640 1 -- start start 2024-09-21T03:42:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77eff77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e8102a60 0x7f77e819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77eff77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e81083a0 0x7f77e819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77eff77640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77e8108cf0 0x7f77e819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77eff77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8113a80 con 0x7f77e81083a0 2024-09-21T03:42:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77eff77640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8113900 con 0x7f77e8102a60 2024-09-21T03:42:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77eff77640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8113c00 con 0x7f77e8108cf0 2024-09-21T03:42:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77ed512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e81083a0 0x7f77e819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77ed512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e81083a0 0x7f77e819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50896/0 (socket says 172.21.15.112:50896) 2024-09-21T03:42:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77ed512640 1 -- 172.21.15.112:0/3122691269 learned_addr learned my addr 172.21.15.112:0/3122691269 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:05.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77edd13640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e8102a60 0x7f77e819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77ee514640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77e8108cf0 0x7f77e819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77ed512640 1 -- 172.21.15.112:0/3122691269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77e8108cf0 msgr2=0x7f77e819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77ed512640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77e8108cf0 0x7f77e819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77ed512640 1 -- 172.21.15.112:0/3122691269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e8102a60 msgr2=0x7f77e819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77ed512640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e8102a60 0x7f77e819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77ed512640 1 -- 172.21.15.112:0/3122691269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e81a0400 con 0x7f77e81083a0 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.781+0000 7f77edd13640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e8102a60 0x7f77e819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.782+0000 7f77ed512640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e81083a0 0x7f77e819aa30 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f77d800d770 tx=0x7f77d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.782+0000 7f77deffd640 1 -- 172.21.15.112:0/3122691269 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8017070 con 0x7f77e81083a0 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.782+0000 7f77deffd640 1 -- 172.21.15.112:0/3122691269 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77d8009c30 con 0x7f77e81083a0 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.782+0000 7f77eff77640 1 -- 172.21.15.112:0/3122691269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e81a06f0 con 0x7f77e81083a0 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.782+0000 7f77deffd640 1 -- 172.21.15.112:0/3122691269 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d800f620 con 0x7f77e81083a0 2024-09-21T03:42:05.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.782+0000 7f77eff77640 1 -- 172.21.15.112:0/3122691269 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e81a6e50 con 0x7f77e81083a0 2024-09-21T03:42:05.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.786+0000 7f77f64eb740 1 Processor -- start 2024-09-21T03:42:05.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.786+0000 7f77f64eb740 1 -- start start 2024-09-21T03:42:05.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.786+0000 7f77deffd640 1 -- 172.21.15.112:0/3122691269 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f77d801a070 con 0x7f77e81083a0 2024-09-21T03:42:05.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.786+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cb66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.786+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cb6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.786+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cb7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.786+0000 7f77f64eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558b15c2cea0 con 0x558b158ee390 2024-09-21T03:42:05.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.786+0000 7f77f64eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558b15c1c2d0 con 0x558b1590f070 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.786+0000 7f77f64eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558b15c200e0 con 0x558b158e61b0 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77ee514640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cb66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77ee514640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cb66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41256/0 (socket says 172.21.15.112:41256) 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77ee514640 1 -- 172.21.15.112:0/3526301020 learned_addr learned my addr 172.21.15.112:0/3526301020 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77ed512640 1 --2- 172.21.15.112:0/3526301020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cb7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77edd13640 1 --2- 172.21.15.112:0/3526301020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cb6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77ee514640 1 -- 172.21.15.112:0/3526301020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 msgr2=0x558b15cb6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77ee514640 1 --2- 172.21.15.112:0/3526301020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cb6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77ee514640 1 -- 172.21.15.112:0/3526301020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 msgr2=0x558b15cb7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77ee514640 1 --2- 172.21.15.112:0/3526301020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cb7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77ee514640 1 -- 172.21.15.112:0/3526301020 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b1597a650 con 0x558b1590f070 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77edd13640 1 --2- 172.21.15.112:0/3526301020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cb6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77deffd640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77d40b6680 0x7f77d40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.787+0000 7f77ee514640 1 --2- 172.21.15.112:0/3526301020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cb66a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77e00098b0 tx=0x7f77e00098e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3e22c153c17f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.788+0000 7f77deffd640 1 -- 172.21.15.112:0/3122691269 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8514+0+0 (secure 0 0 0) 0x7f77d80e71b0 con 0x7f77e81083a0 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.788+0000 7f77edd13640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77d40b6680 0x7f77d40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.788+0000 7f77d3fff640 1 -- 172.21.15.112:0/3526301020 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0031cf0 con 0x558b1590f070 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.788+0000 7f77d3fff640 1 -- 172.21.15.112:0/3526301020 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77e0036030 con 0x558b1590f070 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.788+0000 7f77d3fff640 1 -- 172.21.15.112:0/3526301020 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e00042a0 con 0x558b1590f070 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.789+0000 7f77f64eb740 1 -- 172.21.15.112:0/3526301020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 msgr2=0x558b15cb66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.789+0000 7f77edd13640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77d40b6680 0x7f77d40b8b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x558b15cb5320 tx=0x7f77e400b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.789+0000 7f77f64eb740 1 --2- 172.21.15.112:0/3526301020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cb66a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77e00098b0 tx=0x7f77e00098e0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.789+0000 7f77f64eb740 1 -- 172.21.15.112:0/3526301020 shutdown_connections 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.789+0000 7f77f64eb740 1 --2- 172.21.15.112:0/3526301020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cb7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.789+0000 7f77f64eb740 1 --2- 172.21.15.112:0/3526301020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cb6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.789+0000 7f77f64eb740 1 --2- 172.21.15.112:0/3526301020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cb66a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.789+0000 7f77f64eb740 1 -- 172.21.15.112:0/3526301020 >> 172.21.15.112:0/3526301020 conn(0x558b15903460 msgr2=0x558b15cb4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.789+0000 7f77f64eb740 1 -- 172.21.15.112:0/3526301020 shutdown_connections 2024-09-21T03:42:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.789+0000 7f77f64eb740 1 -- 172.21.15.112:0/3526301020 wait complete. 2024-09-21T03:42:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.790+0000 7f77f64eb740 1 -- start start 2024-09-21T03:42:05.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.790+0000 7f77f64eb740 1 Processor -- start 2024-09-21T03:42:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.790+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cc6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.790+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cc6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.790+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cc6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77f64eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558b15a7d700 con 0x558b158ee390 2024-09-21T03:42:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77f64eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558b15bb7160 con 0x558b1590f070 2024-09-21T03:42:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77f64eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558b15cc8fa0 con 0x558b158e61b0 2024-09-21T03:42:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77f64eb740 10 client.0 _ll_register_callbacks cb 0x558b15bd09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77ed512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cc6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77ed512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cc6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50922/0 (socket says 172.21.15.112:50922) 2024-09-21T03:42:05.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77ed512640 1 -- 172.21.15.112:0/229691300 learned_addr learned my addr 172.21.15.112:0/229691300 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77edd13640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cc6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77ee514640 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cc6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77ed512640 1 -- 172.21.15.112:0/229691300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 msgr2=0x558b15cc6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77ed512640 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cc6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77ed512640 1 -- 172.21.15.112:0/229691300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 msgr2=0x558b15cc6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77ed512640 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cc6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77ed512640 1 -- 172.21.15.112:0/229691300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b15be4c90 con 0x558b158ee390 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77ed512640 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cc6660 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f77d80a9620 tx=0x7f77d80e4e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41eb836d9256505e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.791+0000 7f77d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77d17fa640 1 -- 172.21.15.112:0/229691300 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8021030 con 0x558b158ee390 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77d17fa640 1 -- 172.21.15.112:0/229691300 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8514+0+0 (secure 0 0 0) 0x7f77d80a8bb0 con 0x558b158ee390 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77f64eb740 1 -- 172.21.15.112:0/229691300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77e0036030 con 0x558b158ee390 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77d17fa640 1 -- 172.21.15.112:0/229691300 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f77d80ace20 con 0x558b158ee390 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77f64eb740 10 client.17643 fetch_fsmap learned FSMap version 32 2024-09-21T03:42:05.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77f64eb740 1 -- 172.21.15.112:0/229691300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558b15cd4d00 con 0x558b158ee390 2024-09-21T03:42:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77d17fa640 1 -- 172.21.15.112:0/229691300 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77e0036030 con 0x558b158ee390 2024-09-21T03:42:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77d17fa640 1 -- 172.21.15.112:0/229691300 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f77d80abd20 con 0x558b158ee390 2024-09-21T03:42:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77f64eb740 10 client.17643 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:42:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77f64eb740 10 client.17643 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:42:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77f64eb740 20 client.17643 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.792+0000 7f77f64eb740 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x558b158cd9f0 0x558b15cd8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.793+0000 7f77f64eb740 4 client.17643 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.793+0000 7f77f64eb740 1 -- 172.21.15.112:0/229691300 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b15c2edf0 con 0x558b158cd9f0 2024-09-21T03:42:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.793+0000 7f77edd13640 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x558b158cd9f0 0x558b15cd8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.793+0000 7f77d17fa640 10 client.17643 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:42:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.793+0000 7f77edd13640 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x558b158cd9f0 0x558b15cd8220 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33d28488b4ad230f server_cookie=4e4cfa75066b9bbc in_seq=0 out_seq=0 2024-09-21T03:42:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.794+0000 7f77d17fa640 1 -- 172.21.15.112:0/229691300 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b15c2edf0 con 0x558b158cd9f0 2024-09-21T03:42:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.794+0000 7f77d17fa640 10 client.17643 handle_command_reply: tid=0 2024-09-21T03:42:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.794+0000 7f77f64eb740 1 client.17643 shutdown 2024-09-21T03:42:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77f64eb740 1 -- 172.21.15.112:0/229691300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 msgr2=0x558b15cc6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77f64eb740 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cc6660 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f77d80a9620 tx=0x7f77d80e4e20 comp rx=0 tx=0).stop 2024-09-21T03:42:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77f64eb740 1 -- 172.21.15.112:0/229691300 shutdown_connections 2024-09-21T03:42:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77f64eb740 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x558b158cd9f0 0x558b15cd8220 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77f64eb740 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cc6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77f64eb740 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cc6660 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77d17fa640 0 client.17643 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:42:05.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77f64eb740 1 --2- 172.21.15.112:0/229691300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cc6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77f64eb740 1 -- 172.21.15.112:0/229691300 >> 172.21.15.112:0/229691300 conn(0x558b15903460 msgr2=0x558b15cb8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77f64eb740 1 -- 172.21.15.112:0/229691300 shutdown_connections 2024-09-21T03:42:05.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77f64eb740 1 -- 172.21.15.112:0/229691300 wait complete. 2024-09-21T03:42:05.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.795+0000 7f77f64eb740 20 client.17643 trim_cache size 0 max 16384 2024-09-21T03:42:05.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.810+0000 7f77f64eb740 1 Processor -- start 2024-09-21T03:42:05.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77f64eb740 1 -- start start 2024-09-21T03:42:05.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cc8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cb7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cc3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77f64eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81afe70 con 0x558b158ee390 2024-09-21T03:42:05.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77f64eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81ad6e0 con 0x558b1590f070 2024-09-21T03:42:05.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77f64eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81ab550 con 0x558b158e61b0 2024-09-21T03:42:05.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77ed512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cb7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77edd13640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cc8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77ed512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cb7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50938/0 (socket says 172.21.15.112:50938) 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77edd13640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cc8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:49512/0 (socket says 172.21.15.112:49512) 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77ed512640 1 -- 172.21.15.112:0/923927991 learned_addr learned my addr 172.21.15.112:0/923927991 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.811+0000 7f77ee514640 1 --2- 172.21.15.112:0/923927991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cc3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.812+0000 7f77ed512640 1 -- 172.21.15.112:0/923927991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 msgr2=0x558b15cc8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.812+0000 7f77ed512640 1 --2- 172.21.15.112:0/923927991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cc8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.812+0000 7f77ed512640 1 -- 172.21.15.112:0/923927991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 msgr2=0x558b15cc3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.812+0000 7f77ed512640 1 --2- 172.21.15.112:0/923927991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cc3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.812+0000 7f77ed512640 1 -- 172.21.15.112:0/923927991 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b15cb7490 con 0x558b158ee390 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.812+0000 7f77ee514640 1 --2- 172.21.15.112:0/923927991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cc3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.812+0000 7f77edd13640 1 --2- 172.21.15.112:0/923927991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cc8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.812+0000 7f77ed512640 1 --2- 172.21.15.112:0/923927991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cb7de0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f77d8012040 tx=0x7f77d80a94d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c904e7b5a19d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.812+0000 7f77d37fe640 1 -- 172.21.15.112:0/923927991 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8021070 con 0x558b158ee390 2024-09-21T03:42:05.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.812+0000 7f77d37fe640 1 -- 172.21.15.112:0/923927991 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77d80acb60 con 0x558b158ee390 2024-09-21T03:42:05.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.812+0000 7f77d37fe640 1 -- 172.21.15.112:0/923927991 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d80ac7c0 con 0x558b158ee390 2024-09-21T03:42:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.813+0000 7f77f64eb740 1 -- 172.21.15.112:0/923927991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 msgr2=0x558b15cb7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.813+0000 7f77f64eb740 1 --2- 172.21.15.112:0/923927991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cb7de0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f77d8012040 tx=0x7f77d80a94d0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.813+0000 7f77f64eb740 1 -- 172.21.15.112:0/923927991 shutdown_connections 2024-09-21T03:42:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.813+0000 7f77f64eb740 1 --2- 172.21.15.112:0/923927991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b1590f070 0x558b15cc3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.813+0000 7f77f64eb740 1 --2- 172.21.15.112:0/923927991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15cb7de0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.813+0000 7f77f64eb740 1 --2- 172.21.15.112:0/923927991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b158e61b0 0x558b15cc8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.813+0000 7f77f64eb740 1 -- 172.21.15.112:0/923927991 >> 172.21.15.112:0/923927991 conn(0x558b15903460 msgr2=0x558b15cb76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.813+0000 7f77f64eb740 1 -- 172.21.15.112:0/923927991 shutdown_connections 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.814+0000 7f77f64eb740 1 -- 172.21.15.112:0/923927991 wait complete. 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.814+0000 7f77f64eb740 1 -- start start 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.814+0000 7f77f64eb740 1 Processor -- start 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b158e61b0 0x558b15ccab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15ccb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77f64eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b1590f070 0x558b15d3bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77f64eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81abba0 con 0x558b158ee390 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77f64eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81ad010 con 0x558b158e61b0 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77f64eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81ac270 con 0x558b1590f070 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77f64eb740 10 client.0 _ll_register_callbacks cb 0x7f77d8010b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77ed512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15ccb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77ed512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15ccb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50946/0 (socket says 172.21.15.112:50946) 2024-09-21T03:42:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77ed512640 1 -- 172.21.15.112:0/1602216616 learned_addr learned my addr 172.21.15.112:0/1602216616 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77edd13640 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b158e61b0 0x558b15ccab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77ed512640 1 -- 172.21.15.112:0/1602216616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b1590f070 msgr2=0x558b15d3bb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:42:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77ed512640 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b1590f070 0x558b15d3bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77ed512640 1 -- 172.21.15.112:0/1602216616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b158e61b0 msgr2=0x558b15ccab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77ed512640 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b158e61b0 0x558b15ccab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77ed512640 1 -- 172.21.15.112:0/1602216616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b15d3c200 con 0x558b158ee390 2024-09-21T03:42:05.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.815+0000 7f77edd13640 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b158e61b0 0x558b15ccab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:05.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.816+0000 7f77ed512640 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15ccb150 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f77d80acfe0 tx=0x7f77d80b2ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b342dedc0961c4a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:05.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.816+0000 7f77d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:42:05.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.816+0000 7f77d17fa640 1 -- 172.21.15.112:0/1602216616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8021070 con 0x558b158ee390 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.816+0000 7f77f64eb740 1 -- 172.21.15.112:0/1602216616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77d80acb60 con 0x558b158ee390 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.816+0000 7f77d17fa640 1 -- 172.21.15.112:0/1602216616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8514+0+0 (secure 0 0 0) 0x7f77d80942f0 con 0x558b158ee390 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.816+0000 7f77d17fa640 1 -- 172.21.15.112:0/1602216616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f77d8014020 con 0x558b158ee390 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.816+0000 7f77f64eb740 10 client.17649 fetch_fsmap learned FSMap version 32 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.816+0000 7f77f64eb740 1 -- 172.21.15.112:0/1602216616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558b15d43520 con 0x558b158ee390 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.817+0000 7f77d17fa640 1 -- 172.21.15.112:0/1602216616 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77d80acb60 con 0x558b158ee390 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.817+0000 7f77d17fa640 1 -- 172.21.15.112:0/1602216616 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f77d8010460 con 0x558b158ee390 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.817+0000 7f77f64eb740 10 client.17649 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.817+0000 7f77f64eb740 10 client.17649 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.817+0000 7f77f64eb740 20 client.17649 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.817+0000 7f77f64eb740 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x558b158a2da0 0x558b15d45ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.817+0000 7f77f64eb740 4 client.17649 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.817+0000 7f77f64eb740 1 -- 172.21.15.112:0/1602216616 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b15c2a8a0 con 0x558b158a2da0 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.817+0000 7f77edd13640 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x558b158a2da0 0x558b15d45ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:05.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77d17fa640 10 client.17649 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77edd13640 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x558b158a2da0 0x558b15d45ac0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=849febf0abf8989a server_cookie=3dc35a8151c93096 in_seq=0 out_seq=0 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77d17fa640 1 -- 172.21.15.112:0/1602216616 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b15c2a8a0 con 0x558b158a2da0 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77d17fa640 10 client.17649 handle_command_reply: tid=0 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77f64eb740 1 client.17649 shutdown 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77f64eb740 1 -- 172.21.15.112:0/1602216616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 msgr2=0x558b15ccb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77f64eb740 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15ccb150 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f77d80acfe0 tx=0x7f77d80b2ea0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77f64eb740 1 -- 172.21.15.112:0/1602216616 shutdown_connections 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77f64eb740 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x558b158a2da0 0x558b15d45ac0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77f64eb740 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b1590f070 0x558b15d3bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77d17fa640 0 client.17649 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77f64eb740 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b158ee390 0x558b15ccb150 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77f64eb740 1 --2- 172.21.15.112:0/1602216616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b158e61b0 0x558b15ccab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77f64eb740 1 -- 172.21.15.112:0/1602216616 >> 172.21.15.112:0/1602216616 conn(0x558b15903460 msgr2=0x558b15cb8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77f64eb740 1 -- 172.21.15.112:0/1602216616 shutdown_connections 2024-09-21T03:42:05.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.818+0000 7f77f64eb740 1 -- 172.21.15.112:0/1602216616 wait complete. 2024-09-21T03:42:05.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.819+0000 7f77f64eb740 20 client.17649 trim_cache size 0 max 16384 2024-09-21T03:42:05.825 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:42:05.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.819+0000 7f77eff77640 1 -- 172.21.15.112:0/3122691269 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77d40b6680 msgr2=0x7f77d40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.819+0000 7f77eff77640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77d40b6680 0x7f77d40b8b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x558b15cb5320 tx=0x7f77e400b410 comp rx=0 tx=0).stop 2024-09-21T03:42:05.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.819+0000 7f77eff77640 1 -- 172.21.15.112:0/3122691269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e81083a0 msgr2=0x7f77e819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:05.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.819+0000 7f77eff77640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e81083a0 0x7f77e819aa30 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f77d800d770 tx=0x7f77d800dc30 comp rx=0 tx=0).stop 2024-09-21T03:42:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.820+0000 7f77eff77640 1 -- 172.21.15.112:0/3122691269 shutdown_connections 2024-09-21T03:42:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.820+0000 7f77eff77640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f77d40b6680 0x7f77d40b8b90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.820+0000 7f77eff77640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77e8108cf0 0x7f77e819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.820+0000 7f77eff77640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77e81083a0 0x7f77e819aa30 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.820+0000 7f77eff77640 1 --2- 172.21.15.112:0/3122691269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77e8102a60 0x7f77e819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.820+0000 7f77eff77640 1 -- 172.21.15.112:0/3122691269 >> 172.21.15.112:0/3122691269 conn(0x7f77e8079730 msgr2=0x7f77e81052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.820+0000 7f77eff77640 1 -- 172.21.15.112:0/3122691269 shutdown_connections 2024-09-21T03:42:05.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:05.820+0000 7f77eff77640 1 -- 172.21.15.112:0/3122691269 wait complete. 2024-09-21T03:42:06.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:06 smithi112 ceph-mon[31442]: pgmap v725: 97 pgs: 97 active+clean; 949 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 237 op/s 2024-09-21T03:42:06.541 INFO:tasks.workunit.client.0.smithi112.stdout: 10 3439 62.38 MB/sec warmup 6 sec latency 2052.763 ms 2024-09-21T03:42:06.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:06 smithi203 ceph-mon[34723]: pgmap v725: 97 pgs: 97 active+clean; 949 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 237 op/s 2024-09-21T03:42:06.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:06 smithi187 ceph-mon[34603]: pgmap v725: 97 pgs: 97 active+clean; 949 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 237 op/s 2024-09-21T03:42:06.764 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:42:07.042 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:07.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.530+0000 7f74f40de640 1 -- 172.21.15.112:0/1905923539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ec103b80 msgr2=0x7f74ec103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.530+0000 7f74f40de640 1 --2- 172.21.15.112:0/1905923539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ec103b80 0x7f74ec103f90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009900 tx=0x7f74e802a410 comp rx=0 tx=0).stop 2024-09-21T03:42:07.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.530+0000 7f74f40de640 1 -- 172.21.15.112:0/1905923539 shutdown_connections 2024-09-21T03:42:07.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.530+0000 7f74f40de640 1 --2- 172.21.15.112:0/1905923539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74ec109e10 0x7f74ec110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.530+0000 7f74f40de640 1 --2- 172.21.15.112:0/1905923539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74ec1094c0 0x7f74ec1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.530+0000 7f74f40de640 1 --2- 172.21.15.112:0/1905923539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ec103b80 0x7f74ec103f90 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.530+0000 7f74f40de640 1 -- 172.21.15.112:0/1905923539 >> 172.21.15.112:0/1905923539 conn(0x7f74ec0ff610 msgr2=0x7f74ec101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.530+0000 7f74f40de640 1 -- 172.21.15.112:0/1905923539 shutdown_connections 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.530+0000 7f74f40de640 1 -- 172.21.15.112:0/1905923539 wait complete. 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f40de640 1 Processor -- start 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f40de640 1 -- start start 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f40de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ec103b80 0x7f74ec19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f40de640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74ec1094c0 0x7f74ec19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f40de640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74ec109e10 0x7f74ec19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f40de640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec113ad0 con 0x7f74ec103b80 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f40de640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec113950 con 0x7f74ec109e10 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f40de640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec113c50 con 0x7f74ec1094c0 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f1e7a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ec103b80 0x7f74ec19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f1e7a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ec103b80 0x7f74ec19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50972/0 (socket says 172.21.15.112:50972) 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f1e7a640 1 -- 172.21.15.112:0/208516091 learned_addr learned my addr 172.21.15.112:0/208516091 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f1679640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74ec1094c0 0x7f74ec19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f267b640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74ec109e10 0x7f74ec19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f1e7a640 1 -- 172.21.15.112:0/208516091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74ec1094c0 msgr2=0x7f74ec19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f1e7a640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74ec1094c0 0x7f74ec19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f1e7a640 1 -- 172.21.15.112:0/208516091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74ec109e10 msgr2=0x7f74ec19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f1e7a640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74ec109e10 0x7f74ec19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f1e7a640 1 -- 172.21.15.112:0/208516091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ec1a0370 con 0x7f74ec103b80 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.531+0000 7f74f1679640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74ec1094c0 0x7f74ec19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.532+0000 7f74f1e7a640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ec103b80 0x7f74ec19a4c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f74e8005940 tx=0x7f74e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.532+0000 7f74e2ffd640 1 -- 172.21.15.112:0/208516091 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e803f030 con 0x7f74ec103b80 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.532+0000 7f74f40de640 1 -- 172.21.15.112:0/208516091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ec1a0600 con 0x7f74ec103b80 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.532+0000 7f74e2ffd640 1 -- 172.21.15.112:0/208516091 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e8003aa0 con 0x7f74ec103b80 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.532+0000 7f74e2ffd640 1 -- 172.21.15.112:0/208516091 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8035070 con 0x7f74ec103b80 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.532+0000 7f74f40de640 1 -- 172.21.15.112:0/208516091 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ec1a0b30 con 0x7f74ec103b80 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74fa652740 1 Processor -- start 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74fa652740 1 -- start start 2024-09-21T03:42:07.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74e2ffd640 1 -- 172.21.15.112:0/208516091 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f74e8042030 con 0x7f74ec103b80 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74fa652740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88749070 0x557e88af06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74fa652740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e887201b0 0x557e88af0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74fa652740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88728390 0x557e88af1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74fa652740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e88a66ea0 con 0x557e88749070 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74fa652740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557e88a562d0 con 0x557e88728390 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74fa652740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557e88a5a0e0 con 0x557e887201b0 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74f267b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88749070 0x557e88af06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74f267b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88749070 0x557e88af06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50988/0 (socket says 172.21.15.112:50988) 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74f267b640 1 -- 172.21.15.112:0/4150002575 learned_addr learned my addr 172.21.15.112:0/4150002575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74f1e7a640 1 --2- 172.21.15.112:0/4150002575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e887201b0 0x557e88af0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.535+0000 7f74f1679640 1 --2- 172.21.15.112:0/4150002575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88728390 0x557e88af1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74f267b640 1 -- 172.21.15.112:0/4150002575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e887201b0 msgr2=0x557e88af0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74f267b640 1 --2- 172.21.15.112:0/4150002575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e887201b0 0x557e88af0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74f267b640 1 -- 172.21.15.112:0/4150002575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88728390 msgr2=0x557e88af1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74f267b640 1 --2- 172.21.15.112:0/4150002575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88728390 0x557e88af1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74f267b640 1 -- 172.21.15.112:0/4150002575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e887b4650 con 0x557e88749070 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74f1e7a640 1 --2- 172.21.15.112:0/4150002575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e887201b0 0x557e88af0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74e2ffd640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74d80b66b0 0x7f74d80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74f267b640 1 --2- 172.21.15.112:0/4150002575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88749070 0x557e88af06a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f74e4007f40 tx=0x7f74e400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f7fd24763b3e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74f1e7a640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74d80b66b0 0x7f74d80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74d7fff640 1 -- 172.21.15.112:0/4150002575 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4010900 con 0x557e88749070 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74d7fff640 1 -- 172.21.15.112:0/4150002575 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e40183f0 con 0x557e88749070 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74e2ffd640 1 -- 172.21.15.112:0/208516091 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8514+0+0 (secure 0 0 0) 0x7f74e8107af0 con 0x7f74ec103b80 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.536+0000 7f74d7fff640 1 -- 172.21.15.112:0/4150002575 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4017600 con 0x557e88749070 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.537+0000 7f74fa652740 1 -- 172.21.15.112:0/4150002575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88749070 msgr2=0x557e88af06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.537+0000 7f74f1e7a640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74d80b66b0 0x7f74d80b8bc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f74e81052f0 tx=0x7f74e810b850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.537+0000 7f74fa652740 1 --2- 172.21.15.112:0/4150002575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88749070 0x557e88af06a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f74e4007f40 tx=0x7f74e400f850 comp rx=0 tx=0).stop 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.537+0000 7f74fa652740 1 -- 172.21.15.112:0/4150002575 shutdown_connections 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.537+0000 7f74fa652740 1 --2- 172.21.15.112:0/4150002575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88728390 0x557e88af1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.537+0000 7f74fa652740 1 --2- 172.21.15.112:0/4150002575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e887201b0 0x557e88af0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.537+0000 7f74fa652740 1 --2- 172.21.15.112:0/4150002575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88749070 0x557e88af06a0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.537+0000 7f74fa652740 1 -- 172.21.15.112:0/4150002575 >> 172.21.15.112:0/4150002575 conn(0x557e8873d460 msgr2=0x557e88aee590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.537+0000 7f74fa652740 1 -- 172.21.15.112:0/4150002575 shutdown_connections 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.537+0000 7f74fa652740 1 -- 172.21.15.112:0/4150002575 wait complete. 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74fa652740 1 -- start start 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74fa652740 1 Processor -- start 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74fa652740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e887201b0 0x557e88afbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74fa652740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88728390 0x557e88afc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74fa652740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88749070 0x557e88afc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74fa652740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e888b7700 con 0x557e88728390 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74fa652740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557e889f1160 con 0x557e887201b0 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74fa652740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557e88b03290 con 0x557e88749070 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74fa652740 10 client.0 _ll_register_callbacks cb 0x557e88a0a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74f1679640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88728390 0x557e88afc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74f1679640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88728390 0x557e88afc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50992/0 (socket says 172.21.15.112:50992) 2024-09-21T03:42:07.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.538+0000 7f74f1679640 1 -- 172.21.15.112:0/2621518056 learned_addr learned my addr 172.21.15.112:0/2621518056 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74f267b640 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88749070 0x557e88afc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74f267b640 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e887201b0 0x557e88afbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74f1679640 1 -- 172.21.15.112:0/2621518056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88749070 msgr2=0x557e88afc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74f1679640 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88749070 0x557e88afc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74f1679640 1 -- 172.21.15.112:0/2621518056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e887201b0 msgr2=0x557e88afbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74f1679640 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e887201b0 0x557e88afbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74f1679640 1 -- 172.21.15.112:0/2621518056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e88a1ec90 con 0x557e88728390 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74f1679640 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88728390 0x557e88afc1e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0129e0 tx=0x7f74dc012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4eb33dda78b23e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74d57fa640 1 -- 172.21.15.112:0/2621518056 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc004050 con 0x557e88728390 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74d57fa640 1 -- 172.21.15.112:0/2621518056 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8514+0+0 (secure 0 0 0) 0x7f74dc0041f0 con 0x557e88728390 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74fa652740 1 -- 172.21.15.112:0/2621518056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e40183f0 con 0x557e88728390 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74d57fa640 1 -- 172.21.15.112:0/2621518056 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74dc0043f0 con 0x557e88728390 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74fa652740 10 client.17661 fetch_fsmap learned FSMap version 32 2024-09-21T03:42:07.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.539+0000 7f74fa652740 1 -- 172.21.15.112:0/2621518056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557e88b0ef40 con 0x557e88728390 2024-09-21T03:42:07.549 INFO:tasks.workunit.client.0.smithi112.stdout: 10 3652 55.10 MB/sec warmup 7 sec latency 3052.795 ms 2024-09-21T03:42:07.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.540+0000 7f74d57fa640 1 -- 172.21.15.112:0/2621518056 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e40183f0 con 0x557e88728390 2024-09-21T03:42:07.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.540+0000 7f74d57fa640 1 -- 172.21.15.112:0/2621518056 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f74dc003c40 con 0x557e88728390 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.540+0000 7f74fa652740 10 client.17661 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.540+0000 7f74fa652740 10 client.17661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.540+0000 7f74fa652740 20 client.17661 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.540+0000 7f74fa652740 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557e887079f0 0x557e88b12450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.540+0000 7f74fa652740 4 client.17661 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.540+0000 7f74fa652740 1 -- 172.21.15.112:0/2621518056 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e88a68df0 con 0x557e887079f0 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.540+0000 7f74f267b640 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557e887079f0 0x557e88b12450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.541+0000 7f74d57fa640 10 client.17661 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.541+0000 7f74f267b640 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557e887079f0 0x557e88b12450 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ee010dc55677fa0 server_cookie=81cc1da9f830a02d in_seq=0 out_seq=0 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.541+0000 7f74d57fa640 1 -- 172.21.15.112:0/2621518056 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557e88a68df0 con 0x557e887079f0 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.541+0000 7f74d57fa640 10 client.17661 handle_command_reply: tid=0 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 1 client.17661 shutdown 2024-09-21T03:42:07.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 1 -- 172.21.15.112:0/2621518056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88728390 msgr2=0x557e88afc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88728390 0x557e88afc1e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0129e0 tx=0x7f74dc012ea0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 1 -- 172.21.15.112:0/2621518056 shutdown_connections 2024-09-21T03:42:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557e887079f0 0x557e88b12450 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88749070 0x557e88afc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e88728390 0x557e88afc1e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 1 --2- 172.21.15.112:0/2621518056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e887201b0 0x557e88afbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74d57fa640 0 client.17661 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:42:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 1 -- 172.21.15.112:0/2621518056 >> 172.21.15.112:0/2621518056 conn(0x557e8873d460 msgr2=0x557e88af3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 1 -- 172.21.15.112:0/2621518056 shutdown_connections 2024-09-21T03:42:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 1 -- 172.21.15.112:0/2621518056 wait complete. 2024-09-21T03:42:07.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.542+0000 7f74fa652740 20 client.17661 trim_cache size 0 max 16384 2024-09-21T03:42:07.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.558+0000 7f74fa652740 1 Processor -- start 2024-09-21T03:42:07.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.558+0000 7f74fa652740 1 -- start start 2024-09-21T03:42:07.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.558+0000 7f74fa652740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 0x557e88afd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.558+0000 7f74fa652740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 0x557e88af8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.558+0000 7f74fa652740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88749070 0x557e88af93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.558+0000 7f74fa652740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1afde0 con 0x557e887201b0 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.558+0000 7f74fa652740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1ad650 con 0x557e88749070 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.558+0000 7f74fa652740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1ab4c0 con 0x557e88728390 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.558+0000 7f74f267b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 0x557e88afd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.558+0000 7f74f267b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 0x557e88afd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51000/0 (socket says 172.21.15.112:51000) 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.559+0000 7f74f267b640 1 -- 172.21.15.112:0/1692053779 learned_addr learned my addr 172.21.15.112:0/1692053779 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.559+0000 7f74f1679640 1 --2- 172.21.15.112:0/1692053779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 0x557e88af8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.559+0000 7f74f267b640 1 -- 172.21.15.112:0/1692053779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 msgr2=0x557e88af8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.559+0000 7f74f267b640 1 --2- 172.21.15.112:0/1692053779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 0x557e88af8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.559+0000 7f74f267b640 1 -- 172.21.15.112:0/1692053779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88749070 msgr2=0x557e88af93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.559+0000 7f74f267b640 1 --2- 172.21.15.112:0/1692053779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88749070 0x557e88af93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.559+0000 7f74f267b640 1 -- 172.21.15.112:0/1692053779 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e88aee680 con 0x557e887201b0 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.559+0000 7f74f267b640 1 --2- 172.21.15.112:0/1692053779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 0x557e88afd8d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f74e400fd50 tx=0x7f74e4018de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526ce3430bc65583 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.559+0000 7f74d77fe640 1 -- 172.21.15.112:0/1692053779 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4033070 con 0x557e887201b0 2024-09-21T03:42:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.559+0000 7f74d77fe640 1 -- 172.21.15.112:0/1692053779 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e4007940 con 0x557e887201b0 2024-09-21T03:42:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.559+0000 7f74d77fe640 1 -- 172.21.15.112:0/1692053779 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4027450 con 0x557e887201b0 2024-09-21T03:42:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.560+0000 7f74fa652740 1 -- 172.21.15.112:0/1692053779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 msgr2=0x557e88afd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.560+0000 7f74fa652740 1 --2- 172.21.15.112:0/1692053779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 0x557e88afd8d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f74e400fd50 tx=0x7f74e4018de0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.560+0000 7f74fa652740 1 -- 172.21.15.112:0/1692053779 shutdown_connections 2024-09-21T03:42:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.560+0000 7f74fa652740 1 --2- 172.21.15.112:0/1692053779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88749070 0x557e88af93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.560+0000 7f74fa652740 1 --2- 172.21.15.112:0/1692053779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 0x557e88af8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.560+0000 7f74fa652740 1 --2- 172.21.15.112:0/1692053779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 0x557e88afd8d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.560+0000 7f74fa652740 1 -- 172.21.15.112:0/1692053779 >> 172.21.15.112:0/1692053779 conn(0x557e8873d460 msgr2=0x557e88afd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.560+0000 7f74fa652740 1 -- 172.21.15.112:0/1692053779 shutdown_connections 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.560+0000 7f74fa652740 1 -- 172.21.15.112:0/1692053779 wait complete. 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.561+0000 7f74fa652740 1 -- start start 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.561+0000 7f74fa652740 1 Processor -- start 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74fa652740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 0x557e88b02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74fa652740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 0x557e88b08560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74fa652740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88749070 0x557e88b08c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74f1679640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 0x557e88b08560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74f1679640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 0x557e88b08560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:49576/0 (socket says 172.21.15.112:49576) 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74f1679640 1 -- 172.21.15.112:0/636869187 learned_addr learned my addr 172.21.15.112:0/636869187 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74fa652740 1 -- 172.21.15.112:0/636869187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1abb10 con 0x557e887201b0 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74fa652740 1 -- 172.21.15.112:0/636869187 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1acf80 con 0x557e88749070 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74fa652740 1 -- 172.21.15.112:0/636869187 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1ac1e0 con 0x557e88728390 2024-09-21T03:42:07.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74f267b640 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88749070 0x557e88b08c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74f267b640 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 0x557e88b02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74f1679640 1 -- 172.21.15.112:0/636869187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88749070 msgr2=0x557e88b08c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:07.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74f1679640 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88749070 0x557e88b08c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74f1679640 1 -- 172.21.15.112:0/636869187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 msgr2=0x557e88b02b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:07.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74f1679640 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 0x557e88b02b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74f1679640 1 -- 172.21.15.112:0/636869187 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e88afffd0 con 0x557e88728390 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.562+0000 7f74f267b640 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 0x557e88b02b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.563+0000 7f74fa652740 10 client.0 _ll_register_callbacks cb 0x7f74dc009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.563+0000 7f74f1679640 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 0x557e88b08560 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00a010 tx=0x7f74dc00ff60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d42337b557c7a75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.563+0000 7f74fa652740 1 -- 172.21.15.112:0/636869187 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e4007940 con 0x557e88728390 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.563+0000 7f74d57fa640 10 client.27223 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.563+0000 7f74d57fa640 1 -- 172.21.15.112:0/636869187 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc029040 con 0x557e88728390 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.563+0000 7f74d57fa640 1 -- 172.21.15.112:0/636869187 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8514+0+0 (secure 0 0 0) 0x7f74dc005180 con 0x557e88728390 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.563+0000 7f74d57fa640 1 -- 172.21.15.112:0/636869187 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74dc0053c0 con 0x557e88728390 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.563+0000 7f74fa652740 10 client.27223 fetch_fsmap learned FSMap version 32 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.563+0000 7f74fa652740 1 -- 172.21.15.112:0/636869187 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({fsmap=32}) -- 0x557e88b81760 con 0x557e88728390 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.564+0000 7f74d57fa640 1 -- 172.21.15.112:0/636869187 <== mon.1 v2:172.21.15.203:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f74dc0106f0 con 0x557e88728390 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.564+0000 7f74fa652740 10 client.27223 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:42:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.564+0000 7f74fa652740 10 client.27223 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.564+0000 7f74fa652740 20 client.27223 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.564+0000 7f74fa652740 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557e886dcda0 0x557e88b83e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.564+0000 7f74fa652740 4 client.27223 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.564+0000 7f74fa652740 1 -- 172.21.15.112:0/636869187 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f74e4007940 con 0x557e886dcda0 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.565+0000 7f74f267b640 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557e886dcda0 0x557e88b83e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.565+0000 7f74d57fa640 10 client.27223 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.565+0000 7f74f267b640 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557e886dcda0 0x557e88b83e40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0448b9547413fd6 server_cookie=a2b3bafb00a531a3 in_seq=0 out_seq=0 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.565+0000 7f74d57fa640 1 -- 172.21.15.112:0/636869187 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f74e4007940 con 0x557e886dcda0 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.565+0000 7f74d57fa640 10 client.27223 handle_command_reply: tid=0 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.565+0000 7f74fa652740 1 client.27223 shutdown 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74fa652740 1 -- 172.21.15.112:0/636869187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 msgr2=0x557e88b08560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74fa652740 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 0x557e88b08560 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00a010 tx=0x7f74dc00ff60 comp rx=0 tx=0).stop 2024-09-21T03:42:07.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74fa652740 1 -- 172.21.15.112:0/636869187 shutdown_connections 2024-09-21T03:42:07.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74fa652740 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x557e886dcda0 0x557e88b83e40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74fa652740 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e88749070 0x557e88b08c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74fa652740 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557e88728390 0x557e88b08560 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74fa652740 1 --2- 172.21.15.112:0/636869187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e887201b0 0x557e88b02b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74fa652740 1 -- 172.21.15.112:0/636869187 >> 172.21.15.112:0/636869187 conn(0x557e8873d460 msgr2=0x557e88afd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74d57fa640 0 client.27223 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:42:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74fa652740 1 -- 172.21.15.112:0/636869187 shutdown_connections 2024-09-21T03:42:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74fa652740 1 -- 172.21.15.112:0/636869187 wait complete. 2024-09-21T03:42:07.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.566+0000 7f74fa652740 20 client.27223 trim_cache size 0 max 16384 2024-09-21T03:42:07.571 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:42:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 -- 172.21.15.112:0/208516091 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74d80b66b0 msgr2=0x7f74d80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74d80b66b0 0x7f74d80b8bc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f74e81052f0 tx=0x7f74e810b850 comp rx=0 tx=0).stop 2024-09-21T03:42:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 -- 172.21.15.112:0/208516091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ec103b80 msgr2=0x7f74ec19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ec103b80 0x7f74ec19a4c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f74e8005940 tx=0x7f74e8004270 comp rx=0 tx=0).stop 2024-09-21T03:42:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 -- 172.21.15.112:0/208516091 shutdown_connections 2024-09-21T03:42:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f74d80b66b0 0x7f74d80b8bc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74ec109e10 0x7f74ec19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74ec1094c0 0x7f74ec19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 --2- 172.21.15.112:0/208516091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74ec103b80 0x7f74ec19a4c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 -- 172.21.15.112:0/208516091 >> 172.21.15.112:0/208516091 conn(0x7f74ec0ff610 msgr2=0x7f74ec101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:07.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 -- 172.21.15.112:0/208516091 shutdown_connections 2024-09-21T03:42:07.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:07.567+0000 7f74f40de640 1 -- 172.21.15.112:0/208516091 wait complete. 2024-09-21T03:42:08.365 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:42:08.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:08 smithi112 ceph-mon[31442]: pgmap v726: 97 pgs: 97 active+clean; 965 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 309 op/s 2024-09-21T03:42:08.541 INFO:tasks.workunit.client.0.smithi112.stdout: 10 3837 48.35 MB/sec warmup 8 sec latency 4052.825 ms 2024-09-21T03:42:08.621 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:08.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:08 smithi203 ceph-mon[34723]: pgmap v726: 97 pgs: 97 active+clean; 965 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 309 op/s 2024-09-21T03:42:08.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:08 smithi187 ceph-mon[34603]: pgmap v726: 97 pgs: 97 active+clean; 965 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 309 op/s 2024-09-21T03:42:09.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.123+0000 7fd2bf65a640 1 -- 172.21.15.112:0/389907914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b81052d0 msgr2=0x7fd2b810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.123+0000 7fd2bf65a640 1 --2- 172.21.15.112:0/389907914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b81052d0 0x7fd2b810d910 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009900 tx=0x7fd2ac02a410 comp rx=0 tx=0).stop 2024-09-21T03:42:09.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.123+0000 7fd2bf65a640 1 -- 172.21.15.112:0/389907914 shutdown_connections 2024-09-21T03:42:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.123+0000 7fd2bf65a640 1 --2- 172.21.15.112:0/389907914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b810de50 0x7fd2b81102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.123+0000 7fd2bf65a640 1 --2- 172.21.15.112:0/389907914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b81052d0 0x7fd2b810d910 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.123+0000 7fd2bf65a640 1 --2- 172.21.15.112:0/389907914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2b81048a0 0x7fd2b8104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.123+0000 7fd2bf65a640 1 -- 172.21.15.112:0/389907914 >> 172.21.15.112:0/389907914 conn(0x7fd2b8079730 msgr2=0x7fd2b8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.123+0000 7fd2bf65a640 1 -- 172.21.15.112:0/389907914 shutdown_connections 2024-09-21T03:42:09.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.123+0000 7fd2bf65a640 1 -- 172.21.15.112:0/389907914 wait complete. 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.124+0000 7fd2bf65a640 1 Processor -- start 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.124+0000 7fd2bf65a640 1 -- start start 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.124+0000 7fd2bf65a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b81048a0 0x7fd2b819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.124+0000 7fd2bf65a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2b81052d0 0x7fd2b819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.124+0000 7fd2bf65a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b810de50 0x7fd2b819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.124+0000 7fd2bd3f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b81048a0 0x7fd2b819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.124+0000 7fd2bd3f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b81048a0 0x7fd2b819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51032/0 (socket says 172.21.15.112:51032) 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.124+0000 7fd2bd3f6640 1 -- 172.21.15.112:0/2494289627 learned_addr learned my addr 172.21.15.112:0/2494289627 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.124+0000 7fd2bf65a640 1 -- 172.21.15.112:0/2494289627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b8113a60 con 0x7fd2b81048a0 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.124+0000 7fd2bf65a640 1 -- 172.21.15.112:0/2494289627 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b81138e0 con 0x7fd2b81052d0 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2b81052d0 0x7fd2b819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bf65a640 1 -- 172.21.15.112:0/2494289627 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b8113be0 con 0x7fd2b810de50 2024-09-21T03:42:09.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bdbf7640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b810de50 0x7fd2b819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bd3f6640 1 -- 172.21.15.112:0/2494289627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b810de50 msgr2=0x7fd2b819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b810de50 0x7fd2b819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bd3f6640 1 -- 172.21.15.112:0/2494289627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2b81052d0 msgr2=0x7fd2b819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2b81052d0 0x7fd2b819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bd3f6640 1 -- 172.21.15.112:0/2494289627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b81a03e0 con 0x7fd2b81048a0 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2b81052d0 0x7fd2b819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b81048a0 0x7fd2b819a470 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800d770 tx=0x7fd2a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2b67fc640 1 -- 172.21.15.112:0/2494289627 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8017070 con 0x7fd2b81048a0 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2b67fc640 1 -- 172.21.15.112:0/2494289627 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2a8009c30 con 0x7fd2b81048a0 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2b67fc640 1 -- 172.21.15.112:0/2494289627 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8003cc0 con 0x7fd2b81048a0 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bf65a640 1 -- 172.21.15.112:0/2494289627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b81a06d0 con 0x7fd2b81048a0 2024-09-21T03:42:09.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.125+0000 7fd2bf65a640 1 -- 172.21.15.112:0/2494289627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b81a6e30 con 0x7fd2b81048a0 2024-09-21T03:42:09.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.128+0000 7fd2b67fc640 1 -- 172.21.15.112:0/2494289627 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fd2a8010460 con 0x7fd2b81048a0 2024-09-21T03:42:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2c5bce740 1 Processor -- start 2024-09-21T03:42:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2c5bce740 1 -- start start 2024-09-21T03:42:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2b67fc640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd2a00b6680 0x7fd2a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951e01070 0x5609521a82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521a8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951de0390 0x5609521a8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56095211eea0 con 0x560951de0390 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56095210e2d0 con 0x560951dd81b0 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5609521120e0 con 0x560951e01070 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2bdbf7640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd2a00b6680 0x7fd2a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2b67fc640 1 -- 172.21.15.112:0/2494289627 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8514+0+0 (secure 0 0 0) 0x7fd2a80e7060 con 0x7fd2b81048a0 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2bdbf7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951de0390 0x5609521a8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2bdbf7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951de0390 0x5609521a8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51036/0 (socket says 172.21.15.112:51036) 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2bdbf7640 1 -- 172.21.15.112:0/3968534740 learned_addr learned my addr 172.21.15.112:0/3968534740 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/3968534740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951e01070 0x5609521a82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.129+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/3968534740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521a8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.130+0000 7fd2bdbf7640 1 -- 172.21.15.112:0/3968534740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951e01070 msgr2=0x5609521a82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:09.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.130+0000 7fd2bdbf7640 1 --2- 172.21.15.112:0/3968534740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951e01070 0x5609521a82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.130+0000 7fd2bdbf7640 1 -- 172.21.15.112:0/3968534740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 msgr2=0x5609521a8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.130+0000 7fd2bdbf7640 1 --2- 172.21.15.112:0/3968534740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521a8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.130+0000 7fd2bdbf7640 1 -- 172.21.15.112:0/3968534740 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560951e6c650 con 0x560951de0390 2024-09-21T03:42:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.130+0000 7fd2bdbf7640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd2a00b6680 0x7fd2a00b8b90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0011d00 tx=0x7fd2b0011440 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.130+0000 7fd2bdbf7640 1 --2- 172.21.15.112:0/3968534740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951de0390 0x5609521a8d40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0011670 tx=0x7fd2b001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa899e6cd811745 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.130+0000 7fd29f7fe640 1 -- 172.21.15.112:0/3968534740 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0024070 con 0x560951de0390 2024-09-21T03:42:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.130+0000 7fd29f7fe640 1 -- 172.21.15.112:0/3968534740 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2b0011df0 con 0x560951de0390 2024-09-21T03:42:09.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.130+0000 7fd29f7fe640 1 -- 172.21.15.112:0/3968534740 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001fd80 con 0x560951de0390 2024-09-21T03:42:09.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.131+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3968534740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951de0390 msgr2=0x5609521a8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.131+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/3968534740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951de0390 0x5609521a8d40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0011670 tx=0x7fd2b001d000 comp rx=0 tx=0).stop 2024-09-21T03:42:09.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.131+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3968534740 shutdown_connections 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.131+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/3968534740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951de0390 0x5609521a8d40 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.131+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/3968534740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521a8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.131+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/3968534740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951e01070 0x5609521a82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.131+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3968534740 >> 172.21.15.112:0/3968534740 conn(0x560951df5460 msgr2=0x5609521a61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.131+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3968534740 shutdown_connections 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.131+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3968534740 wait complete. 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.131+0000 7fd2c5bce740 1 -- start start 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2c5bce740 1 Processor -- start 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951e01070 0x5609521b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560951f6f700 con 0x560951e01070 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5609520a9160 con 0x560951dd81b0 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5609521b6a40 con 0x560951de0390 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2bdbf7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951e01070 0x5609521b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2c5bce740 10 client.0 _ll_register_callbacks cb 0x5609520c29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2bdbf7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951e01070 0x5609521b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51040/0 (socket says 172.21.15.112:51040) 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2bdbf7640 1 -- 172.21.15.112:0/3897768389 learned_addr learned my addr 172.21.15.112:0/3897768389 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.132+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd2bcbf5640 1 -- 172.21.15.112:0/3897768389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 msgr2=0x5609521b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd2bcbf5640 1 -- 172.21.15.112:0/3897768389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951e01070 msgr2=0x5609521b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951e01070 0x5609521b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd2bcbf5640 1 -- 172.21.15.112:0/3897768389 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609520d6c90 con 0x560951de0390 2024-09-21T03:42:09.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd2bdbf7640 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951e01070 0x5609521b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b3e70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac006010 tx=0x7fd2ac031810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d871c2dcc90b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd29cff9640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd29cff9640 1 -- 172.21.15.112:0/3897768389 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac035040 con 0x560951de0390 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3897768389 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2b0011df0 con 0x560951de0390 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd29cff9640 1 -- 172.21.15.112:0/3897768389 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8514+0+0 (secure 0 0 0) 0x7fd2ac004240 con 0x560951de0390 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd29cff9640 1 -- 172.21.15.112:0/3897768389 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd2ac004440 con 0x560951de0390 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd2c5bce740 10 client.27235 fetch_fsmap learned FSMap version 32 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.133+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3897768389 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5609521c2730 con 0x560951de0390 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.134+0000 7fd29cff9640 1 -- 172.21.15.112:0/3897768389 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2b0011df0 con 0x560951de0390 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.134+0000 7fd29cff9640 1 -- 172.21.15.112:0/3897768389 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd2ac004c70 con 0x560951de0390 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.134+0000 7fd2c5bce740 10 client.27235 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.134+0000 7fd2c5bce740 10 client.27235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.134+0000 7fd2c5bce740 20 client.27235 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.134+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560951dbf9f0 0x5609521c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.134+0000 7fd2c5bce740 4 client.27235 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:09.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.134+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3897768389 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560952120df0 con 0x560951dbf9f0 2024-09-21T03:42:09.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.134+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560951dbf9f0 0x5609521c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.134+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560951dbf9f0 0x5609521c5b50 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8135c5e7ddfc869 server_cookie=fb16de3948577cd0 in_seq=0 out_seq=0 2024-09-21T03:42:09.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.134+0000 7fd29cff9640 10 client.27235 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.138+0000 7fd29cff9640 1 -- 172.21.15.112:0/3897768389 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560952120df0 con 0x560951dbf9f0 2024-09-21T03:42:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.138+0000 7fd29cff9640 10 client.27235 handle_command_reply: tid=0 2024-09-21T03:42:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.138+0000 7fd2c5bce740 1 client.27235 shutdown 2024-09-21T03:42:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.138+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3897768389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 msgr2=0x5609521b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.138+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b3e70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac006010 tx=0x7fd2ac031810 comp rx=0 tx=0).stop 2024-09-21T03:42:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.138+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3897768389 shutdown_connections 2024-09-21T03:42:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.138+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560951dbf9f0 0x5609521c5b50 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.138+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951e01070 0x5609521b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.138+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b3e70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.138+0000 7fd29cff9640 0 client.27235 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:09.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.138+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/3897768389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.139+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3897768389 >> 172.21.15.112:0/3897768389 conn(0x560951df5460 msgr2=0x5609521ab450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:09.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.139+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3897768389 shutdown_connections 2024-09-21T03:42:09.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.139+0000 7fd2c5bce740 1 -- 172.21.15.112:0/3897768389 wait complete. 2024-09-21T03:42:09.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.139+0000 7fd2c5bce740 20 client.27235 trim_cache size 0 max 16384 2024-09-21T03:42:09.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2c5bce740 1 Processor -- start 2024-09-21T03:42:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2c5bce740 1 -- start start 2024-09-21T03:42:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dbf9f0 0x5609521af110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dd81b0 0x5609521b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b81afe30 con 0x560951dd81b0 2024-09-21T03:42:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b81ad6a0 con 0x560951dbf9f0 2024-09-21T03:42:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b81ab510 con 0x560951de0390 2024-09-21T03:42:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2bcbf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dd81b0 0x5609521b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2bcbf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dd81b0 0x5609521b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51050/0 (socket says 172.21.15.112:51050) 2024-09-21T03:42:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2bcbf5640 1 -- 172.21.15.112:0/1789897765 learned_addr learned my addr 172.21.15.112:0/1789897765 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.153+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/1789897765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dbf9f0 0x5609521af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.154+0000 7fd2bdbf7640 1 --2- 172.21.15.112:0/1789897765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.154+0000 7fd2bcbf5640 1 -- 172.21.15.112:0/1789897765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 msgr2=0x5609521b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.154+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/1789897765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.154+0000 7fd2bcbf5640 1 -- 172.21.15.112:0/1789897765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dbf9f0 msgr2=0x5609521af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.154+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/1789897765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dbf9f0 0x5609521af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.154+0000 7fd2bcbf5640 1 -- 172.21.15.112:0/1789897765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609521a0460 con 0x560951dd81b0 2024-09-21T03:42:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.154+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/1789897765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dbf9f0 0x5609521af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.154+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/1789897765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dd81b0 0x5609521b5560 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0046c0 tx=0x7fd2ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39ca35b92e1d060 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:09.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.154+0000 7fd29effd640 1 -- 172.21.15.112:0/1789897765 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac035030 con 0x560951dd81b0 2024-09-21T03:42:09.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.154+0000 7fd29effd640 1 -- 172.21.15.112:0/1789897765 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2ac054070 con 0x560951dd81b0 2024-09-21T03:42:09.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.154+0000 7fd29effd640 1 -- 172.21.15.112:0/1789897765 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac004d70 con 0x560951dd81b0 2024-09-21T03:42:09.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.155+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1789897765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dd81b0 msgr2=0x5609521b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.155+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/1789897765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dd81b0 0x5609521b5560 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0046c0 tx=0x7fd2ac000bc0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.155+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1789897765 shutdown_connections 2024-09-21T03:42:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.155+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/1789897765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.155+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/1789897765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dd81b0 0x5609521b5560 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.155+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/1789897765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dbf9f0 0x5609521af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.155+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1789897765 >> 172.21.15.112:0/1789897765 conn(0x560951df5460 msgr2=0x5609521b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.155+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1789897765 shutdown_connections 2024-09-21T03:42:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.155+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1789897765 wait complete. 2024-09-21T03:42:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2c5bce740 1 -- start start 2024-09-21T03:42:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2c5bce740 1 Processor -- start 2024-09-21T03:42:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dbf9f0 0x5609521bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2c5bce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b81abb60 con 0x560951dbf9f0 2024-09-21T03:42:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b81acfd0 con 0x560951dd81b0 2024-09-21T03:42:09.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2c5bce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b81ac230 con 0x560951de0390 2024-09-21T03:42:09.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2c5bce740 10 client.0 _ll_register_callbacks cb 0x7fd2ac009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:09.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2bd3f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dbf9f0 0x5609521bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2bd3f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dbf9f0 0x5609521bc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51058/0 (socket says 172.21.15.112:51058) 2024-09-21T03:42:09.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2bd3f6640 1 -- 172.21.15.112:0/1901302433 learned_addr learned my addr 172.21.15.112:0/1901302433 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:09.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.156+0000 7fd2bdbf7640 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd2bd3f6640 1 -- 172.21.15.112:0/1901302433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 msgr2=0x5609521b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:09.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd2bd3f6640 1 -- 172.21.15.112:0/1901302433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 msgr2=0x5609521b78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:09.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521b78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd2bd3f6640 1 -- 172.21.15.112:0/1901302433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609521b8520 con 0x560951dbf9f0 2024-09-21T03:42:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521b78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd2bd3f6640 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dbf9f0 0x5609521bc3f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80e4810 tx=0x7fd2a80d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b31826cb230350 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd29cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:42:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd29cff9640 1 -- 172.21.15.112:0/1901302433 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a805d070 con 0x560951dbf9f0 2024-09-21T03:42:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd29cff9640 1 -- 172.21.15.112:0/1901302433 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8514+0+0 (secure 0 0 0) 0x7fd2a805d210 con 0x560951dbf9f0 2024-09-21T03:42:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1901302433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2ac054070 con 0x560951dbf9f0 2024-09-21T03:42:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd29cff9640 1 -- 172.21.15.112:0/1901302433 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd2a8020040 con 0x560951dbf9f0 2024-09-21T03:42:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd2c5bce740 10 client.17682 fetch_fsmap learned FSMap version 32 2024-09-21T03:42:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1901302433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560952230380 con 0x560951dbf9f0 2024-09-21T03:42:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.157+0000 7fd29cff9640 1 -- 172.21.15.112:0/1901302433 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2ac054070 con 0x560951dbf9f0 2024-09-21T03:42:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.158+0000 7fd29cff9640 1 -- 172.21.15.112:0/1901302433 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd2a80b1050 con 0x560951dbf9f0 2024-09-21T03:42:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.158+0000 7fd2c5bce740 10 client.17682 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:42:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.158+0000 7fd2c5bce740 10 client.17682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.158+0000 7fd2c5bce740 20 client.17682 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.158+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560951e01070 0x560952232b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.158+0000 7fd2c5bce740 4 client.17682 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:42:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.158+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1901302433 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56095211c8a0 con 0x560951e01070 2024-09-21T03:42:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.158+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560951e01070 0x560952232b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.158+0000 7fd29cff9640 10 client.17682 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.158+0000 7fd2bcbf5640 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560951e01070 0x560952232b40 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa849642a33a32d2 server_cookie=f0772bd27de32b24 in_seq=0 out_seq=0 2024-09-21T03:42:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd29cff9640 1 -- 172.21.15.112:0/1901302433 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56095211c8a0 con 0x560951e01070 2024-09-21T03:42:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd29cff9640 10 client.17682 handle_command_reply: tid=0 2024-09-21T03:42:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 1 client.17682 shutdown 2024-09-21T03:42:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1901302433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dbf9f0 msgr2=0x5609521bc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dbf9f0 0x5609521bc3f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80e4810 tx=0x7fd2a80d0040 comp rx=0 tx=0).stop 2024-09-21T03:42:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1901302433 shutdown_connections 2024-09-21T03:42:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x560951e01070 0x560952232b40 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560951de0390 0x5609521b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560951dd81b0 0x5609521b78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd29cff9640 0 client.17682 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 1 --2- 172.21.15.112:0/1901302433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560951dbf9f0 0x5609521bc3f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1901302433 >> 172.21.15.112:0/1901302433 conn(0x560951df5460 msgr2=0x5609521a92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1901302433 shutdown_connections 2024-09-21T03:42:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 1 -- 172.21.15.112:0/1901302433 wait complete. 2024-09-21T03:42:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.160+0000 7fd2c5bce740 20 client.17682 trim_cache size 0 max 16384 2024-09-21T03:42:09.178 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:42:09.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 -- 172.21.15.112:0/2494289627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd2a00b6680 msgr2=0x7fd2a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd2a00b6680 0x7fd2a00b8b90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0011d00 tx=0x7fd2b0011440 comp rx=0 tx=0).stop 2024-09-21T03:42:09.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 -- 172.21.15.112:0/2494289627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b81048a0 msgr2=0x7fd2b819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:09.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b81048a0 0x7fd2b819a470 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800d770 tx=0x7fd2a800dc30 comp rx=0 tx=0).stop 2024-09-21T03:42:09.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 -- 172.21.15.112:0/2494289627 shutdown_connections 2024-09-21T03:42:09.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fd2a00b6680 0x7fd2a00b8b90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b810de50 0x7fd2b819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2b81052d0 0x7fd2b819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 --2- 172.21.15.112:0/2494289627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2b81048a0 0x7fd2b819a470 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:09.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 -- 172.21.15.112:0/2494289627 >> 172.21.15.112:0/2494289627 conn(0x7fd2b8079730 msgr2=0x7fd2b8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:09.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 -- 172.21.15.112:0/2494289627 shutdown_connections 2024-09-21T03:42:09.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:09.161+0000 7fd2bf65a640 1 -- 172.21.15.112:0/2494289627 wait complete. 2024-09-21T03:42:09.541 INFO:tasks.workunit.client.0.smithi112.stdout: 10 4057 43.41 MB/sec warmup 9 sec latency 5052.858 ms 2024-09-21T03:42:10.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:10 smithi112 ceph-mon[31442]: pgmap v727: 97 pgs: 97 active+clean; 965 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 290 op/s 2024-09-21T03:42:10.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:10 smithi112 ceph-mon[31442]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T03:42:10.541 INFO:tasks.workunit.client.0.smithi112.stdout: 10 4613 43.22 MB/sec warmup 10 sec latency 6052.886 ms 2024-09-21T03:42:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:10 smithi203 ceph-mon[34723]: pgmap v727: 97 pgs: 97 active+clean; 965 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 290 op/s 2024-09-21T03:42:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:10 smithi203 ceph-mon[34723]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T03:42:10.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:10 smithi187 ceph-mon[34603]: pgmap v727: 97 pgs: 97 active+clean; 965 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 290 op/s 2024-09-21T03:42:10.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:10 smithi187 ceph-mon[34603]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T03:42:11.088 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:42:11.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:42:11.353 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:11.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 5424 40.75 MB/sec warmup 11 sec latency 7053.007 ms 2024-09-21T03:42:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.868+0000 7f846be48640 1 -- 172.21.15.112:0/1384190839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8464108d00 msgr2=0x7f846410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.868+0000 7f846be48640 1 --2- 172.21.15.112:0/1384190839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8464108d00 0x7f846410f530 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f845c009ca0 tx=0x7f845c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.868+0000 7f846be48640 1 -- 172.21.15.112:0/1384190839 shutdown_connections 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.868+0000 7f846be48640 1 --2- 172.21.15.112:0/1384190839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8464108d00 0x7f846410f530 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.868+0000 7f846be48640 1 --2- 172.21.15.112:0/1384190839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84641083b0 0x7f84641087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.868+0000 7f846be48640 1 --2- 172.21.15.112:0/1384190839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8464103180 0x7f8464103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.868+0000 7f846be48640 1 -- 172.21.15.112:0/1384190839 >> 172.21.15.112:0/1384190839 conn(0x7f8464077c60 msgr2=0x7f84640780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.868+0000 7f846be48640 1 -- 172.21.15.112:0/1384190839 shutdown_connections 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.868+0000 7f846be48640 1 -- 172.21.15.112:0/1384190839 wait complete. 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f846be48640 1 Processor -- start 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f846be48640 1 -- start start 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f846be48640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8464103180 0x7f846419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f846be48640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84641083b0 0x7f846419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f846be48640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8464108d00 0x7f846419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f846be48640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84641129c0 con 0x7f84641083b0 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f846be48640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8464112840 con 0x7f8464108d00 2024-09-21T03:42:11.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f846be48640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8464112b40 con 0x7f8464103180 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f84693e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84641083b0 0x7f846419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f84693e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84641083b0 0x7f846419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51076/0 (socket says 172.21.15.112:51076) 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f84693e3640 1 -- 172.21.15.112:0/3329367523 learned_addr learned my addr 172.21.15.112:0/3329367523 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.869+0000 7f8469be4640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8464103180 0x7f846419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f846a3e5640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8464108d00 0x7f846419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f84693e3640 1 -- 172.21.15.112:0/3329367523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8464103180 msgr2=0x7f846419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f84693e3640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8464103180 0x7f846419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f84693e3640 1 -- 172.21.15.112:0/3329367523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8464108d00 msgr2=0x7f846419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f84693e3640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8464108d00 0x7f846419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f84693e3640 1 -- 172.21.15.112:0/3329367523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84641a0410 con 0x7f84641083b0 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f84693e3640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84641083b0 0x7f846419aa40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f845400e9f0 tx=0x7f845400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f845affd640 1 -- 172.21.15.112:0/3329367523 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845400ccc0 con 0x7f84641083b0 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f846be48640 1 -- 172.21.15.112:0/3329367523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84641a0700 con 0x7f84641083b0 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f845affd640 1 -- 172.21.15.112:0/3329367523 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84540108e0 con 0x7f84641083b0 2024-09-21T03:42:11.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f845affd640 1 -- 172.21.15.112:0/3329367523 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454018860 con 0x7f84641083b0 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.870+0000 7f846be48640 1 -- 172.21.15.112:0/3329367523 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84641a6e60 con 0x7f84641083b0 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.873+0000 7f845affd640 1 -- 172.21.15.112:0/3329367523 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8454010430 con 0x7f84641083b0 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f84723bc740 1 Processor -- start 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f84723bc740 1 -- start start 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f845affd640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f84500b6680 0x7f84500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f846a3e5640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f84500b6680 0x7f84500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f845affd640 1 -- 172.21.15.112:0/3329367523 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f84540e7a80 con 0x7f84641083b0 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f84723bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 0x5565ae6b6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f84723bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f84723bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 0x5565ae6b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f84723bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5565ae62ce60 con 0x5565ae2ee670 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f84723bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5565ae61c290 con 0x5565ae30f350 2024-09-21T03:42:11.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f84723bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5565ae6200a0 con 0x5565ae2e6490 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f8469be4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 0x5565ae6b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f84693e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f8469be4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 0x5565ae6b6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41436/0 (socket says 172.21.15.112:41436) 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.874+0000 7f8469be4640 1 -- 172.21.15.112:0/1490454368 learned_addr learned my addr 172.21.15.112:0/1490454368 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.875+0000 7f846a3e5640 1 --2- 172.21.15.112:0/1490454368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 0x5565ae6b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.875+0000 7f846a3e5640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f84500b6680 0x7f84500b8b90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f845c006010 tx=0x7f845c009800 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.875+0000 7f8469be4640 1 -- 172.21.15.112:0/1490454368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 msgr2=0x5565ae6b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.875+0000 7f8469be4640 1 --2- 172.21.15.112:0/1490454368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6b6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.875+0000 7f8469be4640 1 -- 172.21.15.112:0/1490454368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 msgr2=0x5565ae6b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.875+0000 7f8469be4640 1 --2- 172.21.15.112:0/1490454368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 0x5565ae6b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.875+0000 7f8469be4640 1 -- 172.21.15.112:0/1490454368 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565ae37a930 con 0x5565ae30f350 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.875+0000 7f8469be4640 1 --2- 172.21.15.112:0/1490454368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 0x5565ae6b6430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8460009d20 tx=0x7f8460011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86baa53a74ba6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.875+0000 7f844ffff640 1 -- 172.21.15.112:0/1490454368 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846001e070 con 0x5565ae30f350 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.875+0000 7f844ffff640 1 -- 172.21.15.112:0/1490454368 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8460019ae0 con 0x5565ae30f350 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.875+0000 7f844ffff640 1 -- 172.21.15.112:0/1490454368 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846000ea80 con 0x5565ae30f350 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.876+0000 7f84723bc740 1 -- 172.21.15.112:0/1490454368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 msgr2=0x5565ae6b6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.876+0000 7f84723bc740 1 --2- 172.21.15.112:0/1490454368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 0x5565ae6b6430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8460009d20 tx=0x7f8460011810 comp rx=0 tx=0).stop 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.876+0000 7f84723bc740 1 -- 172.21.15.112:0/1490454368 shutdown_connections 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.876+0000 7f84723bc740 1 --2- 172.21.15.112:0/1490454368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 0x5565ae6b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.876+0000 7f84723bc740 1 --2- 172.21.15.112:0/1490454368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.876+0000 7f84723bc740 1 --2- 172.21.15.112:0/1490454368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 0x5565ae6b6430 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.876+0000 7f84723bc740 1 -- 172.21.15.112:0/1490454368 >> 172.21.15.112:0/1490454368 conn(0x5565ae303740 msgr2=0x5565ae6b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.876+0000 7f84723bc740 1 -- 172.21.15.112:0/1490454368 shutdown_connections 2024-09-21T03:42:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.876+0000 7f84723bc740 1 -- 172.21.15.112:0/1490454368 wait complete. 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.877+0000 7f84723bc740 1 -- start start 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.877+0000 7f84723bc740 1 Processor -- start 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.877+0000 7f84723bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.877+0000 7f84723bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae2ee670 0x5565ae6c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.877+0000 7f84723bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 0x5565ae6d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.877+0000 7f84723bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5565ae47d890 con 0x5565ae30f350 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.877+0000 7f84723bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5565ae5b6d40 con 0x5565ae2ee670 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.877+0000 7f84723bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5565ae6c8f70 con 0x5565ae2e6490 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.877+0000 7f84723bc740 10 client.0 _ll_register_callbacks cb 0x5565ae61ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.877+0000 7f8469be4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f8469be4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6c63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:49696/0 (socket says 172.21.15.112:49696) 2024-09-21T03:42:11.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f8469be4640 1 -- 172.21.15.112:0/797060269 learned_addr learned my addr 172.21.15.112:0/797060269 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f846a3e5640 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 0x5565ae6d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f84693e3640 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae2ee670 0x5565ae6c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f8469be4640 1 -- 172.21.15.112:0/797060269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae2ee670 msgr2=0x5565ae6c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f8469be4640 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae2ee670 0x5565ae6c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f8469be4640 1 -- 172.21.15.112:0/797060269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 msgr2=0x5565ae6d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f8469be4640 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 0x5565ae6d22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f8469be4640 1 -- 172.21.15.112:0/797060269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565ae6d2810 con 0x5565ae2e6490 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f8469be4640 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6c63c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8460005e00 tx=0x7f8460004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e58bd45ae81e904 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f844d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f844d7fa640 1 -- 172.21.15.112:0/797060269 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460027020 con 0x5565ae2e6490 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f84723bc740 1 -- 172.21.15.112:0/797060269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8460019ae0 con 0x5565ae2e6490 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f844d7fa640 1 -- 172.21.15.112:0/797060269 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f8460019630 con 0x5565ae2e6490 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.878+0000 7f844d7fa640 1 -- 172.21.15.112:0/797060269 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8460032050 con 0x5565ae2e6490 2024-09-21T03:42:11.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.879+0000 7f84723bc740 10 client.27253 fetch_fsmap learned FSMap version 32 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.879+0000 7f84723bc740 1 -- 172.21.15.112:0/797060269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5565ae6d5140 con 0x5565ae2e6490 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.879+0000 7f844d7fa640 1 -- 172.21.15.112:0/797060269 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8460019ae0 con 0x5565ae2e6490 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.879+0000 7f844d7fa640 1 -- 172.21.15.112:0/797060269 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f846000eb70 con 0x5565ae2e6490 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.879+0000 7f84723bc740 10 client.27253 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.879+0000 7f84723bc740 10 client.27253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.879+0000 7f84723bc740 20 client.27253 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.879+0000 7f84723bc740 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5565ae2cdcd0 0x5565ae6d84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.879+0000 7f84723bc740 4 client.27253 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.879+0000 7f84723bc740 1 -- 172.21.15.112:0/797060269 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565ae62edb0 con 0x5565ae2cdcd0 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.879+0000 7f84693e3640 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5565ae2cdcd0 0x5565ae6d84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.880+0000 7f84693e3640 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5565ae2cdcd0 0x5565ae6d84a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb55ccfd8eda314 server_cookie=b393ba0e80ebf915 in_seq=0 out_seq=0 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.880+0000 7f844d7fa640 10 client.27253 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.883+0000 7f844d7fa640 1 -- 172.21.15.112:0/797060269 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5565ae62edb0 con 0x5565ae2cdcd0 2024-09-21T03:42:11.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.883+0000 7f844d7fa640 10 client.27253 handle_command_reply: tid=0 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.883+0000 7f84723bc740 1 client.27253 shutdown 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.883+0000 7f84723bc740 1 -- 172.21.15.112:0/797060269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 msgr2=0x5565ae6c63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.883+0000 7f84723bc740 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6c63c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8460005e00 tx=0x7f8460004270 comp rx=0 tx=0).stop 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.884+0000 7f84723bc740 1 -- 172.21.15.112:0/797060269 shutdown_connections 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.884+0000 7f84723bc740 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5565ae2cdcd0 0x5565ae6d84a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.884+0000 7f84723bc740 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 0x5565ae6d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.884+0000 7f84723bc740 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae2ee670 0x5565ae6c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.884+0000 7f844d7fa640 0 client.27253 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.884+0000 7f84723bc740 1 --2- 172.21.15.112:0/797060269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6c63c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.884+0000 7f84723bc740 1 -- 172.21.15.112:0/797060269 >> 172.21.15.112:0/797060269 conn(0x5565ae303740 msgr2=0x5565ae6b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.884+0000 7f84723bc740 1 -- 172.21.15.112:0/797060269 shutdown_connections 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.884+0000 7f84723bc740 1 -- 172.21.15.112:0/797060269 wait complete. 2024-09-21T03:42:11.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.884+0000 7f84723bc740 20 client.27253 trim_cache size 0 max 16384 2024-09-21T03:42:11.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.901+0000 7f84723bc740 1 Processor -- start 2024-09-21T03:42:11.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f84723bc740 1 -- start start 2024-09-21T03:42:11.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f84723bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae2e6490 0x5565ae6d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f84723bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2ee670 0x5565ae6b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f84723bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 0x5565ae6c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f84723bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84641afec0 con 0x5565ae30f350 2024-09-21T03:42:11.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f84723bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f84641ad730 con 0x5565ae2e6490 2024-09-21T03:42:11.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f84723bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f84641ab5a0 con 0x5565ae2ee670 2024-09-21T03:42:11.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f846a3e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 0x5565ae6c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f846a3e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 0x5565ae6c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51100/0 (socket says 172.21.15.112:51100) 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f846a3e5640 1 -- 172.21.15.112:0/3459595165 learned_addr learned my addr 172.21.15.112:0/3459595165 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f84693e3640 1 --2- 172.21.15.112:0/3459595165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2ee670 0x5565ae6b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f8469be4640 1 --2- 172.21.15.112:0/3459595165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae2e6490 0x5565ae6d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f846a3e5640 1 -- 172.21.15.112:0/3459595165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2ee670 msgr2=0x5565ae6b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f846a3e5640 1 --2- 172.21.15.112:0/3459595165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2ee670 0x5565ae6b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f846a3e5640 1 -- 172.21.15.112:0/3459595165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae2e6490 msgr2=0x5565ae6d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f846a3e5640 1 --2- 172.21.15.112:0/3459595165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae2e6490 0x5565ae6d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f846a3e5640 1 -- 172.21.15.112:0/3459595165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565ae6c46c0 con 0x5565ae30f350 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.902+0000 7f846a3e5640 1 --2- 172.21.15.112:0/3459595165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 0x5565ae6c5920 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f845c031ed0 tx=0x7f845c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e552214350c6a3ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.903+0000 7f844f7fe640 1 -- 172.21.15.112:0/3459595165 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c003970 con 0x5565ae30f350 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.903+0000 7f844f7fe640 1 -- 172.21.15.112:0/3459595165 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f845c036040 con 0x5565ae30f350 2024-09-21T03:42:11.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.903+0000 7f844f7fe640 1 -- 172.21.15.112:0/3459595165 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c004860 con 0x5565ae30f350 2024-09-21T03:42:11.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.903+0000 7f84723bc740 1 -- 172.21.15.112:0/3459595165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 msgr2=0x5565ae6c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.903+0000 7f84723bc740 1 --2- 172.21.15.112:0/3459595165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 0x5565ae6c5920 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f845c031ed0 tx=0x7f845c040000 comp rx=0 tx=0).stop 2024-09-21T03:42:11.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.904+0000 7f84723bc740 1 -- 172.21.15.112:0/3459595165 shutdown_connections 2024-09-21T03:42:11.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.904+0000 7f84723bc740 1 --2- 172.21.15.112:0/3459595165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae30f350 0x5565ae6c5920 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.904+0000 7f84723bc740 1 --2- 172.21.15.112:0/3459595165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2ee670 0x5565ae6b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.904+0000 7f84723bc740 1 --2- 172.21.15.112:0/3459595165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae2e6490 0x5565ae6d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.904+0000 7f84723bc740 1 -- 172.21.15.112:0/3459595165 >> 172.21.15.112:0/3459595165 conn(0x5565ae303740 msgr2=0x5565ae6b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.904+0000 7f84723bc740 1 -- 172.21.15.112:0/3459595165 shutdown_connections 2024-09-21T03:42:11.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.904+0000 7f84723bc740 1 -- 172.21.15.112:0/3459595165 wait complete. 2024-09-21T03:42:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.904+0000 7f84723bc740 1 -- start start 2024-09-21T03:42:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.904+0000 7f84723bc740 1 Processor -- start 2024-09-21T03:42:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84723bc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84723bc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 0x5565ae6ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84723bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 0x5565ae6cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84723bc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84641abbf0 con 0x5565ae2ee670 2024-09-21T03:42:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84723bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f84641ad060 con 0x5565ae30f350 2024-09-21T03:42:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84723bc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f84641ac2c0 con 0x5565ae2e6490 2024-09-21T03:42:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84723bc740 10 client.0 _ll_register_callbacks cb 0x7f846001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:11.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84693e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 0x5565ae6ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f8469be4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84693e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 0x5565ae6ca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51108/0 (socket says 172.21.15.112:51108) 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84693e3640 1 -- 172.21.15.112:0/2497107795 learned_addr learned my addr 172.21.15.112:0/2497107795 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f846a3e5640 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 0x5565ae6cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84693e3640 1 -- 172.21.15.112:0/2497107795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 msgr2=0x5565ae6ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84693e3640 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84693e3640 1 -- 172.21.15.112:0/2497107795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 msgr2=0x5565ae6cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84693e3640 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 0x5565ae6cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84693e3640 1 -- 172.21.15.112:0/2497107795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565ae741f30 con 0x5565ae2ee670 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f84693e3640 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 0x5565ae6ca8e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f845400c6a0 tx=0x7f84540a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e3818feb13efee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f844d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.905+0000 7f844d7fa640 1 -- 172.21.15.112:0/2497107795 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84540adfa0 con 0x5565ae2ee670 2024-09-21T03:42:11.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f844d7fa640 1 -- 172.21.15.112:0/2497107795 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f84540e5420 con 0x5565ae2ee670 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f84723bc740 1 -- 172.21.15.112:0/2497107795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f845c036040 con 0x5565ae2ee670 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f844d7fa640 1 -- 172.21.15.112:0/2497107795 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f84540c8050 con 0x5565ae2ee670 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f84723bc740 10 client.17700 fetch_fsmap learned FSMap version 32 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f84723bc740 1 -- 172.21.15.112:0/2497107795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5565ae743210 con 0x5565ae2ee670 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f844d7fa640 1 -- 172.21.15.112:0/2497107795 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f845c036040 con 0x5565ae2ee670 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f844d7fa640 1 -- 172.21.15.112:0/2497107795 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f84540ac570 con 0x5565ae2ee670 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f84723bc740 10 client.17700 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f84723bc740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f84723bc740 20 client.17700 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f84723bc740 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5565ae2a3080 0x5565ae745950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f84723bc740 4 client.17700 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:42:11.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f84723bc740 1 -- 172.21.15.112:0/2497107795 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5565ae62a860 con 0x5565ae2a3080 2024-09-21T03:42:11.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.906+0000 7f8469be4640 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5565ae2a3080 0x5565ae745950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:11.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.907+0000 7f844d7fa640 10 client.17700 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:11.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.907+0000 7f8469be4640 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5565ae2a3080 0x5565ae745950 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2cdf17d921dcb60 server_cookie=742722ed1a311ee5 in_seq=0 out_seq=0 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f844d7fa640 1 -- 172.21.15.112:0/2497107795 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5565ae62a860 con 0x5565ae2a3080 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f844d7fa640 10 client.17700 handle_command_reply: tid=0 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 1 client.17700 shutdown 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 1 -- 172.21.15.112:0/2497107795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 msgr2=0x5565ae6ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 0x5565ae6ca8e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f845400c6a0 tx=0x7f84540a9040 comp rx=0 tx=0).stop 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 1 -- 172.21.15.112:0/2497107795 shutdown_connections 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5565ae2a3080 0x5565ae745950 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565ae30f350 0x5565ae6cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f844d7fa640 0 client.17700 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5565ae2ee670 0x5565ae6ca8e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 1 --2- 172.21.15.112:0/2497107795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5565ae2e6490 0x5565ae6ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 1 -- 172.21.15.112:0/2497107795 >> 172.21.15.112:0/2497107795 conn(0x5565ae303740 msgr2=0x5565ae6b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 1 -- 172.21.15.112:0/2497107795 shutdown_connections 2024-09-21T03:42:11.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 1 -- 172.21.15.112:0/2497107795 wait complete. 2024-09-21T03:42:11.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.920+0000 7f84723bc740 20 client.17700 trim_cache size 0 max 16384 2024-09-21T03:42:11.924 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:42:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:42:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "4591c632-505e-4006-8d95-6080e5d063e2", 2024-09-21T03:42:11.924 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:42:11.924 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:42:11.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 -- 172.21.15.112:0/3329367523 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f84500b6680 msgr2=0x7f84500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f84500b6680 0x7f84500b8b90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f845c006010 tx=0x7f845c009800 comp rx=0 tx=0).stop 2024-09-21T03:42:11.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 -- 172.21.15.112:0/3329367523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84641083b0 msgr2=0x7f846419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:11.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84641083b0 0x7f846419aa40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f845400e9f0 tx=0x7f845400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 -- 172.21.15.112:0/3329367523 shutdown_connections 2024-09-21T03:42:11.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f84500b6680 0x7f84500b8b90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8464108d00 0x7f846419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84641083b0 0x7f846419aa40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 --2- 172.21.15.112:0/3329367523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8464103180 0x7f846419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:11.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 -- 172.21.15.112:0/3329367523 >> 172.21.15.112:0/3329367523 conn(0x7f8464077c60 msgr2=0x7f8464101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:11.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 -- 172.21.15.112:0/3329367523 shutdown_connections 2024-09-21T03:42:11.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:11.921+0000 7f846be48640 1 -- 172.21.15.112:0/3329367523 wait complete. 2024-09-21T03:42:12.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:12 smithi112 ceph-mon[31442]: pgmap v729: 97 pgs: 97 active+clean; 978 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 400 op/s 2024-09-21T03:42:12.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:12 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:42:12.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:12 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:12.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:12 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:42:12.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 5632 38.15 MB/sec warmup 12 sec latency 8053.051 ms 2024-09-21T03:42:12.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:12 smithi203 ceph-mon[34723]: pgmap v729: 97 pgs: 97 active+clean; 978 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 400 op/s 2024-09-21T03:42:12.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:12 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:42:12.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:12 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:12.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:12 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:42:12.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:12 smithi187 ceph-mon[34603]: pgmap v729: 97 pgs: 97 active+clean; 978 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 400 op/s 2024-09-21T03:42:12.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:12 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:42:12.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:12 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:12.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:12 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:42:12.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:42:13.108 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:13.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 5741 36.22 MB/sec warmup 13 sec latency 9053.087 ms 2024-09-21T03:42:13.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.660+0000 7fa1a51a7640 1 -- 172.21.15.112:0/2961998030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a0102a60 msgr2=0x7fa1a0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.660+0000 7fa1a51a7640 1 --2- 172.21.15.112:0/2961998030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a0102a60 0x7fa1a0102e70 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa190009900 tx=0x7fa19002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.660+0000 7fa1a51a7640 1 -- 172.21.15.112:0/2961998030 shutdown_connections 2024-09-21T03:42:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.660+0000 7fa1a51a7640 1 --2- 172.21.15.112:0/2961998030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1a0108cf0 0x7fa1a010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.660+0000 7fa1a51a7640 1 --2- 172.21.15.112:0/2961998030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a01083a0 0x7fa1a01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.660+0000 7fa1a51a7640 1 --2- 172.21.15.112:0/2961998030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a0102a60 0x7fa1a0102e70 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.660+0000 7fa1a51a7640 1 -- 172.21.15.112:0/2961998030 >> 172.21.15.112:0/2961998030 conn(0x7fa1a0079730 msgr2=0x7fa1a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:13.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.660+0000 7fa1a51a7640 1 -- 172.21.15.112:0/2961998030 shutdown_connections 2024-09-21T03:42:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.660+0000 7fa1a51a7640 1 -- 172.21.15.112:0/2961998030 wait complete. 2024-09-21T03:42:13.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.661+0000 7fa1a51a7640 1 Processor -- start 2024-09-21T03:42:13.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.662+0000 7fa1a51a7640 1 -- start start 2024-09-21T03:42:13.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.662+0000 7fa1a51a7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a0102a60 0x7fa1a0104410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.662+0000 7fa1a51a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1a01083a0 0x7fa1a0104950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.662+0000 7fa1a51a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a0108cf0 0x7fa1a0104e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.662+0000 7fa1a51a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a0112960 con 0x7fa1a0108cf0 2024-09-21T03:42:13.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.662+0000 7fa1a51a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a01127e0 con 0x7fa1a01083a0 2024-09-21T03:42:13.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.662+0000 7fa1a51a7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a0112ae0 con 0x7fa1a0102a60 2024-09-21T03:42:13.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.663+0000 7fa19ed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a0102a60 0x7fa1a0104410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.663+0000 7fa19ed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a0102a60 0x7fa1a0104410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:49734/0 (socket says 172.21.15.112:49734) 2024-09-21T03:42:13.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.663+0000 7fa19ed9d640 1 -- 172.21.15.112:0/1143293946 learned_addr learned my addr 172.21.15.112:0/1143293946 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:13.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.663+0000 7fa19f59e640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a0108cf0 0x7fa1a0104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.663+0000 7fa19e59c640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1a01083a0 0x7fa1a0104950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.663+0000 7fa19ed9d640 1 -- 172.21.15.112:0/1143293946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1a01083a0 msgr2=0x7fa1a0104950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.663+0000 7fa19ed9d640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1a01083a0 0x7fa1a0104950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.663+0000 7fa19ed9d640 1 -- 172.21.15.112:0/1143293946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a0108cf0 msgr2=0x7fa1a0104e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.663+0000 7fa19ed9d640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a0108cf0 0x7fa1a0104e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.663+0000 7fa19ed9d640 1 -- 172.21.15.112:0/1143293946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a01a8820 con 0x7fa1a0102a60 2024-09-21T03:42:13.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.663+0000 7fa19f59e640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a0108cf0 0x7fa1a0104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:13.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.664+0000 7fa19ed9d640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a0102a60 0x7fa1a0104410 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa190005940 tx=0x7fa1900040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:13.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.664+0000 7fa197fff640 1 -- 172.21.15.112:0/1143293946 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19003f030 con 0x7fa1a0102a60 2024-09-21T03:42:13.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.664+0000 7fa1a51a7640 1 -- 172.21.15.112:0/1143293946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a01a8ab0 con 0x7fa1a0102a60 2024-09-21T03:42:13.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.664+0000 7fa1a51a7640 1 -- 172.21.15.112:0/1143293946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a01a8fe0 con 0x7fa1a0102a60 2024-09-21T03:42:13.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.664+0000 7fa197fff640 1 -- 172.21.15.112:0/1143293946 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1900044c0 con 0x7fa1a0102a60 2024-09-21T03:42:13.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.664+0000 7fa197fff640 1 -- 172.21.15.112:0/1143293946 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190035070 con 0x7fa1a0102a60 2024-09-21T03:42:13.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.667+0000 7fa1ab71b740 1 Processor -- start 2024-09-21T03:42:13.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.668+0000 7fa1ab71b740 1 -- start start 2024-09-21T03:42:13.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.668+0000 7fa1ab71b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e9879070 0x55d3e9c20650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.668+0000 7fa1ab71b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c20b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.668+0000 7fa1ab71b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9c210d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.668+0000 7fa1ab71b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9b96e50 con 0x55d3e9858390 2024-09-21T03:42:13.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.668+0000 7fa1ab71b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9b86280 con 0x55d3e98501b0 2024-09-21T03:42:13.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.668+0000 7fa1ab71b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9b8a090 con 0x55d3e9879070 2024-09-21T03:42:13.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.668+0000 7fa19e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9c210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.668+0000 7fa19e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9c210d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51134/0 (socket says 172.21.15.112:51134) 2024-09-21T03:42:13.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.668+0000 7fa19e59c640 1 -- 172.21.15.112:0/303769526 learned_addr learned my addr 172.21.15.112:0/303769526 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:13.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.668+0000 7fa19f59e640 1 --2- 172.21.15.112:0/303769526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e9879070 0x55d3e9c20650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.669+0000 7fa197fff640 1 -- 172.21.15.112:0/1143293946 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa190042030 con 0x7fa1a0102a60 2024-09-21T03:42:13.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.669+0000 7fa19e59c640 1 -- 172.21.15.112:0/303769526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e9879070 msgr2=0x55d3e9c20650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:13.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.669+0000 7fa19e59c640 1 --2- 172.21.15.112:0/303769526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e9879070 0x55d3e9c20650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.669+0000 7fa19e59c640 1 -- 172.21.15.112:0/303769526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 msgr2=0x55d3e9c20b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:42:13.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.669+0000 7fa19e59c640 1 --2- 172.21.15.112:0/303769526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c20b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.669+0000 7fa19e59c640 1 -- 172.21.15.112:0/303769526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e98e4650 con 0x55d3e9858390 2024-09-21T03:42:13.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.669+0000 7fa19e59c640 1 --2- 172.21.15.112:0/303769526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9c210d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa188009bf0 tx=0x7fa188010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd18c6d8d74f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.669+0000 7fa1957fa640 1 -- 172.21.15.112:0/303769526 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18801b480 con 0x55d3e9858390 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.669+0000 7fa1957fa640 1 -- 172.21.15.112:0/303769526 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18801bad0 con 0x55d3e9858390 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.669+0000 7fa1957fa640 1 -- 172.21.15.112:0/303769526 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188012a80 con 0x55d3e9858390 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.670+0000 7fa197fff640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa18c0b6680 0x7fa18c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.670+0000 7fa197fff640 1 -- 172.21.15.112:0/1143293946 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7fa190107960 con 0x7fa1a0102a60 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.670+0000 7fa19f59e640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa18c0b6680 0x7fa18c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.670+0000 7fa1ab71b740 1 -- 172.21.15.112:0/303769526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 msgr2=0x55d3e9c210d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.670+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/303769526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9c210d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa188009bf0 tx=0x7fa188010740 comp rx=0 tx=0).stop 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.671+0000 7fa1ab71b740 1 -- 172.21.15.112:0/303769526 shutdown_connections 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.671+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/303769526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9c210d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.671+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/303769526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c20b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.671+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/303769526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e9879070 0x55d3e9c20650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.671+0000 7fa1ab71b740 1 -- 172.21.15.112:0/303769526 >> 172.21.15.112:0/303769526 conn(0x55d3e986d460 msgr2=0x55d3e9c1e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:13.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.671+0000 7fa19f59e640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa18c0b6680 0x7fa18c0b8b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa198005fd0 tx=0x7fa198009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:13.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.671+0000 7fa1ab71b740 1 -- 172.21.15.112:0/303769526 shutdown_connections 2024-09-21T03:42:13.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.671+0000 7fa1ab71b740 1 -- 172.21.15.112:0/303769526 wait complete. 2024-09-21T03:42:13.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.672+0000 7fa1ab71b740 1 -- start start 2024-09-21T03:42:13.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.672+0000 7fa1ab71b740 1 Processor -- start 2024-09-21T03:42:13.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.673+0000 7fa1ab71b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c34550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.673+0000 7fa1ab71b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9c34a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.673+0000 7fa19ed9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c34550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.673+0000 7fa19ed9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c34550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41504/0 (socket says 172.21.15.112:41504) 2024-09-21T03:42:13.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.673+0000 7fa19ed9d640 1 -- 172.21.15.112:0/2265706663 learned_addr learned my addr 172.21.15.112:0/2265706663 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:13.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.673+0000 7fa19e59c640 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9c34a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.673+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e9879070 0x55d3e9c40920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.673+0000 7fa1ab71b740 1 -- 172.21.15.112:0/2265706663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d3e99e7700 con 0x55d3e9858390 2024-09-21T03:42:13.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.673+0000 7fa1ab71b740 1 -- 172.21.15.112:0/2265706663 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9b21110 con 0x55d3e98501b0 2024-09-21T03:42:13.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.673+0000 7fa1ab71b740 1 -- 172.21.15.112:0/2265706663 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9c37560 con 0x55d3e9879070 2024-09-21T03:42:13.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.673+0000 7fa1ab71b740 10 client.0 _ll_register_callbacks cb 0x55d3e9b3a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:13.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.674+0000 7fa19f59e640 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e9879070 0x55d3e9c40920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.674+0000 7fa19ed9d640 1 -- 172.21.15.112:0/2265706663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e9879070 msgr2=0x55d3e9c40920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:13.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.674+0000 7fa19ed9d640 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e9879070 0x55d3e9c40920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.674+0000 7fa19ed9d640 1 -- 172.21.15.112:0/2265706663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 msgr2=0x55d3e9c34a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:13.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.674+0000 7fa19ed9d640 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9c34a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.674+0000 7fa19ed9d640 1 -- 172.21.15.112:0/2265706663 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e9b4ec40 con 0x55d3e98501b0 2024-09-21T03:42:13.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.675+0000 7fa19ed9d640 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c34550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa190105190 tx=0x7fa1900c9f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=253f31a29aab29fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.675+0000 7fa1867fc640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:42:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.676+0000 7fa1867fc640 1 -- 172.21.15.112:0/2265706663 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900d2070 con 0x55d3e98501b0 2024-09-21T03:42:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.676+0000 7fa1867fc640 1 -- 172.21.15.112:0/2265706663 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7fa1900d6f70 con 0x55d3e98501b0 2024-09-21T03:42:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.676+0000 7fa1ab71b740 1 -- 172.21.15.112:0/2265706663 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa18801bad0 con 0x55d3e98501b0 2024-09-21T03:42:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.676+0000 7fa1867fc640 1 -- 172.21.15.112:0/2265706663 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa1900cd130 con 0x55d3e98501b0 2024-09-21T03:42:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.676+0000 7fa1ab71b740 10 client.27248 fetch_fsmap learned FSMap version 32 2024-09-21T03:42:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.677+0000 7fa1ab71b740 1 -- 172.21.15.112:0/2265706663 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d3e9c43350 con 0x55d3e98501b0 2024-09-21T03:42:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.677+0000 7fa1867fc640 1 -- 172.21.15.112:0/2265706663 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa190038560 con 0x55d3e98501b0 2024-09-21T03:42:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.677+0000 7fa1867fc640 1 -- 172.21.15.112:0/2265706663 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa1900c9130 con 0x55d3e98501b0 2024-09-21T03:42:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.677+0000 7fa1ab71b740 10 client.27248 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:42:13.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.677+0000 7fa1ab71b740 10 client.27248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.677+0000 7fa1ab71b740 20 client.27248 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.677+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d3e98379f0 0x55d3e9c46730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.677+0000 7fa1ab71b740 4 client.27248 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.677+0000 7fa1ab71b740 1 -- 172.21.15.112:0/2265706663 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa18801bad0 con 0x55d3e98379f0 2024-09-21T03:42:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.677+0000 7fa19e59c640 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d3e98379f0 0x55d3e9c46730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.678+0000 7fa1867fc640 10 client.27248 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:13.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.678+0000 7fa19e59c640 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d3e98379f0 0x55d3e9c46730 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66ca69c04aa1832e server_cookie=e19e0dfe18dcbefc in_seq=0 out_seq=0 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1867fc640 1 -- 172.21.15.112:0/2265706663 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa18801bad0 con 0x55d3e98379f0 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1867fc640 10 client.27248 handle_command_reply: tid=0 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 1 client.27248 shutdown 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 1 -- 172.21.15.112:0/2265706663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 msgr2=0x55d3e9c34550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c34550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa190105190 tx=0x7fa1900c9f60 comp rx=0 tx=0).stop 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 1 -- 172.21.15.112:0/2265706663 shutdown_connections 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d3e98379f0 0x55d3e9c46730 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1867fc640 0 client.27248 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e9879070 0x55d3e9c40920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9c34a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/2265706663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c34550 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 1 -- 172.21.15.112:0/2265706663 >> 172.21.15.112:0/2265706663 conn(0x55d3e986d460 msgr2=0x55d3e9c237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:13.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 1 -- 172.21.15.112:0/2265706663 shutdown_connections 2024-09-21T03:42:13.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 1 -- 172.21.15.112:0/2265706663 wait complete. 2024-09-21T03:42:13.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.682+0000 7fa1ab71b740 20 client.27248 trim_cache size 0 max 16384 2024-09-21T03:42:13.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.708+0000 7fa1ab71b740 1 Processor -- start 2024-09-21T03:42:13.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.710+0000 7fa1ab71b740 1 -- start start 2024-09-21T03:42:13.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.711+0000 7fa1ab71b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 0x55d3e9c34a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.711+0000 7fa1ab71b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e98501b0 0x55d3e9c1db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.711+0000 7fa1ab71b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e9858390 0x55d3e9c1e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.711+0000 7fa1ab71b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa180010f60 con 0x55d3e98501b0 2024-09-21T03:42:13.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.711+0000 7fa1ab71b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa18000e7d0 con 0x55d3e9858390 2024-09-21T03:42:13.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.711+0000 7fa1ab71b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa18000c640 con 0x55d3e98379f0 2024-09-21T03:42:13.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.711+0000 7fa19ed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 0x55d3e9c34a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.711+0000 7fa19ed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 0x55d3e9c34a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:49742/0 (socket says 172.21.15.112:49742) 2024-09-21T03:42:13.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.711+0000 7fa19f59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e9858390 0x55d3e9c1e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.711+0000 7fa19ed9d640 1 -- 172.21.15.112:0/1940217166 learned_addr learned my addr 172.21.15.112:0/1940217166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:13.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.711+0000 7fa19e59c640 1 --2- 172.21.15.112:0/1940217166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e98501b0 0x55d3e9c1db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.712+0000 7fa19f59e640 1 -- 172.21.15.112:0/1940217166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 msgr2=0x55d3e9c34a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:13.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.712+0000 7fa19f59e640 1 --2- 172.21.15.112:0/1940217166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 0x55d3e9c34a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.712+0000 7fa19f59e640 1 -- 172.21.15.112:0/1940217166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e98501b0 msgr2=0x55d3e9c1db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:13.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.712+0000 7fa19f59e640 1 --2- 172.21.15.112:0/1940217166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e98501b0 0x55d3e9c1db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.712+0000 7fa19f59e640 1 -- 172.21.15.112:0/1940217166 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e9c1f370 con 0x55d3e9858390 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.712+0000 7fa19e59c640 1 --2- 172.21.15.112:0/1940217166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e98501b0 0x55d3e9c1db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.712+0000 7fa19f59e640 1 --2- 172.21.15.112:0/1940217166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e9858390 0x55d3e9c1e0b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa198010740 tx=0x7fa198015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=674b76ed79ad9b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.712+0000 7fa194ff9640 1 -- 172.21.15.112:0/1940217166 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198003970 con 0x55d3e9858390 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.712+0000 7fa194ff9640 1 -- 172.21.15.112:0/1940217166 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa198004030 con 0x55d3e9858390 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.712+0000 7fa194ff9640 1 -- 172.21.15.112:0/1940217166 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19801c690 con 0x55d3e9858390 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.713+0000 7fa1ab71b740 1 -- 172.21.15.112:0/1940217166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e9858390 msgr2=0x55d3e9c1e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.713+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/1940217166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e9858390 0x55d3e9c1e0b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa198010740 tx=0x7fa198015000 comp rx=0 tx=0).stop 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.713+0000 7fa1ab71b740 1 -- 172.21.15.112:0/1940217166 shutdown_connections 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.713+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/1940217166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e9858390 0x55d3e9c1e0b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.713+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/1940217166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e98501b0 0x55d3e9c1db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.713+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/1940217166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 0x55d3e9c34a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.713+0000 7fa1ab71b740 1 -- 172.21.15.112:0/1940217166 >> 172.21.15.112:0/1940217166 conn(0x55d3e986d460 msgr2=0x55d3e9c34300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.713+0000 7fa1ab71b740 1 -- 172.21.15.112:0/1940217166 shutdown_connections 2024-09-21T03:42:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.713+0000 7fa1ab71b740 1 -- 172.21.15.112:0/1940217166 wait complete. 2024-09-21T03:42:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.714+0000 7fa1ab71b740 1 -- start start 2024-09-21T03:42:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.714+0000 7fa1ab71b740 1 Processor -- start 2024-09-21T03:42:13.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.714+0000 7fa1ab71b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 0x55d3e9c38c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.714+0000 7fa1ab71b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c39170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa1ab71b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9cb0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa1ab71b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa18000cc90 con 0x55d3e9858390 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa1ab71b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa18000e100 con 0x55d3e98501b0 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa1ab71b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa18000d360 con 0x55d3e98379f0 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa1ab71b740 10 client.0 _ll_register_callbacks cb 0x7fa190005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa19ed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 0x55d3e9c38c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa19ed9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 0x55d3e9c38c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:49744/0 (socket says 172.21.15.112:49744) 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa19ed9d640 1 -- 172.21.15.112:0/3541304761 learned_addr learned my addr 172.21.15.112:0/3541304761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa19e59c640 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c39170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa19f59e640 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9cb0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa19ed9d640 1 -- 172.21.15.112:0/3541304761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 msgr2=0x55d3e9c39170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:13.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa19ed9d640 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c39170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa19ed9d640 1 -- 172.21.15.112:0/3541304761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 msgr2=0x55d3e9cb0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa19ed9d640 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9cb0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa19ed9d640 1 -- 172.21.15.112:0/3541304761 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e9cb0670 con 0x55d3e98379f0 2024-09-21T03:42:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.715+0000 7fa19e59c640 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c39170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.716+0000 7fa19ed9d640 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 0x55d3e9c38c30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa1900cd770 tx=0x7fa1900ced20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be26bb57c912ffad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.716+0000 7fa1867fc640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:42:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.716+0000 7fa1867fc640 1 -- 172.21.15.112:0/3541304761 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900d2040 con 0x55d3e98379f0 2024-09-21T03:42:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.716+0000 7fa1ab71b740 1 -- 172.21.15.112:0/3541304761 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa198004030 con 0x55d3e98379f0 2024-09-21T03:42:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.716+0000 7fa1867fc640 1 -- 172.21.15.112:0/3541304761 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7fa1900cca40 con 0x55d3e98379f0 2024-09-21T03:42:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.716+0000 7fa1867fc640 1 -- 172.21.15.112:0/3541304761 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa1900ccf50 con 0x55d3e98379f0 2024-09-21T03:42:13.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.716+0000 7fa1ab71b740 10 client.27277 fetch_fsmap learned FSMap version 32 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.716+0000 7fa1ab71b740 1 -- 172.21.15.112:0/3541304761 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d3e9cb17a0 con 0x55d3e98379f0 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.716+0000 7fa1867fc640 1 -- 172.21.15.112:0/3541304761 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa198004030 con 0x55d3e98379f0 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.717+0000 7fa1867fc640 1 -- 172.21.15.112:0/3541304761 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa1900d3450 con 0x55d3e98379f0 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.717+0000 7fa1ab71b740 10 client.27277 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.717+0000 7fa1ab71b740 10 client.27277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.717+0000 7fa1ab71b740 20 client.27277 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.717+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d3e9879070 0x55d3e9cb3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.717+0000 7fa1ab71b740 4 client.27277 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.717+0000 7fa1ab71b740 1 -- 172.21.15.112:0/3541304761 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d3e9b98da0 con 0x55d3e9879070 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.717+0000 7fa19e59c640 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d3e9879070 0x55d3e9cb3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.717+0000 7fa1867fc640 10 client.27277 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:13.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.718+0000 7fa19e59c640 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d3e9879070 0x55d3e9cb3e20 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925132acd8698ea3 server_cookie=2f5f061cd1994f46 in_seq=0 out_seq=0 2024-09-21T03:42:13.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1867fc640 1 -- 172.21.15.112:0/3541304761 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6257 (crc 0 0 0) 0x55d3e9b98da0 con 0x55d3e9879070 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1867fc640 10 client.27277 handle_command_reply: tid=0 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1ab71b740 1 client.27277 shutdown 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1ab71b740 1 -- 172.21.15.112:0/3541304761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 msgr2=0x55d3e9c38c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 0x55d3e9c38c30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa1900cd770 tx=0x7fa1900ced20 comp rx=0 tx=0).stop 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1ab71b740 1 -- 172.21.15.112:0/3541304761 shutdown_connections 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d3e9879070 0x55d3e9cb3e20 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d3e9858390 0x55d3e9cb0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3e98501b0 0x55d3e9c39170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1867fc640 0 client.27277 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1ab71b740 1 --2- 172.21.15.112:0/3541304761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e98379f0 0x55d3e9c38c30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.723+0000 7fa1ab71b740 1 -- 172.21.15.112:0/3541304761 >> 172.21.15.112:0/3541304761 conn(0x55d3e986d460 msgr2=0x55d3e9c35a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.724+0000 7fa1ab71b740 1 -- 172.21.15.112:0/3541304761 shutdown_connections 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.724+0000 7fa1ab71b740 1 -- 172.21.15.112:0/3541304761 wait complete. 2024-09-21T03:42:13.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.724+0000 7fa1ab71b740 20 client.27277 trim_cache size 0 max 16384 2024-09-21T03:42:13.728 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:42:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "scrub active (92 inodes in the stack)", 2024-09-21T03:42:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:42:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "4591c632-505e-4006-8d95-6080e5d063e2": { 2024-09-21T03:42:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "4591c632-505e-4006-8d95-6080e5d063e2", 2024-09-21T03:42:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:42:13.728 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "4591c632-505e-4006-8d95-6080e5d063e2": { 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:42:11.919443+0000", 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c": { 2024-09-21T03:42:13.729 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:45.957537+0000", 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "3d0e4442-3ee3-44bd-99e2-f0a81392c8c4": { 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:41:28.881491+0000", 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:42:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:42:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:42:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:42:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.732 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:42:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:42:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.733 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:42:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:42:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.734 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:42:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:42:13.735 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:42:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:42:13.736 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.737 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:42:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:42:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:42:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:42:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.739 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:42:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:42:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.740 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:42:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:42:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.741 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:42:13.742 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:42:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.743 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:42:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:42:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:42:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:42:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.745 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:42:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:42:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.746 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:42:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:42:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.747 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:42:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:42:13.748 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:42:13.749 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:42:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:13.750 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:42:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:42:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:13.751 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:42:13.752 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:42:13.752 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:42:13.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.724+0000 7fa1a51a7640 1 -- 172.21.15.112:0/1143293946 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa18c0b6680 msgr2=0x7fa18c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.724+0000 7fa1a51a7640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa18c0b6680 0x7fa18c0b8b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa198005fd0 tx=0x7fa198009500 comp rx=0 tx=0).stop 2024-09-21T03:42:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.724+0000 7fa1a51a7640 1 -- 172.21.15.112:0/1143293946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a0102a60 msgr2=0x7fa1a0104410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.724+0000 7fa1a51a7640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a0102a60 0x7fa1a0104410 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa190005940 tx=0x7fa1900040c0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.725+0000 7fa1a51a7640 1 -- 172.21.15.112:0/1143293946 shutdown_connections 2024-09-21T03:42:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.725+0000 7fa1a51a7640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fa18c0b6680 0x7fa18c0b8b90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.725+0000 7fa1a51a7640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1a0108cf0 0x7fa1a0104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.725+0000 7fa1a51a7640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1a01083a0 0x7fa1a0104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.725+0000 7fa1a51a7640 1 --2- 172.21.15.112:0/1143293946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a0102a60 0x7fa1a0104410 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:13.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.725+0000 7fa1a51a7640 1 -- 172.21.15.112:0/1143293946 >> 172.21.15.112:0/1143293946 conn(0x7fa1a0079730 msgr2=0x7fa1a0106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:13.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.725+0000 7fa1a51a7640 1 -- 172.21.15.112:0/1143293946 shutdown_connections 2024-09-21T03:42:13.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:13.725+0000 7fa1a51a7640 1 -- 172.21.15.112:0/1143293946 wait complete. 2024-09-21T03:42:14.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:14 smithi112 ceph-mon[31442]: pgmap v730: 97 pgs: 97 active+clean; 982 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-09-21T03:42:14.541 INFO:tasks.workunit.client.0.smithi112.stdout: 10 6015 35.69 MB/sec warmup 14 sec latency 9798.168 ms 2024-09-21T03:42:14.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:14 smithi203 ceph-mon[34723]: pgmap v730: 97 pgs: 97 active+clean; 982 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-09-21T03:42:14.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:14 smithi187 ceph-mon[34603]: pgmap v730: 97 pgs: 97 active+clean; 982 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-09-21T03:42:14.738 INFO:tasks.cephfs.filesystem:scrub status for tag:4591c632-505e-4006-8d95-6080e5d063e2 - {'path': '/', 'tag': '4591c632-505e-4006-8d95-6080e5d063e2', 'options': 'recursive,force'} 2024-09-21T03:42:15.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:15 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:15.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:15 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:42:15.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:15 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:15.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 6784 36.88 MB/sec warmup 15 sec latency 88.992 ms 2024-09-21T03:42:15.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:15 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:15.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:15 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:42:15.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:15 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:15 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:15 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:42:15.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:15 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:16.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:16 smithi112 ceph-mon[31442]: pgmap v731: 97 pgs: 97 active+clean; 990 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 318 op/s 2024-09-21T03:42:16.541 INFO:tasks.workunit.client.0.smithi112.stdout: 10 7254 35.44 MB/sec warmup 16 sec latency 152.572 ms 2024-09-21T03:42:16.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:16 smithi203 ceph-mon[34723]: pgmap v731: 97 pgs: 97 active+clean; 990 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 318 op/s 2024-09-21T03:42:16.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:16 smithi187 ceph-mon[34603]: pgmap v731: 97 pgs: 97 active+clean; 990 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 318 op/s 2024-09-21T03:42:17.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 8063 35.70 MB/sec warmup 17 sec latency 136.890 ms 2024-09-21T03:42:18.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:18 smithi112 ceph-mon[31442]: pgmap v732: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 16 MiB/s wr, 362 op/s 2024-09-21T03:42:18.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 8781 36.13 MB/sec warmup 18 sec latency 256.401 ms 2024-09-21T03:42:18.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:18 smithi203 ceph-mon[34723]: pgmap v732: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 16 MiB/s wr, 362 op/s 2024-09-21T03:42:18.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:18 smithi187 ceph-mon[34603]: pgmap v732: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 16 MiB/s wr, 362 op/s 2024-09-21T03:42:19.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:19 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:42:19.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:19 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:19.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 9566 35.50 MB/sec warmup 19 sec latency 796.517 ms 2024-09-21T03:42:19.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:19 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:42:19.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:19 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:19.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:19 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:42:19.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:19 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:20.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:20 smithi112 ceph-mon[31442]: pgmap v733: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 17 MiB/s wr, 389 op/s 2024-09-21T03:42:20.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:20 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:20.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:20 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:20.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 10681 37.78 MB/sec warmup 20 sec latency 60.849 ms 2024-09-21T03:42:20.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:20 smithi203 ceph-mon[34723]: pgmap v733: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 17 MiB/s wr, 389 op/s 2024-09-21T03:42:20.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:20 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:20.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:20 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:20.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:20 smithi187 ceph-mon[34603]: pgmap v733: 97 pgs: 97 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 17 MiB/s wr, 389 op/s 2024-09-21T03:42:20.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:20 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:20.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:20 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:21.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:42:21.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 11351 38.15 MB/sec warmup 21 sec latency 224.506 ms 2024-09-21T03:42:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:42:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:42:21.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:42:21.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:42:21.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:42:22.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:22 smithi112 ceph-mon[31442]: pgmap v734: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 477 op/s 2024-09-21T03:42:22.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 11888 37.04 MB/sec warmup 22 sec latency 89.000 ms 2024-09-21T03:42:22.562 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:22 smithi203 ceph-mon[34723]: pgmap v734: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 477 op/s 2024-09-21T03:42:22.580 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:22 smithi187 ceph-mon[34603]: pgmap v734: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 477 op/s 2024-09-21T03:42:23.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 12899 37.44 MB/sec warmup 23 sec latency 71.479 ms 2024-09-21T03:42:24.538 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:24 smithi112 ceph-mon[31442]: pgmap v735: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 16 MiB/s wr, 408 op/s 2024-09-21T03:42:24.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 14029 38.76 MB/sec warmup 24 sec latency 53.442 ms 2024-09-21T03:42:24.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:24 smithi203 ceph-mon[34723]: pgmap v735: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 16 MiB/s wr, 408 op/s 2024-09-21T03:42:24.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:24 smithi187 ceph-mon[34603]: pgmap v735: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 16 MiB/s wr, 408 op/s 2024-09-21T03:42:25.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 14854 38.96 MB/sec warmup 25 sec latency 110.762 ms 2024-09-21T03:42:26.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:26 smithi112 ceph-mon[31442]: pgmap v736: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 481 op/s 2024-09-21T03:42:26.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 15924 39.64 MB/sec warmup 26 sec latency 65.796 ms 2024-09-21T03:42:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:26 smithi203 ceph-mon[34723]: pgmap v736: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 481 op/s 2024-09-21T03:42:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:26 smithi187 ceph-mon[34603]: pgmap v736: 97 pgs: 97 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 481 op/s 2024-09-21T03:42:27.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 16820 40.33 MB/sec warmup 27 sec latency 74.676 ms 2024-09-21T03:42:28.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:28 smithi112 ceph-mon[31442]: pgmap v737: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 682 op/s 2024-09-21T03:42:28.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 17543 39.82 MB/sec warmup 28 sec latency 113.419 ms 2024-09-21T03:42:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:28 smithi203 ceph-mon[34723]: pgmap v737: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 682 op/s 2024-09-21T03:42:28.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:28 smithi187 ceph-mon[34603]: pgmap v737: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 682 op/s 2024-09-21T03:42:29.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:29 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:42:29.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 18450 39.99 MB/sec warmup 29 sec latency 115.436 ms 2024-09-21T03:42:29.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:29 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:42:29.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:29 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:42:30.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:30 smithi112 ceph-mon[31442]: pgmap v738: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.8 MiB/s wr, 526 op/s 2024-09-21T03:42:30.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:30 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:42:30.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:30 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:42:30.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 19524 41.20 MB/sec warmup 30 sec latency 293.099 ms 2024-09-21T03:42:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:30 smithi203 ceph-mon[34723]: pgmap v738: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.8 MiB/s wr, 526 op/s 2024-09-21T03:42:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:30 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:42:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:30 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:42:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:30 smithi187 ceph-mon[34603]: pgmap v738: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.8 MiB/s wr, 526 op/s 2024-09-21T03:42:30.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:30 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:42:30.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:30 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:42:31.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 20075 40.72 MB/sec warmup 31 sec latency 59.637 ms 2024-09-21T03:42:32.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:32 smithi112 ceph-mon[31442]: pgmap v739: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-09-21T03:42:32.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 20810 40.42 MB/sec warmup 32 sec latency 147.483 ms 2024-09-21T03:42:32.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:32 smithi203 ceph-mon[34723]: pgmap v739: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-09-21T03:42:32.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:32 smithi187 ceph-mon[34603]: pgmap v739: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-09-21T03:42:33.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 21749 40.51 MB/sec warmup 33 sec latency 150.347 ms 2024-09-21T03:42:34.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 22785 41.36 MB/sec warmup 34 sec latency 42.198 ms 2024-09-21T03:42:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:34 smithi203 ceph-mon[34723]: pgmap v740: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 581 op/s 2024-09-21T03:42:34.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:34 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:34.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:34 smithi187 ceph-mon[34603]: pgmap v740: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 581 op/s 2024-09-21T03:42:34.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:34 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:34 smithi112 ceph-mon[31442]: pgmap v740: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 581 op/s 2024-09-21T03:42:34.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:34 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:42:35.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 23573 41.55 MB/sec warmup 35 sec latency 75.169 ms 2024-09-21T03:42:36.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 24303 41.36 MB/sec warmup 36 sec latency 210.685 ms 2024-09-21T03:42:36.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:36 smithi203 ceph-mon[34723]: pgmap v741: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-09-21T03:42:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:36 smithi187 ceph-mon[34603]: pgmap v741: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-09-21T03:42:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:36 smithi112 ceph-mon[31442]: pgmap v741: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-09-21T03:42:37.542 INFO:tasks.workunit.client.0.smithi112.stdout: 10 25044 41.04 MB/sec warmup 37 sec latency 80.251 ms 2024-09-21T03:42:38.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 25771 41.23 MB/sec warmup 38 sec latency 175.249 ms 2024-09-21T03:42:38.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:38 smithi203 ceph-mon[34723]: pgmap v742: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 705 op/s 2024-09-21T03:42:38.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:38 smithi187 ceph-mon[34603]: pgmap v742: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 705 op/s 2024-09-21T03:42:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:38 smithi112 ceph-mon[31442]: pgmap v742: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 705 op/s 2024-09-21T03:42:39.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 26557 41.06 MB/sec warmup 39 sec latency 67.857 ms 2024-09-21T03:42:40.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 27415 41.35 MB/sec warmup 40 sec latency 66.962 ms 2024-09-21T03:42:40.589 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:40 smithi112 ceph-mon[31442]: pgmap v743: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 16 MiB/s wr, 524 op/s 2024-09-21T03:42:40.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:40 smithi203 ceph-mon[34723]: pgmap v743: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 16 MiB/s wr, 524 op/s 2024-09-21T03:42:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:40 smithi187 ceph-mon[34603]: pgmap v743: 97 pgs: 97 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 16 MiB/s wr, 524 op/s 2024-09-21T03:42:41.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 28134 41.15 MB/sec warmup 41 sec latency 66.693 ms 2024-09-21T03:42:42.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 29107 41.40 MB/sec warmup 42 sec latency 39.642 ms 2024-09-21T03:42:42.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:42 smithi203 ceph-mon[34723]: pgmap v744: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 735 op/s 2024-09-21T03:42:42.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:42 smithi187 ceph-mon[34603]: pgmap v744: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 735 op/s 2024-09-21T03:42:42.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:42 smithi112 ceph-mon[31442]: pgmap v744: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 735 op/s 2024-09-21T03:42:43.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 29814 41.45 MB/sec warmup 43 sec latency 218.044 ms 2024-09-21T03:42:44.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 30572 41.32 MB/sec warmup 44 sec latency 130.405 ms 2024-09-21T03:42:44.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:44 smithi203 ceph-mon[34723]: pgmap v745: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 580 op/s 2024-09-21T03:42:44.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:44 smithi187 ceph-mon[34603]: pgmap v745: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 580 op/s 2024-09-21T03:42:44.739 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:42:44.774 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:44 smithi112 ceph-mon[31442]: pgmap v745: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 580 op/s 2024-09-21T03:42:45.081 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:45.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 31435 41.61 MB/sec warmup 45 sec latency 84.614 ms 2024-09-21T03:42:45.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.655+0000 7f22a5be2640 1 -- 172.21.15.112:0/3066693131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a0108d10 msgr2=0x7f22a010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.655+0000 7f22a5be2640 1 --2- 172.21.15.112:0/3066693131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a0108d10 0x7f22a010f540 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2298009ca0 tx=0x7f229802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.655+0000 7f22a5be2640 1 -- 172.21.15.112:0/3066693131 shutdown_connections 2024-09-21T03:42:45.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.655+0000 7f22a5be2640 1 --2- 172.21.15.112:0/3066693131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a0108d10 0x7f22a010f540 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.655+0000 7f22a5be2640 1 --2- 172.21.15.112:0/3066693131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22a01083c0 0x7f22a01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.655+0000 7f22a5be2640 1 --2- 172.21.15.112:0/3066693131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a0102a80 0x7f22a0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.655+0000 7f22a5be2640 1 -- 172.21.15.112:0/3066693131 >> 172.21.15.112:0/3066693131 conn(0x7f22a0079740 msgr2=0x7f22a0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.655+0000 7f22a5be2640 1 -- 172.21.15.112:0/3066693131 shutdown_connections 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.655+0000 7f22a5be2640 1 -- 172.21.15.112:0/3066693131 wait complete. 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.656+0000 7f22a5be2640 1 Processor -- start 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.656+0000 7f22a5be2640 1 -- start start 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.656+0000 7f22a5be2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22a0102a80 0x7f22a0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.656+0000 7f22a5be2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a01083c0 0x7f22a01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f22a5be2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a0108d10 0x7f22a019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f22a5be2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22a01129e0 con 0x7f22a01083c0 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f22a5be2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0112860 con 0x7f22a0102a80 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f22a5be2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0112b60 con 0x7f22a0108d10 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f229effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a01083c0 0x7f22a01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f229ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a0108d10 0x7f22a019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f229effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a01083c0 0x7f22a01998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39862/0 (socket says 172.21.15.112:39862) 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f229ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a0108d10 0x7f22a019eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:44158/0 (socket says 172.21.15.112:44158) 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f229effd640 1 -- 172.21.15.112:0/863797389 learned_addr learned my addr 172.21.15.112:0/863797389 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f229ffff640 1 -- 172.21.15.112:0/863797389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22a0102a80 msgr2=0x7f22a0199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f229ffff640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22a0102a80 0x7f22a0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f229ffff640 1 -- 172.21.15.112:0/863797389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a01083c0 msgr2=0x7f22a01998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f229ffff640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a01083c0 0x7f22a01998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f229ffff640 1 -- 172.21.15.112:0/863797389 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a019f2a0 con 0x7f22a0108d10 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.657+0000 7f229effd640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a01083c0 0x7f22a01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.658+0000 7f229ffff640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a0108d10 0x7f22a019eb70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f229802ade0 tx=0x7f22980316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.658+0000 7f229cff9640 1 -- 172.21.15.112:0/863797389 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229804c070 con 0x7f22a0108d10 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.658+0000 7f22a5be2640 1 -- 172.21.15.112:0/863797389 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a019f530 con 0x7f22a0108d10 2024-09-21T03:42:45.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.658+0000 7f22a5be2640 1 -- 172.21.15.112:0/863797389 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a006b820 con 0x7f22a0108d10 2024-09-21T03:42:45.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.658+0000 7f229cff9640 1 -- 172.21.15.112:0/863797389 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229803cbd0 con 0x7f22a0108d10 2024-09-21T03:42:45.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.658+0000 7f229cff9640 1 -- 172.21.15.112:0/863797389 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298004040 con 0x7f22a0108d10 2024-09-21T03:42:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.661+0000 7f22ac156740 1 Processor -- start 2024-09-21T03:42:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.662+0000 7f229cff9640 1 -- 172.21.15.112:0/863797389 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f229803c720 con 0x7f22a0108d10 2024-09-21T03:42:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.662+0000 7f22ac156740 1 -- start start 2024-09-21T03:42:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f22ac156740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbdb5070 0x55d4cc15c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f22ac156740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc15cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f22ac156740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 0x55d4cc15d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f22ac156740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d4cc0d2e50 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f22ac156740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d4cc0c2280 con 0x55d4cbdb5070 2024-09-21T03:42:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f22ac156740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d4cc0c6090 con 0x55d4cbd94390 2024-09-21T03:42:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc15cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbdb5070 0x55d4cc15c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbdb5070 0x55d4cc15c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35056/0 (socket says 172.21.15.112:35056) 2024-09-21T03:42:45.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc15cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39872/0 (socket says 172.21.15.112:39872) 2024-09-21T03:42:45.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229effd640 1 -- 172.21.15.112:0/3278404596 learned_addr learned my addr 172.21.15.112:0/3278404596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:45.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229effd640 1 --2- 172.21.15.112:0/3278404596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 0x55d4cc15d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229f7fe640 1 -- 172.21.15.112:0/3278404596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 msgr2=0x55d4cc15d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229f7fe640 1 --2- 172.21.15.112:0/3278404596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 0x55d4cc15d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229f7fe640 1 -- 172.21.15.112:0/3278404596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbdb5070 msgr2=0x55d4cc15c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229f7fe640 1 --2- 172.21.15.112:0/3278404596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbdb5070 0x55d4cc15c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229f7fe640 1 -- 172.21.15.112:0/3278404596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4cbe20650 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.663+0000 7f229cff9640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f22880b6700 0x7f22880b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.664+0000 7f229f7fe640 1 --2- 172.21.15.112:0/3278404596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc15cb90 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2290009750 tx=0x7f2290009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11b9cfb33528400 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.664+0000 7f229cff9640 1 -- 172.21.15.112:0/863797389 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f229810bcd0 con 0x7f22a0108d10 2024-09-21T03:42:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.664+0000 7f229effd640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f22880b6700 0x7f22880b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.664+0000 7f2285ffb640 1 -- 172.21.15.112:0/3278404596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290015020 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.664+0000 7f2285ffb640 1 -- 172.21.15.112:0/3278404596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2290011420 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.664+0000 7f2285ffb640 1 -- 172.21.15.112:0/3278404596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290009e60 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.665+0000 7f22ac156740 1 -- 172.21.15.112:0/3278404596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbd8c1b0 msgr2=0x55d4cc15cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.665+0000 7f22ac156740 1 --2- 172.21.15.112:0/3278404596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc15cb90 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2290009750 tx=0x7f2290009c10 comp rx=0 tx=0).stop 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.665+0000 7f229effd640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f22880b6700 0x7f22880b8c10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2294009fd0 tx=0x7f2294015040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.665+0000 7f22ac156740 1 -- 172.21.15.112:0/3278404596 shutdown_connections 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.665+0000 7f22ac156740 1 --2- 172.21.15.112:0/3278404596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 0x55d4cc15d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.665+0000 7f22ac156740 1 --2- 172.21.15.112:0/3278404596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc15cb90 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.665+0000 7f22ac156740 1 --2- 172.21.15.112:0/3278404596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbdb5070 0x55d4cc15c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.665+0000 7f22ac156740 1 -- 172.21.15.112:0/3278404596 >> 172.21.15.112:0/3278404596 conn(0x55d4cbda9460 msgr2=0x55d4cc15a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.665+0000 7f22ac156740 1 -- 172.21.15.112:0/3278404596 shutdown_connections 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.666+0000 7f22ac156740 1 -- 172.21.15.112:0/3278404596 wait complete. 2024-09-21T03:42:45.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.666+0000 7f22ac156740 1 -- start start 2024-09-21T03:42:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.667+0000 7f22ac156740 1 Processor -- start 2024-09-21T03:42:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.667+0000 7f22ac156740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc167cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.667+0000 7f22ac156740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd94390 0x55d4cc168210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.667+0000 7f229f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc167cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.667+0000 7f22ac156740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc168770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.667+0000 7f22ac156740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d4cbf23700 con 0x55d4cbdb5070 2024-09-21T03:42:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.667+0000 7f229f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc167cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:44172/0 (socket says 172.21.15.112:44172) 2024-09-21T03:42:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.667+0000 7f22ac156740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d4cc05d110 con 0x55d4cbd94390 2024-09-21T03:42:45.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.667+0000 7f22ac156740 1 -- 172.21.15.112:0/3761499066 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d4cc16ae80 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.667+0000 7f229f7fe640 1 -- 172.21.15.112:0/3761499066 learned_addr learned my addr 172.21.15.112:0/3761499066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.668+0000 7f229effd640 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd94390 0x55d4cc168210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.668+0000 7f229ffff640 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc168770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.668+0000 7f22ac156740 10 client.0 _ll_register_callbacks cb 0x55d4cc076990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.668+0000 7f229f7fe640 1 -- 172.21.15.112:0/3761499066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd94390 msgr2=0x55d4cc168210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.668+0000 7f229f7fe640 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd94390 0x55d4cc168210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.668+0000 7f229f7fe640 1 -- 172.21.15.112:0/3761499066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 msgr2=0x55d4cc168770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.668+0000 7f229f7fe640 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc168770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.668+0000 7f229f7fe640 1 -- 172.21.15.112:0/3761499066 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4cc08ac40 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.668+0000 7f229f7fe640 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc167cd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2290009720 tx=0x7f2290011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d68478334bd085eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.668+0000 7f227effd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.669+0000 7f227effd640 1 -- 172.21.15.112:0/3761499066 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229000bad0 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.669+0000 7f227effd640 1 -- 172.21.15.112:0/3761499066 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f229000bc70 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.669+0000 7f22ac156740 1 -- 172.21.15.112:0/3761499066 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2290011420 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.669+0000 7f227effd640 1 -- 172.21.15.112:0/3761499066 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2290009e60 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.669+0000 7f22ac156740 10 client.27286 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.669+0000 7f22ac156740 1 -- 172.21.15.112:0/3761499066 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d4cc176b70 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.669+0000 7f227effd640 1 -- 172.21.15.112:0/3761499066 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229002b8e0 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.669+0000 7f227effd640 1 -- 172.21.15.112:0/3761499066 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f229001a420 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.670+0000 7f22ac156740 10 client.27286 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.670+0000 7f22ac156740 10 client.27286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.670+0000 7f22ac156740 20 client.27286 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.670+0000 7f22ac156740 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d4cbd739f0 0x55d4cc17a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.670+0000 7f22ac156740 4 client.27286 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.670+0000 7f22ac156740 1 -- 172.21.15.112:0/3761499066 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2290011420 con 0x55d4cbd739f0 2024-09-21T03:42:45.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.670+0000 7f229effd640 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d4cbd739f0 0x55d4cc17a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.670+0000 7f227effd640 10 client.27286 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:45.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.671+0000 7f229effd640 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d4cbd739f0 0x55d4cc17a500 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6aee796af14b3c3 server_cookie=82e595d3520a99cd in_seq=0 out_seq=0 2024-09-21T03:42:45.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.672+0000 7f227effd640 1 -- 172.21.15.112:0/3761499066 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f2290011420 con 0x55d4cbd739f0 2024-09-21T03:42:45.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.672+0000 7f227effd640 10 client.27286 handle_command_reply: tid=0 2024-09-21T03:42:45.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.672+0000 7f22ac156740 1 client.27286 shutdown 2024-09-21T03:42:45.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f22ac156740 1 -- 172.21.15.112:0/3761499066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd8c1b0 msgr2=0x55d4cc167cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f22ac156740 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc167cd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2290009720 tx=0x7f2290011800 comp rx=0 tx=0).stop 2024-09-21T03:42:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f22ac156740 1 -- 172.21.15.112:0/3761499066 shutdown_connections 2024-09-21T03:42:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f22ac156740 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d4cbd739f0 0x55d4cc17a500 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f22ac156740 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc168770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f22ac156740 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd94390 0x55d4cc168210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f22ac156740 1 --2- 172.21.15.112:0/3761499066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc167cd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f22ac156740 1 -- 172.21.15.112:0/3761499066 >> 172.21.15.112:0/3761499066 conn(0x55d4cbda9460 msgr2=0x55d4cc15f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f227effd640 0 client.27286 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f22ac156740 1 -- 172.21.15.112:0/3761499066 shutdown_connections 2024-09-21T03:42:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f22ac156740 1 -- 172.21.15.112:0/3761499066 wait complete. 2024-09-21T03:42:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.673+0000 7f22ac156740 20 client.27286 trim_cache size 0 max 16384 2024-09-21T03:42:45.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.705+0000 7f22ac156740 1 Processor -- start 2024-09-21T03:42:45.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.705+0000 7f22ac156740 1 -- start start 2024-09-21T03:42:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.705+0000 7f22ac156740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc15e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.705+0000 7f22ac156740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd94390 0x55d4cc169900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.705+0000 7f22ac156740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.705+0000 7f22ac156740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f228c011070 con 0x55d4cbdb5070 2024-09-21T03:42:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.706+0000 7f22ac156740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f228c00e8b0 con 0x55d4cbd94390 2024-09-21T03:42:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.706+0000 7f22ac156740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f228c00c720 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.706+0000 7f229ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.706+0000 7f229ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39894/0 (socket says 172.21.15.112:39894) 2024-09-21T03:42:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.706+0000 7f229ffff640 1 -- 172.21.15.112:0/3508951116 learned_addr learned my addr 172.21.15.112:0/3508951116 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:45.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.706+0000 7f229effd640 1 --2- 172.21.15.112:0/3508951116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd94390 0x55d4cc169900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.706+0000 7f229ffff640 1 -- 172.21.15.112:0/3508951116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd8c1b0 msgr2=0x55d4cc15e040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:42:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.706+0000 7f229ffff640 1 --2- 172.21.15.112:0/3508951116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc15e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.706+0000 7f229ffff640 1 -- 172.21.15.112:0/3508951116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd94390 msgr2=0x55d4cc169900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.706+0000 7f229ffff640 1 --2- 172.21.15.112:0/3508951116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd94390 0x55d4cc169900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.706+0000 7f229ffff640 1 -- 172.21.15.112:0/3508951116 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4cc174100 con 0x55d4cbdb5070 2024-09-21T03:42:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.707+0000 7f229ffff640 1 --2- 172.21.15.112:0/3508951116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc159b70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f2298005cc0 tx=0x7f229802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef81ef3ab93a634 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.707+0000 7f22857fa640 1 -- 172.21.15.112:0/3508951116 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229803ca10 con 0x55d4cbdb5070 2024-09-21T03:42:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.708+0000 7f22857fa640 1 -- 172.21.15.112:0/3508951116 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2298036020 con 0x55d4cbdb5070 2024-09-21T03:42:45.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.708+0000 7f22857fa640 1 -- 172.21.15.112:0/3508951116 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980db570 con 0x55d4cbdb5070 2024-09-21T03:42:45.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.709+0000 7f22ac156740 1 -- 172.21.15.112:0/3508951116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 msgr2=0x55d4cc159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.709+0000 7f22ac156740 1 --2- 172.21.15.112:0/3508951116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc159b70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f2298005cc0 tx=0x7f229802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.709+0000 7f22ac156740 1 -- 172.21.15.112:0/3508951116 shutdown_connections 2024-09-21T03:42:45.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.709+0000 7f22ac156740 1 --2- 172.21.15.112:0/3508951116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc159b70 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.709+0000 7f22ac156740 1 --2- 172.21.15.112:0/3508951116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd94390 0x55d4cc169900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.709+0000 7f22ac156740 1 --2- 172.21.15.112:0/3508951116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.709+0000 7f22ac156740 1 -- 172.21.15.112:0/3508951116 >> 172.21.15.112:0/3508951116 conn(0x55d4cbda9460 msgr2=0x55d4cc1691f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.709+0000 7f22ac156740 1 -- 172.21.15.112:0/3508951116 shutdown_connections 2024-09-21T03:42:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.710+0000 7f22ac156740 1 -- 172.21.15.112:0/3508951116 wait complete. 2024-09-21T03:42:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.710+0000 7f22ac156740 1 -- start start 2024-09-21T03:42:45.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.711+0000 7f22ac156740 1 Processor -- start 2024-09-21T03:42:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.711+0000 7f22ac156740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc16c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.712+0000 7f22ac156740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 0x55d4cc16c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.712+0000 7f22ac156740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc16ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.712+0000 7f22ac156740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f228c00cd70 con 0x55d4cbdb5070 2024-09-21T03:42:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.712+0000 7f22ac156740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f228c00e1e0 con 0x55d4cbd8c1b0 2024-09-21T03:42:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.712+0000 7f22ac156740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f228c00d440 con 0x55d4cbd94390 2024-09-21T03:42:45.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.712+0000 7f22ac156740 10 client.0 _ll_register_callbacks cb 0x7f2294009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.713+0000 7f229effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 0x55d4cc16c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.713+0000 7f229effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 0x55d4cc16c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:44180/0 (socket says 172.21.15.112:44180) 2024-09-21T03:42:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.713+0000 7f229effd640 1 -- 172.21.15.112:0/510503214 learned_addr learned my addr 172.21.15.112:0/510503214 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.713+0000 7f229effd640 1 -- 172.21.15.112:0/510503214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd8c1b0 msgr2=0x55d4cc16c240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:42:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.713+0000 7f229effd640 1 --2- 172.21.15.112:0/510503214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc16c240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.713+0000 7f229effd640 1 -- 172.21.15.112:0/510503214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 msgr2=0x55d4cc16ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:45.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.713+0000 7f229effd640 1 --2- 172.21.15.112:0/510503214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc16ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.713+0000 7f229effd640 1 -- 172.21.15.112:0/510503214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4cc1e3fb0 con 0x55d4cbd94390 2024-09-21T03:42:45.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.714+0000 7f229effd640 1 --2- 172.21.15.112:0/510503214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 0x55d4cc16c780 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22940058c0 tx=0x7f2294005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da0a6700d9bab96f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:45.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.714+0000 7f227effd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:42:45.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.715+0000 7f227effd640 1 -- 172.21.15.112:0/510503214 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294037060 con 0x55d4cbd94390 2024-09-21T03:42:45.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.715+0000 7f22ac156740 1 -- 172.21.15.112:0/510503214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2298036020 con 0x55d4cbd94390 2024-09-21T03:42:45.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.715+0000 7f227effd640 1 -- 172.21.15.112:0/510503214 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f2294009040 con 0x55d4cbd94390 2024-09-21T03:42:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.716+0000 7f227effd640 1 -- 172.21.15.112:0/510503214 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f22940038c0 con 0x55d4cbd94390 2024-09-21T03:42:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.716+0000 7f22ac156740 10 client.27289 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.716+0000 7f22ac156740 1 -- 172.21.15.112:0/510503214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d4cc1e52d0 con 0x55d4cbd94390 2024-09-21T03:42:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.717+0000 7f227effd640 1 -- 172.21.15.112:0/510503214 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2298036020 con 0x55d4cbd94390 2024-09-21T03:42:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.717+0000 7f227effd640 1 -- 172.21.15.112:0/510503214 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f229401d040 con 0x55d4cbd94390 2024-09-21T03:42:45.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.717+0000 7f22ac156740 10 client.27289 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.717+0000 7f22ac156740 10 client.27289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.717+0000 7f22ac156740 20 client.27289 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.717+0000 7f22ac156740 1 --2- 172.21.15.112:0/510503214 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d4cbd48da0 0x55d4cc1e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.717+0000 7f22ac156740 4 client.27289 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:42:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.717+0000 7f22ac156740 1 -- 172.21.15.112:0/510503214 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d4cc0d4da0 con 0x55d4cbd48da0 2024-09-21T03:42:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.717+0000 7f229f7fe640 1 --2- 172.21.15.112:0/510503214 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d4cbd48da0 0x55d4cc1e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.718+0000 7f227effd640 10 client.27289 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:45.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.719+0000 7f229f7fe640 1 --2- 172.21.15.112:0/510503214 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d4cbd48da0 0x55d4cc1e7ac0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=598a8b1ec73e7668 server_cookie=47dbb93a354f965 in_seq=0 out_seq=0 2024-09-21T03:42:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f227effd640 1 -- 172.21.15.112:0/510503214 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6054 (crc 0 0 0) 0x55d4cc0d4da0 con 0x55d4cbd48da0 2024-09-21T03:42:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f227effd640 10 client.27289 handle_command_reply: tid=0 2024-09-21T03:42:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f22ac156740 1 client.27289 shutdown 2024-09-21T03:42:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f22ac156740 1 -- 172.21.15.112:0/510503214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 msgr2=0x55d4cc16c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f22ac156740 1 --2- 172.21.15.112:0/510503214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 0x55d4cc16c780 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22940058c0 tx=0x7f2294005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f22ac156740 1 -- 172.21.15.112:0/510503214 shutdown_connections 2024-09-21T03:42:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f22ac156740 1 --2- 172.21.15.112:0/510503214 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55d4cbd48da0 0x55d4cc1e7ac0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f22ac156740 1 --2- 172.21.15.112:0/510503214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d4cbdb5070 0x55d4cc16ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f22ac156740 1 --2- 172.21.15.112:0/510503214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4cbd94390 0x55d4cc16c780 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f22ac156740 1 --2- 172.21.15.112:0/510503214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4cbd8c1b0 0x55d4cc16c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f22ac156740 1 -- 172.21.15.112:0/510503214 >> 172.21.15.112:0/510503214 conn(0x55d4cbda9460 msgr2=0x55d4cc15de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f227effd640 0 client.27289 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:45.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f22ac156740 1 -- 172.21.15.112:0/510503214 shutdown_connections 2024-09-21T03:42:45.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.721+0000 7f22ac156740 1 -- 172.21.15.112:0/510503214 wait complete. 2024-09-21T03:42:45.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.722+0000 7f22ac156740 20 client.27289 trim_cache size 0 max 16384 2024-09-21T03:42:45.725 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:42:45.725 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:42:45.725 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:42:45.725 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:42:45.725 INFO:teuthology.orchestra.run.smithi112.stdout: "4591c632-505e-4006-8d95-6080e5d063e2": { 2024-09-21T03:42:45.725 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:42:11.919443+0000", 2024-09-21T03:42:45.725 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.725 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.725 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.725 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.725 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c": { 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:45.957537+0000", 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: "3d0e4442-3ee3-44bd-99e2-f0a81392c8c4": { 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:41:28.881491+0000", 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.726 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:42:45.727 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.728 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:42:45.730 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.732 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.733 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.734 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.735 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.736 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:42:45.737 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.724+0000 7f22a5be2640 1 -- 172.21.15.112:0/863797389 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f22880b6700 msgr2=0x7f22880b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.724+0000 7f22a5be2640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f22880b6700 0x7f22880b8c10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2294009fd0 tx=0x7f2294015040 comp rx=0 tx=0).stop 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.724+0000 7f22a5be2640 1 -- 172.21.15.112:0/863797389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a0108d10 msgr2=0x7f22a019eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.724+0000 7f22a5be2640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a0108d10 0x7f22a019eb70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f229802ade0 tx=0x7f22980316f0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.725+0000 7f22a5be2640 1 -- 172.21.15.112:0/863797389 shutdown_connections 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.725+0000 7f22a5be2640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f22880b6700 0x7f22880b8c10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.725+0000 7f22a5be2640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a0108d10 0x7f22a019eb70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.725+0000 7f22a5be2640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22a01083c0 0x7f22a01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.725+0000 7f22a5be2640 1 --2- 172.21.15.112:0/863797389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22a0102a80 0x7f22a0199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.725+0000 7f22a5be2640 1 -- 172.21.15.112:0/863797389 >> 172.21.15.112:0/863797389 conn(0x7f22a0079740 msgr2=0x7f22a0104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.725+0000 7f22a5be2640 1 -- 172.21.15.112:0/863797389 shutdown_connections 2024-09-21T03:42:45.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:45.725+0000 7f22a5be2640 1 -- 172.21.15.112:0/863797389 wait complete. 2024-09-21T03:42:46.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 32269 41.68 MB/sec warmup 46 sec latency 115.233 ms 2024-09-21T03:42:46.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:46 smithi203 ceph-mon[34723]: pgmap v746: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 18 MiB/s wr, 587 op/s 2024-09-21T03:42:46.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:46 smithi187 ceph-mon[34603]: pgmap v746: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 18 MiB/s wr, 587 op/s 2024-09-21T03:42:46.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:46 smithi112 ceph-mon[31442]: pgmap v746: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 18 MiB/s wr, 587 op/s 2024-09-21T03:42:47.013 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:42:47.014 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:42:47.344 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:47.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 32953 41.50 MB/sec warmup 47 sec latency 72.604 ms 2024-09-21T03:42:47.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.853+0000 7efcd4110640 1 -- 172.21.15.112:0/3038372340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efccc102ac0 msgr2=0x7efccc102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.853+0000 7efcd4110640 1 --2- 172.21.15.112:0/3038372340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efccc102ac0 0x7efccc102ed0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009900 tx=0x7efcc802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.853+0000 7efcd4110640 1 -- 172.21.15.112:0/3038372340 shutdown_connections 2024-09-21T03:42:47.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.853+0000 7efcd4110640 1 --2- 172.21.15.112:0/3038372340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efccc108d00 0x7efccc10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.853+0000 7efcd4110640 1 --2- 172.21.15.112:0/3038372340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efccc1083b0 0x7efccc1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.853+0000 7efcd4110640 1 --2- 172.21.15.112:0/3038372340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efccc102ac0 0x7efccc102ed0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.853+0000 7efcd4110640 1 -- 172.21.15.112:0/3038372340 >> 172.21.15.112:0/3038372340 conn(0x7efccc079790 msgr2=0x7efccc079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.853+0000 7efcd4110640 1 -- 172.21.15.112:0/3038372340 shutdown_connections 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.853+0000 7efcd4110640 1 -- 172.21.15.112:0/3038372340 wait complete. 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.854+0000 7efcd4110640 1 Processor -- start 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.854+0000 7efcd4110640 1 -- start start 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd4110640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efccc102ac0 0x7efccc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd4110640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efccc1083b0 0x7efccc199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd4110640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efccc108d00 0x7efccc19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd4110640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1129c0 con 0x7efccc1083b0 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd4110640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efccc112840 con 0x7efccc108d00 2024-09-21T03:42:47.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd4110640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efccc112b40 con 0x7efccc102ac0 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd16ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efccc1083b0 0x7efccc199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd16ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efccc1083b0 0x7efccc199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39926/0 (socket says 172.21.15.112:39926) 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd16ab640 1 -- 172.21.15.112:0/769065062 learned_addr learned my addr 172.21.15.112:0/769065062 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd26ad640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efccc108d00 0x7efccc19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd1eac640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efccc102ac0 0x7efccc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd16ab640 1 -- 172.21.15.112:0/769065062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efccc102ac0 msgr2=0x7efccc1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd16ab640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efccc102ac0 0x7efccc1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd16ab640 1 -- 172.21.15.112:0/769065062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efccc108d00 msgr2=0x7efccc19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd16ab640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efccc108d00 0x7efccc19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.855+0000 7efcd16ab640 1 -- 172.21.15.112:0/769065062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc19f360 con 0x7efccc1083b0 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.856+0000 7efcd16ab640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efccc1083b0 0x7efccc199990 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00d740 tx=0x7efcbc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.856+0000 7efcc2ffd640 1 -- 172.21.15.112:0/769065062 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc017070 con 0x7efccc1083b0 2024-09-21T03:42:47.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.856+0000 7efcd4110640 1 -- 172.21.15.112:0/769065062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efccc19f650 con 0x7efccc1083b0 2024-09-21T03:42:47.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.856+0000 7efcc2ffd640 1 -- 172.21.15.112:0/769065062 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcbc00de50 con 0x7efccc1083b0 2024-09-21T03:42:47.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.856+0000 7efcc2ffd640 1 -- 172.21.15.112:0/769065062 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc00f620 con 0x7efccc1083b0 2024-09-21T03:42:47.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.856+0000 7efcd4110640 1 -- 172.21.15.112:0/769065062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efccc06b820 con 0x7efccc1083b0 2024-09-21T03:42:47.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.858+0000 7efcd4110640 1 -- 172.21.15.112:0/769065062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efccc102ed0 con 0x7efccc1083b0 2024-09-21T03:42:47.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.860+0000 7efcc2ffd640 1 -- 172.21.15.112:0/769065062 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7efcbc01a070 con 0x7efccc1083b0 2024-09-21T03:42:47.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.861+0000 7efcc2ffd640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efcb80b6680 0x7efcb80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:47.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.861+0000 7efcc2ffd640 1 -- 172.21.15.112:0/769065062 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7efcbc0e7070 con 0x7efccc1083b0 2024-09-21T03:42:47.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.861+0000 7efcd1eac640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efcb80b6680 0x7efcb80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:47.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.862+0000 7efcd1eac640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efcb80b6680 0x7efcb80b8b90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efcc8031440 tx=0x7efcc803c040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:47.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:47.867+0000 7efcc2ffd640 1 -- 172.21.15.112:0/769065062 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efcbc0b19d0 con 0x7efccc1083b0 2024-09-21T03:42:48.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.131+0000 7efcd4110640 1 -- 172.21.15.112:0/769065062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efccc06c300 con 0x7efccc1083b0 2024-09-21T03:42:48.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.132+0000 7efcc2ffd640 1 -- 172.21.15.112:0/769065062 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11936 (secure 0 0 0) 0x7efcbc0b6910 con 0x7efccc1083b0 2024-09-21T03:42:48.134 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:42:48.134 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":35,"btime":"2024-09-21T03:42:33:280889+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:42:32.317716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:42:48.135 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 35 2024-09-21T03:42:48.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 -- 172.21.15.112:0/769065062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efcb80b6680 msgr2=0x7efcb80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efcb80b6680 0x7efcb80b8b90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efcc8031440 tx=0x7efcc803c040 comp rx=0 tx=0).stop 2024-09-21T03:42:48.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 -- 172.21.15.112:0/769065062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efccc1083b0 msgr2=0x7efccc199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:48.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efccc1083b0 0x7efccc199990 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00d740 tx=0x7efcbc00dc00 comp rx=0 tx=0).stop 2024-09-21T03:42:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 -- 172.21.15.112:0/769065062 shutdown_connections 2024-09-21T03:42:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efcb80b6680 0x7efcb80b8b90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efccc108d00 0x7efccc19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efccc1083b0 0x7efccc199990 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 --2- 172.21.15.112:0/769065062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efccc102ac0 0x7efccc1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 -- 172.21.15.112:0/769065062 >> 172.21.15.112:0/769065062 conn(0x7efccc079790 msgr2=0x7efccc106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 -- 172.21.15.112:0/769065062 shutdown_connections 2024-09-21T03:42:48.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:48.135+0000 7efcd4110640 1 -- 172.21.15.112:0/769065062 wait complete. 2024-09-21T03:42:48.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:48 smithi112 ceph-mon[31442]: pgmap v747: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 741 op/s 2024-09-21T03:42:48.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:48 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/769065062' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:48.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 33573 41.27 MB/sec warmup 48 sec latency 123.003 ms 2024-09-21T03:42:48.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:48 smithi203 ceph-mon[34723]: pgmap v747: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 741 op/s 2024-09-21T03:42:48.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:48 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/769065062' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:48.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:48 smithi187 ceph-mon[34603]: pgmap v747: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 741 op/s 2024-09-21T03:42:48.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:48 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/769065062' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:42:48.960 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:42:49.267 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:49.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 34455 41.14 MB/sec warmup 49 sec latency 97.481 ms 2024-09-21T03:42:50.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.033+0000 7f66abbe7640 1 -- 172.21.15.112:0/1378659653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a4102a60 msgr2=0x7f66a4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.033+0000 7f66abbe7640 1 --2- 172.21.15.112:0/1378659653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a4102a60 0x7f66a4102e70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009900 tx=0x7f66a002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:50.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.033+0000 7f66abbe7640 1 -- 172.21.15.112:0/1378659653 shutdown_connections 2024-09-21T03:42:50.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.033+0000 7f66abbe7640 1 --2- 172.21.15.112:0/1378659653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66a4108cf0 0x7f66a410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.033+0000 7f66abbe7640 1 --2- 172.21.15.112:0/1378659653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41083a0 0x7f66a41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.033+0000 7f66abbe7640 1 --2- 172.21.15.112:0/1378659653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a4102a60 0x7f66a4102e70 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.033+0000 7f66abbe7640 1 -- 172.21.15.112:0/1378659653 >> 172.21.15.112:0/1378659653 conn(0x7f66a4079730 msgr2=0x7f66a4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.034+0000 7f66abbe7640 1 -- 172.21.15.112:0/1378659653 shutdown_connections 2024-09-21T03:42:50.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.034+0000 7f66abbe7640 1 -- 172.21.15.112:0/1378659653 wait complete. 2024-09-21T03:42:50.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.034+0000 7f66abbe7640 1 Processor -- start 2024-09-21T03:42:50.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66abbe7640 1 -- start start 2024-09-21T03:42:50.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66abbe7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66a4102a60 0x7f66a4073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66abbe7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41083a0 0x7f66a4073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66abbe7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a4108cf0 0x7f66a40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66abbe7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4112960 con 0x7f66a4108cf0 2024-09-21T03:42:50.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66abbe7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41127e0 con 0x7f66a4102a60 2024-09-21T03:42:50.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66abbe7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4112ae0 con 0x7f66a41083a0 2024-09-21T03:42:50.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66aa184640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a4108cf0 0x7f66a40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66a9983640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66a4102a60 0x7f66a4073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66a9182640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41083a0 0x7f66a4073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66aa184640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a4108cf0 0x7f66a40740f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39942/0 (socket says 172.21.15.112:39942) 2024-09-21T03:42:50.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66aa184640 1 -- 172.21.15.112:0/1650826039 learned_addr learned my addr 172.21.15.112:0/1650826039 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:50.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.035+0000 7f66a9182640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41083a0 0x7f66a4073bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:44212/0 (socket says 172.21.15.112:44212) 2024-09-21T03:42:50.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.036+0000 7f66a9983640 1 -- 172.21.15.112:0/1650826039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41083a0 msgr2=0x7f66a4073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.036+0000 7f66a9983640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41083a0 0x7f66a4073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.036+0000 7f66a9983640 1 -- 172.21.15.112:0/1650826039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a4108cf0 msgr2=0x7f66a40740f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.036+0000 7f66a9983640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a4108cf0 0x7f66a40740f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.036+0000 7f66a9983640 1 -- 172.21.15.112:0/1650826039 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a41b0f40 con 0x7f66a4102a60 2024-09-21T03:42:50.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.036+0000 7f66a9983640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66a4102a60 0x7f66a4073670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f66a002a930 tx=0x7f66a0031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.036+0000 7f669affd640 1 -- 172.21.15.112:0/1650826039 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a003f030 con 0x7f66a4102a60 2024-09-21T03:42:50.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.036+0000 7f66abbe7640 1 -- 172.21.15.112:0/1650826039 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a41b11d0 con 0x7f66a4102a60 2024-09-21T03:42:50.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.036+0000 7f66abbe7640 1 -- 172.21.15.112:0/1650826039 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a41b1700 con 0x7f66a4102a60 2024-09-21T03:42:50.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.037+0000 7f669affd640 1 -- 172.21.15.112:0/1650826039 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a0004490 con 0x7f66a4102a60 2024-09-21T03:42:50.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.037+0000 7f669affd640 1 -- 172.21.15.112:0/1650826039 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0035070 con 0x7f66a4102a60 2024-09-21T03:42:50.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66b215b740 1 Processor -- start 2024-09-21T03:42:50.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66b215b740 1 -- start start 2024-09-21T03:42:50.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f669affd640 1 -- 172.21.15.112:0/1650826039 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f66a0042030 con 0x7f66a4102a60 2024-09-21T03:42:50.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66b215b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 0x5614525622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66b215b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614521921b0 0x561452562800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66b215b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 0x561452562d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66b215b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5614524d8ea0 con 0x56145219a390 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66b215b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5614524c82d0 con 0x5614521921b0 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66b215b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5614524cc0e0 con 0x5614521bb070 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66a9983640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614521921b0 0x561452562800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66aa184640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 0x5614525622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66aa184640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 0x5614525622c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:44216/0 (socket says 172.21.15.112:44216) 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66a9983640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614521921b0 0x561452562800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35114/0 (socket says 172.21.15.112:35114) 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66aa184640 1 -- 172.21.15.112:0/4192141094 learned_addr learned my addr 172.21.15.112:0/4192141094 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.040+0000 7f66a9182640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 0x561452562d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.041+0000 7f66aa184640 1 -- 172.21.15.112:0/4192141094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614521921b0 msgr2=0x561452562800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.041+0000 7f66aa184640 1 --2- 172.21.15.112:0/4192141094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614521921b0 0x561452562800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.041+0000 7f66aa184640 1 -- 172.21.15.112:0/4192141094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 msgr2=0x561452562d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.041+0000 7f66aa184640 1 --2- 172.21.15.112:0/4192141094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 0x561452562d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.041+0000 7f66aa184640 1 -- 172.21.15.112:0/4192141094 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561452226650 con 0x5614521bb070 2024-09-21T03:42:50.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.041+0000 7f66a9182640 1 --2- 172.21.15.112:0/4192141094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 0x561452562d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:50.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.041+0000 7f66aa184640 1 --2- 172.21.15.112:0/4192141094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 0x5614525622c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f669c011570 tx=0x7f669c011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c5fdaf060d9340 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.041+0000 7f669affd640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f66900b6700 0x7f66900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.041+0000 7f668ffff640 1 -- 172.21.15.112:0/4192141094 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c004b20 con 0x5614521bb070 2024-09-21T03:42:50.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.041+0000 7f668ffff640 1 -- 172.21.15.112:0/4192141094 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669c004cc0 con 0x5614521bb070 2024-09-21T03:42:50.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.042+0000 7f668ffff640 1 -- 172.21.15.112:0/4192141094 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c005490 con 0x5614521bb070 2024-09-21T03:42:50.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.042+0000 7f669affd640 1 -- 172.21.15.112:0/1650826039 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f66a01076f0 con 0x7f66a4102a60 2024-09-21T03:42:50.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.042+0000 7f66a9983640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f66900b6700 0x7f66900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.043+0000 7f66b215b740 1 -- 172.21.15.112:0/4192141094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 msgr2=0x5614525622c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.043+0000 7f66a9983640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f66900b6700 0x7f66900b8c10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f66a0104f20 tx=0x7f66a010d550 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.043+0000 7f66b215b740 1 --2- 172.21.15.112:0/4192141094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 0x5614525622c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f669c011570 tx=0x7f669c011a30 comp rx=0 tx=0).stop 2024-09-21T03:42:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.043+0000 7f66b215b740 1 -- 172.21.15.112:0/4192141094 shutdown_connections 2024-09-21T03:42:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.043+0000 7f66b215b740 1 --2- 172.21.15.112:0/4192141094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 0x561452562d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.043+0000 7f66b215b740 1 --2- 172.21.15.112:0/4192141094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614521921b0 0x561452562800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.043+0000 7f66b215b740 1 --2- 172.21.15.112:0/4192141094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 0x5614525622c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.043+0000 7f66b215b740 1 -- 172.21.15.112:0/4192141094 >> 172.21.15.112:0/4192141094 conn(0x5614521af460 msgr2=0x5614525601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.043+0000 7f66b215b740 1 -- 172.21.15.112:0/4192141094 shutdown_connections 2024-09-21T03:42:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.043+0000 7f66b215b740 1 -- 172.21.15.112:0/4192141094 wait complete. 2024-09-21T03:42:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.044+0000 7f66b215b740 1 -- start start 2024-09-21T03:42:50.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.044+0000 7f66b215b740 1 Processor -- start 2024-09-21T03:42:50.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.045+0000 7f66b215b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 0x56145256d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.045+0000 7f66b215b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 0x56145256dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.045+0000 7f66aa184640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 0x56145256d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.045+0000 7f66aa184640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 0x56145256d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:44228/0 (socket says 172.21.15.112:44228) 2024-09-21T03:42:50.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.045+0000 7f66aa184640 1 -- 172.21.15.112:0/1470659599 learned_addr learned my addr 172.21.15.112:0/1470659599 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:50.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.045+0000 7f66b215b740 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614521bb070 0x56145256e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.045+0000 7f66b215b740 1 -- 172.21.15.112:0/1470659599 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561452329700 con 0x5614521bb070 2024-09-21T03:42:50.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.045+0000 7f66b215b740 1 -- 172.21.15.112:0/1470659599 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561452463160 con 0x56145219a390 2024-09-21T03:42:50.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.045+0000 7f66b215b740 1 -- 172.21.15.112:0/1470659599 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561452570aa0 con 0x5614521921b0 2024-09-21T03:42:50.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.045+0000 7f66a9182640 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 0x56145256dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.045+0000 7f66aa184640 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614521bb070 0x56145256e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f66b215b740 10 client.0 _ll_register_callbacks cb 0x56145247c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:50.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f66aa184640 1 -- 172.21.15.112:0/1470659599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 msgr2=0x56145256dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f66aa184640 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 0x56145256dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f66aa184640 1 -- 172.21.15.112:0/1470659599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614521bb070 msgr2=0x56145256e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f66aa184640 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614521bb070 0x56145256e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f66aa184640 1 -- 172.21.15.112:0/1470659599 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561452490c90 con 0x5614521921b0 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f66b215b740 1 -- 172.21.15.112:0/1470659599 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669c004cc0 con 0x5614521921b0 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f66aa184640 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 0x56145256d8a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f669c009a60 tx=0x7f669c0099b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97612a17faa2d9c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f668d7fa640 10 client.27307 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f668d7fa640 1 -- 172.21.15.112:0/1470659599 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c033050 con 0x5614521921b0 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f668d7fa640 1 -- 172.21.15.112:0/1470659599 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f669c005190 con 0x5614521921b0 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f668d7fa640 1 -- 172.21.15.112:0/1470659599 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f669c029070 con 0x5614521921b0 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f66b215b740 10 client.27307 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.046+0000 7f66b215b740 1 -- 172.21.15.112:0/1470659599 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({fsmap=35}) -- 0x56145257c5e0 con 0x5614521921b0 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.047+0000 7f668d7fa640 1 -- 172.21.15.112:0/1470659599 <== mon.1 v2:172.21.15.203:3300/0 4 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f669c0053d0 con 0x5614521921b0 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.047+0000 7f66b215b740 10 client.27307 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.047+0000 7f66b215b740 10 client.27307 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:42:50.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.047+0000 7f66b215b740 20 client.27307 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:50.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.047+0000 7f66b215b740 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5614521799f0 0x56145257fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.047+0000 7f66b215b740 4 client.27307 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:50.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.047+0000 7f66b215b740 1 -- 172.21.15.112:0/1470659599 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614524dadf0 con 0x5614521799f0 2024-09-21T03:42:50.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.047+0000 7f66a9182640 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5614521799f0 0x56145257fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.048+0000 7f668d7fa640 10 client.27307 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:42:50.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.048+0000 7f66a9182640 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5614521799f0 0x56145257fc40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3370b38c0a754288 server_cookie=2d5c0b3236568447 in_seq=0 out_seq=0 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f668d7fa640 1 -- 172.21.15.112:0/1470659599 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5614524dadf0 con 0x5614521799f0 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f668d7fa640 10 client.27307 handle_command_reply: tid=0 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 1 client.27307 shutdown 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 1 -- 172.21.15.112:0/1470659599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 msgr2=0x56145256d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 0x56145256d8a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f669c009a60 tx=0x7f669c0099b0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 1 -- 172.21.15.112:0/1470659599 shutdown_connections 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x5614521799f0 0x56145257fc40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614521bb070 0x56145256e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 0x56145256dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 1 --2- 172.21.15.112:0/1470659599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 0x56145256d8a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f668d7fa640 0 client.27307 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 1 -- 172.21.15.112:0/1470659599 >> 172.21.15.112:0/1470659599 conn(0x5614521af460 msgr2=0x561452565450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 1 -- 172.21.15.112:0/1470659599 shutdown_connections 2024-09-21T03:42:50.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 1 -- 172.21.15.112:0/1470659599 wait complete. 2024-09-21T03:42:50.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.051+0000 7f66b215b740 20 client.27307 trim_cache size 0 max 16384 2024-09-21T03:42:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.070+0000 7f66b215b740 1 Processor -- start 2024-09-21T03:42:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66b215b740 1 -- start start 2024-09-21T03:42:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66b215b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 0x56145256f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66b215b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 0x56145256aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66b215b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614521bb070 0x56145256afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66b215b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6688010fe0 con 0x56145219a390 2024-09-21T03:42:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66b215b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f668800e820 con 0x5614521bb070 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66b215b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f668800c690 con 0x5614521921b0 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66a9182640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 0x56145256aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66a9182640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 0x56145256aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39956/0 (socket says 172.21.15.112:39956) 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66a9182640 1 -- 172.21.15.112:0/3580680118 learned_addr learned my addr 172.21.15.112:0/3580680118 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66aa184640 1 --2- 172.21.15.112:0/3580680118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614521bb070 0x56145256afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66aa184640 1 --2- 172.21.15.112:0/3580680118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 0x56145256f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66a9182640 1 -- 172.21.15.112:0/3580680118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 msgr2=0x56145256f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66a9182640 1 --2- 172.21.15.112:0/3580680118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 0x56145256f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66a9182640 1 -- 172.21.15.112:0/3580680118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614521bb070 msgr2=0x56145256afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66a9182640 1 --2- 172.21.15.112:0/3580680118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614521bb070 0x56145256afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66a9182640 1 -- 172.21.15.112:0/3580680118 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614525630d0 con 0x56145219a390 2024-09-21T03:42:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.071+0000 7f66aa184640 1 --2- 172.21.15.112:0/3580680118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 0x56145256f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:42:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.072+0000 7f66a9182640 1 --2- 172.21.15.112:0/3580680118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 0x56145256aa90 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x561452580370 tx=0x7f6694005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9583c09f23a96fde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.072+0000 7f668f7fe640 1 -- 172.21.15.112:0/3580680118 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669401e6b0 con 0x56145219a390 2024-09-21T03:42:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.072+0000 7f668f7fe640 1 -- 172.21.15.112:0/3580680118 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669401ed00 con 0x56145219a390 2024-09-21T03:42:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.072+0000 7f668f7fe640 1 -- 172.21.15.112:0/3580680118 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694024dd0 con 0x56145219a390 2024-09-21T03:42:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.073+0000 7f66b215b740 1 -- 172.21.15.112:0/3580680118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 msgr2=0x56145256aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.073+0000 7f66b215b740 1 --2- 172.21.15.112:0/3580680118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 0x56145256aa90 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x561452580370 tx=0x7f6694005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.073+0000 7f66b215b740 1 -- 172.21.15.112:0/3580680118 shutdown_connections 2024-09-21T03:42:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.073+0000 7f66b215b740 1 --2- 172.21.15.112:0/3580680118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614521bb070 0x56145256afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.073+0000 7f66b215b740 1 --2- 172.21.15.112:0/3580680118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145219a390 0x56145256aa90 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.073+0000 7f66b215b740 1 --2- 172.21.15.112:0/3580680118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521921b0 0x56145256f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.073+0000 7f66b215b740 1 -- 172.21.15.112:0/3580680118 >> 172.21.15.112:0/3580680118 conn(0x5614521af460 msgr2=0x56145256edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.073+0000 7f66b215b740 1 -- 172.21.15.112:0/3580680118 shutdown_connections 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.073+0000 7f66b215b740 1 -- 172.21.15.112:0/3580680118 wait complete. 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.073+0000 7f66b215b740 1 -- start start 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66b215b740 1 Processor -- start 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66b215b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614521921b0 0x561452571dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66b215b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 0x561452572310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66b215b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 0x561452572850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66b215b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f668800cce0 con 0x5614521921b0 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66b215b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f668800e150 con 0x56145219a390 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66b215b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f668800d3b0 con 0x5614521bb070 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66b215b740 10 client.0 _ll_register_callbacks cb 0x7f669c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66a9182640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 0x561452572310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66a9182640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 0x561452572310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35138/0 (socket says 172.21.15.112:35138) 2024-09-21T03:42:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66a9182640 1 -- 172.21.15.112:0/4138375047 learned_addr learned my addr 172.21.15.112:0/4138375047 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.074+0000 7f66aa184640 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614521921b0 0x561452571dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.075+0000 7f66a9182640 1 -- 172.21.15.112:0/4138375047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 msgr2=0x561452572850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.075+0000 7f66aa184640 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 0x561452572850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.075+0000 7f66a9182640 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 0x561452572850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.075+0000 7f66a9182640 1 -- 172.21.15.112:0/4138375047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614521921b0 msgr2=0x561452571dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.075+0000 7f66a9182640 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614521921b0 0x561452571dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.075+0000 7f66a9182640 1 -- 172.21.15.112:0/4138375047 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614525e96b0 con 0x56145219a390 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.075+0000 7f66aa184640 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 0x561452572850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.075+0000 7f66a9182640 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 0x561452572310 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6694009710 tx=0x7f66940244d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485b66892fe50f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.075+0000 7f668d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.076+0000 7f668d7fa640 1 -- 172.21.15.112:0/4138375047 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669402a040 con 0x56145219a390 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.076+0000 7f668d7fa640 1 -- 172.21.15.112:0/4138375047 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f6694024a50 con 0x56145219a390 2024-09-21T03:42:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.076+0000 7f66b215b740 1 -- 172.21.15.112:0/4138375047 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669401ed00 con 0x56145219a390 2024-09-21T03:42:50.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.076+0000 7f668d7fa640 1 -- 172.21.15.112:0/4138375047 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6694024c50 con 0x56145219a390 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.077+0000 7f66b215b740 10 client.27290 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.077+0000 7f66b215b740 1 -- 172.21.15.112:0/4138375047 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5614525ea8a0 con 0x56145219a390 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.077+0000 7f668d7fa640 1 -- 172.21.15.112:0/4138375047 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669401ed00 con 0x56145219a390 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.077+0000 7f668d7fa640 1 -- 172.21.15.112:0/4138375047 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f669402d070 con 0x56145219a390 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.077+0000 7f66b215b740 10 client.27290 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.077+0000 7f66b215b740 10 client.27290 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.077+0000 7f66b215b740 20 client.27290 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.077+0000 7f66b215b740 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56145214eda0 0x5614525ed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.077+0000 7f66b215b740 4 client.27290 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.077+0000 7f66b215b740 1 -- 172.21.15.112:0/4138375047 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614524d68a0 con 0x56145214eda0 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.078+0000 7f66aa184640 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56145214eda0 0x5614525ed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.079+0000 7f668d7fa640 10 client.27290 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:42:50.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.079+0000 7f66aa184640 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56145214eda0 0x5614525ed0b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1187451f49afcac server_cookie=8e46e17565dea9f3 in_seq=0 out_seq=0 2024-09-21T03:42:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f668d7fa640 1 -- 172.21.15.112:0/4138375047 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614524d68a0 con 0x56145214eda0 2024-09-21T03:42:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f668d7fa640 10 client.27290 handle_command_reply: tid=0 2024-09-21T03:42:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f66b215b740 1 client.27290 shutdown 2024-09-21T03:42:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f66b215b740 1 -- 172.21.15.112:0/4138375047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 msgr2=0x561452572310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f66b215b740 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 0x561452572310 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6694009710 tx=0x7f66940244d0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f66b215b740 1 -- 172.21.15.112:0/4138375047 shutdown_connections 2024-09-21T03:42:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f66b215b740 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x56145214eda0 0x5614525ed0b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f668d7fa640 0 client.27290 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:42:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f66b215b740 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5614521bb070 0x561452572850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f66b215b740 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145219a390 0x561452572310 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f66b215b740 1 --2- 172.21.15.112:0/4138375047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614521921b0 0x561452571dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.083+0000 7f66b215b740 1 -- 172.21.15.112:0/4138375047 >> 172.21.15.112:0/4138375047 conn(0x5614521af460 msgr2=0x56145256ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.084+0000 7f66b215b740 1 -- 172.21.15.112:0/4138375047 shutdown_connections 2024-09-21T03:42:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.084+0000 7f66b215b740 1 -- 172.21.15.112:0/4138375047 wait complete. 2024-09-21T03:42:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.084+0000 7f66b215b740 20 client.27290 trim_cache size 0 max 16384 2024-09-21T03:42:50.087 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:42:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 -- 172.21.15.112:0/1650826039 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f66900b6700 msgr2=0x7f66900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f66900b6700 0x7f66900b8c10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f66a0104f20 tx=0x7f66a010d550 comp rx=0 tx=0).stop 2024-09-21T03:42:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 -- 172.21.15.112:0/1650826039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66a4102a60 msgr2=0x7f66a4073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66a4102a60 0x7f66a4073670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f66a002a930 tx=0x7f66a0031a20 comp rx=0 tx=0).stop 2024-09-21T03:42:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 -- 172.21.15.112:0/1650826039 shutdown_connections 2024-09-21T03:42:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f66900b6700 0x7f66900b8c10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a4108cf0 0x7f66a40740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41083a0 0x7f66a4073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 --2- 172.21.15.112:0/1650826039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66a4102a60 0x7f66a4073670 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 -- 172.21.15.112:0/1650826039 >> 172.21.15.112:0/1650826039 conn(0x7f66a4079730 msgr2=0x7f66a4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 -- 172.21.15.112:0/1650826039 shutdown_connections 2024-09-21T03:42:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:50.085+0000 7f66abbe7640 1 -- 172.21.15.112:0/1650826039 wait complete. 2024-09-21T03:42:50.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 35607 41.96 MB/sec warmup 50 sec latency 45.632 ms 2024-09-21T03:42:50.601 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:50 smithi112 ceph-mon[31442]: pgmap v748: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.6 MiB/s wr, 594 op/s 2024-09-21T03:42:50.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:50 smithi203 ceph-mon[34723]: pgmap v748: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.6 MiB/s wr, 594 op/s 2024-09-21T03:42:50.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:50 smithi187 ceph-mon[34603]: pgmap v748: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.6 MiB/s wr, 594 op/s 2024-09-21T03:42:51.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:42:51.338 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:51.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 36220 41.92 MB/sec warmup 51 sec latency 179.002 ms 2024-09-21T03:42:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.046+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1256844235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f741083a0 msgr2=0x7f2f741087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.046+0000 7f2f7aac5640 1 --2- 172.21.15.112:0/1256844235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f741083a0 0x7f2f741087b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009900 tx=0x7f2f6002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.046+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1256844235 shutdown_connections 2024-09-21T03:42:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.046+0000 7f2f7aac5640 1 --2- 172.21.15.112:0/1256844235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f74108cf0 0x7f2f7410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.046+0000 7f2f7aac5640 1 --2- 172.21.15.112:0/1256844235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f741083a0 0x7f2f741087b0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.046+0000 7f2f7aac5640 1 --2- 172.21.15.112:0/1256844235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f74102a60 0x7f2f74102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.046+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1256844235 >> 172.21.15.112:0/1256844235 conn(0x7f2f74079730 msgr2=0x7f2f74079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:52.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.046+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1256844235 shutdown_connections 2024-09-21T03:42:52.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.046+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1256844235 wait complete. 2024-09-21T03:42:52.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.047+0000 7f2f7aac5640 1 Processor -- start 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.047+0000 7f2f7aac5640 1 -- start start 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.047+0000 7f2f7aac5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f74102a60 0x7f2f7419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.047+0000 7f2f7aac5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f741083a0 0x7f2f7419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.047+0000 7f2f7aac5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f74108cf0 0x7f2f7419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.047+0000 7f2f7aac5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f74112960 con 0x7f2f741083a0 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.047+0000 7f2f7aac5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f741127e0 con 0x7f2f74108cf0 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.047+0000 7f2f7aac5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f74112ae0 con 0x7f2f74102a60 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f73fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f741083a0 0x7f2f7419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f73fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f741083a0 0x7f2f7419aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39986/0 (socket says 172.21.15.112:39986) 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f73fff640 1 -- 172.21.15.112:0/1245737732 learned_addr learned my addr 172.21.15.112:0/1245737732 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f78861640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f74102a60 0x7f2f7419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f79062640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f74108cf0 0x7f2f7419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f73fff640 1 -- 172.21.15.112:0/1245737732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f74102a60 msgr2=0x7f2f7419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f73fff640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f74102a60 0x7f2f7419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f73fff640 1 -- 172.21.15.112:0/1245737732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f74108cf0 msgr2=0x7f2f7419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f73fff640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f74108cf0 0x7f2f7419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f73fff640 1 -- 172.21.15.112:0/1245737732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f741a0410 con 0x7f2f741083a0 2024-09-21T03:42:52.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f73fff640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f741083a0 0x7f2f7419aa40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2f60032fd0 tx=0x7f2f60031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:52.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.048+0000 7f2f71ffb640 1 -- 172.21.15.112:0/1245737732 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6003f030 con 0x7f2f741083a0 2024-09-21T03:42:52.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.049+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1245737732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f741a06a0 con 0x7f2f741083a0 2024-09-21T03:42:52.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.049+0000 7f2f71ffb640 1 -- 172.21.15.112:0/1245737732 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f60004490 con 0x7f2f741083a0 2024-09-21T03:42:52.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.049+0000 7f2f71ffb640 1 -- 172.21.15.112:0/1245737732 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60035070 con 0x7f2f741083a0 2024-09-21T03:42:52.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.049+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1245737732 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f740ff550 con 0x7f2f741083a0 2024-09-21T03:42:52.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f81039740 1 Processor -- start 2024-09-21T03:42:52.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f81039740 1 -- start start 2024-09-21T03:42:52.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f71ffb640 1 -- 172.21.15.112:0/1245737732 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2f60042030 con 0x7f2f741083a0 2024-09-21T03:42:52.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f81039740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37d3070 0x5563a3b7a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f81039740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b7abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f81039740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37b2390 0x5563a3b7b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f81039740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5563a3af0ea0 con 0x5563a37aa1b0 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f81039740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5563a3ae02d0 con 0x5563a37b2390 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f81039740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5563a3ae40e0 con 0x5563a37d3070 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f78861640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b7abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f78861640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b7abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39994/0 (socket says 172.21.15.112:39994) 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f78861640 1 -- 172.21.15.112:0/3843016312 learned_addr learned my addr 172.21.15.112:0/3843016312 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f79062640 1 --2- 172.21.15.112:0/3843016312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37d3070 0x5563a3b7a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.052+0000 7f2f73fff640 1 --2- 172.21.15.112:0/3843016312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37b2390 0x5563a3b7b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.053+0000 7f2f78861640 1 -- 172.21.15.112:0/3843016312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37d3070 msgr2=0x5563a3b7a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.053+0000 7f2f78861640 1 --2- 172.21.15.112:0/3843016312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37d3070 0x5563a3b7a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.053+0000 7f2f78861640 1 -- 172.21.15.112:0/3843016312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37b2390 msgr2=0x5563a3b7b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.053+0000 7f2f78861640 1 --2- 172.21.15.112:0/3843016312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37b2390 0x5563a3b7b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.053+0000 7f2f78861640 1 -- 172.21.15.112:0/3843016312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a383e650 con 0x5563a37aa1b0 2024-09-21T03:42:52.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.053+0000 7f2f78861640 1 --2- 172.21.15.112:0/3843016312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b7abe0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009d20 tx=0x7f2f68011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fc99c3c5b5e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:52.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.053+0000 7f2f71ffb640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f640b6680 0x7f2f640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.053+0000 7f2f79062640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f640b6680 0x7f2f640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.053+0000 7f2f71ffb640 1 -- 172.21.15.112:0/1245737732 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f2f60107a30 con 0x7f2f741083a0 2024-09-21T03:42:52.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.053+0000 7f2f5effd640 1 -- 172.21.15.112:0/3843016312 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6801e070 con 0x5563a37aa1b0 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.054+0000 7f2f5effd640 1 -- 172.21.15.112:0/3843016312 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f68019ae0 con 0x5563a37aa1b0 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.054+0000 7f2f5effd640 1 -- 172.21.15.112:0/3843016312 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6800eac0 con 0x5563a37aa1b0 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.055+0000 7f2f81039740 1 -- 172.21.15.112:0/3843016312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 msgr2=0x5563a3b7abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.055+0000 7f2f79062640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f640b6680 0x7f2f640b8b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c005fd0 tx=0x7f2f6c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.055+0000 7f2f81039740 1 --2- 172.21.15.112:0/3843016312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b7abe0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009d20 tx=0x7f2f68011810 comp rx=0 tx=0).stop 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.055+0000 7f2f81039740 1 -- 172.21.15.112:0/3843016312 shutdown_connections 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.055+0000 7f2f81039740 1 --2- 172.21.15.112:0/3843016312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37b2390 0x5563a3b7b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.055+0000 7f2f81039740 1 --2- 172.21.15.112:0/3843016312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b7abe0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.055+0000 7f2f81039740 1 --2- 172.21.15.112:0/3843016312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37d3070 0x5563a3b7a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.055+0000 7f2f81039740 1 -- 172.21.15.112:0/3843016312 >> 172.21.15.112:0/3843016312 conn(0x5563a37c7460 msgr2=0x5563a3b78590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.055+0000 7f2f81039740 1 -- 172.21.15.112:0/3843016312 shutdown_connections 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.055+0000 7f2f81039740 1 -- 172.21.15.112:0/3843016312 wait complete. 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.055+0000 7f2f81039740 1 -- start start 2024-09-21T03:42:52.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.056+0000 7f2f81039740 1 Processor -- start 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.056+0000 7f2f81039740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37aa1b0 0x5563a3b85d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.056+0000 7f2f81039740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b86260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.056+0000 7f2f81039740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37d3070 0x5563a3b92170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.056+0000 7f2f81039740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5563a3941700 con 0x5563a37b2390 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.056+0000 7f2f81039740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5563a3a7b160 con 0x5563a37d3070 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.056+0000 7f2f81039740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5563a3b88e80 con 0x5563a37aa1b0 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.056+0000 7f2f81039740 10 client.0 _ll_register_callbacks cb 0x5563a3a949e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f73fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b86260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f73fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b86260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40004/0 (socket says 172.21.15.112:40004) 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f73fff640 1 -- 172.21.15.112:0/634470837 learned_addr learned my addr 172.21.15.112:0/634470837 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f79062640 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37d3070 0x5563a3b92170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f79062640 1 -- 172.21.15.112:0/634470837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37aa1b0 msgr2=0x5563a3b85d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:52.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f79062640 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37aa1b0 0x5563a3b85d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f79062640 1 -- 172.21.15.112:0/634470837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 msgr2=0x5563a3b86260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:52.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f79062640 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b86260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f79062640 1 -- 172.21.15.112:0/634470837 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a3aa8c90 con 0x5563a37d3070 2024-09-21T03:42:52.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f73fff640 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b86260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:52.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f79062640 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37d3070 0x5563a3b92170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c011e90 tx=0x7f2f6c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6cd0e0805779322 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:52.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f53fff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:42:52.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.057+0000 7f2f53fff640 1 -- 172.21.15.112:0/634470837 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00f870 con 0x5563a37d3070 2024-09-21T03:42:52.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.058+0000 7f2f81039740 1 -- 172.21.15.112:0/634470837 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f68019ae0 con 0x5563a37d3070 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.058+0000 7f2f53fff640 1 -- 172.21.15.112:0/634470837 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f2f6c00fa10 con 0x5563a37d3070 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.058+0000 7f2f53fff640 1 -- 172.21.15.112:0/634470837 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2f6c00fc10 con 0x5563a37d3070 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.058+0000 7f2f81039740 10 client.27302 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.058+0000 7f2f81039740 1 -- 172.21.15.112:0/634470837 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5563a3b94c00 con 0x5563a37d3070 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.058+0000 7f2f53fff640 1 -- 172.21.15.112:0/634470837 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f68019ae0 con 0x5563a37d3070 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.059+0000 7f2f53fff640 1 -- 172.21.15.112:0/634470837 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f2f6c026070 con 0x5563a37d3070 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.060+0000 7f2f81039740 10 client.27302 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.060+0000 7f2f81039740 10 client.27302 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.060+0000 7f2f81039740 20 client.27302 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.060+0000 7f2f81039740 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5563a37919f0 0x5563a3b981b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.060+0000 7f2f81039740 4 client.27302 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.060+0000 7f2f81039740 1 -- 172.21.15.112:0/634470837 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563a3af2df0 con 0x5563a37919f0 2024-09-21T03:42:52.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.060+0000 7f2f78861640 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5563a37919f0 0x5563a3b981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.061+0000 7f2f78861640 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5563a37919f0 0x5563a3b981b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c00b780d61c5c41 server_cookie=b76eebe048c6d9be in_seq=0 out_seq=0 2024-09-21T03:42:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.061+0000 7f2f53fff640 10 client.27302 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:42:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.062+0000 7f2f53fff640 1 -- 172.21.15.112:0/634470837 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563a3af2df0 con 0x5563a37919f0 2024-09-21T03:42:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.062+0000 7f2f53fff640 10 client.27302 handle_command_reply: tid=0 2024-09-21T03:42:52.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.062+0000 7f2f81039740 1 client.27302 shutdown 2024-09-21T03:42:52.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.062+0000 7f2f81039740 1 -- 172.21.15.112:0/634470837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37d3070 msgr2=0x5563a3b92170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.062+0000 7f2f81039740 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37d3070 0x5563a3b92170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c011e90 tx=0x7f2f6c004ab0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.063+0000 7f2f81039740 1 -- 172.21.15.112:0/634470837 shutdown_connections 2024-09-21T03:42:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.063+0000 7f2f81039740 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5563a37919f0 0x5563a3b981b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.063+0000 7f2f81039740 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37d3070 0x5563a3b92170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.063+0000 7f2f53fff640 0 client.27302 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:42:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.063+0000 7f2f81039740 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b86260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.063+0000 7f2f81039740 1 --2- 172.21.15.112:0/634470837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37aa1b0 0x5563a3b85d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.063+0000 7f2f81039740 1 -- 172.21.15.112:0/634470837 >> 172.21.15.112:0/634470837 conn(0x5563a37c7460 msgr2=0x5563a3b7d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.063+0000 7f2f81039740 1 -- 172.21.15.112:0/634470837 shutdown_connections 2024-09-21T03:42:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.063+0000 7f2f81039740 1 -- 172.21.15.112:0/634470837 wait complete. 2024-09-21T03:42:52.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.064+0000 7f2f81039740 20 client.27302 trim_cache size 0 max 16384 2024-09-21T03:42:52.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f81039740 1 Processor -- start 2024-09-21T03:42:52.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f81039740 1 -- start start 2024-09-21T03:42:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f81039740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37919f0 0x5563a3b861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f81039740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37aa1b0 0x5563a3b82f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f81039740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b83450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f81039740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f741b41f0 con 0x5563a37b2390 2024-09-21T03:42:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f81039740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f741b1a30 con 0x5563a37919f0 2024-09-21T03:42:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f81039740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f741af8a0 con 0x5563a37aa1b0 2024-09-21T03:42:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f79062640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b83450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f79062640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b83450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40016/0 (socket says 172.21.15.112:40016) 2024-09-21T03:42:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f79062640 1 -- 172.21.15.112:0/937829303 learned_addr learned my addr 172.21.15.112:0/937829303 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:52.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.085+0000 7f2f73fff640 1 --2- 172.21.15.112:0/937829303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37aa1b0 0x5563a3b82f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.086+0000 7f2f78861640 1 --2- 172.21.15.112:0/937829303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37919f0 0x5563a3b861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.086+0000 7f2f79062640 1 -- 172.21.15.112:0/937829303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37aa1b0 msgr2=0x5563a3b82f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:52.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.086+0000 7f2f79062640 1 --2- 172.21.15.112:0/937829303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37aa1b0 0x5563a3b82f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.086+0000 7f2f79062640 1 -- 172.21.15.112:0/937829303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37919f0 msgr2=0x5563a3b861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:52.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.086+0000 7f2f79062640 1 --2- 172.21.15.112:0/937829303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37919f0 0x5563a3b861e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.086+0000 7f2f79062640 1 -- 172.21.15.112:0/937829303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a3b92820 con 0x5563a37b2390 2024-09-21T03:42:52.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.086+0000 7f2f73fff640 1 --2- 172.21.15.112:0/937829303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37aa1b0 0x5563a3b82f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:52.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.086+0000 7f2f79062640 1 --2- 172.21.15.112:0/937829303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b83450 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c01dad0 tx=0x7f2f6c009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8067fbbc5462c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:52.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.086+0000 7f2f5e7fc640 1 -- 172.21.15.112:0/937829303 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c035070 con 0x5563a37b2390 2024-09-21T03:42:52.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.087+0000 7f2f5e7fc640 1 -- 172.21.15.112:0/937829303 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f6c01dd20 con 0x5563a37b2390 2024-09-21T03:42:52.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.087+0000 7f2f5e7fc640 1 -- 172.21.15.112:0/937829303 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0316b0 con 0x5563a37b2390 2024-09-21T03:42:52.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.087+0000 7f2f81039740 1 -- 172.21.15.112:0/937829303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 msgr2=0x5563a3b83450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.087+0000 7f2f81039740 1 --2- 172.21.15.112:0/937829303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b83450 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c01dad0 tx=0x7f2f6c009630 comp rx=0 tx=0).stop 2024-09-21T03:42:52.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.088+0000 7f2f81039740 1 -- 172.21.15.112:0/937829303 shutdown_connections 2024-09-21T03:42:52.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.088+0000 7f2f81039740 1 --2- 172.21.15.112:0/937829303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37b2390 0x5563a3b83450 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.088+0000 7f2f81039740 1 --2- 172.21.15.112:0/937829303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37aa1b0 0x5563a3b82f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.088+0000 7f2f81039740 1 --2- 172.21.15.112:0/937829303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37919f0 0x5563a3b861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.088+0000 7f2f81039740 1 -- 172.21.15.112:0/937829303 >> 172.21.15.112:0/937829303 conn(0x5563a37c7460 msgr2=0x5563a3b85ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.088+0000 7f2f81039740 1 -- 172.21.15.112:0/937829303 shutdown_connections 2024-09-21T03:42:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.088+0000 7f2f81039740 1 -- 172.21.15.112:0/937829303 wait complete. 2024-09-21T03:42:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.088+0000 7f2f81039740 1 -- start start 2024-09-21T03:42:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.089+0000 7f2f81039740 1 Processor -- start 2024-09-21T03:42:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.089+0000 7f2f81039740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37919f0 0x5563a3b8e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.089+0000 7f2f81039740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b89d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.089+0000 7f2f81039740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37b2390 0x5563a3b8a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.089+0000 7f2f81039740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f741afef0 con 0x5563a37aa1b0 2024-09-21T03:42:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.089+0000 7f2f81039740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f741b1360 con 0x5563a37919f0 2024-09-21T03:42:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.089+0000 7f2f81039740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f741b05c0 con 0x5563a37b2390 2024-09-21T03:42:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.089+0000 7f2f81039740 10 client.0 _ll_register_callbacks cb 0x7f2f6c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:52.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.090+0000 7f2f73fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b89d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.090+0000 7f2f73fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b89d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40026/0 (socket says 172.21.15.112:40026) 2024-09-21T03:42:52.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.090+0000 7f2f73fff640 1 -- 172.21.15.112:0/265485911 learned_addr learned my addr 172.21.15.112:0/265485911 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:52.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.090+0000 7f2f73fff640 1 -- 172.21.15.112:0/265485911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37b2390 msgr2=0x5563a3b8a240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:42:52.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.090+0000 7f2f73fff640 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37b2390 0x5563a3b8a240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.090+0000 7f2f73fff640 1 -- 172.21.15.112:0/265485911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37919f0 msgr2=0x5563a3b8e850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:42:52.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.091+0000 7f2f78861640 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37919f0 0x5563a3b8e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.091+0000 7f2f73fff640 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37919f0 0x5563a3b8e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.091+0000 7f2f73fff640 1 -- 172.21.15.112:0/265485911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a3b8a980 con 0x5563a37aa1b0 2024-09-21T03:42:52.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.091+0000 7f2f78861640 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37919f0 0x5563a3b8e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:52.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.091+0000 7f2f73fff640 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b89d00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2f601057b0 tx=0x7f2f6002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ca15394349f5c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:52.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.091+0000 7f2f53fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:42:52.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.092+0000 7f2f53fff640 1 -- 172.21.15.112:0/265485911 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60049040 con 0x5563a37aa1b0 2024-09-21T03:42:52.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.092+0000 7f2f53fff640 1 -- 172.21.15.112:0/265485911 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f2f600cebe0 con 0x5563a37aa1b0 2024-09-21T03:42:52.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.092+0000 7f2f81039740 1 -- 172.21.15.112:0/265485911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f6c01dd20 con 0x5563a37aa1b0 2024-09-21T03:42:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.092+0000 7f2f53fff640 1 -- 172.21.15.112:0/265485911 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2f600cd2b0 con 0x5563a37aa1b0 2024-09-21T03:42:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.092+0000 7f2f81039740 10 client.17769 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.092+0000 7f2f81039740 1 -- 172.21.15.112:0/265485911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5563a3c02aa0 con 0x5563a37aa1b0 2024-09-21T03:42:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.093+0000 7f2f53fff640 1 -- 172.21.15.112:0/265485911 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f600d2170 con 0x5563a37aa1b0 2024-09-21T03:42:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.093+0000 7f2f53fff640 1 -- 172.21.15.112:0/265485911 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f2f600c92b0 con 0x5563a37aa1b0 2024-09-21T03:42:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.093+0000 7f2f81039740 10 client.17769 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.093+0000 7f2f81039740 10 client.17769 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:42:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.093+0000 7f2f81039740 20 client.17769 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.093+0000 7f2f81039740 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5563a37d3070 0x5563a3c05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.093+0000 7f2f81039740 4 client.17769 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:42:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.093+0000 7f2f81039740 1 -- 172.21.15.112:0/265485911 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2f6c01dd20 con 0x5563a37d3070 2024-09-21T03:42:52.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.094+0000 7f2f78861640 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5563a37d3070 0x5563a3c05270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:52.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.100+0000 7f2f78861640 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5563a37d3070 0x5563a3c05270 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7b94d364e15acd7 server_cookie=9e7563e1600db1c4 in_seq=0 out_seq=0 2024-09-21T03:42:52.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.100+0000 7f2f53fff640 10 client.17769 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:42:52.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f53fff640 1 -- 172.21.15.112:0/265485911 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2f6c01dd20 con 0x5563a37d3070 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f53fff640 10 client.17769 handle_command_reply: tid=0 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f81039740 1 client.17769 shutdown 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f81039740 1 -- 172.21.15.112:0/265485911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 msgr2=0x5563a3b89d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f81039740 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b89d00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2f601057b0 tx=0x7f2f6002a410 comp rx=0 tx=0).stop 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f81039740 1 -- 172.21.15.112:0/265485911 shutdown_connections 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f81039740 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5563a37d3070 0x5563a3c05270 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f81039740 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563a37b2390 0x5563a3b8a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f81039740 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563a37aa1b0 0x5563a3b89d00 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f81039740 1 --2- 172.21.15.112:0/265485911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563a37919f0 0x5563a3b8e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f81039740 1 -- 172.21.15.112:0/265485911 >> 172.21.15.112:0/265485911 conn(0x5563a37c7460 msgr2=0x5563a3b7d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.107+0000 7f2f53fff640 0 client.17769 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.108+0000 7f2f81039740 1 -- 172.21.15.112:0/265485911 shutdown_connections 2024-09-21T03:42:52.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.108+0000 7f2f81039740 1 -- 172.21.15.112:0/265485911 wait complete. 2024-09-21T03:42:52.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.108+0000 7f2f81039740 20 client.17769 trim_cache size 0 max 16384 2024-09-21T03:42:52.111 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:42:52.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.108+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1245737732 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f640b6680 msgr2=0x7f2f640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.108+0000 7f2f7aac5640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f640b6680 0x7f2f640b8b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c005fd0 tx=0x7f2f6c009500 comp rx=0 tx=0).stop 2024-09-21T03:42:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.108+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1245737732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f741083a0 msgr2=0x7f2f7419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.108+0000 7f2f7aac5640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f741083a0 0x7f2f7419aa40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2f60032fd0 tx=0x7f2f60031a20 comp rx=0 tx=0).stop 2024-09-21T03:42:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.108+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1245737732 shutdown_connections 2024-09-21T03:42:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.108+0000 7f2f7aac5640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f640b6680 0x7f2f640b8b90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.108+0000 7f2f7aac5640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f74108cf0 0x7f2f7419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.108+0000 7f2f7aac5640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f741083a0 0x7f2f7419aa40 secure :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2f60032fd0 tx=0x7f2f60031a20 comp rx=0 tx=0).stop 2024-09-21T03:42:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.109+0000 7f2f7aac5640 1 --2- 172.21.15.112:0/1245737732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f74102a60 0x7f2f7419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.109+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1245737732 >> 172.21.15.112:0/1245737732 conn(0x7f2f74079730 msgr2=0x7f2f7406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.109+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1245737732 shutdown_connections 2024-09-21T03:42:52.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:52.109+0000 7f2f7aac5640 1 -- 172.21.15.112:0/1245737732 wait complete. 2024-09-21T03:42:52.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 36696 41.39 MB/sec warmup 52 sec latency 120.599 ms 2024-09-21T03:42:52.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:52 smithi203 ceph-mon[34723]: pgmap v749: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 766 op/s 2024-09-21T03:42:52.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:52 smithi187 ceph-mon[34603]: pgmap v749: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 766 op/s 2024-09-21T03:42:52.687 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:52 smithi112 ceph-mon[31442]: pgmap v749: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 766 op/s 2024-09-21T03:42:52.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:42:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:53.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 37484 41.27 MB/sec warmup 53 sec latency 77.379 ms 2024-09-21T03:42:53.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.742+0000 7f7cd969a640 1 -- 172.21.15.112:0/199264357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cd4102a60 msgr2=0x7f7cd4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.742+0000 7f7cd969a640 1 --2- 172.21.15.112:0/199264357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cd4102a60 0x7f7cd4102e70 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009900 tx=0x7f7cc802a410 comp rx=0 tx=0).stop 2024-09-21T03:42:53.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.742+0000 7f7cd969a640 1 -- 172.21.15.112:0/199264357 shutdown_connections 2024-09-21T03:42:53.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.742+0000 7f7cd969a640 1 --2- 172.21.15.112:0/199264357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cd4108cf0 0x7f7cd410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.742+0000 7f7cd969a640 1 --2- 172.21.15.112:0/199264357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cd41083a0 0x7f7cd41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.742+0000 7f7cd969a640 1 --2- 172.21.15.112:0/199264357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cd4102a60 0x7f7cd4102e70 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.742+0000 7f7cd969a640 1 -- 172.21.15.112:0/199264357 >> 172.21.15.112:0/199264357 conn(0x7f7cd4079730 msgr2=0x7f7cd4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.742+0000 7f7cd969a640 1 -- 172.21.15.112:0/199264357 shutdown_connections 2024-09-21T03:42:53.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.742+0000 7f7cd969a640 1 -- 172.21.15.112:0/199264357 wait complete. 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.743+0000 7f7cd969a640 1 Processor -- start 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.743+0000 7f7cd969a640 1 -- start start 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.743+0000 7f7cd969a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cd4102a60 0x7f7cd419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.743+0000 7f7cd969a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cd41083a0 0x7f7cd419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.743+0000 7f7cd969a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cd4108cf0 0x7f7cd419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.743+0000 7f7cd969a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4112960 con 0x7f7cd4102a60 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.743+0000 7f7cd969a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd41127e0 con 0x7f7cd4108cf0 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.743+0000 7f7cd969a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4112ae0 con 0x7f7cd41083a0 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.743+0000 7f7cd2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cd4102a60 0x7f7cd419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd27fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cd41083a0 0x7f7cd419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cd4108cf0 0x7f7cd419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cd4102a60 0x7f7cd419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40056/0 (socket says 172.21.15.112:40056) 2024-09-21T03:42:53.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cd4108cf0 0x7f7cd419fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35218/0 (socket says 172.21.15.112:35218) 2024-09-21T03:42:53.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd27fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cd41083a0 0x7f7cd419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:44286/0 (socket says 172.21.15.112:44286) 2024-09-21T03:42:53.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd2ffd640 1 -- 172.21.15.112:0/251079289 learned_addr learned my addr 172.21.15.112:0/251079289 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:53.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd37fe640 1 -- 172.21.15.112:0/251079289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cd41083a0 msgr2=0x7f7cd419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cd41083a0 0x7f7cd419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd37fe640 1 -- 172.21.15.112:0/251079289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cd4102a60 msgr2=0x7f7cd419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cd4102a60 0x7f7cd419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd37fe640 1 -- 172.21.15.112:0/251079289 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd41a03b0 con 0x7f7cd4108cf0 2024-09-21T03:42:53.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cd4108cf0 0x7f7cd419fc80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009fd0 tx=0x7f7ccc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cbbfff640 1 -- 172.21.15.112:0/251079289 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00ce20 con 0x7f7cd4108cf0 2024-09-21T03:42:53.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd969a640 1 -- 172.21.15.112:0/251079289 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd41a06a0 con 0x7f7cd4108cf0 2024-09-21T03:42:53.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cd969a640 1 -- 172.21.15.112:0/251079289 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd41a6e00 con 0x7f7cd4108cf0 2024-09-21T03:42:53.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cbbfff640 1 -- 172.21.15.112:0/251079289 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ccc005020 con 0x7f7cd4108cf0 2024-09-21T03:42:53.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.744+0000 7f7cbbfff640 1 -- 172.21.15.112:0/251079289 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc010690 con 0x7f7cd4108cf0 2024-09-21T03:42:53.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.747+0000 7f7cdfc0e740 1 Processor -- start 2024-09-21T03:42:53.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.747+0000 7f7cbbfff640 1 -- 172.21.15.112:0/251079289 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7ccc004b20 con 0x7f7cd4108cf0 2024-09-21T03:42:53.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cdfc0e740 1 -- start start 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c00be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 0x55c1d0c01120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c1d0b76ea0 con 0x55c1d0838390 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c1d0b662d0 con 0x55c1d0859070 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c1d0b6a0e0 con 0x55c1d08301b0 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cd27fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cd2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c00be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cd27fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:35232/0 (socket says 172.21.15.112:35232) 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cd2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c00be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:44302/0 (socket says 172.21.15.112:44302) 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cd27fc640 1 -- 172.21.15.112:0/3504903166 learned_addr learned my addr 172.21.15.112:0/3504903166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:53.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cd27fc640 1 --2- 172.21.15.112:0/3504903166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 0x55c1d0c01120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cd2ffd640 1 -- 172.21.15.112:0/3504903166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 msgr2=0x55c1d0c006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cd2ffd640 1 --2- 172.21.15.112:0/3504903166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cd2ffd640 1 -- 172.21.15.112:0/3504903166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 msgr2=0x55c1d0c01120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cd2ffd640 1 --2- 172.21.15.112:0/3504903166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 0x55c1d0c01120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cd2ffd640 1 -- 172.21.15.112:0/3504903166 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d08c4650 con 0x55c1d08301b0 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.748+0000 7f7cbbfff640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7cbc0b6700 0x7f7cbc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.749+0000 7f7cd27fc640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7cbc0b6700 0x7f7cbc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.749+0000 7f7cd2ffd640 1 --2- 172.21.15.112:0/3504903166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c00be0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8004770 tx=0x7f7cc8031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=218a15e5db1f6a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.749+0000 7f7cbbfff640 1 -- 172.21.15.112:0/251079289 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f7ccc0e7bc0 con 0x7f7cd4108cf0 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.749+0000 7f7cb97fa640 1 -- 172.21.15.112:0/3504903166 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8040070 con 0x55c1d08301b0 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.749+0000 7f7cb97fa640 1 -- 172.21.15.112:0/3504903166 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc8004030 con 0x55c1d08301b0 2024-09-21T03:42:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.749+0000 7f7cb97fa640 1 -- 172.21.15.112:0/3504903166 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8035040 con 0x55c1d08301b0 2024-09-21T03:42:53.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.750+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/3504903166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 msgr2=0x55c1d0c00be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.750+0000 7f7cd27fc640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7cbc0b6700 0x7f7cbc0b8c10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009fd0 tx=0x7f7cc4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.750+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/3504903166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c00be0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8004770 tx=0x7f7cc8031b10 comp rx=0 tx=0).stop 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.750+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/3504903166 shutdown_connections 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.750+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/3504903166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 0x55c1d0c01120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.750+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/3504903166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c00be0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.750+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/3504903166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.750+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/3504903166 >> 172.21.15.112:0/3504903166 conn(0x55c1d084d460 msgr2=0x55c1d0bfe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.750+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/3504903166 shutdown_connections 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.750+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/3504903166 wait complete. 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cdfc0e740 1 -- start start 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cdfc0e740 1 Processor -- start 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d08301b0 0x55c1d0c100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d0838390 0x55c1d0c105e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c10b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c1d09c7700 con 0x55c1d08301b0 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c1d0b01160 con 0x55c1d0859070 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c1d0c13180 con 0x55c1d0838390 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cd2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d08301b0 0x55c1d0c100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cdfc0e740 10 client.0 _ll_register_callbacks cb 0x55c1d0b1a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cd2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d08301b0 0x55c1d0c100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40080/0 (socket says 172.21.15.112:40080) 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cd2ffd640 1 -- 172.21.15.112:0/4215466284 learned_addr learned my addr 172.21.15.112:0/4215466284 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c10b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.751+0000 7f7cd27fc640 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d0838390 0x55c1d0c105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cd37fe640 1 -- 172.21.15.112:0/4215466284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d0838390 msgr2=0x55c1d0c105e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d0838390 0x55c1d0c105e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cd37fe640 1 -- 172.21.15.112:0/4215466284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d08301b0 msgr2=0x55c1d0c100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d08301b0 0x55c1d0c100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cd37fe640 1 -- 172.21.15.112:0/4215466284 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d0b2ec90 con 0x55c1d0859070 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cd2ffd640 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d08301b0 0x55c1d0c100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:53.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c10b20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0e5340 tx=0x7f7ccc0ad1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b47c01d5a0cb8e89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/4215466284 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc0b22e0 con 0x55c1d0859070 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/4215466284 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc8004030 con 0x55c1d0859070 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/4215466284 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f7ccc0acca0 con 0x55c1d0859070 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/4215466284 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7ccc0b2520 con 0x55c1d0859070 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cdfc0e740 10 client.27314 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.752+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/4215466284 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c1d0c1eec0 con 0x55c1d0859070 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.753+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/4215466284 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc8004030 con 0x55c1d0859070 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.753+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/4215466284 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f7ccc0ac7a0 con 0x55c1d0859070 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.753+0000 7f7cdfc0e740 10 client.27314 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.753+0000 7f7cdfc0e740 10 client.27314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.753+0000 7f7cdfc0e740 20 client.27314 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.753+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c1d08179f0 0x55c1d0c22390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.753+0000 7f7cdfc0e740 4 client.27314 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:53.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.753+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/4215466284 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1d0b78df0 con 0x55c1d08179f0 2024-09-21T03:42:53.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.753+0000 7f7cd2ffd640 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c1d08179f0 0x55c1d0c22390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.754+0000 7f7cb6ff5640 10 client.27314 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:53.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.754+0000 7f7cd2ffd640 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c1d08179f0 0x55c1d0c22390 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb638ab73ac51f03 server_cookie=22cc2ef48b0d572b in_seq=0 out_seq=0 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.754+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/4215466284 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c1d0b78df0 con 0x55c1d08179f0 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.754+0000 7f7cb6ff5640 10 client.27314 handle_command_reply: tid=0 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 1 client.27314 shutdown 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/4215466284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 msgr2=0x55c1d0c10b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c10b20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0e5340 tx=0x7f7ccc0ad1b0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/4215466284 shutdown_connections 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c1d08179f0 0x55c1d0c22390 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c10b20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d0838390 0x55c1d0c105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cb6ff5640 0 client.27314 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/4215466284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d08301b0 0x55c1d0c100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/4215466284 >> 172.21.15.112:0/4215466284 conn(0x55c1d084d460 msgr2=0x55c1d0c02090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/4215466284 shutdown_connections 2024-09-21T03:42:53.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/4215466284 wait complete. 2024-09-21T03:42:53.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.755+0000 7f7cdfc0e740 20 client.27314 trim_cache size 0 max 16384 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.771+0000 7f7cdfc0e740 1 Processor -- start 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.771+0000 7f7cdfc0e740 1 -- start start 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.771+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c01de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.771+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 0x55c1d0c0d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.771+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c0d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.771+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc0011040 con 0x55c1d0838390 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.771+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc000e880 con 0x55c1d0859070 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.771+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc000c6f0 con 0x55c1d08301b0 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 0x55c1d0c0d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 0x55c1d0c0d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40084/0 (socket says 172.21.15.112:40084) 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd27fc640 1 -- 172.21.15.112:0/2161543144 learned_addr learned my addr 172.21.15.112:0/2161543144 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/2161543144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c0d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd2ffd640 1 --2- 172.21.15.112:0/2161543144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c01de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd27fc640 1 -- 172.21.15.112:0/2161543144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 msgr2=0x55c1d0c01de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd27fc640 1 --2- 172.21.15.112:0/2161543144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c01de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd27fc640 1 -- 172.21.15.112:0/2161543144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 msgr2=0x55c1d0c0d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd27fc640 1 --2- 172.21.15.112:0/2161543144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c0d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd27fc640 1 -- 172.21.15.112:0/2161543144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d0c11140 con 0x55c1d0838390 2024-09-21T03:42:53.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/2161543144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c0d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:53.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cd27fc640 1 --2- 172.21.15.112:0/2161543144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 0x55c1d0c0d290 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4003640 tx=0x7f7cc4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfe0cdf5808e521 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cb8ff9640 1 -- 172.21.15.112:0/2161543144 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4003d30 con 0x55c1d0838390 2024-09-21T03:42:53.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cb8ff9640 1 -- 172.21.15.112:0/2161543144 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc40054d0 con 0x55c1d0838390 2024-09-21T03:42:53.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.772+0000 7f7cb8ff9640 1 -- 172.21.15.112:0/2161543144 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4004d60 con 0x55c1d0838390 2024-09-21T03:42:53.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.774+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/2161543144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 msgr2=0x55c1d0c0d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.774+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/2161543144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 0x55c1d0c0d290 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4003640 tx=0x7f7cc4016000 comp rx=0 tx=0).stop 2024-09-21T03:42:53.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.774+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/2161543144 shutdown_connections 2024-09-21T03:42:53.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.774+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/2161543144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0859070 0x55c1d0c0d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.774+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/2161543144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0838390 0x55c1d0c0d290 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.774+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/2161543144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c01de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.774+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/2161543144 >> 172.21.15.112:0/2161543144 conn(0x55c1d084d460 msgr2=0x55c1d0c016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.774+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/2161543144 shutdown_connections 2024-09-21T03:42:53.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.774+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/2161543144 wait complete. 2024-09-21T03:42:53.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.774+0000 7f7cdfc0e740 1 -- start start 2024-09-21T03:42:53.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.774+0000 7f7cdfc0e740 1 Processor -- start 2024-09-21T03:42:53.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c14be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0838390 0x55c1d0c15120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cdfc0e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0859070 0x55c1d0c8bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc000cd40 con 0x55c1d0859070 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc000e1b0 con 0x55c1d0838390 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cdfc0e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc000d410 con 0x55c1d08301b0 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cdfc0e740 10 client.0 _ll_register_callbacks cb 0x7f7ccc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cd37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0859070 0x55c1d0c8bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cd37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0859070 0x55c1d0c8bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40096/0 (socket says 172.21.15.112:40096) 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cd2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c14be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cd37fe640 1 -- 172.21.15.112:0/1670297695 learned_addr learned my addr 172.21.15.112:0/1670297695 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cd27fc640 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0838390 0x55c1d0c15120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cd37fe640 1 -- 172.21.15.112:0/1670297695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 msgr2=0x55c1d0c14be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c14be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cd37fe640 1 -- 172.21.15.112:0/1670297695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0838390 msgr2=0x55c1d0c15120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0838390 0x55c1d0c15120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cd37fe640 1 -- 172.21.15.112:0/1670297695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d0c8c2d0 con 0x55c1d0859070 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.775+0000 7f7cd2ffd640 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c14be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.776+0000 7f7cd37fe640 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0859070 0x55c1d0c8bd90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00c760 tx=0x7f7ccc009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cefbbb7ca5d3396 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.776+0000 7f7cb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.776+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/1670297695 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc0ad390 con 0x55c1d0859070 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.776+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/1670297695 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f7ccc0acea0 con 0x55c1d0859070 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.776+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/1670297695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc40054d0 con 0x55c1d0859070 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.777+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/1670297695 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7ccc08c050 con 0x55c1d0859070 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.777+0000 7f7cdfc0e740 10 client.17787 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:53.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.777+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/1670297695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c1d0c8d550 con 0x55c1d0859070 2024-09-21T03:42:53.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.777+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/1670297695 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc40054d0 con 0x55c1d0859070 2024-09-21T03:42:53.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.777+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/1670297695 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f7ccc0b3830 con 0x55c1d0859070 2024-09-21T03:42:53.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.778+0000 7f7cdfc0e740 10 client.17787 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:53.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.778+0000 7f7cdfc0e740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:53.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.778+0000 7f7cdfc0e740 20 client.17787 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:53.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.778+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c1d07ecda0 0x55c1d0c8fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:53.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.778+0000 7f7cdfc0e740 4 client.17787 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:42:53.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.778+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/1670297695 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1d0b748a0 con 0x55c1d07ecda0 2024-09-21T03:42:53.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.779+0000 7f7cd2ffd640 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c1d07ecda0 0x55c1d0c8fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:53.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.779+0000 7f7cb6ff5640 10 client.17787 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:53.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.779+0000 7f7cd2ffd640 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c1d07ecda0 0x55c1d0c8fc00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64231e7aeac12dde server_cookie=aa6a52352acf9e50 in_seq=0 out_seq=0 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.782+0000 7f7cb6ff5640 1 -- 172.21.15.112:0/1670297695 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1d0b748a0 con 0x55c1d07ecda0 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.782+0000 7f7cb6ff5640 10 client.17787 handle_command_reply: tid=0 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.782+0000 7f7cdfc0e740 1 client.17787 shutdown 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.783+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/1670297695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0859070 msgr2=0x55c1d0c8bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.783+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0859070 0x55c1d0c8bd90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00c760 tx=0x7f7ccc009590 comp rx=0 tx=0).stop 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.783+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/1670297695 shutdown_connections 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.783+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55c1d07ecda0 0x55c1d0c8fc00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.783+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c1d0859070 0x55c1d0c8bd90 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.783+0000 7f7cb6ff5640 0 client.17787 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.783+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1d0838390 0x55c1d0c15120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.783+0000 7f7cdfc0e740 1 --2- 172.21.15.112:0/1670297695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c1d08301b0 0x55c1d0c14be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.783+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/1670297695 >> 172.21.15.112:0/1670297695 conn(0x55c1d084d460 msgr2=0x55c1d0c114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.784+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/1670297695 shutdown_connections 2024-09-21T03:42:53.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.784+0000 7f7cdfc0e740 1 -- 172.21.15.112:0/1670297695 wait complete. 2024-09-21T03:42:53.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.784+0000 7f7cdfc0e740 20 client.17787 trim_cache size 0 max 16384 2024-09-21T03:42:53.786 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:42:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 -- 172.21.15.112:0/251079289 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7cbc0b6700 msgr2=0x7f7cbc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7cbc0b6700 0x7f7cbc0b8c10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009fd0 tx=0x7f7cc4005f90 comp rx=0 tx=0).stop 2024-09-21T03:42:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 -- 172.21.15.112:0/251079289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cd4108cf0 msgr2=0x7f7cd419fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cd4108cf0 0x7f7cd419fc80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009fd0 tx=0x7f7ccc00ef10 comp rx=0 tx=0).stop 2024-09-21T03:42:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 -- 172.21.15.112:0/251079289 shutdown_connections 2024-09-21T03:42:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7cbc0b6700 0x7f7cbc0b8c10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cd4108cf0 0x7f7cd419fc80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cd41083a0 0x7f7cd419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 --2- 172.21.15.112:0/251079289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cd4102a60 0x7f7cd419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:53.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 -- 172.21.15.112:0/251079289 >> 172.21.15.112:0/251079289 conn(0x7f7cd4079730 msgr2=0x7f7cd4101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:53.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 -- 172.21.15.112:0/251079289 shutdown_connections 2024-09-21T03:42:53.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:53.785+0000 7f7cd969a640 1 -- 172.21.15.112:0/251079289 wait complete. 2024-09-21T03:42:54.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:54 smithi112 ceph-mon[31442]: pgmap v750: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 607 op/s 2024-09-21T03:42:54.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 38274 41.21 MB/sec warmup 54 sec latency 307.982 ms 2024-09-21T03:42:54.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:54 smithi203 ceph-mon[34723]: pgmap v750: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 607 op/s 2024-09-21T03:42:54.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:54 smithi187 ceph-mon[34603]: pgmap v750: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 607 op/s 2024-09-21T03:42:55.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 38997 41.43 MB/sec warmup 55 sec latency 118.136 ms 2024-09-21T03:42:55.639 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:42:55.640 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:42:55.947 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:56.328 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:56 smithi112 ceph-mon[31442]: pgmap v751: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 576 op/s 2024-09-21T03:42:56.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 39672 41.43 MB/sec warmup 56 sec latency 80.412 ms 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.588+0000 7f87663b9640 1 -- 172.21.15.112:0/2067222870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87601052d0 msgr2=0x7f876010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.588+0000 7f87663b9640 1 --2- 172.21.15.112:0/2067222870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87601052d0 0x7f876010d910 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8748009900 tx=0x7f874802a410 comp rx=0 tx=0).stop 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.588+0000 7f87663b9640 1 -- 172.21.15.112:0/2067222870 shutdown_connections 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.588+0000 7f87663b9640 1 --2- 172.21.15.112:0/2067222870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f876010de50 0x7f87601102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.588+0000 7f87663b9640 1 --2- 172.21.15.112:0/2067222870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87601052d0 0x7f876010d910 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.588+0000 7f87663b9640 1 --2- 172.21.15.112:0/2067222870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87601048a0 0x7f8760104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.588+0000 7f87663b9640 1 -- 172.21.15.112:0/2067222870 >> 172.21.15.112:0/2067222870 conn(0x7f8760079730 msgr2=0x7f8760079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.588+0000 7f87663b9640 1 -- 172.21.15.112:0/2067222870 shutdown_connections 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.588+0000 7f87663b9640 1 -- 172.21.15.112:0/2067222870 wait complete. 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.589+0000 7f87663b9640 1 Processor -- start 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.589+0000 7f87663b9640 1 -- start start 2024-09-21T03:42:56.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f87663b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87601048a0 0x7f876019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f87663b9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87601052d0 0x7f876019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f87663b9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876010de50 0x7f876019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f87663b9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8760113a60 con 0x7f87601048a0 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f87663b9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f87601138e0 con 0x7f876010de50 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f87663b9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8760113be0 con 0x7f87601052d0 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f875ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87601048a0 0x7f876019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f875ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87601048a0 0x7f876019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47266/0 (socket says 172.21.15.112:47266) 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f875ffff640 1 -- 172.21.15.112:0/2628693180 learned_addr learned my addr 172.21.15.112:0/2628693180 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f8757fff640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87601052d0 0x7f876019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f8764956640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876010de50 0x7f876019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f875ffff640 1 -- 172.21.15.112:0/2628693180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87601052d0 msgr2=0x7f876019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f875ffff640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87601052d0 0x7f876019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f875ffff640 1 -- 172.21.15.112:0/2628693180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876010de50 msgr2=0x7f876019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:56.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f875ffff640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876010de50 0x7f876019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f875ffff640 1 -- 172.21.15.112:0/2628693180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87601a03e0 con 0x7f87601048a0 2024-09-21T03:42:56.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.590+0000 7f875ffff640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87601048a0 0x7f876019a470 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f875000e9c0 tx=0x7f875000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:56.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.591+0000 7f875dffb640 1 -- 172.21.15.112:0/2628693180 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875000cd80 con 0x7f87601048a0 2024-09-21T03:42:56.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.591+0000 7f87663b9640 1 -- 172.21.15.112:0/2628693180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87601a06d0 con 0x7f87601048a0 2024-09-21T03:42:56.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.591+0000 7f87663b9640 1 -- 172.21.15.112:0/2628693180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87601a6e30 con 0x7f87601048a0 2024-09-21T03:42:56.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.591+0000 7f875dffb640 1 -- 172.21.15.112:0/2628693180 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87500035f0 con 0x7f87601048a0 2024-09-21T03:42:56.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.591+0000 7f875dffb640 1 -- 172.21.15.112:0/2628693180 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750010690 con 0x7f87601048a0 2024-09-21T03:42:56.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.594+0000 7f875dffb640 1 -- 172.21.15.112:0/2628693180 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f87500108f0 con 0x7f87601048a0 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.594+0000 7f876c92d740 1 Processor -- start 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.594+0000 7f876c92d740 1 -- start start 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.594+0000 7f876c92d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8d8350 0x55efecc7f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f876c92d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc7f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f876c92d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8b7670 0x55efecc7feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f876c92d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55efecbf5e60 con 0x55efec8d8350 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f876c92d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55efecbe5290 con 0x55efec8af490 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f876c92d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55efecbe90a0 con 0x55efec8b7670 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f8764956640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8d8350 0x55efecc7f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f8757fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8b7670 0x55efecc7feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f8757fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8b7670 0x55efecc7feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:47484/0 (socket says 172.21.15.112:47484) 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f8757fff640 1 -- 172.21.15.112:0/2434603551 learned_addr learned my addr 172.21.15.112:0/2434603551 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:56.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f875ffff640 1 --2- 172.21.15.112:0/2434603551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc7f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f8757fff640 1 -- 172.21.15.112:0/2434603551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 msgr2=0x55efecc7f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:56.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f8757fff640 1 --2- 172.21.15.112:0/2434603551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc7f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f8757fff640 1 -- 172.21.15.112:0/2434603551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8d8350 msgr2=0x55efecc7f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:56.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.595+0000 7f8757fff640 1 --2- 172.21.15.112:0/2434603551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8d8350 0x55efecc7f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.596+0000 7f875dffb640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f874c0b6680 0x7f874c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.596+0000 7f8757fff640 1 -- 172.21.15.112:0/2434603551 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efec943930 con 0x55efec8b7670 2024-09-21T03:42:56.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.596+0000 7f8764956640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f874c0b6680 0x7f874c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.596+0000 7f875dffb640 1 -- 172.21.15.112:0/2628693180 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f87500e7960 con 0x7f87601048a0 2024-09-21T03:42:56.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.596+0000 7f8757fff640 1 --2- 172.21.15.112:0/2434603551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8b7670 0x55efecc7feb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8748009900 tx=0x7f8748032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ba7390c50c08b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:56.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.596+0000 7f8764956640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f874c0b6680 0x7f874c0b8b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8758005fd0 tx=0x7f8758009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:56.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.596+0000 7f87567fc640 1 -- 172.21.15.112:0/2434603551 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748040070 con 0x55efec8b7670 2024-09-21T03:42:56.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.597+0000 7f87567fc640 1 -- 172.21.15.112:0/2434603551 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8748004030 con 0x55efec8b7670 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.597+0000 7f87567fc640 1 -- 172.21.15.112:0/2434603551 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748035040 con 0x55efec8b7670 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.597+0000 7f876c92d740 1 -- 172.21.15.112:0/2434603551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8b7670 msgr2=0x55efecc7feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.597+0000 7f876c92d740 1 --2- 172.21.15.112:0/2434603551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8b7670 0x55efecc7feb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8748009900 tx=0x7f8748032b20 comp rx=0 tx=0).stop 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.598+0000 7f876c92d740 1 -- 172.21.15.112:0/2434603551 shutdown_connections 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.598+0000 7f876c92d740 1 --2- 172.21.15.112:0/2434603551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8b7670 0x55efecc7feb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.598+0000 7f876c92d740 1 --2- 172.21.15.112:0/2434603551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc7f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.598+0000 7f876c92d740 1 --2- 172.21.15.112:0/2434603551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8d8350 0x55efecc7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.598+0000 7f876c92d740 1 -- 172.21.15.112:0/2434603551 >> 172.21.15.112:0/2434603551 conn(0x55efec8cc740 msgr2=0x55efecc7d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.598+0000 7f876c92d740 1 -- 172.21.15.112:0/2434603551 shutdown_connections 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.598+0000 7f876c92d740 1 -- 172.21.15.112:0/2434603551 wait complete. 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.598+0000 7f876c92d740 1 -- start start 2024-09-21T03:42:56.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.598+0000 7f876c92d740 1 Processor -- start 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f876c92d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8af490 0x55efecc8af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f876c92d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8b7670 0x55efecc8b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f876c92d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8d8350 0x55efecc96f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f876c92d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55efeca46890 con 0x55efec8af490 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f876c92d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55efecb7fd40 con 0x55efec8b7670 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f876c92d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55efecc8dbb0 con 0x55efec8d8350 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f876c92d740 10 client.0 _ll_register_callbacks cb 0x55efecbe3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f8757fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8b7670 0x55efecc8b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f875ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8af490 0x55efecc8af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f8764956640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8d8350 0x55efecc96f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f8757fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8b7670 0x55efecc8b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:39722/0 (socket says 172.21.15.112:39722) 2024-09-21T03:42:56.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f8757fff640 1 -- 172.21.15.112:0/604065161 learned_addr learned my addr 172.21.15.112:0/604065161 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:56.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f8764956640 1 -- 172.21.15.112:0/604065161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8b7670 msgr2=0x55efecc8b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:56.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f8764956640 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8b7670 0x55efecc8b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f8764956640 1 -- 172.21.15.112:0/604065161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8af490 msgr2=0x55efecc8af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:56.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f8764956640 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8af490 0x55efecc8af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.599+0000 7f8764956640 1 -- 172.21.15.112:0/604065161 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efecc97440 con 0x55efec8d8350 2024-09-21T03:42:56.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.600+0000 7f875ffff640 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8af490 0x55efecc8af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:56.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.600+0000 7f8764956640 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8d8350 0x55efecc96f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8758011e90 tx=0x7f8758004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4f71d7d838929b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:56.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.600+0000 7f873f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:42:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.600+0000 7f873f7fe640 1 -- 172.21.15.112:0/604065161 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875800f870 con 0x55efec8d8350 2024-09-21T03:42:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.600+0000 7f873f7fe640 1 -- 172.21.15.112:0/604065161 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f875800fa10 con 0x55efec8d8350 2024-09-21T03:42:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.600+0000 7f876c92d740 1 -- 172.21.15.112:0/604065161 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8748004030 con 0x55efec8d8350 2024-09-21T03:42:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.600+0000 7f873f7fe640 1 -- 172.21.15.112:0/604065161 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f875800fc10 con 0x55efec8d8350 2024-09-21T03:42:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.600+0000 7f876c92d740 10 client.27352 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.600+0000 7f876c92d740 1 -- 172.21.15.112:0/604065161 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55efecca2590 con 0x55efec8d8350 2024-09-21T03:42:56.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.601+0000 7f873f7fe640 1 -- 172.21.15.112:0/604065161 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f875800fdb0 con 0x55efec8d8350 2024-09-21T03:42:56.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.601+0000 7f873f7fe640 1 -- 172.21.15.112:0/604065161 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f8758026070 con 0x55efec8d8350 2024-09-21T03:42:56.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.601+0000 7f876c92d740 10 client.27352 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:56.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.601+0000 7f876c92d740 10 client.27352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:56.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.601+0000 7f876c92d740 20 client.27352 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:56.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.601+0000 7f876c92d740 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55efec896cd0 0x55efecca5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.601+0000 7f876c92d740 4 client.27352 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.601+0000 7f876c92d740 1 -- 172.21.15.112:0/604065161 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8748004030 con 0x55efec896cd0 2024-09-21T03:42:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.601+0000 7f875ffff640 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55efec896cd0 0x55efecca5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.602+0000 7f873f7fe640 10 client.27352 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.602+0000 7f875ffff640 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55efec896cd0 0x55efecca5c50 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efcf67bf740ec3fa server_cookie=cae146176bffa9e3 in_seq=0 out_seq=0 2024-09-21T03:42:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f873f7fe640 1 -- 172.21.15.112:0/604065161 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8748004030 con 0x55efec896cd0 2024-09-21T03:42:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f873f7fe640 10 client.27352 handle_command_reply: tid=0 2024-09-21T03:42:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 1 client.27352 shutdown 2024-09-21T03:42:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 1 -- 172.21.15.112:0/604065161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8d8350 msgr2=0x55efecc96f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:56.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8d8350 0x55efecc96f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8758011e90 tx=0x7f8758004ab0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 1 -- 172.21.15.112:0/604065161 shutdown_connections 2024-09-21T03:42:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55efec896cd0 0x55efecca5c50 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8d8350 0x55efecc96f00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8b7670 0x55efecc8b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f873f7fe640 0 client.27352 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 1 --2- 172.21.15.112:0/604065161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8af490 0x55efecc8af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 1 -- 172.21.15.112:0/604065161 >> 172.21.15.112:0/604065161 conn(0x55efec8cc740 msgr2=0x55efecc825c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:56.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 1 -- 172.21.15.112:0/604065161 shutdown_connections 2024-09-21T03:42:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 1 -- 172.21.15.112:0/604065161 wait complete. 2024-09-21T03:42:56.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.603+0000 7f876c92d740 20 client.27352 trim_cache size 0 max 16384 2024-09-21T03:42:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f876c92d740 1 Processor -- start 2024-09-21T03:42:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f876c92d740 1 -- start start 2024-09-21T03:42:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f876c92d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8cc740 0x55efecc85550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f876c92d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec896cd0 0x55efecc96e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f876c92d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc89600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f876c92d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8740011080 con 0x55efec8cc740 2024-09-21T03:42:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f876c92d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f874000e8c0 con 0x55efec8af490 2024-09-21T03:42:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f876c92d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f874000c730 con 0x55efec896cd0 2024-09-21T03:42:56.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f875ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8cc740 0x55efecc85550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f875ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8cc740 0x55efecc85550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47284/0 (socket says 172.21.15.112:47284) 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f875ffff640 1 -- 172.21.15.112:0/19675558 learned_addr learned my addr 172.21.15.112:0/19675558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f8757fff640 1 --2- 172.21.15.112:0/19675558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec896cd0 0x55efecc96e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.623+0000 7f8764956640 1 --2- 172.21.15.112:0/19675558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc89600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.624+0000 7f875ffff640 1 -- 172.21.15.112:0/19675558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec896cd0 msgr2=0x55efecc96e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.624+0000 7f875ffff640 1 --2- 172.21.15.112:0/19675558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec896cd0 0x55efecc96e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.624+0000 7f875ffff640 1 -- 172.21.15.112:0/19675558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 msgr2=0x55efecc89600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.624+0000 7f875ffff640 1 --2- 172.21.15.112:0/19675558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc89600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.624+0000 7f875ffff640 1 -- 172.21.15.112:0/19675558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efecc89280 con 0x55efec8cc740 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.624+0000 7f875ffff640 1 --2- 172.21.15.112:0/19675558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8cc740 0x55efecc85550 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f8750009250 tx=0x7f87500e9e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7457f4d6a5334e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.624+0000 7f8755ffb640 1 -- 172.21.15.112:0/19675558 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87500b2070 con 0x55efec8cc740 2024-09-21T03:42:56.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.624+0000 7f8755ffb640 1 -- 172.21.15.112:0/19675558 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87500ad2f0 con 0x55efec8cc740 2024-09-21T03:42:56.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.624+0000 7f8755ffb640 1 -- 172.21.15.112:0/19675558 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87500aed70 con 0x55efec8cc740 2024-09-21T03:42:56.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.625+0000 7f876c92d740 1 -- 172.21.15.112:0/19675558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8cc740 msgr2=0x55efecc85550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:56.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.625+0000 7f876c92d740 1 --2- 172.21.15.112:0/19675558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8cc740 0x55efecc85550 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f8750009250 tx=0x7f87500e9e70 comp rx=0 tx=0).stop 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.625+0000 7f876c92d740 1 -- 172.21.15.112:0/19675558 shutdown_connections 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.625+0000 7f876c92d740 1 --2- 172.21.15.112:0/19675558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc89600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.625+0000 7f876c92d740 1 --2- 172.21.15.112:0/19675558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec896cd0 0x55efecc96e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.625+0000 7f876c92d740 1 --2- 172.21.15.112:0/19675558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec8cc740 0x55efecc85550 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.625+0000 7f876c92d740 1 -- 172.21.15.112:0/19675558 >> 172.21.15.112:0/19675558 conn(0x55efecc7d360 msgr2=0x55efecc85ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.625+0000 7f876c92d740 1 -- 172.21.15.112:0/19675558 shutdown_connections 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.626+0000 7f876c92d740 1 -- 172.21.15.112:0/19675558 wait complete. 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.626+0000 7f876c92d740 1 -- start start 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.626+0000 7f876c92d740 1 Processor -- start 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f876c92d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec896cd0 0x55efecc8f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f876c92d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc8f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f876c92d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8cc740 0x55efecd0f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f876c92d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f874000cd80 con 0x55efec896cd0 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f875ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec896cd0 0x55efecc8f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f876c92d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f874000e1f0 con 0x55efec8af490 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f875ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec896cd0 0x55efecc8f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47300/0 (socket says 172.21.15.112:47300) 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f876c92d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f874000d450 con 0x55efec8cc740 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f875ffff640 1 -- 172.21.15.112:0/3406664434 learned_addr learned my addr 172.21.15.112:0/3406664434 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f8764956640 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8cc740 0x55efecd0f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f8757fff640 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc8f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f876c92d740 10 client.0 _ll_register_callbacks cb 0x7f8758009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f8764956640 1 -- 172.21.15.112:0/3406664434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 msgr2=0x55efecc8f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f8764956640 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc8f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f8764956640 1 -- 172.21.15.112:0/3406664434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec896cd0 msgr2=0x55efecc8f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f8764956640 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec896cd0 0x55efecc8f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.627+0000 7f8764956640 1 -- 172.21.15.112:0/3406664434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efecd0fb50 con 0x55efec8cc740 2024-09-21T03:42:56.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f8764956640 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8cc740 0x55efecd0f610 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8758009690 tx=0x7f87580093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a1d23fef5dcdee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:56.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f873f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:42:56.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f873f7fe640 1 -- 172.21.15.112:0/3406664434 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758035070 con 0x55efec8cc740 2024-09-21T03:42:56.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f873f7fe640 1 -- 172.21.15.112:0/3406664434 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7f8758028910 con 0x55efec8cc740 2024-09-21T03:42:56.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f876c92d740 1 -- 172.21.15.112:0/3406664434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87500ad2f0 con 0x55efec8cc740 2024-09-21T03:42:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f873f7fe640 1 -- 172.21.15.112:0/3406664434 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f875801d470 con 0x55efec8cc740 2024-09-21T03:42:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f876c92d740 10 client.27358 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f876c92d740 1 -- 172.21.15.112:0/3406664434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55efecd10c40 con 0x55efec8cc740 2024-09-21T03:42:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f873f7fe640 1 -- 172.21.15.112:0/3406664434 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87500ad2f0 con 0x55efec8cc740 2024-09-21T03:42:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f873f7fe640 1 -- 172.21.15.112:0/3406664434 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f8758026040 con 0x55efec8cc740 2024-09-21T03:42:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f876c92d740 10 client.27358 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.628+0000 7f876c92d740 10 client.27358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.629+0000 7f876c92d740 20 client.27358 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.629+0000 7f876c92d740 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55efec8b7670 0x55efecd13480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:56.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.629+0000 7f876c92d740 4 client.27358 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:42:56.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.629+0000 7f876c92d740 1 -- 172.21.15.112:0/3406664434 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55efecbf7db0 con 0x55efec8b7670 2024-09-21T03:42:56.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.629+0000 7f875ffff640 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55efec8b7670 0x55efecd13480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:56.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.629+0000 7f873f7fe640 10 client.27358 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:56.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.629+0000 7f875ffff640 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55efec8b7670 0x55efecd13480 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645911b695d4c89 server_cookie=3bbb28c689f81d72 in_seq=0 out_seq=0 2024-09-21T03:42:56.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f873f7fe640 1 -- 172.21.15.112:0/3406664434 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55efecbf7db0 con 0x55efec8b7670 2024-09-21T03:42:56.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f873f7fe640 10 client.27358 handle_command_reply: tid=0 2024-09-21T03:42:56.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f876c92d740 1 client.27358 shutdown 2024-09-21T03:42:56.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f876c92d740 1 -- 172.21.15.112:0/3406664434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8cc740 msgr2=0x55efecd0f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:56.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f876c92d740 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8cc740 0x55efecd0f610 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8758009690 tx=0x7f87580093b0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f876c92d740 1 -- 172.21.15.112:0/3406664434 shutdown_connections 2024-09-21T03:42:56.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f876c92d740 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55efec8b7670 0x55efecd13480 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f876c92d740 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55efec8cc740 0x55efecd0f610 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f876c92d740 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efec8af490 0x55efecc8f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f873f7fe640 0 client.27358 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:56.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f876c92d740 1 --2- 172.21.15.112:0/3406664434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55efec896cd0 0x55efecc8f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f876c92d740 1 -- 172.21.15.112:0/3406664434 >> 172.21.15.112:0/3406664434 conn(0x55efecc7d360 msgr2=0x55efecc85ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:56.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f876c92d740 1 -- 172.21.15.112:0/3406664434 shutdown_connections 2024-09-21T03:42:56.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.630+0000 7f876c92d740 1 -- 172.21.15.112:0/3406664434 wait complete. 2024-09-21T03:42:56.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f876c92d740 20 client.27358 trim_cache size 0 max 16384 2024-09-21T03:42:56.639 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:42:56.639 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:42:56.639 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "33db2ca2-efca-4efa-81a7-253c9075949e", 2024-09-21T03:42:56.639 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:42:56.639 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:42:56.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 -- 172.21.15.112:0/2628693180 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f874c0b6680 msgr2=0x7f874c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:56.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f874c0b6680 0x7f874c0b8b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8758005fd0 tx=0x7f8758009500 comp rx=0 tx=0).stop 2024-09-21T03:42:56.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 -- 172.21.15.112:0/2628693180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87601048a0 msgr2=0x7f876019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:56.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87601048a0 0x7f876019a470 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f875000e9c0 tx=0x7f875000ee80 comp rx=0 tx=0).stop 2024-09-21T03:42:56.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 -- 172.21.15.112:0/2628693180 shutdown_connections 2024-09-21T03:42:56.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f874c0b6680 0x7f874c0b8b90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876010de50 0x7f876019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87601052d0 0x7f876019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 --2- 172.21.15.112:0/2628693180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87601048a0 0x7f876019a470 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:56.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 -- 172.21.15.112:0/2628693180 >> 172.21.15.112:0/2628693180 conn(0x7f8760079730 msgr2=0x7f87600ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:56.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 -- 172.21.15.112:0/2628693180 shutdown_connections 2024-09-21T03:42:56.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:56.631+0000 7f87663b9640 1 -- 172.21.15.112:0/2628693180 wait complete. 2024-09-21T03:42:56.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:56 smithi203 ceph-mon[34723]: pgmap v751: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 576 op/s 2024-09-21T03:42:56.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:56 smithi187 ceph-mon[34603]: pgmap v751: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 576 op/s 2024-09-21T03:42:57.362 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:42:57.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:57 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:42:57.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:57 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:57.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:57 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:42:57.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:57 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:42:57.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 39870 40.80 MB/sec warmup 57 sec latency 355.746 ms 2024-09-21T03:42:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:57 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:42:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:57 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:57 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:42:57.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:57 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:42:57.665 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:42:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:57 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:42:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:57 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:57 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:42:57.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:57 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:42:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.233+0000 7fea2977e640 1 -- 172.21.15.112:0/2295264478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea241053f0 msgr2=0x7fea24105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.233+0000 7fea2977e640 1 --2- 172.21.15.112:0/2295264478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea241053f0 0x7fea24105800 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fea14009900 tx=0x7fea1402a410 comp rx=0 tx=0).stop 2024-09-21T03:42:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.233+0000 7fea2977e640 1 -- 172.21.15.112:0/2295264478 shutdown_connections 2024-09-21T03:42:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.233+0000 7fea2977e640 1 --2- 172.21.15.112:0/2295264478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea24102630 0x7fea24106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.233+0000 7fea2977e640 1 --2- 172.21.15.112:0/2295264478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea24101ce0 0x7fea241020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.233+0000 7fea2977e640 1 --2- 172.21.15.112:0/2295264478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea241053f0 0x7fea24105800 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.233+0000 7fea2977e640 1 -- 172.21.15.112:0/2295264478 >> 172.21.15.112:0/2295264478 conn(0x7fea24077c60 msgr2=0x7fea240780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.233+0000 7fea2977e640 1 -- 172.21.15.112:0/2295264478 shutdown_connections 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.233+0000 7fea2977e640 1 -- 172.21.15.112:0/2295264478 wait complete. 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea2977e640 1 Processor -- start 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea2977e640 1 -- start start 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea2977e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea24101ce0 0x7fea241a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea2977e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea24102630 0x7fea241a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea2977e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea241053f0 0x7fea241a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea2977e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea2411a080 con 0x7fea24101ce0 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea2977e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea24119f00 con 0x7fea24102630 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea2977e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea2411a200 con 0x7fea241053f0 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea24101ce0 0x7fea241a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea22ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea24101ce0 0x7fea241a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47316/0 (socket says 172.21.15.112:47316) 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea22ffd640 1 -- 172.21.15.112:0/188817344 learned_addr learned my addr 172.21.15.112:0/188817344 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea227fc640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea24102630 0x7fea241a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea237fe640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea241053f0 0x7fea241a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea22ffd640 1 -- 172.21.15.112:0/188817344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea241053f0 msgr2=0x7fea241a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea22ffd640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea241053f0 0x7fea241a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea22ffd640 1 -- 172.21.15.112:0/188817344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea24102630 msgr2=0x7fea241a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea22ffd640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea24102630 0x7fea241a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.234+0000 7fea22ffd640 1 -- 172.21.15.112:0/188817344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea241a6a60 con 0x7fea24101ce0 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.235+0000 7fea22ffd640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea24101ce0 0x7fea241a0af0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fea14005940 tx=0x7fea140040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.235+0000 7fea1bfff640 1 -- 172.21.15.112:0/188817344 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1403f030 con 0x7fea24101ce0 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.235+0000 7fea1bfff640 1 -- 172.21.15.112:0/188817344 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea140044c0 con 0x7fea24101ce0 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.235+0000 7fea2977e640 1 -- 172.21.15.112:0/188817344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea241a6cf0 con 0x7fea24101ce0 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.235+0000 7fea1bfff640 1 -- 172.21.15.112:0/188817344 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14035070 con 0x7fea24101ce0 2024-09-21T03:42:58.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.235+0000 7fea2977e640 1 -- 172.21.15.112:0/188817344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea241ad4b0 con 0x7fea24101ce0 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.238+0000 7fea2fcf2740 1 Processor -- start 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.238+0000 7fea2fcf2740 1 -- start start 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 0x5618b988c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 0x5618b988cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b988d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5618b9802e50 con 0x5618b94bc1b0 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5618b97f2280 con 0x5618b94c4390 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5618b97f6090 con 0x5618b94e5070 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea237fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 0x5618b988c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea237fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 0x5618b988c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:47540/0 (socket says 172.21.15.112:47540) 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea237fe640 1 -- 172.21.15.112:0/1813318979 learned_addr learned my addr 172.21.15.112:0/1813318979 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea1bfff640 1 -- 172.21.15.112:0/188817344 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fea14042030 con 0x7fea24101ce0 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea227fc640 1 --2- 172.21.15.112:0/1813318979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b988d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea237fe640 1 -- 172.21.15.112:0/1813318979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 msgr2=0x5618b988d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea237fe640 1 --2- 172.21.15.112:0/1813318979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b988d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea22ffd640 1 --2- 172.21.15.112:0/1813318979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 0x5618b988cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea237fe640 1 -- 172.21.15.112:0/1813318979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 msgr2=0x5618b988cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea237fe640 1 --2- 172.21.15.112:0/1813318979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 0x5618b988cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea237fe640 1 -- 172.21.15.112:0/1813318979 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618b9550650 con 0x5618b94e5070 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.239+0000 7fea22ffd640 1 --2- 172.21.15.112:0/1813318979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 0x5618b988cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.240+0000 7fea237fe640 1 --2- 172.21.15.112:0/1813318979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 0x5618b988c650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009f50 tx=0x7fea1c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9de3eda646815321 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.240+0000 7fea1bfff640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fea100b6680 0x7fea100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.240+0000 7fea197fa640 1 -- 172.21.15.112:0/1813318979 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c004b20 con 0x5618b94e5070 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.240+0000 7fea22ffd640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fea100b6680 0x7fea100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.240+0000 7fea1bfff640 1 -- 172.21.15.112:0/188817344 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7fea14107a80 con 0x7fea24101ce0 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.241+0000 7fea197fa640 1 -- 172.21.15.112:0/1813318979 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea1c004cc0 con 0x5618b94e5070 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.241+0000 7fea197fa640 1 -- 172.21.15.112:0/1813318979 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c005020 con 0x5618b94e5070 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.242+0000 7fea22ffd640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fea100b6680 0x7fea100b8b90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fea141052b0 tx=0x7fea1403b4f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.242+0000 7fea2fcf2740 1 -- 172.21.15.112:0/1813318979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 msgr2=0x5618b988c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.242+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/1813318979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 0x5618b988c650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009f50 tx=0x7fea1c0118d0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.242+0000 7fea2fcf2740 1 -- 172.21.15.112:0/1813318979 shutdown_connections 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.242+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/1813318979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b988d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.242+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/1813318979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 0x5618b988cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.242+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/1813318979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 0x5618b988c650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.242+0000 7fea2fcf2740 1 -- 172.21.15.112:0/1813318979 >> 172.21.15.112:0/1813318979 conn(0x5618b94d9460 msgr2=0x5618b988a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.242+0000 7fea2fcf2740 1 -- 172.21.15.112:0/1813318979 shutdown_connections 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.242+0000 7fea2fcf2740 1 -- 172.21.15.112:0/1813318979 wait complete. 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.242+0000 7fea2fcf2740 1 -- start start 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea2fcf2740 1 Processor -- start 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94bc1b0 0x5618b9897cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9898230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94e5070 0x5618b98987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5618b9653700 con 0x5618b94e5070 2024-09-21T03:42:58.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5618b978d110 con 0x5618b94c4390 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5618b989ae30 con 0x5618b94bc1b0 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea2fcf2740 10 client.0 _ll_register_callbacks cb 0x5618b97a6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea227fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9898230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94e5070 0x5618b98987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea227fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9898230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:39770/0 (socket says 172.21.15.112:39770) 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.243+0000 7fea227fc640 1 -- 172.21.15.112:0/3990997533 learned_addr learned my addr 172.21.15.112:0/3990997533 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.244+0000 7fea227fc640 1 -- 172.21.15.112:0/3990997533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94bc1b0 msgr2=0x5618b9897cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.244+0000 7fea227fc640 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94bc1b0 0x5618b9897cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.244+0000 7fea227fc640 1 -- 172.21.15.112:0/3990997533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94e5070 msgr2=0x5618b98987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.244+0000 7fea227fc640 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94e5070 0x5618b98987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.244+0000 7fea227fc640 1 -- 172.21.15.112:0/3990997533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618b97bac40 con 0x5618b94c4390 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.244+0000 7fea237fe640 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94e5070 0x5618b98987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.244+0000 7fea227fc640 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9898230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fea0c012700 tx=0x7fea0c012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=371573ff66900228 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.244+0000 7fea0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.244+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3990997533 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c004050 con 0x5618b94c4390 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.244+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3990997533 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7fea0c0041f0 con 0x5618b94c4390 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.244+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3990997533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1c004cc0 con 0x5618b94c4390 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.245+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3990997533 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fea0c0043f0 con 0x5618b94c4390 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.245+0000 7fea2fcf2740 10 client.27350 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.245+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3990997533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5618b98a6ad0 con 0x5618b94c4390 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.245+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3990997533 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea1c004cc0 con 0x5618b94c4390 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.245+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3990997533 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fea0c003b50 con 0x5618b94c4390 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.245+0000 7fea2fcf2740 10 client.27350 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.245+0000 7fea2fcf2740 10 client.27350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.245+0000 7fea2fcf2740 20 client.27350 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.246+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5618b94a39f0 0x5618b98aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.246+0000 7fea2fcf2740 4 client.27350 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:42:58.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.246+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3990997533 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618b9804da0 con 0x5618b94a39f0 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.246+0000 7fea237fe640 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5618b94a39f0 0x5618b98aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.246+0000 7fea0a7fc640 10 client.27350 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.247+0000 7fea237fe640 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5618b94a39f0 0x5618b98aa3e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f02c1579fbdb5f7 server_cookie=b23def2fb74f77e6 in_seq=0 out_seq=0 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.252+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3990997533 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5618b9804da0 con 0x5618b94a39f0 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.252+0000 7fea0a7fc640 10 client.27350 handle_command_reply: tid=0 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.252+0000 7fea2fcf2740 1 client.27350 shutdown 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.253+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3990997533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 msgr2=0x5618b9898230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.253+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9898230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fea0c012700 tx=0x7fea0c012bc0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.253+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3990997533 shutdown_connections 2024-09-21T03:42:58.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.253+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5618b94a39f0 0x5618b98aa3e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.253+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94e5070 0x5618b98987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.253+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9898230 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.253+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3990997533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94bc1b0 0x5618b9897cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.253+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3990997533 >> 172.21.15.112:0/3990997533 conn(0x5618b94d9460 msgr2=0x5618b988f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.253+0000 7fea0a7fc640 0 client.27350 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.253+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3990997533 shutdown_connections 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.253+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3990997533 wait complete. 2024-09-21T03:42:58.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.254+0000 7fea2fcf2740 20 client.27350 trim_cache size 0 max 16384 2024-09-21T03:42:58.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.277+0000 7fea2fcf2740 1 Processor -- start 2024-09-21T03:42:58.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.277+0000 7fea2fcf2740 1 -- start start 2024-09-21T03:42:58.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 0x5618b9885280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9897aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 0x5618b9897fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea241b85b0 con 0x5618b94bc1b0 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea241b5df0 con 0x5618b94c4390 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea241b3c60 con 0x5618b94e5070 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 0x5618b9885280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea227fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9897aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea227fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9897aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:39782/0 (socket says 172.21.15.112:39782) 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 0x5618b9885280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47336/0 (socket says 172.21.15.112:47336) 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea227fc640 1 -- 172.21.15.112:0/1626085737 learned_addr learned my addr 172.21.15.112:0/1626085737 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea237fe640 1 --2- 172.21.15.112:0/1626085737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 0x5618b9897fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea227fc640 1 -- 172.21.15.112:0/1626085737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 msgr2=0x5618b9897fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea227fc640 1 --2- 172.21.15.112:0/1626085737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 0x5618b9897fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea227fc640 1 -- 172.21.15.112:0/1626085737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 msgr2=0x5618b9885280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea227fc640 1 --2- 172.21.15.112:0/1626085737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 0x5618b9885280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea227fc640 1 -- 172.21.15.112:0/1626085737 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618b9893580 con 0x5618b94c4390 2024-09-21T03:42:58.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.278+0000 7fea237fe640 1 --2- 172.21.15.112:0/1626085737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 0x5618b9885280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.279+0000 7fea227fc640 1 --2- 172.21.15.112:0/1626085737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9897aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea0c003ef0 tx=0x7fea0c0108b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd3497961ac9aade server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.279+0000 7fea18ff9640 1 -- 172.21.15.112:0/1626085737 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c033070 con 0x5618b94c4390 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.279+0000 7fea18ff9640 1 -- 172.21.15.112:0/1626085737 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea0c010e00 con 0x5618b94c4390 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.279+0000 7fea18ff9640 1 -- 172.21.15.112:0/1626085737 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c029040 con 0x5618b94c4390 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.280+0000 7fea2fcf2740 1 -- 172.21.15.112:0/1626085737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 msgr2=0x5618b9897aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.280+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/1626085737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9897aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea0c003ef0 tx=0x7fea0c0108b0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.280+0000 7fea2fcf2740 1 -- 172.21.15.112:0/1626085737 shutdown_connections 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.280+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/1626085737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94e5070 0x5618b9897fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.280+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/1626085737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94c4390 0x5618b9897aa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.280+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/1626085737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94bc1b0 0x5618b9885280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.280+0000 7fea2fcf2740 1 -- 172.21.15.112:0/1626085737 >> 172.21.15.112:0/1626085737 conn(0x5618b94d9460 msgr2=0x5618b9896570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.280+0000 7fea2fcf2740 1 -- 172.21.15.112:0/1626085737 shutdown_connections 2024-09-21T03:42:58.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.280+0000 7fea2fcf2740 1 -- 172.21.15.112:0/1626085737 wait complete. 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea2fcf2740 1 -- start start 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea2fcf2740 1 Processor -- start 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94bc1b0 0x5618b989b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94c4390 0x5618b989bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea2fcf2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94e5070 0x5618b9912d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea241b42b0 con 0x5618b94c4390 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea241b5720 con 0x5618b94e5070 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea2fcf2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea241b4980 con 0x5618b94bc1b0 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea2fcf2740 10 client.0 _ll_register_callbacks cb 0x7fea0c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94c4390 0x5618b989bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94c4390 0x5618b989bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47352/0 (socket says 172.21.15.112:47352) 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea227fc640 1 -- 172.21.15.112:0/3675251659 learned_addr learned my addr 172.21.15.112:0/3675251659 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea237fe640 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94bc1b0 0x5618b989b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.281+0000 7fea237fe640 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94e5070 0x5618b9912d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea227fc640 1 -- 172.21.15.112:0/3675251659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94bc1b0 msgr2=0x5618b989b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea227fc640 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94bc1b0 0x5618b989b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea227fc640 1 -- 172.21.15.112:0/3675251659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94e5070 msgr2=0x5618b9912d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:42:58.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea227fc640 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94e5070 0x5618b9912d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea227fc640 1 -- 172.21.15.112:0/3675251659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618b99132d0 con 0x5618b94c4390 2024-09-21T03:42:58.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea227fc640 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94c4390 0x5618b989bdc0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0104a0 tx=0x7fea0c010680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4a5a43b7b26999 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:42:58.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3675251659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c033040 con 0x5618b94c4390 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3675251659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea0c010e00 con 0x5618b94c4390 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3675251659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8470+0+0 (secure 0 0 0) 0x7fea0c004000 con 0x5618b94c4390 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3675251659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fea0c029040 con 0x5618b94c4390 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea2fcf2740 10 client.17820 fetch_fsmap learned FSMap version 35 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.282+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3675251659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5618b9914530 con 0x5618b94c4390 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.283+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3675251659 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea0c010e00 con 0x5618b94c4390 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.283+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3675251659 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fea0c036020 con 0x5618b94c4390 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.283+0000 7fea2fcf2740 10 client.17820 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.283+0000 7fea2fcf2740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.283+0000 7fea2fcf2740 20 client.17820 populate_metadata read hostname 'smithi112' 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.283+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5618b9478da0 0x5618b9916c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.283+0000 7fea2fcf2740 4 client.17820 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:42:58.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.283+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3675251659 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5618b9800850 con 0x5618b9478da0 2024-09-21T03:42:58.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.283+0000 7fea237fe640 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5618b9478da0 0x5618b9916c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:42:58.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.284+0000 7fea0a7fc640 10 client.17820 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:58.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.284+0000 7fea237fe640 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5618b9478da0 0x5618b9916c00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d00a1b41cee20cf7 server_cookie=b810ebb3dd840807 in_seq=0 out_seq=0 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea0a7fc640 1 -- 172.21.15.112:0/3675251659 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6534 (crc 0 0 0) 0x5618b9800850 con 0x5618b9478da0 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea0a7fc640 10 client.17820 handle_command_reply: tid=0 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea2fcf2740 1 client.17820 shutdown 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3675251659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94c4390 msgr2=0x5618b989bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94c4390 0x5618b989bdc0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0104a0 tx=0x7fea0c010680 comp rx=0 tx=0).stop 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3675251659 shutdown_connections 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5618b9478da0 0x5618b9916c00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea0a7fc640 0 client.17820 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618b94e5070 0x5618b9912d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618b94c4390 0x5618b989bdc0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea2fcf2740 1 --2- 172.21.15.112:0/3675251659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5618b94bc1b0 0x5618b989b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3675251659 >> 172.21.15.112:0/3675251659 conn(0x5618b94d9460 msgr2=0x5618b9897aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3675251659 shutdown_connections 2024-09-21T03:42:58.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.287+0000 7fea2fcf2740 1 -- 172.21.15.112:0/3675251659 wait complete. 2024-09-21T03:42:58.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.288+0000 7fea2fcf2740 20 client.17820 trim_cache size 0 max 16384 2024-09-21T03:42:58.292 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:42:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:42:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:42:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: "33db2ca2-efca-4efa-81a7-253c9075949e": { 2024-09-21T03:42:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "33db2ca2-efca-4efa-81a7-253c9075949e", 2024-09-21T03:42:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:42:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "33db2ca2-efca-4efa-81a7-253c9075949e": { 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:42:56.630350+0000", 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "4591c632-505e-4006-8d95-6080e5d063e2": { 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:42:11.919443+0000", 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c": { 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:45.957537+0000", 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "3d0e4442-3ee3-44bd-99e2-f0a81392c8c4": { 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:41:28.881491+0000", 2024-09-21T03:42:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:42:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:42:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:42:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:42:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:42:58.305 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:42:58.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.289+0000 7fea2977e640 1 -- 172.21.15.112:0/188817344 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fea100b6680 msgr2=0x7fea100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.289+0000 7fea2977e640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fea100b6680 0x7fea100b8b90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fea141052b0 tx=0x7fea1403b4f0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.289+0000 7fea2977e640 1 -- 172.21.15.112:0/188817344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea24101ce0 msgr2=0x7fea241a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:42:58.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.289+0000 7fea2977e640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea24101ce0 0x7fea241a0af0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fea14005940 tx=0x7fea140040c0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.289+0000 7fea2977e640 1 -- 172.21.15.112:0/188817344 shutdown_connections 2024-09-21T03:42:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.289+0000 7fea2977e640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fea100b6680 0x7fea100b8b90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.289+0000 7fea2977e640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea241053f0 0x7fea241a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.289+0000 7fea2977e640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea24102630 0x7fea241a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.289+0000 7fea2977e640 1 --2- 172.21.15.112:0/188817344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea24101ce0 0x7fea241a0af0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:42:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.289+0000 7fea2977e640 1 -- 172.21.15.112:0/188817344 >> 172.21.15.112:0/188817344 conn(0x7fea24077c60 msgr2=0x7fea24100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:42:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.290+0000 7fea2977e640 1 -- 172.21.15.112:0/188817344 shutdown_connections 2024-09-21T03:42:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:42:58.290+0000 7fea2977e640 1 -- 172.21.15.112:0/188817344 wait complete. 2024-09-21T03:42:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:58 smithi112 ceph-mon[31442]: pgmap v752: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 848 op/s 2024-09-21T03:42:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:58 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:42:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:58 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:58 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:58.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:42:58 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:58.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 40713 40.79 MB/sec warmup 58 sec latency 67.033 ms 2024-09-21T03:42:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:58 smithi203 ceph-mon[34723]: pgmap v752: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 848 op/s 2024-09-21T03:42:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:58 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:42:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:58 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:58 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:42:58 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:58 smithi187 ceph-mon[34603]: pgmap v752: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 848 op/s 2024-09-21T03:42:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:58 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:42:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:58 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:58 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:42:58 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:42:59.147 INFO:tasks.cephfs.filesystem:scrub status for tag:33db2ca2-efca-4efa-81a7-253c9075949e - {'path': '/', 'tag': '33db2ca2-efca-4efa-81a7-253c9075949e', 'options': 'recursive,force'} 2024-09-21T03:42:59.543 INFO:tasks.workunit.client.0.smithi112.stdout: 10 41334 40.57 MB/sec warmup 59 sec latency 286.600 ms 2024-09-21T03:43:00.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 41777 40.32 MB/sec warmup 60 sec latency 417.004 ms 2024-09-21T03:43:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:00 smithi203 ceph-mon[34723]: pgmap v753: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 632 op/s 2024-09-21T03:43:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:00 smithi187 ceph-mon[34603]: pgmap v753: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 632 op/s 2024-09-21T03:43:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:00 smithi112 ceph-mon[31442]: pgmap v753: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 632 op/s 2024-09-21T03:43:01.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 41777 39.66 MB/sec warmup 61 sec latency 1417.046 ms 2024-09-21T03:43:02.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 41909 39.18 MB/sec warmup 62 sec latency 2284.385 ms 2024-09-21T03:43:02.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:02 smithi203 ceph-mon[34723]: pgmap v754: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 803 op/s 2024-09-21T03:43:02.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:02 smithi187 ceph-mon[34603]: pgmap v754: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 803 op/s 2024-09-21T03:43:02.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:02 smithi112 ceph-mon[31442]: pgmap v754: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 803 op/s 2024-09-21T03:43:03.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 42377 39.02 MB/sec warmup 63 sec latency 95.502 ms 2024-09-21T03:43:04.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 42797 38.82 MB/sec warmup 64 sec latency 318.498 ms 2024-09-21T03:43:04.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:04 smithi203 ceph-mon[34723]: pgmap v755: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-09-21T03:43:04.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:04 smithi187 ceph-mon[34603]: pgmap v755: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-09-21T03:43:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:04 smithi112 ceph-mon[31442]: pgmap v755: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-09-21T03:43:05.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 43481 38.78 MB/sec warmup 65 sec latency 54.444 ms 2024-09-21T03:43:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:05 smithi203 ceph-mon[34723]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T03:43:05.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:05 smithi187 ceph-mon[34603]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T03:43:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:05 smithi112 ceph-mon[31442]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T03:43:06.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 44232 38.73 MB/sec warmup 66 sec latency 76.000 ms 2024-09-21T03:43:06.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:06 smithi203 ceph-mon[34723]: pgmap v757: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 790 op/s 2024-09-21T03:43:06.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:06 smithi187 ceph-mon[34603]: pgmap v757: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 790 op/s 2024-09-21T03:43:06.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:06 smithi112 ceph-mon[31442]: pgmap v757: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 790 op/s 2024-09-21T03:43:07.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 45104 38.90 MB/sec warmup 67 sec latency 57.229 ms 2024-09-21T03:43:08.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 45843 38.85 MB/sec warmup 68 sec latency 61.483 ms 2024-09-21T03:43:08.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:08 smithi203 ceph-mon[34723]: pgmap v758: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 838 op/s 2024-09-21T03:43:08.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:08 smithi187 ceph-mon[34603]: pgmap v758: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 838 op/s 2024-09-21T03:43:08.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:08 smithi112 ceph-mon[31442]: pgmap v758: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 838 op/s 2024-09-21T03:43:09.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 46418 38.82 MB/sec warmup 69 sec latency 434.271 ms 2024-09-21T03:43:09.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:09 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:43:09.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:09 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:43:09.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:09 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:43:10.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 47095 38.76 MB/sec warmup 70 sec latency 58.091 ms 2024-09-21T03:43:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:10 smithi203 ceph-mon[34723]: pgmap v759: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 846 op/s 2024-09-21T03:43:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:10 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:43:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:10 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:43:10.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:10 smithi203 ceph-mon[34723]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T03:43:10.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:10 smithi187 ceph-mon[34603]: pgmap v759: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 846 op/s 2024-09-21T03:43:10.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:10 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:43:10.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:10 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:43:10.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:10 smithi187 ceph-mon[34603]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T03:43:10.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:10 smithi112 ceph-mon[31442]: pgmap v759: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 846 op/s 2024-09-21T03:43:10.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:10 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:43:10.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:10 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:43:10.688 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:10 smithi112 ceph-mon[31442]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T03:43:11.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 47650 38.63 MB/sec warmup 71 sec latency 223.629 ms 2024-09-21T03:43:12.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:12 smithi112 ceph-mon[31442]: pgmap v761: 97 pgs: 97 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 996 op/s 2024-09-21T03:43:12.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 48507 38.67 MB/sec warmup 72 sec latency 55.853 ms 2024-09-21T03:43:12.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:12 smithi203 ceph-mon[34723]: pgmap v761: 97 pgs: 97 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 996 op/s 2024-09-21T03:43:12.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:12 smithi187 ceph-mon[34603]: pgmap v761: 97 pgs: 97 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 996 op/s 2024-09-21T03:43:13.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 49499 38.91 MB/sec warmup 73 sec latency 53.496 ms 2024-09-21T03:43:14.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 50001 38.81 MB/sec warmup 74 sec latency 396.845 ms 2024-09-21T03:43:14.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:14 smithi203 ceph-mon[34723]: pgmap v762: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1k op/s 2024-09-21T03:43:14.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:14 smithi187 ceph-mon[34603]: pgmap v762: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1k op/s 2024-09-21T03:43:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:14 smithi112 ceph-mon[31442]: pgmap v762: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1k op/s 2024-09-21T03:43:15.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 50567 38.74 MB/sec warmup 75 sec latency 108.693 ms 2024-09-21T03:43:16.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 51416 38.67 MB/sec warmup 76 sec latency 58.282 ms 2024-09-21T03:43:16.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:16 smithi203 ceph-mon[34723]: pgmap v763: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 883 op/s 2024-09-21T03:43:16.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:16 smithi187 ceph-mon[34603]: pgmap v763: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 883 op/s 2024-09-21T03:43:16.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:16 smithi112 ceph-mon[31442]: pgmap v763: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 883 op/s 2024-09-21T03:43:17.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 52312 38.82 MB/sec warmup 77 sec latency 72.308 ms 2024-09-21T03:43:18.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 53096 38.89 MB/sec warmup 78 sec latency 78.612 ms 2024-09-21T03:43:18.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:18 smithi203 ceph-mon[34723]: pgmap v764: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 741 op/s 2024-09-21T03:43:18.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:18 smithi187 ceph-mon[34603]: pgmap v764: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 741 op/s 2024-09-21T03:43:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:18 smithi112 ceph-mon[31442]: pgmap v764: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 741 op/s 2024-09-21T03:43:19.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 53589 38.75 MB/sec warmup 79 sec latency 266.132 ms 2024-09-21T03:43:20.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 54242 38.74 MB/sec warmup 80 sec latency 66.088 ms 2024-09-21T03:43:20.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:20 smithi203 ceph-mon[34723]: pgmap v765: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-21T03:43:20.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:20 smithi187 ceph-mon[34603]: pgmap v765: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-21T03:43:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:20 smithi112 ceph-mon[31442]: pgmap v765: 97 pgs: 97 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 775 op/s 2024-09-21T03:43:21.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 55020 38.79 MB/sec warmup 81 sec latency 47.925 ms 2024-09-21T03:43:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:21.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:43:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:21.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:43:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:43:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:43:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:43:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:43:22.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:22 smithi112 ceph-mon[31442]: pgmap v766: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 20 MiB/s wr, 906 op/s 2024-09-21T03:43:22.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 55779 38.77 MB/sec warmup 82 sec latency 58.445 ms 2024-09-21T03:43:22.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:22 smithi203 ceph-mon[34723]: pgmap v766: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 20 MiB/s wr, 906 op/s 2024-09-21T03:43:22.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:22 smithi187 ceph-mon[34603]: pgmap v766: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 20 MiB/s wr, 906 op/s 2024-09-21T03:43:23.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 56636 38.97 MB/sec warmup 83 sec latency 57.711 ms 2024-09-21T03:43:24.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 57283 38.75 MB/sec warmup 84 sec latency 204.799 ms 2024-09-21T03:43:24.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:24 smithi203 ceph-mon[34723]: pgmap v767: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 686 op/s 2024-09-21T03:43:24.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:24 smithi187 ceph-mon[34603]: pgmap v767: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 686 op/s 2024-09-21T03:43:24.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:24 smithi112 ceph-mon[31442]: pgmap v767: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 686 op/s 2024-09-21T03:43:25.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 58146 38.85 MB/sec warmup 85 sec latency 55.725 ms 2024-09-21T03:43:26.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 58931 39.03 MB/sec warmup 86 sec latency 106.863 ms 2024-09-21T03:43:26.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:26 smithi203 ceph-mon[34723]: pgmap v768: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 676 op/s 2024-09-21T03:43:26.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:26 smithi187 ceph-mon[34603]: pgmap v768: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 676 op/s 2024-09-21T03:43:26.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:26 smithi112 ceph-mon[31442]: pgmap v768: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 676 op/s 2024-09-21T03:43:27.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 59603 38.95 MB/sec warmup 87 sec latency 45.725 ms 2024-09-21T03:43:28.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 60370 38.95 MB/sec warmup 88 sec latency 57.894 ms 2024-09-21T03:43:28.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:28 smithi203 ceph-mon[34723]: pgmap v769: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 870 op/s 2024-09-21T03:43:28.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:28 smithi187 ceph-mon[34603]: pgmap v769: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 870 op/s 2024-09-21T03:43:28.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:28 smithi112 ceph-mon[31442]: pgmap v769: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 18 MiB/s wr, 870 op/s 2024-09-21T03:43:29.148 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:43:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:29.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 61102 38.93 MB/sec warmup 89 sec latency 176.907 ms 2024-09-21T03:43:30.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.076+0000 7fcfc2b15640 1 -- 172.21.15.112:0/4135936926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfbc1094c0 msgr2=0x7fcfbc1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.076+0000 7fcfc2b15640 1 --2- 172.21.15.112:0/4135936926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfbc1094c0 0x7fcfbc1098d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009900 tx=0x7fcfa802a410 comp rx=0 tx=0).stop 2024-09-21T03:43:30.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.076+0000 7fcfc2b15640 1 -- 172.21.15.112:0/4135936926 shutdown_connections 2024-09-21T03:43:30.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.076+0000 7fcfc2b15640 1 --2- 172.21.15.112:0/4135936926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfbc109e10 0x7fcfbc1106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.076+0000 7fcfc2b15640 1 --2- 172.21.15.112:0/4135936926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfbc1094c0 0x7fcfbc1098d0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.076+0000 7fcfc2b15640 1 --2- 172.21.15.112:0/4135936926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc102010 0x7fcfbc102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.076+0000 7fcfc2b15640 1 -- 172.21.15.112:0/4135936926 >> 172.21.15.112:0/4135936926 conn(0x7fcfbc077c60 msgr2=0x7fcfbc0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.077+0000 7fcfc2b15640 1 -- 172.21.15.112:0/4135936926 shutdown_connections 2024-09-21T03:43:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.077+0000 7fcfc2b15640 1 -- 172.21.15.112:0/4135936926 wait complete. 2024-09-21T03:43:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.078+0000 7fcfc2b15640 1 Processor -- start 2024-09-21T03:43:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.078+0000 7fcfc2b15640 1 -- start start 2024-09-21T03:43:30.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.079+0000 7fcfc2b15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfbc102010 0x7fcfbc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.079+0000 7fcfc2b15640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfbc1094c0 0x7fcfbc19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.079+0000 7fcfc2b15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc109e10 0x7fcfbc19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.079+0000 7fcfc2b15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc113b10 con 0x7fcfbc102010 2024-09-21T03:43:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.079+0000 7fcfbbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfbc1094c0 0x7fcfbc19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.080+0000 7fcfc2b15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc113990 con 0x7fcfbc109e10 2024-09-21T03:43:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.080+0000 7fcfc10b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc109e10 0x7fcfbc19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.080+0000 7fcfc10b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc109e10 0x7fcfbc19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:52888/0 (socket says 172.21.15.112:52888) 2024-09-21T03:43:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.080+0000 7fcfc10b2640 1 -- 172.21.15.112:0/3388079904 learned_addr learned my addr 172.21.15.112:0/3388079904 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.080+0000 7fcfc2b15640 1 -- 172.21.15.112:0/3388079904 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc113c90 con 0x7fcfbc1094c0 2024-09-21T03:43:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.080+0000 7fcfc08b1640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfbc102010 0x7fcfbc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.081+0000 7fcfc10b2640 1 -- 172.21.15.112:0/3388079904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfbc1094c0 msgr2=0x7fcfbc19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.081+0000 7fcfc10b2640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfbc1094c0 0x7fcfbc19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.081+0000 7fcfc10b2640 1 -- 172.21.15.112:0/3388079904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfbc102010 msgr2=0x7fcfbc19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.081+0000 7fcfc10b2640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfbc102010 0x7fcfbc19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.081+0000 7fcfc10b2640 1 -- 172.21.15.112:0/3388079904 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc1a03d0 con 0x7fcfbc109e10 2024-09-21T03:43:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.081+0000 7fcfc10b2640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc109e10 0x7fcfbc19fca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400eb10 tx=0x7fcfb400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:30.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.082+0000 7fcfb9ffb640 1 -- 172.21.15.112:0/3388079904 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4004b20 con 0x7fcfbc109e10 2024-09-21T03:43:30.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.082+0000 7fcfb9ffb640 1 -- 172.21.15.112:0/3388079904 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb4004cc0 con 0x7fcfbc109e10 2024-09-21T03:43:30.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.082+0000 7fcfc2b15640 1 -- 172.21.15.112:0/3388079904 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc1a06c0 con 0x7fcfbc109e10 2024-09-21T03:43:30.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.082+0000 7fcfb9ffb640 1 -- 172.21.15.112:0/3388079904 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4010720 con 0x7fcfbc109e10 2024-09-21T03:43:30.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.082+0000 7fcfc2b15640 1 -- 172.21.15.112:0/3388079904 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc102e90 con 0x7fcfbc109e10 2024-09-21T03:43:30.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.085+0000 7fcfb9ffb640 1 -- 172.21.15.112:0/3388079904 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fcfb40108c0 con 0x7fcfbc109e10 2024-09-21T03:43:30.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.086+0000 7fcfc9089740 1 Processor -- start 2024-09-21T03:43:30.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.086+0000 7fcfc9089740 1 -- start start 2024-09-21T03:43:30.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.086+0000 7fcfc9089740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x5596654f1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.086+0000 7fcfc9089740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x5596654f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfc9089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 0x5596654f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfc9089740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559665467e50 con 0x559665129390 2024-09-21T03:43:30.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfc9089740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559665457280 con 0x5596651211b0 2024-09-21T03:43:30.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfc9089740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55966545b090 con 0x55966514a070 2024-09-21T03:43:30.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfb9ffb640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcfac0038b0 0x7fcfac005dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfbbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x5596654f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfbbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x5596654f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:56676/0 (socket says 172.21.15.112:56676) 2024-09-21T03:43:30.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfbbfff640 1 -- 172.21.15.112:0/4214756887 learned_addr learned my addr 172.21.15.112:0/4214756887 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:30.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfc08b1640 1 --2- 172.21.15.112:0/4214756887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x5596654f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfb9ffb640 1 -- 172.21.15.112:0/3388079904 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fcfb40e7d70 con 0x7fcfbc109e10 2024-09-21T03:43:30.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfbbfff640 1 -- 172.21.15.112:0/4214756887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 msgr2=0x5596654f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:30.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfbbfff640 1 --2- 172.21.15.112:0/4214756887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x5596654f1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfbbfff640 1 -- 172.21.15.112:0/4214756887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 msgr2=0x5596654f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:30.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfbbfff640 1 --2- 172.21.15.112:0/4214756887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 0x5596654f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.087+0000 7fcfbbfff640 1 -- 172.21.15.112:0/4214756887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596651b5650 con 0x55966514a070 2024-09-21T03:43:30.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.088+0000 7fcfbbfff640 1 --2- 172.21.15.112:0/4214756887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x5596654f1650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009900 tx=0x7fcfa8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a2e44bdbe8f829e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:30.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.088+0000 7fcfa6ffd640 1 -- 172.21.15.112:0/4214756887 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8041020 con 0x55966514a070 2024-09-21T03:43:30.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.088+0000 7fcfa6ffd640 1 -- 172.21.15.112:0/4214756887 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa8004430 con 0x55966514a070 2024-09-21T03:43:30.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.088+0000 7fcfa6ffd640 1 -- 172.21.15.112:0/4214756887 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8035040 con 0x55966514a070 2024-09-21T03:43:30.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.089+0000 7fcfc9089740 1 -- 172.21.15.112:0/4214756887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 msgr2=0x5596654f1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.089+0000 7fcfc9089740 1 --2- 172.21.15.112:0/4214756887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x5596654f1650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009900 tx=0x7fcfa8004040 comp rx=0 tx=0).stop 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.089+0000 7fcfc10b2640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcfac0038b0 0x7fcfac005dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.089+0000 7fcfc9089740 1 -- 172.21.15.112:0/4214756887 shutdown_connections 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.089+0000 7fcfc9089740 1 --2- 172.21.15.112:0/4214756887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 0x5596654f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.089+0000 7fcfc9089740 1 --2- 172.21.15.112:0/4214756887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x5596654f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.089+0000 7fcfc9089740 1 --2- 172.21.15.112:0/4214756887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x5596654f1650 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.089+0000 7fcfc9089740 1 -- 172.21.15.112:0/4214756887 >> 172.21.15.112:0/4214756887 conn(0x55966513e460 msgr2=0x5596654ef540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.090+0000 7fcfc9089740 1 -- 172.21.15.112:0/4214756887 shutdown_connections 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.090+0000 7fcfc9089740 1 -- 172.21.15.112:0/4214756887 wait complete. 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.090+0000 7fcfc9089740 1 -- start start 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.090+0000 7fcfc9089740 1 Processor -- start 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.091+0000 7fcfc9089740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x5596654fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.091+0000 7fcfc9089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 0x5596654fd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.091+0000 7fcfc9089740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x55966551a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.091+0000 7fcfbbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x5596654fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.091+0000 7fcfbbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x5596654fcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:52902/0 (socket says 172.21.15.112:52902) 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfbbfff640 1 -- 172.21.15.112:0/1091150166 learned_addr learned my addr 172.21.15.112:0/1091150166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfc9089740 1 -- 172.21.15.112:0/1091150166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5596652b8700 con 0x559665129390 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfc9089740 1 -- 172.21.15.112:0/1091150166 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5596653f2110 con 0x5596651211b0 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfc9089740 1 -- 172.21.15.112:0/1091150166 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5596654ffdd0 con 0x55966514a070 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfbbfff640 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x55966551a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfc9089740 10 client.0 _ll_register_callbacks cb 0x55966540b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfc08b1640 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 0x5596654fd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfbbfff640 1 -- 172.21.15.112:0/1091150166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 msgr2=0x55966551a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfbbfff640 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x55966551a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfbbfff640 1 -- 172.21.15.112:0/1091150166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 msgr2=0x5596654fd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfbbfff640 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 0x5596654fd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfbbfff640 1 -- 172.21.15.112:0/1091150166 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55966541fc40 con 0x5596651211b0 2024-09-21T03:43:30.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.092+0000 7fcfbbfff640 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x5596654fcd50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcfa802a450 tx=0x7fcfa8004ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f18da4f0ce5cca93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.093+0000 7fcf9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.093+0000 7fcf9bfff640 1 -- 172.21.15.112:0/1091150166 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8041030 con 0x5596651211b0 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.093+0000 7fcf9bfff640 1 -- 172.21.15.112:0/1091150166 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fcfa80037f0 con 0x5596651211b0 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.093+0000 7fcfc10b2640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcfac0038b0 0x7fcfac005dc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40e55a0 tx=0x7fcfb4000f40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.093+0000 7fcfc9089740 1 -- 172.21.15.112:0/1091150166 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa8004430 con 0x5596651211b0 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.093+0000 7fcf9bfff640 1 -- 172.21.15.112:0/1091150166 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcfa8035040 con 0x5596651211b0 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.093+0000 7fcfc9089740 10 client.27365 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.094+0000 7fcfc9089740 1 -- 172.21.15.112:0/1091150166 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55966551cb20 con 0x5596651211b0 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.094+0000 7fcf9bfff640 1 -- 172.21.15.112:0/1091150166 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa80314a0 con 0x5596651211b0 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.094+0000 7fcf9bfff640 1 -- 172.21.15.112:0/1091150166 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fcfa803a040 con 0x5596651211b0 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.094+0000 7fcfc9089740 10 client.27365 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.094+0000 7fcfc9089740 10 client.27365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.095+0000 7fcfc9089740 20 client.27365 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.095+0000 7fcfc9089740 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5596651089f0 0x55966551fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.095+0000 7fcfc9089740 4 client.27365 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:30.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.095+0000 7fcfc9089740 1 -- 172.21.15.112:0/1091150166 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcfa8004430 con 0x5596651089f0 2024-09-21T03:43:30.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.095+0000 7fcfc08b1640 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5596651089f0 0x55966551fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.096+0000 7fcf9bfff640 10 client.27365 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:30.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.096+0000 7fcfc08b1640 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5596651089f0 0x55966551fe80 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd70a722a830374 server_cookie=30544081955d9ece in_seq=0 out_seq=0 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcf9bfff640 1 -- 172.21.15.112:0/1091150166 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fcfa8004430 con 0x5596651089f0 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcf9bfff640 10 client.27365 handle_command_reply: tid=0 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcfc9089740 1 client.27365 shutdown 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcfc9089740 1 -- 172.21.15.112:0/1091150166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 msgr2=0x5596654fcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcfc9089740 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x5596654fcd50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcfa802a450 tx=0x7fcfa8004ba0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcfc9089740 1 -- 172.21.15.112:0/1091150166 shutdown_connections 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcfc9089740 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5596651089f0 0x55966551fe80 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcf9bfff640 0 client.27365 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcfc9089740 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x55966551a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcfc9089740 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 0x5596654fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcfc9089740 1 --2- 172.21.15.112:0/1091150166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x5596654fcd50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.102+0000 7fcfc9089740 1 -- 172.21.15.112:0/1091150166 >> 172.21.15.112:0/1091150166 conn(0x55966513e460 msgr2=0x5596654f47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.103+0000 7fcfc9089740 1 -- 172.21.15.112:0/1091150166 shutdown_connections 2024-09-21T03:43:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.103+0000 7fcfc9089740 1 -- 172.21.15.112:0/1091150166 wait complete. 2024-09-21T03:43:30.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.103+0000 7fcfc9089740 20 client.27365 trim_cache size 0 max 16384 2024-09-21T03:43:30.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.131+0000 7fcfc9089740 1 Processor -- start 2024-09-21T03:43:30.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.131+0000 7fcfc9089740 1 -- start start 2024-09-21T03:43:30.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.132+0000 7fcfc9089740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5596651211b0 0x5596654fd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.132+0000 7fcfc9089740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559665129390 0x55966551a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.132+0000 7fcfc9089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55966514a070 0x5596654eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.132+0000 7fcfc9089740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc1a75d0 con 0x55966514a070 2024-09-21T03:43:30.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.132+0000 7fcfc9089740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc0736e0 con 0x559665129390 2024-09-21T03:43:30.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.132+0000 7fcfc9089740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc071550 con 0x5596651211b0 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.132+0000 7fcfc08b1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559665129390 0x55966551a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.132+0000 7fcfc08b1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559665129390 0x55966551a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:52916/0 (socket says 172.21.15.112:52916) 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.132+0000 7fcfc08b1640 1 -- 172.21.15.112:0/4124725109 learned_addr learned my addr 172.21.15.112:0/4124725109 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.132+0000 7fcfbbfff640 1 --2- 172.21.15.112:0/4124725109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55966514a070 0x5596654eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.133+0000 7fcfc08b1640 1 -- 172.21.15.112:0/4124725109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5596651211b0 msgr2=0x5596654fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.133+0000 7fcfc08b1640 1 --2- 172.21.15.112:0/4124725109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5596651211b0 0x5596654fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.133+0000 7fcfc08b1640 1 -- 172.21.15.112:0/4124725109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55966514a070 msgr2=0x5596654eeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.133+0000 7fcfc08b1640 1 --2- 172.21.15.112:0/4124725109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55966514a070 0x5596654eeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.133+0000 7fcfc08b1640 1 -- 172.21.15.112:0/4124725109 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596654f2480 con 0x559665129390 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.133+0000 7fcfc08b1640 1 --2- 172.21.15.112:0/4124725109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559665129390 0x55966551a0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009a20 tx=0x7fcfb0013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e70c6d1226eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.133+0000 7fcfa67fc640 1 -- 172.21.15.112:0/4124725109 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0009d30 con 0x559665129390 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.133+0000 7fcfa67fc640 1 -- 172.21.15.112:0/4124725109 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb00130d0 con 0x559665129390 2024-09-21T03:43:30.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.133+0000 7fcfa67fc640 1 -- 172.21.15.112:0/4124725109 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0003cc0 con 0x559665129390 2024-09-21T03:43:30.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.135+0000 7fcfc9089740 1 -- 172.21.15.112:0/4124725109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559665129390 msgr2=0x55966551a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.135+0000 7fcfc9089740 1 --2- 172.21.15.112:0/4124725109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559665129390 0x55966551a0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009a20 tx=0x7fcfb0013830 comp rx=0 tx=0).stop 2024-09-21T03:43:30.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.135+0000 7fcfc9089740 1 -- 172.21.15.112:0/4124725109 shutdown_connections 2024-09-21T03:43:30.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.135+0000 7fcfc9089740 1 --2- 172.21.15.112:0/4124725109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55966514a070 0x5596654eeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.135+0000 7fcfc9089740 1 --2- 172.21.15.112:0/4124725109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559665129390 0x55966551a0c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.135+0000 7fcfc9089740 1 --2- 172.21.15.112:0/4124725109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5596651211b0 0x5596654fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.135+0000 7fcfc9089740 1 -- 172.21.15.112:0/4124725109 >> 172.21.15.112:0/4124725109 conn(0x55966513e460 msgr2=0x5596654fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:30.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.135+0000 7fcfc9089740 1 -- 172.21.15.112:0/4124725109 shutdown_connections 2024-09-21T03:43:30.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.135+0000 7fcfc9089740 1 -- 172.21.15.112:0/4124725109 wait complete. 2024-09-21T03:43:30.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.136+0000 7fcfc9089740 1 -- start start 2024-09-21T03:43:30.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.136+0000 7fcfc9089740 1 Processor -- start 2024-09-21T03:43:30.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.137+0000 7fcfc9089740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x559665523670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.137+0000 7fcfc9089740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 0x559665520b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.137+0000 7fcfbbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x559665523670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.137+0000 7fcfbbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x559665523670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:52924/0 (socket says 172.21.15.112:52924) 2024-09-21T03:43:30.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.137+0000 7fcfbbfff640 1 -- 172.21.15.112:0/3570516467 learned_addr learned my addr 172.21.15.112:0/3570516467 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:30.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.137+0000 7fcfc9089740 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x559665521080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.137+0000 7fcfc9089740 1 -- 172.21.15.112:0/3570516467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc071ba0 con 0x559665129390 2024-09-21T03:43:30.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.137+0000 7fcfc9089740 1 -- 172.21.15.112:0/3570516467 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc073010 con 0x5596651211b0 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.137+0000 7fcfc9089740 1 -- 172.21.15.112:0/3570516467 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc072270 con 0x55966514a070 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.137+0000 7fcfc08b1640 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 0x559665520b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.137+0000 7fcfc9089740 10 client.0 _ll_register_callbacks cb 0x7fcfb40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.138+0000 7fcfbbfff640 1 -- 172.21.15.112:0/3570516467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 msgr2=0x559665521080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.138+0000 7fcfbbfff640 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x559665521080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.138+0000 7fcfbbfff640 1 -- 172.21.15.112:0/3570516467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 msgr2=0x559665520b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.138+0000 7fcfbbfff640 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 0x559665520b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.138+0000 7fcfbbfff640 1 -- 172.21.15.112:0/3570516467 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596655217f0 con 0x5596651211b0 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.138+0000 7fcfbbfff640 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x559665523670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8005590 tx=0x7fcfa8003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e5e2bd0b7a560b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.138+0000 7fcf9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.138+0000 7fcf9bfff640 1 -- 172.21.15.112:0/3570516467 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8041040 con 0x5596651211b0 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.138+0000 7fcf9bfff640 1 -- 172.21.15.112:0/3570516467 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fcfa8003f40 con 0x5596651211b0 2024-09-21T03:43:30.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.138+0000 7fcfc9089740 1 -- 172.21.15.112:0/3570516467 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb00130d0 con 0x5596651211b0 2024-09-21T03:43:30.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.139+0000 7fcf9bfff640 1 -- 172.21.15.112:0/3570516467 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcfa8004410 con 0x5596651211b0 2024-09-21T03:43:30.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.139+0000 7fcfc9089740 10 client.27371 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:30.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.139+0000 7fcfc9089740 1 -- 172.21.15.112:0/3570516467 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55966558a510 con 0x5596651211b0 2024-09-21T03:43:30.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.139+0000 7fcf9bfff640 1 -- 172.21.15.112:0/3570516467 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb00130d0 con 0x5596651211b0 2024-09-21T03:43:30.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.139+0000 7fcf9bfff640 1 -- 172.21.15.112:0/3570516467 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fcfa80314a0 con 0x5596651211b0 2024-09-21T03:43:30.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.139+0000 7fcfc9089740 10 client.27371 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:30.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.139+0000 7fcfc9089740 10 client.27371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:43:30.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.139+0000 7fcfc9089740 20 client.27371 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:30.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.139+0000 7fcfc9089740 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5596650ddda0 0x55966558ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:30.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.139+0000 7fcfc9089740 4 client.27371 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:43:30.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.139+0000 7fcfc9089740 1 -- 172.21.15.112:0/3570516467 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559665469da0 con 0x5596650ddda0 2024-09-21T03:43:30.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.140+0000 7fcfc08b1640 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5596650ddda0 0x55966558ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:30.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.141+0000 7fcfc08b1640 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5596650ddda0 0x55966558ccb0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d6b50c30b84e9 server_cookie=c49066b1b64dddec in_seq=0 out_seq=0 2024-09-21T03:43:30.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.141+0000 7fcf9bfff640 10 client.27371 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:30.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.143+0000 7fcf9bfff640 1 -- 172.21.15.112:0/3570516467 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6338 (crc 0 0 0) 0x559665469da0 con 0x5596650ddda0 2024-09-21T03:43:30.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.143+0000 7fcf9bfff640 10 client.27371 handle_command_reply: tid=0 2024-09-21T03:43:30.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.143+0000 7fcfc9089740 1 client.27371 shutdown 2024-09-21T03:43:30.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.144+0000 7fcfc9089740 1 -- 172.21.15.112:0/3570516467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 msgr2=0x559665523670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.144+0000 7fcfc9089740 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x559665523670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8005590 tx=0x7fcfa8003e80 comp rx=0 tx=0).stop 2024-09-21T03:43:30.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.144+0000 7fcfc9089740 1 -- 172.21.15.112:0/3570516467 shutdown_connections 2024-09-21T03:43:30.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.144+0000 7fcfc9089740 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5596650ddda0 0x55966558ccb0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.144+0000 7fcfc9089740 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55966514a070 0x559665521080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.144+0000 7fcfc9089740 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559665129390 0x559665520b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.144+0000 7fcfc9089740 1 --2- 172.21.15.112:0/3570516467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5596651211b0 0x559665523670 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.144+0000 7fcf9bfff640 0 client.27371 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.144+0000 7fcfc9089740 1 -- 172.21.15.112:0/3570516467 >> 172.21.15.112:0/3570516467 conn(0x55966513e460 msgr2=0x5596654f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.145+0000 7fcfc9089740 1 -- 172.21.15.112:0/3570516467 shutdown_connections 2024-09-21T03:43:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.145+0000 7fcfc9089740 1 -- 172.21.15.112:0/3570516467 wait complete. 2024-09-21T03:43:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.145+0000 7fcfc9089740 20 client.27371 trim_cache size 0 max 16384 2024-09-21T03:43:30.148 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:43:30.148 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:43:30.148 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:43:30.148 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:43:30.148 INFO:teuthology.orchestra.run.smithi112.stdout: "33db2ca2-efca-4efa-81a7-253c9075949e": { 2024-09-21T03:43:30.148 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:42:56.630350+0000", 2024-09-21T03:43:30.148 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "4591c632-505e-4006-8d95-6080e5d063e2": { 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:42:11.919443+0000", 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c": { 2024-09-21T03:43:30.149 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:45.957537+0000", 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "3d0e4442-3ee3-44bd-99e2-f0a81392c8c4": { 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:41:28.881491+0000", 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.151 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.152 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.153 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.154 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.155 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.156 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.157 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.158 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:30.159 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:30.160 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:30.160 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:30.160 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:30.160 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:43:30.160 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:43:30.160 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.146+0000 7fcfc2b15640 1 -- 172.21.15.112:0/3388079904 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcfac0038b0 msgr2=0x7fcfac005dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.146+0000 7fcfc2b15640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcfac0038b0 0x7fcfac005dc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40e55a0 tx=0x7fcfb4000f40 comp rx=0 tx=0).stop 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.146+0000 7fcfc2b15640 1 -- 172.21.15.112:0/3388079904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc109e10 msgr2=0x7fcfbc19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.146+0000 7fcfc2b15640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc109e10 0x7fcfbc19fca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400eb10 tx=0x7fcfb400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.146+0000 7fcfc2b15640 1 -- 172.21.15.112:0/3388079904 shutdown_connections 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.146+0000 7fcfc2b15640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fcfac0038b0 0x7fcfac005dc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.146+0000 7fcfc2b15640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc109e10 0x7fcfbc19fca0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.146+0000 7fcfc2b15640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfbc1094c0 0x7fcfbc19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.146+0000 7fcfc2b15640 1 --2- 172.21.15.112:0/3388079904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfbc102010 0x7fcfbc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.146+0000 7fcfc2b15640 1 -- 172.21.15.112:0/3388079904 >> 172.21.15.112:0/3388079904 conn(0x7fcfbc077c60 msgr2=0x7fcfbc107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.147+0000 7fcfc2b15640 1 -- 172.21.15.112:0/3388079904 shutdown_connections 2024-09-21T03:43:30.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:30.147+0000 7fcfc2b15640 1 -- 172.21.15.112:0/3388079904 wait complete. 2024-09-21T03:43:30.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 61902 39.00 MB/sec warmup 90 sec latency 81.357 ms 2024-09-21T03:43:30.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:30 smithi203 ceph-mon[34723]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 658 op/s 2024-09-21T03:43:30.668 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:30 smithi112 ceph-mon[31442]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 658 op/s 2024-09-21T03:43:30.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:30 smithi187 ceph-mon[34603]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 658 op/s 2024-09-21T03:43:31.263 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:43:31.263 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:43:31.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 62663 39.00 MB/sec warmup 91 sec latency 115.178 ms 2024-09-21T03:43:31.607 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:32.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.201+0000 7fb903fff640 1 -- 172.21.15.112:0/99641827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9041081b0 msgr2=0x7fb9041085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:32.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.201+0000 7fb903fff640 1 --2- 172.21.15.112:0/99641827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9041081b0 0x7fb9041085c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009900 tx=0x7fb8ec02a410 comp rx=0 tx=0).stop 2024-09-21T03:43:32.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.201+0000 7fb903fff640 1 -- 172.21.15.112:0/99641827 shutdown_connections 2024-09-21T03:43:32.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.201+0000 7fb903fff640 1 --2- 172.21.15.112:0/99641827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb904108b00 0x7fb90410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:32.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.201+0000 7fb903fff640 1 --2- 172.21.15.112:0/99641827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9041081b0 0x7fb9041085c0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:32.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.201+0000 7fb903fff640 1 --2- 172.21.15.112:0/99641827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb904102870 0x7fb904102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:32.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.201+0000 7fb903fff640 1 -- 172.21.15.112:0/99641827 >> 172.21.15.112:0/99641827 conn(0x7fb9040fe2c0 msgr2=0x7fb904100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:32.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.201+0000 7fb903fff640 1 -- 172.21.15.112:0/99641827 shutdown_connections 2024-09-21T03:43:32.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.201+0000 7fb903fff640 1 -- 172.21.15.112:0/99641827 wait complete. 2024-09-21T03:43:32.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.202+0000 7fb903fff640 1 Processor -- start 2024-09-21T03:43:32.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.202+0000 7fb903fff640 1 -- start start 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.202+0000 7fb903fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb904102870 0x7fb90419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.202+0000 7fb903fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9041081b0 0x7fb90419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.202+0000 7fb903fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb904108b00 0x7fb9041a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.202+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb904102870 0x7fb90419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.202+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb904102870 0x7fb90419c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49488/0 (socket says 172.21.15.112:49488) 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb902ffd640 1 -- 172.21.15.112:0/83389890 learned_addr learned my addr 172.21.15.112:0/83389890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb904112770 con 0x7fb904102870 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041125f0 con 0x7fb9041081b0 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb8fbfff640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9041081b0 0x7fb90419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041128f0 con 0x7fb904108b00 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb9037fe640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb904108b00 0x7fb9041a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:32.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb8fbfff640 1 -- 172.21.15.112:0/83389890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb904108b00 msgr2=0x7fb9041a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:32.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb8fbfff640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb904108b00 0x7fb9041a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:32.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb8fbfff640 1 -- 172.21.15.112:0/83389890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb904102870 msgr2=0x7fb90419c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:32.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb8fbfff640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb904102870 0x7fb90419c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:32.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb8fbfff640 1 -- 172.21.15.112:0/83389890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9041a2410 con 0x7fb9041081b0 2024-09-21T03:43:32.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.203+0000 7fb8fbfff640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9041081b0 0x7fb90419ca40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec02a930 tx=0x7fb8ec031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:32.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.204+0000 7fb90880f640 1 -- 172.21.15.112:0/83389890 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec03f070 con 0x7fb9041081b0 2024-09-21T03:43:32.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.204+0000 7fb90880f640 1 -- 172.21.15.112:0/83389890 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ec004490 con 0x7fb9041081b0 2024-09-21T03:43:32.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.204+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9041a26a0 con 0x7fb9041081b0 2024-09-21T03:43:32.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.204+0000 7fb90880f640 1 -- 172.21.15.112:0/83389890 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec035070 con 0x7fb9041081b0 2024-09-21T03:43:32.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.204+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9041091e0 con 0x7fb9041081b0 2024-09-21T03:43:32.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.205+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb904103af0 con 0x7fb9041081b0 2024-09-21T03:43:32.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.207+0000 7fb90880f640 1 -- 172.21.15.112:0/83389890 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb8ec0037f0 con 0x7fb9041081b0 2024-09-21T03:43:32.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.208+0000 7fb90880f640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8f00b6700 0x7fb8f00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:32.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.208+0000 7fb90880f640 1 -- 172.21.15.112:0/83389890 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fb8ec107b60 con 0x7fb9041081b0 2024-09-21T03:43:32.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.209+0000 7fb902ffd640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8f00b6700 0x7fb8f00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:32.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.209+0000 7fb902ffd640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8f00b6700 0x7fb8f00b8c10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4014fd0 tx=0x7fb8f4016040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:32.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.213+0000 7fb90880f640 1 -- 172.21.15.112:0/83389890 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8ec0d2460 con 0x7fb9041081b0 2024-09-21T03:43:32.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.503+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9040ffde0 con 0x7fb9041081b0 2024-09-21T03:43:32.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.504+0000 7fb90880f640 1 -- 172.21.15.112:0/83389890 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11936 (secure 0 0 0) 0x7fb8ec0d73a0 con 0x7fb9041081b0 2024-09-21T03:43:32.506 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:43:32.506 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":35,"btime":"2024-09-21T03:42:33:280889+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:42:32.317716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:43:32.506 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 35 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.506+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8f00b6700 msgr2=0x7fb8f00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.506+0000 7fb903fff640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8f00b6700 0x7fb8f00b8c10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4014fd0 tx=0x7fb8f4016040 comp rx=0 tx=0).stop 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.506+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9041081b0 msgr2=0x7fb90419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.506+0000 7fb903fff640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9041081b0 0x7fb90419ca40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec02a930 tx=0x7fb8ec031a20 comp rx=0 tx=0).stop 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.506+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 shutdown_connections 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.506+0000 7fb903fff640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8f00b6700 0x7fb8f00b8c10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.506+0000 7fb903fff640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb904108b00 0x7fb9041a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.506+0000 7fb903fff640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9041081b0 0x7fb90419ca40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.507+0000 7fb903fff640 1 --2- 172.21.15.112:0/83389890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb904102870 0x7fb90419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.507+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 >> 172.21.15.112:0/83389890 conn(0x7fb9040fe2c0 msgr2=0x7fb904107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.507+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 shutdown_connections 2024-09-21T03:43:32.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:32.507+0000 7fb903fff640 1 -- 172.21.15.112:0/83389890 wait complete. 2024-09-21T03:43:32.544 INFO:tasks.workunit.client.0.smithi112.stdout: 10 63357 39.07 MB/sec warmup 92 sec latency 88.591 ms 2024-09-21T03:43:32.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:32 smithi203 ceph-mon[34723]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 831 op/s 2024-09-21T03:43:32.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:32 smithi187 ceph-mon[34603]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 831 op/s 2024-09-21T03:43:32.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:32 smithi112 ceph-mon[31442]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 831 op/s 2024-09-21T03:43:33.433 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:43:33.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 64073 39.02 MB/sec warmup 93 sec latency 79.223 ms 2024-09-21T03:43:33.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:33 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/83389890' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:33.742 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:33.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:33 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/83389890' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:33.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:33 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/83389890' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:43:34.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.310+0000 7fc01165e640 1 -- 172.21.15.112:0/2769821886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103b90 msgr2=0x7fc00c103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.310+0000 7fc01165e640 1 --2- 172.21.15.112:0/2769821886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103b90 0x7fc00c103fa0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009900 tx=0x7fbffc02a410 comp rx=0 tx=0).stop 2024-09-21T03:43:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.310+0000 7fc01165e640 1 -- 172.21.15.112:0/2769821886 shutdown_connections 2024-09-21T03:43:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.310+0000 7fc01165e640 1 --2- 172.21.15.112:0/2769821886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc00c109e20 0x7fc00c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.310+0000 7fc01165e640 1 --2- 172.21.15.112:0/2769821886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c1094d0 0x7fc00c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.310+0000 7fc01165e640 1 --2- 172.21.15.112:0/2769821886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103b90 0x7fc00c103fa0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.310+0000 7fc01165e640 1 -- 172.21.15.112:0/2769821886 >> 172.21.15.112:0/2769821886 conn(0x7fc00c0ff600 msgr2=0x7fc00c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.310+0000 7fc01165e640 1 -- 172.21.15.112:0/2769821886 shutdown_connections 2024-09-21T03:43:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.310+0000 7fc01165e640 1 -- 172.21.15.112:0/2769821886 wait complete. 2024-09-21T03:43:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.311+0000 7fc01165e640 1 Processor -- start 2024-09-21T03:43:34.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.311+0000 7fc01165e640 1 -- start start 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc01165e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103b90 0x7fc00c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc01165e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc00c1094d0 0x7fc00c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc01165e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c109e20 0x7fc00c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc01165e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c113ae0 con 0x7fc00c103b90 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc01165e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c113960 con 0x7fc00c109e20 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc01165e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c113c60 con 0x7fc00c1094d0 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc00affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103b90 0x7fc00c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc00affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103b90 0x7fc00c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49494/0 (socket says 172.21.15.112:49494) 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc00affd640 1 -- 172.21.15.112:0/3051951532 learned_addr learned my addr 172.21.15.112:0/3051951532 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc00a7fc640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc00c1094d0 0x7fc00c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c109e20 0x7fc00c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc00affd640 1 -- 172.21.15.112:0/3051951532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc00c1094d0 msgr2=0x7fc00c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc00affd640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc00c1094d0 0x7fc00c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc00affd640 1 -- 172.21.15.112:0/3051951532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c109e20 msgr2=0x7fc00c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc00affd640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c109e20 0x7fc00c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.312+0000 7fc00affd640 1 -- 172.21.15.112:0/3051951532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c1a0440 con 0x7fc00c103b90 2024-09-21T03:43:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.313+0000 7fc00affd640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103b90 0x7fc00c19a4d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbffc005940 tx=0x7fbffc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.313+0000 7fbff3fff640 1 -- 172.21.15.112:0/3051951532 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc03f030 con 0x7fc00c103b90 2024-09-21T03:43:34.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.313+0000 7fbff3fff640 1 -- 172.21.15.112:0/3051951532 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbffc0044c0 con 0x7fc00c103b90 2024-09-21T03:43:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.313+0000 7fc01165e640 1 -- 172.21.15.112:0/3051951532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c1a06d0 con 0x7fc00c103b90 2024-09-21T03:43:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.313+0000 7fbff3fff640 1 -- 172.21.15.112:0/3051951532 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc035070 con 0x7fc00c103b90 2024-09-21T03:43:34.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.313+0000 7fc01165e640 1 -- 172.21.15.112:0/3051951532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1a6e90 con 0x7fc00c103b90 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.317+0000 7fbff3fff640 1 -- 172.21.15.112:0/3051951532 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fbffc042030 con 0x7fc00c103b90 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.317+0000 7fc017bd2740 1 Processor -- start 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.317+0000 7fc017bd2740 1 -- start start 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.317+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ec342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.317+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec34800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.317+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec34d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.317+0000 7fc017bd2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f49ebaaea0 con 0x55f49e86c390 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.317+0000 7fc017bd2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f49eb9a2d0 con 0x55f49e88d070 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.317+0000 7fc017bd2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f49eb9e0e0 con 0x55f49e8641b0 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.318+0000 7fc00b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ec342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.318+0000 7fc00b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ec342c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:52984/0 (socket says 172.21.15.112:52984) 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.318+0000 7fc00b7fe640 1 -- 172.21.15.112:0/3446199334 learned_addr learned my addr 172.21.15.112:0/3446199334 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:34.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.318+0000 7fc00affd640 1 --2- 172.21.15.112:0/3446199334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec34800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.318+0000 7fc00a7fc640 1 --2- 172.21.15.112:0/3446199334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec34d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.318+0000 7fc00b7fe640 1 -- 172.21.15.112:0/3446199334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 msgr2=0x55f49ec34800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.318+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/3446199334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec34800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.318+0000 7fc00b7fe640 1 -- 172.21.15.112:0/3446199334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 msgr2=0x55f49ec34d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.318+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/3446199334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec34d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.318+0000 7fc00b7fe640 1 -- 172.21.15.112:0/3446199334 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49e8f8650 con 0x55f49e88d070 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.318+0000 7fbff3fff640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbff40b6700 0x7fbff40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.319+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/3446199334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ec342c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc004009fd0 tx=0x7fc0040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b889a9c3cb228a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.319+0000 7fbff3fff640 1 -- 172.21.15.112:0/3051951532 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fbffc1079f0 con 0x7fc00c103b90 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.319+0000 7fbff17fa640 1 -- 172.21.15.112:0/3446199334 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004004d20 con 0x55f49e88d070 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.319+0000 7fbff17fa640 1 -- 172.21.15.112:0/3446199334 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc004004050 con 0x55f49e88d070 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.319+0000 7fbff17fa640 1 -- 172.21.15.112:0/3446199334 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004005280 con 0x55f49e88d070 2024-09-21T03:43:34.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.319+0000 7fc00affd640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbff40b6700 0x7fbff40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.320+0000 7fc017bd2740 1 -- 172.21.15.112:0/3446199334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 msgr2=0x55f49ec342c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.320+0000 7fc017bd2740 1 --2- 172.21.15.112:0/3446199334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ec342c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc004009fd0 tx=0x7fc0040118d0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.320+0000 7fc00affd640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbff40b6700 0x7fbff40b8c10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbffc105220 tx=0x7fbffc10b4c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.320+0000 7fc017bd2740 1 -- 172.21.15.112:0/3446199334 shutdown_connections 2024-09-21T03:43:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.320+0000 7fc017bd2740 1 --2- 172.21.15.112:0/3446199334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec34d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.320+0000 7fc017bd2740 1 --2- 172.21.15.112:0/3446199334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec34800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.320+0000 7fc017bd2740 1 --2- 172.21.15.112:0/3446199334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ec342c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.320+0000 7fc017bd2740 1 -- 172.21.15.112:0/3446199334 >> 172.21.15.112:0/3446199334 conn(0x55f49e881460 msgr2=0x55f49ec321b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.320+0000 7fc017bd2740 1 -- 172.21.15.112:0/3446199334 shutdown_connections 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.320+0000 7fc017bd2740 1 -- 172.21.15.112:0/3446199334 wait complete. 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc017bd2740 1 -- start start 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc017bd2740 1 Processor -- start 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec3f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec3fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ec4bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc017bd2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f49e9fb700 con 0x55f49e86c390 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc017bd2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f49eb35160 con 0x55f49e88d070 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc017bd2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f49ec42aa0 con 0x55f49e8641b0 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc017bd2740 10 client.0 _ll_register_callbacks cb 0x55f49eb4e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc00a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec3fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc00b7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec3f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc00a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec3fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49512/0 (socket says 172.21.15.112:49512) 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc00a7fc640 1 -- 172.21.15.112:0/1322352795 learned_addr learned my addr 172.21.15.112:0/1322352795 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:34.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec3f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:56738/0 (socket says 172.21.15.112:56738) 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.321+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ec4bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.322+0000 7fc00a7fc640 1 -- 172.21.15.112:0/1322352795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 msgr2=0x55f49ec3f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.322+0000 7fc00a7fc640 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec3f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.322+0000 7fc00a7fc640 1 -- 172.21.15.112:0/1322352795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 msgr2=0x55f49ec4bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.322+0000 7fc00a7fc640 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ec4bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.322+0000 7fc00a7fc640 1 -- 172.21.15.112:0/1322352795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49eb62c90 con 0x55f49e86c390 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.322+0000 7fc00a7fc640 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec3fe80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc000012700 tx=0x7fc000012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53be0f827714a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.322+0000 7fbfeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.322+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1322352795 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000004050 con 0x55f49e86c390 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.322+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1322352795 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fc0000041f0 con 0x55f49e86c390 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.322+0000 7fc017bd2740 1 -- 172.21.15.112:0/1322352795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc004004050 con 0x55f49e86c390 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.323+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1322352795 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc0000043f0 con 0x55f49e86c390 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.323+0000 7fc017bd2740 10 client.17853 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.323+0000 7fc017bd2740 1 -- 172.21.15.112:0/1322352795 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f49ec4dba0 con 0x55f49e86c390 2024-09-21T03:43:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.323+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1322352795 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc004004050 con 0x55f49e86c390 2024-09-21T03:43:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.323+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1322352795 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fc000003b60 con 0x55f49e86c390 2024-09-21T03:43:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.323+0000 7fc017bd2740 10 client.17853 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.323+0000 7fc017bd2740 10 client.17853 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:43:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.323+0000 7fc017bd2740 20 client.17853 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.323+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f49e84b9f0 0x55f49ec4f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.323+0000 7fc017bd2740 4 client.17853 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.323+0000 7fc017bd2740 1 -- 172.21.15.112:0/1322352795 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f49ebacdf0 con 0x55f49e84b9f0 2024-09-21T03:43:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.324+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f49e84b9f0 0x55f49ec4f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.324+0000 7fbfeeff5640 10 client.17853 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:43:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.324+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f49e84b9f0 0x55f49ec4f4f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f0cc2c2015c00c4 server_cookie=b292be1db5b71309 in_seq=0 out_seq=0 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1322352795 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f49ebacdf0 con 0x55f49e84b9f0 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fbfeeff5640 10 client.17853 handle_command_reply: tid=0 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fc017bd2740 1 client.17853 shutdown 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fc017bd2740 1 -- 172.21.15.112:0/1322352795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 msgr2=0x55f49ec3fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec3fe80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc000012700 tx=0x7fc000012bc0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fc017bd2740 1 -- 172.21.15.112:0/1322352795 shutdown_connections 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f49e84b9f0 0x55f49ec4f4f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ec4bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec3fe80 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1322352795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec3f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fc017bd2740 1 -- 172.21.15.112:0/1322352795 >> 172.21.15.112:0/1322352795 conn(0x55f49e881460 msgr2=0x55f49ec374a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:34.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.329+0000 7fbfeeff5640 0 client.17853 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:43:34.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.330+0000 7fc017bd2740 1 -- 172.21.15.112:0/1322352795 shutdown_connections 2024-09-21T03:43:34.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.330+0000 7fc017bd2740 1 -- 172.21.15.112:0/1322352795 wait complete. 2024-09-21T03:43:34.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.330+0000 7fc017bd2740 20 client.17853 trim_cache size 0 max 16384 2024-09-21T03:43:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc017bd2740 1 Processor -- start 2024-09-21T03:43:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc017bd2740 1 -- start start 2024-09-21T03:43:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e8641b0 0x55f49ec3fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e86c390 0x55f49ec4bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e88d070 0x55f49ec4fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc017bd2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1afdd0 con 0x55f49e88d070 2024-09-21T03:43:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc017bd2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ad610 con 0x55f49e8641b0 2024-09-21T03:43:34.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc017bd2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ab480 con 0x55f49e86c390 2024-09-21T03:43:34.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc00b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e8641b0 0x55f49ec3fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc00a7fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e86c390 0x55f49ec4bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc00b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e8641b0 0x55f49ec3fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:53006/0 (socket says 172.21.15.112:53006) 2024-09-21T03:43:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc00b7fe640 1 -- 172.21.15.112:0/3075712997 learned_addr learned my addr 172.21.15.112:0/3075712997 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.345+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/3075712997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e88d070 0x55f49ec4fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.346+0000 7fc00b7fe640 1 -- 172.21.15.112:0/3075712997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e86c390 msgr2=0x55f49ec4bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.346+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/3075712997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e86c390 0x55f49ec4bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.346+0000 7fc00b7fe640 1 -- 172.21.15.112:0/3075712997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e8641b0 msgr2=0x55f49ec3fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.346+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/3075712997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e8641b0 0x55f49ec3fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.346+0000 7fc00b7fe640 1 -- 172.21.15.112:0/3075712997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49ec36820 con 0x55f49e88d070 2024-09-21T03:43:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.346+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/3075712997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e88d070 0x55f49ec4fcc0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc004006010 tx=0x7fc004009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad961e9a7f10a768 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.346+0000 7fbff0ff9640 1 -- 172.21.15.112:0/3075712997 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004003af0 con 0x55f49e88d070 2024-09-21T03:43:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.346+0000 7fbff0ff9640 1 -- 172.21.15.112:0/3075712997 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc004003c90 con 0x55f49e88d070 2024-09-21T03:43:34.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.346+0000 7fbff0ff9640 1 -- 172.21.15.112:0/3075712997 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00400fb10 con 0x55f49e88d070 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.347+0000 7fc017bd2740 1 -- 172.21.15.112:0/3075712997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e88d070 msgr2=0x55f49ec4fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.347+0000 7fc017bd2740 1 --2- 172.21.15.112:0/3075712997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e88d070 0x55f49ec4fcc0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc004006010 tx=0x7fc004009be0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.347+0000 7fc017bd2740 1 -- 172.21.15.112:0/3075712997 shutdown_connections 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.347+0000 7fc017bd2740 1 --2- 172.21.15.112:0/3075712997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e88d070 0x55f49ec4fcc0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.347+0000 7fc017bd2740 1 --2- 172.21.15.112:0/3075712997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e86c390 0x55f49ec4bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.347+0000 7fc017bd2740 1 --2- 172.21.15.112:0/3075712997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e8641b0 0x55f49ec3fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.347+0000 7fc017bd2740 1 -- 172.21.15.112:0/3075712997 >> 172.21.15.112:0/3075712997 conn(0x55f49e881460 msgr2=0x55f49ec3f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.347+0000 7fc017bd2740 1 -- 172.21.15.112:0/3075712997 shutdown_connections 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.347+0000 7fc017bd2740 1 -- 172.21.15.112:0/3075712997 wait complete. 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.348+0000 7fc017bd2740 1 -- start start 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.348+0000 7fc017bd2740 1 Processor -- start 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.348+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec5be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.348+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec5c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.348+0000 7fc017bd2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ecc3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.348+0000 7fc017bd2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1abad0 con 0x55f49e86c390 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.348+0000 7fc017bd2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1acf40 con 0x55f49e88d070 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.348+0000 7fc017bd2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ac1a0 con 0x55f49e8641b0 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.348+0000 7fc017bd2740 10 client.0 _ll_register_callbacks cb 0x7fc0000097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc00a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec5c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc00a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec5c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49532/0 (socket says 172.21.15.112:49532) 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc00a7fc640 1 -- 172.21.15.112:0/1071941709 learned_addr learned my addr 172.21.15.112:0/1071941709 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec5be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ecc3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc00a7fc640 1 -- 172.21.15.112:0/1071941709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 msgr2=0x55f49ec5be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc00a7fc640 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec5be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc00a7fc640 1 -- 172.21.15.112:0/1071941709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 msgr2=0x55f49ecc3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:34.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc00a7fc640 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ecc3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc00a7fc640 1 -- 172.21.15.112:0/1071941709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49ec5cc20 con 0x55f49e86c390 2024-09-21T03:43:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc00a7fc640 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec5c350 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc000003f50 tx=0x7fc00000f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45ed4bf2a4a026a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fbfeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:43:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1071941709 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00002b040 con 0x55f49e86c390 2024-09-21T03:43:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1071941709 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fc000003fd0 con 0x55f49e86c390 2024-09-21T03:43:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.349+0000 7fc017bd2740 1 -- 172.21.15.112:0/1071941709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc004003c90 con 0x55f49e86c390 2024-09-21T03:43:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1071941709 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc0000041d0 con 0x55f49e86c390 2024-09-21T03:43:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fc017bd2740 10 client.17859 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fc017bd2740 1 -- 172.21.15.112:0/1071941709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f49ecc5300 con 0x55f49e86c390 2024-09-21T03:43:34.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1071941709 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc004003c90 con 0x55f49e86c390 2024-09-21T03:43:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1071941709 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fc0000035f0 con 0x55f49e86c390 2024-09-21T03:43:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fc017bd2740 10 client.17859 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fc017bd2740 10 client.17859 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:43:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fc017bd2740 20 client.17859 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f49e820da0 0x55f49ecc7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fc017bd2740 4 client.17859 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:43:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fc017bd2740 1 -- 172.21.15.112:0/1071941709 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f49eba88a0 con 0x55f49e820da0 2024-09-21T03:43:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.350+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f49e820da0 0x55f49ecc7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.351+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f49e820da0 0x55f49ecc7ad0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=834744a0eaba4a18 server_cookie=275e505b246276fc in_seq=0 out_seq=0 2024-09-21T03:43:34.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.351+0000 7fbfeeff5640 10 client.17859 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fbfeeff5640 1 -- 172.21.15.112:0/1071941709 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f49eba88a0 con 0x55f49e820da0 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fbfeeff5640 10 client.17859 handle_command_reply: tid=0 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fc017bd2740 1 client.17859 shutdown 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fc017bd2740 1 -- 172.21.15.112:0/1071941709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 msgr2=0x55f49ec5c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec5c350 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc000003f50 tx=0x7fc00000f820 comp rx=0 tx=0).stop 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fc017bd2740 1 -- 172.21.15.112:0/1071941709 shutdown_connections 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x55f49e820da0 0x55f49ecc7ad0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49e88d070 0x55f49ecc3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49e86c390 0x55f49ec5c350 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fc017bd2740 1 --2- 172.21.15.112:0/1071941709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f49e8641b0 0x55f49ec5be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fc017bd2740 1 -- 172.21.15.112:0/1071941709 >> 172.21.15.112:0/1071941709 conn(0x55f49e881460 msgr2=0x55f49ec4fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fbfeeff5640 0 client.17859 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fc017bd2740 1 -- 172.21.15.112:0/1071941709 shutdown_connections 2024-09-21T03:43:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.352+0000 7fc017bd2740 1 -- 172.21.15.112:0/1071941709 wait complete. 2024-09-21T03:43:34.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.353+0000 7fc017bd2740 20 client.17859 trim_cache size 0 max 16384 2024-09-21T03:43:34.358 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:43:34.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.353+0000 7fc01165e640 1 -- 172.21.15.112:0/3051951532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbff40b6700 msgr2=0x7fbff40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:34.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.353+0000 7fc01165e640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbff40b6700 0x7fbff40b8c10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbffc105220 tx=0x7fbffc10b4c0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.353+0000 7fc01165e640 1 -- 172.21.15.112:0/3051951532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103b90 msgr2=0x7fc00c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:34.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.353+0000 7fc01165e640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103b90 0x7fc00c19a4d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbffc005940 tx=0x7fbffc0040c0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.353+0000 7fc01165e640 1 -- 172.21.15.112:0/3051951532 shutdown_connections 2024-09-21T03:43:34.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.353+0000 7fc01165e640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fbff40b6700 0x7fbff40b8c10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.353+0000 7fc01165e640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c109e20 0x7fc00c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.353+0000 7fc01165e640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc00c1094d0 0x7fc00c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.353+0000 7fc01165e640 1 --2- 172.21.15.112:0/3051951532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103b90 0x7fc00c19a4d0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:34.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.353+0000 7fc01165e640 1 -- 172.21.15.112:0/3051951532 >> 172.21.15.112:0/3051951532 conn(0x7fc00c0ff600 msgr2=0x7fc00c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:34.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.354+0000 7fc01165e640 1 -- 172.21.15.112:0/3051951532 shutdown_connections 2024-09-21T03:43:34.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:34.354+0000 7fc01165e640 1 -- 172.21.15.112:0/3051951532 wait complete. 2024-09-21T03:43:34.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:34 smithi112 ceph-mon[31442]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 668 op/s 2024-09-21T03:43:34.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 64760 38.94 MB/sec warmup 94 sec latency 116.569 ms 2024-09-21T03:43:34.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:34 smithi187 ceph-mon[34603]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 668 op/s 2024-09-21T03:43:34.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:34 smithi203 ceph-mon[34723]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 668 op/s 2024-09-21T03:43:35.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:43:35.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 65683 39.03 MB/sec warmup 95 sec latency 148.593 ms 2024-09-21T03:43:35.609 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.103+0000 7f3e699b1640 1 -- 172.21.15.112:0/2197772168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e64102a60 msgr2=0x7f3e64102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.103+0000 7f3e699b1640 1 --2- 172.21.15.112:0/2197772168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e64102a60 0x7f3e64102e70 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009900 tx=0x7f3e5402a410 comp rx=0 tx=0).stop 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.103+0000 7f3e699b1640 1 -- 172.21.15.112:0/2197772168 shutdown_connections 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.103+0000 7f3e699b1640 1 --2- 172.21.15.112:0/2197772168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e64108cf0 0x7f3e6410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.103+0000 7f3e699b1640 1 --2- 172.21.15.112:0/2197772168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e641083a0 0x7f3e641087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.103+0000 7f3e699b1640 1 --2- 172.21.15.112:0/2197772168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e64102a60 0x7f3e64102e70 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.103+0000 7f3e699b1640 1 -- 172.21.15.112:0/2197772168 >> 172.21.15.112:0/2197772168 conn(0x7f3e64079730 msgr2=0x7f3e64079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.103+0000 7f3e699b1640 1 -- 172.21.15.112:0/2197772168 shutdown_connections 2024-09-21T03:43:36.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.103+0000 7f3e699b1640 1 -- 172.21.15.112:0/2197772168 wait complete. 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.104+0000 7f3e699b1640 1 Processor -- start 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.104+0000 7f3e699b1640 1 -- start start 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.104+0000 7f3e699b1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e64102a60 0x7f3e64199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.104+0000 7f3e699b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e641083a0 0x7f3e641998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.104+0000 7f3e699b1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e64108cf0 0x7f3e6419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.104+0000 7f3e699b1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e64112960 con 0x7f3e641083a0 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.104+0000 7f3e699b1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641127e0 con 0x7f3e64108cf0 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.104+0000 7f3e699b1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e64112ae0 con 0x7f3e64102a60 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.104+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e64102a60 0x7f3e64199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e64102a60 0x7f3e64199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46912/0 (socket says 172.21.15.112:46912) 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e62ffd640 1 -- 172.21.15.112:0/3391488209 learned_addr learned my addr 172.21.15.112:0/3391488209 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e637fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e64108cf0 0x7f3e6419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e627fc640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e641083a0 0x7f3e641998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e637fe640 1 -- 172.21.15.112:0/3391488209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e64102a60 msgr2=0x7f3e64199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e637fe640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e64102a60 0x7f3e64199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e637fe640 1 -- 172.21.15.112:0/3391488209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e641083a0 msgr2=0x7f3e641998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e637fe640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e641083a0 0x7f3e641998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e637fe640 1 -- 172.21.15.112:0/3391488209 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6419f280 con 0x7f3e64108cf0 2024-09-21T03:43:36.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e627fc640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e641083a0 0x7f3e641998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:36.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e637fe640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e64108cf0 0x7f3e6419eb50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fd0 tx=0x7f3e5c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.105+0000 7f3e689af640 1 -- 172.21.15.112:0/3391488209 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00ce20 con 0x7f3e64108cf0 2024-09-21T03:43:36.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.106+0000 7f3e699b1640 1 -- 172.21.15.112:0/3391488209 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6419f570 con 0x7f3e64108cf0 2024-09-21T03:43:36.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.106+0000 7f3e699b1640 1 -- 172.21.15.112:0/3391488209 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e641a5cd0 con 0x7f3e64108cf0 2024-09-21T03:43:36.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.106+0000 7f3e689af640 1 -- 172.21.15.112:0/3391488209 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5c005020 con 0x7f3e64108cf0 2024-09-21T03:43:36.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.106+0000 7f3e689af640 1 -- 172.21.15.112:0/3391488209 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c010690 con 0x7f3e64108cf0 2024-09-21T03:43:36.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.109+0000 7f3e6ff25740 1 Processor -- start 2024-09-21T03:43:36.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e689af640 1 -- 172.21.15.112:0/3391488209 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3e5c004a40 con 0x7f3e64108cf0 2024-09-21T03:43:36.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e6ff25740 1 -- start start 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 0x562d097df740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d0940b390 0x562d097de480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562d09749ea0 con 0x562d094031b0 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562d097392d0 con 0x562d0942c070 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562d0973d0e0 con 0x562d0940b390 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e627fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 0x562d097df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e627fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 0x562d097df740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34074/0 (socket says 172.21.15.112:34074) 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42320/0 (socket says 172.21.15.112:42320) 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.110+0000 7f3e627fc640 1 -- 172.21.15.112:0/2179284868 learned_addr learned my addr 172.21.15.112:0/2179284868 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.111+0000 7f3e627fc640 1 --2- 172.21.15.112:0/2179284868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d0940b390 0x562d097de480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.111+0000 7f3e627fc640 1 -- 172.21.15.112:0/2179284868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d0940b390 msgr2=0x562d097de480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:36.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.111+0000 7f3e627fc640 1 --2- 172.21.15.112:0/2179284868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d0940b390 0x562d097de480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.111+0000 7f3e627fc640 1 -- 172.21.15.112:0/2179284868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 msgr2=0x562d097dfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:36.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.111+0000 7f3e627fc640 1 --2- 172.21.15.112:0/2179284868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.111+0000 7f3e627fc640 1 -- 172.21.15.112:0/2179284868 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d09497650 con 0x562d0942c070 2024-09-21T03:43:36.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.111+0000 7f3e62ffd640 1 --2- 172.21.15.112:0/2179284868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:36.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.111+0000 7f3e689af640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e4c0b6680 0x7f3e4c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.111+0000 7f3e627fc640 1 --2- 172.21.15.112:0/2179284868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 0x562d097df740 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e58013960 tx=0x7f3e58009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9292f44b917fa237 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.111+0000 7f3e689af640 1 -- 172.21.15.112:0/3391488209 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7f3e5c0e7b30 con 0x7f3e64108cf0 2024-09-21T03:43:36.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.111+0000 7f3e62ffd640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e4c0b6680 0x7f3e4c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.112+0000 7f3e49ffb640 1 -- 172.21.15.112:0/2179284868 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58004030 con 0x562d0942c070 2024-09-21T03:43:36.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.112+0000 7f3e49ffb640 1 -- 172.21.15.112:0/2179284868 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5800f940 con 0x562d0942c070 2024-09-21T03:43:36.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.112+0000 7f3e49ffb640 1 -- 172.21.15.112:0/2179284868 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58003a20 con 0x562d0942c070 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.113+0000 7f3e6ff25740 1 -- 172.21.15.112:0/2179284868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 msgr2=0x562d097df740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.113+0000 7f3e62ffd640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e4c0b6680 0x7f3e4c0b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3e54031440 tx=0x7f3e540058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.113+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/2179284868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 0x562d097df740 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e58013960 tx=0x7f3e58009c00 comp rx=0 tx=0).stop 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.113+0000 7f3e6ff25740 1 -- 172.21.15.112:0/2179284868 shutdown_connections 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.113+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/2179284868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d0940b390 0x562d097de480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.113+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/2179284868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.113+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/2179284868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 0x562d097df740 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.113+0000 7f3e6ff25740 1 -- 172.21.15.112:0/2179284868 >> 172.21.15.112:0/2179284868 conn(0x562d09420460 msgr2=0x562d097d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.113+0000 7f3e6ff25740 1 -- 172.21.15.112:0/2179284868 shutdown_connections 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.113+0000 7f3e6ff25740 1 -- 172.21.15.112:0/2179284868 wait complete. 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.113+0000 7f3e6ff25740 1 -- start start 2024-09-21T03:43:36.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e6ff25740 1 Processor -- start 2024-09-21T03:43:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 0x562d097e3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d0940b390 0x562d097e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 0x562d097e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562d0959a700 con 0x562d0940b390 2024-09-21T03:43:36.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562d096d4160 con 0x562d0942c070 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562d097deef0 con 0x562d094031b0 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e6ff25740 10 client.0 _ll_register_callbacks cb 0x562d096ed9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 0x562d097e3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e627fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d0940b390 0x562d097e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 0x562d097e3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46942/0 (socket says 172.21.15.112:46942) 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e62ffd640 1 -- 172.21.15.112:0/3233725180 learned_addr learned my addr 172.21.15.112:0/3233725180 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e627fc640 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d0940b390 0x562d097e3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42336/0 (socket says 172.21.15.112:42336) 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.114+0000 7f3e637fe640 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 0x562d097e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.115+0000 7f3e62ffd640 1 -- 172.21.15.112:0/3233725180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 msgr2=0x562d097e8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.115+0000 7f3e62ffd640 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 0x562d097e8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.115+0000 7f3e62ffd640 1 -- 172.21.15.112:0/3233725180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d0940b390 msgr2=0x562d097e3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.115+0000 7f3e62ffd640 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d0940b390 0x562d097e3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.115+0000 7f3e62ffd640 1 -- 172.21.15.112:0/3233725180 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d09701c90 con 0x562d094031b0 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.115+0000 7f3e62ffd640 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 0x562d097e3980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009a20 tx=0x7f3e54009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af0e55bc7006febe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.115+0000 7f3e42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.115+0000 7f3e42ffd640 1 -- 172.21.15.112:0/3233725180 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5404e070 con 0x562d094031b0 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.115+0000 7f3e42ffd640 1 -- 172.21.15.112:0/3233725180 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7f3e540035f0 con 0x562d094031b0 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.115+0000 7f3e6ff25740 1 -- 172.21.15.112:0/3233725180 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e5800f940 con 0x562d094031b0 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.115+0000 7f3e42ffd640 1 -- 172.21.15.112:0/3233725180 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3e5405a050 con 0x562d094031b0 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.116+0000 7f3e6ff25740 10 client.27421 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.116+0000 7f3e6ff25740 1 -- 172.21.15.112:0/3233725180 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562d097f1f00 con 0x562d094031b0 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.116+0000 7f3e42ffd640 1 -- 172.21.15.112:0/3233725180 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5800f940 con 0x562d094031b0 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.116+0000 7f3e42ffd640 1 -- 172.21.15.112:0/3233725180 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f3e5404a7f0 con 0x562d094031b0 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.116+0000 7f3e6ff25740 10 client.27421 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.116+0000 7f3e6ff25740 10 client.27421 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.116+0000 7f3e6ff25740 20 client.27421 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.116+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562d093ea9f0 0x562d097f5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.116+0000 7f3e6ff25740 4 client.27421 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.116+0000 7f3e6ff25740 1 -- 172.21.15.112:0/3233725180 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d0974bdf0 con 0x562d093ea9f0 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.116+0000 7f3e627fc640 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562d093ea9f0 0x562d097f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.117+0000 7f3e627fc640 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562d093ea9f0 0x562d097f5500 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2defb638f18a89e5 server_cookie=f10df5d152d70c75 in_seq=0 out_seq=0 2024-09-21T03:43:36.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.117+0000 7f3e42ffd640 10 client.27421 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.119+0000 7f3e42ffd640 1 -- 172.21.15.112:0/3233725180 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d0974bdf0 con 0x562d093ea9f0 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.119+0000 7f3e42ffd640 10 client.27421 handle_command_reply: tid=0 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.119+0000 7f3e6ff25740 1 client.27421 shutdown 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e6ff25740 1 -- 172.21.15.112:0/3233725180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 msgr2=0x562d097e3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 0x562d097e3980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009a20 tx=0x7f3e54009260 comp rx=0 tx=0).stop 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e6ff25740 1 -- 172.21.15.112:0/3233725180 shutdown_connections 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562d093ea9f0 0x562d097f5500 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0942c070 0x562d097e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d0940b390 0x562d097e3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e42ffd640 0 client.27421 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/3233725180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 0x562d097e3980 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e6ff25740 1 -- 172.21.15.112:0/3233725180 >> 172.21.15.112:0/3233725180 conn(0x562d09420460 msgr2=0x562d097d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e6ff25740 1 -- 172.21.15.112:0/3233725180 shutdown_connections 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e6ff25740 1 -- 172.21.15.112:0/3233725180 wait complete. 2024-09-21T03:43:36.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.120+0000 7f3e6ff25740 20 client.27421 trim_cache size 0 max 16384 2024-09-21T03:43:36.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.134+0000 7f3e6ff25740 1 Processor -- start 2024-09-21T03:43:36.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.134+0000 7f3e6ff25740 1 -- start start 2024-09-21T03:43:36.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.134+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d093ea9f0 0x562d097d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.134+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.134+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d0940b390 0x562d097dd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.134+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50011080 con 0x562d094031b0 2024-09-21T03:43:36.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.134+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5000e8c0 con 0x562d093ea9f0 2024-09-21T03:43:36.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.134+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5000c730 con 0x562d0940b390 2024-09-21T03:43:36.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e627fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d093ea9f0 0x562d097d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e627fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42342/0 (socket says 172.21.15.112:42342) 2024-09-21T03:43:36.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e627fc640 1 -- 172.21.15.112:0/2555531291 learned_addr learned my addr 172.21.15.112:0/2555531291 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:36.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e637fe640 1 --2- 172.21.15.112:0/2555531291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d0940b390 0x562d097dd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e627fc640 1 -- 172.21.15.112:0/2555531291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d0940b390 msgr2=0x562d097dd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:36.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e627fc640 1 --2- 172.21.15.112:0/2555531291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d0940b390 0x562d097dd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e627fc640 1 -- 172.21.15.112:0/2555531291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d093ea9f0 msgr2=0x562d097d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:36.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e627fc640 1 --2- 172.21.15.112:0/2555531291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d093ea9f0 0x562d097d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e627fc640 1 -- 172.21.15.112:0/2555531291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d097e4690 con 0x562d094031b0 2024-09-21T03:43:36.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e627fc640 1 --2- 172.21.15.112:0/2555531291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dce40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009730 tx=0x7f3e58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdea6ea1dc7b488 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.135+0000 7f3e497fa640 1 -- 172.21.15.112:0/2555531291 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5802f070 con 0x562d094031b0 2024-09-21T03:43:36.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.136+0000 7f3e497fa640 1 -- 172.21.15.112:0/2555531291 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5800fdf0 con 0x562d094031b0 2024-09-21T03:43:36.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.136+0000 7f3e497fa640 1 -- 172.21.15.112:0/2555531291 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58003850 con 0x562d094031b0 2024-09-21T03:43:36.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.137+0000 7f3e6ff25740 1 -- 172.21.15.112:0/2555531291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 msgr2=0x562d097dce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.137+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/2555531291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dce40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009730 tx=0x7f3e58000bc0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.137+0000 7f3e6ff25740 1 -- 172.21.15.112:0/2555531291 shutdown_connections 2024-09-21T03:43:36.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.137+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/2555531291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d0940b390 0x562d097dd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.137+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/2555531291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d094031b0 0x562d097dce40 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.137+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/2555531291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d093ea9f0 0x562d097d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.137+0000 7f3e6ff25740 1 -- 172.21.15.112:0/2555531291 >> 172.21.15.112:0/2555531291 conn(0x562d09420460 msgr2=0x562d097db440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.137+0000 7f3e6ff25740 1 -- 172.21.15.112:0/2555531291 shutdown_connections 2024-09-21T03:43:36.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.137+0000 7f3e6ff25740 1 -- 172.21.15.112:0/2555531291 wait complete. 2024-09-21T03:43:36.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.137+0000 7f3e6ff25740 1 -- start start 2024-09-21T03:43:36.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e6ff25740 1 Processor -- start 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d093ea9f0 0x562d097eb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 0x562d097ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e6ff25740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0940b390 0x562d0985eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5000cd80 con 0x562d093ea9f0 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5000e1f0 con 0x562d0940b390 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e6ff25740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5000d450 con 0x562d094031b0 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e6ff25740 10 client.0 _ll_register_callbacks cb 0x7f3e54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d093ea9f0 0x562d097eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d093ea9f0 0x562d097eb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42356/0 (socket says 172.21.15.112:42356) 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e627fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 0x562d097ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e62ffd640 1 -- 172.21.15.112:0/1219089938 learned_addr learned my addr 172.21.15.112:0/1219089938 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e637fe640 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0940b390 0x562d0985eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e62ffd640 1 -- 172.21.15.112:0/1219089938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 msgr2=0x562d097ebaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:36.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.138+0000 7f3e62ffd640 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 0x562d097ebaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e62ffd640 1 -- 172.21.15.112:0/1219089938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0940b390 msgr2=0x562d0985eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e62ffd640 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0940b390 0x562d0985eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e62ffd640 1 -- 172.21.15.112:0/1219089938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d097ec2d0 con 0x562d093ea9f0 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e637fe640 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0940b390 0x562d0985eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e627fc640 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 0x562d097ebaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e62ffd640 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d093ea9f0 0x562d097eb560 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3e54004770 tx=0x7f3e54004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a228057f5a6e05e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e42ffd640 1 -- 172.21.15.112:0/1219089938 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54062020 con 0x562d093ea9f0 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e42ffd640 1 -- 172.21.15.112:0/1219089938 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7f3e5404a8e0 con 0x562d093ea9f0 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e6ff25740 1 -- 172.21.15.112:0/1219089938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e5800fdf0 con 0x562d093ea9f0 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e42ffd640 1 -- 172.21.15.112:0/1219089938 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3e5406d050 con 0x562d093ea9f0 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e6ff25740 10 client.17877 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.139+0000 7f3e6ff25740 1 -- 172.21.15.112:0/1219089938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562d09860410 con 0x562d093ea9f0 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.140+0000 7f3e42ffd640 1 -- 172.21.15.112:0/1219089938 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5800fdf0 con 0x562d093ea9f0 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.140+0000 7f3e42ffd640 1 -- 172.21.15.112:0/1219089938 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f3e5403d5e0 con 0x562d093ea9f0 2024-09-21T03:43:36.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.140+0000 7f3e6ff25740 10 client.17877 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.140+0000 7f3e6ff25740 10 client.17877 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.140+0000 7f3e6ff25740 20 client.17877 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.140+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562d0942c070 0x562d09862c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.140+0000 7f3e6ff25740 4 client.17877 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.140+0000 7f3e6ff25740 1 -- 172.21.15.112:0/1219089938 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d097478a0 con 0x562d0942c070 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.140+0000 7f3e627fc640 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562d0942c070 0x562d09862c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e42ffd640 10 client.17877 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e627fc640 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562d0942c070 0x562d09862c50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff41490fb1462697 server_cookie=c8d737142f688648 in_seq=0 out_seq=0 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e42ffd640 1 -- 172.21.15.112:0/1219089938 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d097478a0 con 0x562d0942c070 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e42ffd640 10 client.17877 handle_command_reply: tid=0 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e6ff25740 1 client.17877 shutdown 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e6ff25740 1 -- 172.21.15.112:0/1219089938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d093ea9f0 msgr2=0x562d097eb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d093ea9f0 0x562d097eb560 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3e54004770 tx=0x7f3e54004540 comp rx=0 tx=0).stop 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e6ff25740 1 -- 172.21.15.112:0/1219089938 shutdown_connections 2024-09-21T03:43:36.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x562d0942c070 0x562d09862c50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562d0940b390 0x562d0985eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d094031b0 0x562d097ebaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e42ffd640 0 client.17877 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:43:36.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e6ff25740 1 --2- 172.21.15.112:0/1219089938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d093ea9f0 0x562d097eb560 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e6ff25740 1 -- 172.21.15.112:0/1219089938 >> 172.21.15.112:0/1219089938 conn(0x562d09420460 msgr2=0x562d097df6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e6ff25740 1 -- 172.21.15.112:0/1219089938 shutdown_connections 2024-09-21T03:43:36.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.141+0000 7f3e6ff25740 1 -- 172.21.15.112:0/1219089938 wait complete. 2024-09-21T03:43:36.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e6ff25740 20 client.17877 trim_cache size 0 max 16384 2024-09-21T03:43:36.147 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:43:36.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 -- 172.21.15.112:0/3391488209 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e4c0b6680 msgr2=0x7f3e4c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e4c0b6680 0x7f3e4c0b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3e54031440 tx=0x7f3e540058d0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 -- 172.21.15.112:0/3391488209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e64108cf0 msgr2=0x7f3e6419eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:36.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e64108cf0 0x7f3e6419eb50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fd0 tx=0x7f3e5c00ef10 comp rx=0 tx=0).stop 2024-09-21T03:43:36.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 -- 172.21.15.112:0/3391488209 shutdown_connections 2024-09-21T03:43:36.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f3e4c0b6680 0x7f3e4c0b8b90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e64108cf0 0x7f3e6419eb50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e641083a0 0x7f3e641998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 --2- 172.21.15.112:0/3391488209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e64102a60 0x7f3e64199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:36.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 -- 172.21.15.112:0/3391488209 >> 172.21.15.112:0/3391488209 conn(0x7f3e64079730 msgr2=0x7f3e64101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:36.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 -- 172.21.15.112:0/3391488209 shutdown_connections 2024-09-21T03:43:36.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:36.142+0000 7f3e699b1640 1 -- 172.21.15.112:0/3391488209 wait complete. 2024-09-21T03:43:36.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 66592 39.30 MB/sec warmup 96 sec latency 60.703 ms 2024-09-21T03:43:36.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:36 smithi187 ceph-mon[34603]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 622 op/s 2024-09-21T03:43:36.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:36 smithi112 ceph-mon[31442]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 622 op/s 2024-09-21T03:43:36.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:36 smithi203 ceph-mon[34723]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 622 op/s 2024-09-21T03:43:37.370 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:43:37.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 67108 39.10 MB/sec warmup 97 sec latency 109.085 ms 2024-09-21T03:43:37.690 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.278+0000 7f98efef6640 1 -- 172.21.15.112:0/2827583480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e8106bd0 msgr2=0x7f98e8110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.278+0000 7f98efef6640 1 --2- 172.21.15.112:0/2827583480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e8106bd0 0x7f98e8110660 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009ca0 tx=0x7f98e002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.278+0000 7f98efef6640 1 -- 172.21.15.112:0/2827583480 shutdown_connections 2024-09-21T03:43:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.278+0000 7f98efef6640 1 --2- 172.21.15.112:0/2827583480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e8106bd0 0x7f98e8110660 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.278+0000 7f98efef6640 1 --2- 172.21.15.112:0/2827583480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8106280 0x7f98e8106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.278+0000 7f98efef6640 1 --2- 172.21.15.112:0/2827583480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8102030 0x7f98e8102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.278+0000 7f98efef6640 1 -- 172.21.15.112:0/2827583480 >> 172.21.15.112:0/2827583480 conn(0x7f98e8077c60 msgr2=0x7f98e80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.278+0000 7f98efef6640 1 -- 172.21.15.112:0/2827583480 shutdown_connections 2024-09-21T03:43:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.278+0000 7f98efef6640 1 -- 172.21.15.112:0/2827583480 wait complete. 2024-09-21T03:43:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.279+0000 7f98efef6640 1 Processor -- start 2024-09-21T03:43:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.279+0000 7f98efef6640 1 -- start start 2024-09-21T03:43:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.279+0000 7f98efef6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e8102030 0x7f98e8102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.279+0000 7f98efef6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8106280 0x7f98e8103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.279+0000 7f98efef6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8106bd0 0x7f98e8103a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98efef6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8113ac0 con 0x7f98e8102030 2024-09-21T03:43:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98efef6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8113940 con 0x7f98e8106bd0 2024-09-21T03:43:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98efef6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8113c40 con 0x7f98e8106280 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98ee493640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8106bd0 0x7f98e8103a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98ed491640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8106280 0x7f98e8103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98ee493640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8106bd0 0x7f98e8103a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34132/0 (socket says 172.21.15.112:34132) 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98ed491640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8106280 0x7f98e8103510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:46976/0 (socket says 172.21.15.112:46976) 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98ee493640 1 -- 172.21.15.112:0/749618116 learned_addr learned my addr 172.21.15.112:0/749618116 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98edc92640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e8102030 0x7f98e8102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98ee493640 1 -- 172.21.15.112:0/749618116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8106280 msgr2=0x7f98e8103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98ee493640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8106280 0x7f98e8103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98ee493640 1 -- 172.21.15.112:0/749618116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e8102030 msgr2=0x7f98e8102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98ee493640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e8102030 0x7f98e8102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98ee493640 1 -- 172.21.15.112:0/749618116 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e81a8750 con 0x7f98e8106bd0 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.280+0000 7f98ee493640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8106bd0 0x7f98e8103a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f98e0005e30 tx=0x7f98e0031840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.281+0000 7f98deffd640 1 -- 172.21.15.112:0/749618116 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0003b70 con 0x7f98e8106bd0 2024-09-21T03:43:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.281+0000 7f98efef6640 1 -- 172.21.15.112:0/749618116 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e81a89e0 con 0x7f98e8106bd0 2024-09-21T03:43:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.281+0000 7f98efef6640 1 -- 172.21.15.112:0/749618116 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e81a8f70 con 0x7f98e8106bd0 2024-09-21T03:43:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.281+0000 7f98deffd640 1 -- 172.21.15.112:0/749618116 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e0003d10 con 0x7f98e8106bd0 2024-09-21T03:43:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.281+0000 7f98deffd640 1 -- 172.21.15.112:0/749618116 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0030a60 con 0x7f98e8106bd0 2024-09-21T03:43:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.284+0000 7f98f646a740 1 Processor -- start 2024-09-21T03:43:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.285+0000 7f98deffd640 1 -- 172.21.15.112:0/749618116 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f98e0030c00 con 0x7f98e8106bd0 2024-09-21T03:43:38.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.285+0000 7f98f646a740 1 -- start start 2024-09-21T03:43:38.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.286+0000 7f98f646a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a3840070 0x55f8a3befeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.286+0000 7f98f646a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a38171b0 0x55f8a3bf03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.286+0000 7f98f646a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a381f390 0x55f8a3bf0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.286+0000 7f98f646a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f8a3b5dea0 con 0x55f8a381f390 2024-09-21T03:43:38.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.286+0000 7f98f646a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f8a3b4d2d0 con 0x55f8a3840070 2024-09-21T03:43:38.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.286+0000 7f98f646a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f8a3b510e0 con 0x55f8a38171b0 2024-09-21T03:43:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.286+0000 7f98ed491640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a3840070 0x55f8a3befeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.286+0000 7f98ed491640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a3840070 0x55f8a3befeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34138/0 (socket says 172.21.15.112:34138) 2024-09-21T03:43:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.286+0000 7f98ed491640 1 -- 172.21.15.112:0/3065609236 learned_addr learned my addr 172.21.15.112:0/3065609236 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.286+0000 7f98ed491640 1 --2- 172.21.15.112:0/3065609236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a381f390 0x55f8a3bf0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.287+0000 7f98edc92640 1 --2- 172.21.15.112:0/3065609236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a38171b0 0x55f8a3bf03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.287+0000 7f98deffd640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98d40b6780 0x7f98d40b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.287+0000 7f98ed491640 1 -- 172.21.15.112:0/3065609236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a38171b0 msgr2=0x55f8a3bf03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.287+0000 7f98ed491640 1 --2- 172.21.15.112:0/3065609236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a38171b0 0x55f8a3bf03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.287+0000 7f98ed491640 1 -- 172.21.15.112:0/3065609236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a381f390 msgr2=0x55f8a3bf0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.287+0000 7f98ed491640 1 --2- 172.21.15.112:0/3065609236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a381f390 0x55f8a3bf0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.287+0000 7f98ed491640 1 -- 172.21.15.112:0/3065609236 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8a38ab650 con 0x55f8a3840070 2024-09-21T03:43:38.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.287+0000 7f98deffd640 1 -- 172.21.15.112:0/749618116 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7f98e01086b0 con 0x7f98e8106bd0 2024-09-21T03:43:38.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.287+0000 7f98edc92640 1 --2- 172.21.15.112:0/3065609236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a38171b0 0x55f8a3bf03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:38.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.287+0000 7f98ee493640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98d40b6780 0x7f98d40b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.288+0000 7f98ed491640 1 --2- 172.21.15.112:0/3065609236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a3840070 0x55f8a3befeb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009f60 tx=0x7f98d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f7e5a7362d1bf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.288+0000 7f98d3fff640 1 -- 172.21.15.112:0/3065609236 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d801e420 con 0x55f8a3840070 2024-09-21T03:43:38.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.288+0000 7f98d3fff640 1 -- 172.21.15.112:0/3065609236 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d801ea70 con 0x55f8a3840070 2024-09-21T03:43:38.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.288+0000 7f98d3fff640 1 -- 172.21.15.112:0/3065609236 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d800e970 con 0x55f8a3840070 2024-09-21T03:43:38.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.289+0000 7f98f646a740 1 -- 172.21.15.112:0/3065609236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a3840070 msgr2=0x55f8a3befeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.289+0000 7f98ee493640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98d40b6780 0x7f98d40b8c90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f98e0006010 tx=0x7f98e00095c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.289+0000 7f98f646a740 1 --2- 172.21.15.112:0/3065609236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a3840070 0x55f8a3befeb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009f60 tx=0x7f98d80046c0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.289+0000 7f98f646a740 1 -- 172.21.15.112:0/3065609236 shutdown_connections 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.289+0000 7f98f646a740 1 --2- 172.21.15.112:0/3065609236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a381f390 0x55f8a3bf0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.289+0000 7f98f646a740 1 --2- 172.21.15.112:0/3065609236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a38171b0 0x55f8a3bf03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.289+0000 7f98f646a740 1 --2- 172.21.15.112:0/3065609236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a3840070 0x55f8a3befeb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.289+0000 7f98f646a740 1 -- 172.21.15.112:0/3065609236 >> 172.21.15.112:0/3065609236 conn(0x55f8a3834460 msgr2=0x55f8a3bedda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.289+0000 7f98f646a740 1 -- 172.21.15.112:0/3065609236 shutdown_connections 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.289+0000 7f98f646a740 1 -- 172.21.15.112:0/3065609236 wait complete. 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.290+0000 7f98f646a740 1 -- start start 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.290+0000 7f98f646a740 1 Processor -- start 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.290+0000 7f98f646a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 0x55f8a3bfb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.290+0000 7f98f646a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bfbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.290+0000 7f98f646a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a3840070 0x55f8a3c07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.290+0000 7f98f646a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f8a39ae700 con 0x55f8a3840070 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.290+0000 7f98f646a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f8a3ae8160 con 0x55f8a38171b0 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.290+0000 7f98f646a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f8a3bfe690 con 0x55f8a381f390 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.290+0000 7f98f646a740 10 client.0 _ll_register_callbacks cb 0x55f8a3b019e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98ed491640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 0x55f8a3bfb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98ed491640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 0x55f8a3bfb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34148/0 (socket says 172.21.15.112:34148) 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98ed491640 1 -- 172.21.15.112:0/2534441113 learned_addr learned my addr 172.21.15.112:0/2534441113 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98edc92640 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bfbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98ed491640 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a3840070 0x55f8a3c07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98ed491640 1 -- 172.21.15.112:0/2534441113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 msgr2=0x55f8a3bfbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98ed491640 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bfbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98ed491640 1 -- 172.21.15.112:0/2534441113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a3840070 msgr2=0x55f8a3c07a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98ed491640 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a3840070 0x55f8a3c07a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98ed491640 1 -- 172.21.15.112:0/2534441113 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8a3b15c90 con 0x55f8a38171b0 2024-09-21T03:43:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98ed491640 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 0x55f8a3bfb580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005e00 tx=0x7f98d8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfcf70c358ae1913 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:43:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.291+0000 7f98d17fa640 1 -- 172.21.15.112:0/2534441113 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d800ec30 con 0x55f8a38171b0 2024-09-21T03:43:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98f646a740 1 -- 172.21.15.112:0/2534441113 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d801ea70 con 0x55f8a38171b0 2024-09-21T03:43:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98d17fa640 1 -- 172.21.15.112:0/2534441113 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7f98d800edd0 con 0x55f8a38171b0 2024-09-21T03:43:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98d17fa640 1 -- 172.21.15.112:0/2534441113 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f98d8005020 con 0x55f8a38171b0 2024-09-21T03:43:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98f646a740 10 client.27425 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98f646a740 1 -- 172.21.15.112:0/2534441113 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f8a3c0a3f0 con 0x55f8a38171b0 2024-09-21T03:43:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98d17fa640 1 -- 172.21.15.112:0/2534441113 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d801ea70 con 0x55f8a38171b0 2024-09-21T03:43:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98d17fa640 1 -- 172.21.15.112:0/2534441113 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f98d8004c90 con 0x55f8a38171b0 2024-09-21T03:43:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98f646a740 10 client.27425 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98f646a740 10 client.27425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:43:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98f646a740 20 client.27425 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98f646a740 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f8a37fe9f0 0x55f8a3c0d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98f646a740 4 client.27425 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.292+0000 7f98f646a740 1 -- 172.21.15.112:0/2534441113 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8a3b5fdf0 con 0x55f8a37fe9f0 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.293+0000 7f98edc92640 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f8a37fe9f0 0x55f8a3c0d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.293+0000 7f98d17fa640 10 client.27425 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.293+0000 7f98edc92640 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f8a37fe9f0 0x55f8a3c0d810 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c74cc75e663a59bd server_cookie=94658b99dbcc8c47 in_seq=0 out_seq=0 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.294+0000 7f98d17fa640 1 -- 172.21.15.112:0/2534441113 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8a3b5fdf0 con 0x55f8a37fe9f0 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.294+0000 7f98d17fa640 10 client.27425 handle_command_reply: tid=0 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.294+0000 7f98f646a740 1 client.27425 shutdown 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.295+0000 7f98f646a740 1 -- 172.21.15.112:0/2534441113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 msgr2=0x55f8a3bfb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.295+0000 7f98f646a740 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 0x55f8a3bfb580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005e00 tx=0x7f98d8005850 comp rx=0 tx=0).stop 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.295+0000 7f98f646a740 1 -- 172.21.15.112:0/2534441113 shutdown_connections 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.295+0000 7f98f646a740 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f8a37fe9f0 0x55f8a3c0d810 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.295+0000 7f98d17fa640 0 client.27425 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.295+0000 7f98f646a740 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a3840070 0x55f8a3c07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.295+0000 7f98f646a740 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bfbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.295+0000 7f98f646a740 1 --2- 172.21.15.112:0/2534441113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 0x55f8a3bfb580 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.295+0000 7f98f646a740 1 -- 172.21.15.112:0/2534441113 >> 172.21.15.112:0/2534441113 conn(0x55f8a3834460 msgr2=0x55f8a3bf3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.295+0000 7f98f646a740 1 -- 172.21.15.112:0/2534441113 shutdown_connections 2024-09-21T03:43:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.295+0000 7f98f646a740 1 -- 172.21.15.112:0/2534441113 wait complete. 2024-09-21T03:43:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.296+0000 7f98f646a740 20 client.27425 trim_cache size 0 max 16384 2024-09-21T03:43:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.313+0000 7f98f646a740 1 Processor -- start 2024-09-21T03:43:38.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.314+0000 7f98f646a740 1 -- start start 2024-09-21T03:43:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.314+0000 7f98f646a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a38171b0 0x55f8a3bfa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.314+0000 7f98f646a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bfd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.314+0000 7f98f646a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a3840070 0x55f8a3c07910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.314+0000 7f98f646a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc011080 con 0x55f8a38171b0 2024-09-21T03:43:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.314+0000 7f98f646a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc00e8c0 con 0x55f8a3840070 2024-09-21T03:43:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.314+0000 7f98f646a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc00c730 con 0x55f8a381f390 2024-09-21T03:43:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.314+0000 7f98edc92640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bfd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.314+0000 7f98edc92640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bfd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:47006/0 (socket says 172.21.15.112:47006) 2024-09-21T03:43:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.314+0000 7f98edc92640 1 -- 172.21.15.112:0/1086385456 learned_addr learned my addr 172.21.15.112:0/1086385456 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.315+0000 7f98ed491640 1 --2- 172.21.15.112:0/1086385456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a38171b0 0x55f8a3bfa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.315+0000 7f98edc92640 1 -- 172.21.15.112:0/1086385456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a3840070 msgr2=0x55f8a3c07910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.315+0000 7f98edc92640 1 --2- 172.21.15.112:0/1086385456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a3840070 0x55f8a3c07910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.315+0000 7f98edc92640 1 -- 172.21.15.112:0/1086385456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a38171b0 msgr2=0x55f8a3bfa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.315+0000 7f98edc92640 1 --2- 172.21.15.112:0/1086385456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a38171b0 0x55f8a3bfa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.315+0000 7f98edc92640 1 -- 172.21.15.112:0/1086385456 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8a3bf21a0 con 0x55f8a381f390 2024-09-21T03:43:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.315+0000 7f98edc92640 1 --2- 172.21.15.112:0/1086385456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bfd1b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f98e8104a20 tx=0x7f98e4014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fbef894c66a7910 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.315+0000 7f98d37fe640 1 -- 172.21.15.112:0/1086385456 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4004310 con 0x55f8a381f390 2024-09-21T03:43:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.316+0000 7f98d37fe640 1 -- 172.21.15.112:0/1086385456 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f8a3b5fdf0 con 0x55f8a381f390 2024-09-21T03:43:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.316+0000 7f98d37fe640 1 -- 172.21.15.112:0/1086385456 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e401e950 con 0x55f8a381f390 2024-09-21T03:43:38.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.317+0000 7f98f646a740 1 -- 172.21.15.112:0/1086385456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 msgr2=0x55f8a3bfd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.317+0000 7f98f646a740 1 --2- 172.21.15.112:0/1086385456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bfd1b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f98e8104a20 tx=0x7f98e4014040 comp rx=0 tx=0).stop 2024-09-21T03:43:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.317+0000 7f98f646a740 1 -- 172.21.15.112:0/1086385456 shutdown_connections 2024-09-21T03:43:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.317+0000 7f98f646a740 1 --2- 172.21.15.112:0/1086385456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a3840070 0x55f8a3c07910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.317+0000 7f98f646a740 1 --2- 172.21.15.112:0/1086385456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bfd1b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.317+0000 7f98f646a740 1 --2- 172.21.15.112:0/1086385456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a38171b0 0x55f8a3bfa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.317+0000 7f98f646a740 1 -- 172.21.15.112:0/1086385456 >> 172.21.15.112:0/1086385456 conn(0x55f8a3834460 msgr2=0x55f8a3bfcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.317+0000 7f98f646a740 1 -- 172.21.15.112:0/1086385456 shutdown_connections 2024-09-21T03:43:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.317+0000 7f98f646a740 1 -- 172.21.15.112:0/1086385456 wait complete. 2024-09-21T03:43:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.317+0000 7f98f646a740 1 -- start start 2024-09-21T03:43:38.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.318+0000 7f98f646a740 1 Processor -- start 2024-09-21T03:43:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.318+0000 7f98f646a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 0x55f8a3bff980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.318+0000 7f98f646a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.318+0000 7f98f646a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a3840070 0x55f8a3c00400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.318+0000 7f98f646a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc00cd80 con 0x55f8a3840070 2024-09-21T03:43:38.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.318+0000 7f98f646a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc00e1f0 con 0x55f8a38171b0 2024-09-21T03:43:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.318+0000 7f98f646a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc00d450 con 0x55f8a381f390 2024-09-21T03:43:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.318+0000 7f98f646a740 10 client.0 _ll_register_callbacks cb 0x7f98d8009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.319+0000 7f98ed491640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 0x55f8a3bff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.319+0000 7f98ed491640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 0x55f8a3bff980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34174/0 (socket says 172.21.15.112:34174) 2024-09-21T03:43:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.319+0000 7f98edc92640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.319+0000 7f98ed491640 1 -- 172.21.15.112:0/519824464 learned_addr learned my addr 172.21.15.112:0/519824464 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:38.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.319+0000 7f98ed491640 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a3840070 0x55f8a3c00400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.320+0000 7f98edc92640 1 -- 172.21.15.112:0/519824464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 msgr2=0x55f8a3bff980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.320+0000 7f98edc92640 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 0x55f8a3bff980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.320+0000 7f98edc92640 1 -- 172.21.15.112:0/519824464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a3840070 msgr2=0x55f8a3c00400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.320+0000 7f98edc92640 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a3840070 0x55f8a3c00400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.320+0000 7f98edc92640 1 -- 172.21.15.112:0/519824464 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8a3c77070 con 0x55f8a381f390 2024-09-21T03:43:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.320+0000 7f98edc92640 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bffec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55f8a3bee420 tx=0x7f98e401e490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdf08d249fc5ddc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.320+0000 7f98d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:43:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.320+0000 7f98d17fa640 1 -- 172.21.15.112:0/519824464 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e40043a0 con 0x55f8a381f390 2024-09-21T03:43:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.321+0000 7f98f646a740 1 -- 172.21.15.112:0/519824464 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f8a3b5fdf0 con 0x55f8a381f390 2024-09-21T03:43:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.321+0000 7f98d17fa640 1 -- 172.21.15.112:0/519824464 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7f98e40035f0 con 0x55f8a381f390 2024-09-21T03:43:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.321+0000 7f98d17fa640 1 -- 172.21.15.112:0/519824464 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f98e40037f0 con 0x55f8a381f390 2024-09-21T03:43:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.321+0000 7f98f646a740 10 client.27445 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:38.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.321+0000 7f98f646a740 1 -- 172.21.15.112:0/519824464 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f8a3c782f0 con 0x55f8a381f390 2024-09-21T03:43:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.321+0000 7f98d17fa640 1 -- 172.21.15.112:0/519824464 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e4003990 con 0x55f8a381f390 2024-09-21T03:43:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.321+0000 7f98d17fa640 1 -- 172.21.15.112:0/519824464 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f98e40254d0 con 0x55f8a381f390 2024-09-21T03:43:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.322+0000 7f98f646a740 10 client.27445 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.322+0000 7f98f646a740 10 client.27445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:43:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.322+0000 7f98f646a740 20 client.27445 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.322+0000 7f98f646a740 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f8a37d3da0 0x55f8a3c7aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.322+0000 7f98f646a740 4 client.27445 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:43:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.322+0000 7f98f646a740 1 -- 172.21.15.112:0/519824464 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8a3b5fdf0 con 0x55f8a37d3da0 2024-09-21T03:43:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.322+0000 7f98ed491640 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f8a37d3da0 0x55f8a3c7aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:38.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.322+0000 7f98d17fa640 10 client.27445 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.323+0000 7f98ed491640 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f8a37d3da0 0x55f8a3c7aa80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea596a2d7f31bc2c server_cookie=5198c4c742dc5277 in_seq=0 out_seq=0 2024-09-21T03:43:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.324+0000 7f98d17fa640 1 -- 172.21.15.112:0/519824464 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8a3b5fdf0 con 0x55f8a37d3da0 2024-09-21T03:43:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.324+0000 7f98d17fa640 10 client.27445 handle_command_reply: tid=0 2024-09-21T03:43:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.324+0000 7f98f646a740 1 client.27445 shutdown 2024-09-21T03:43:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.324+0000 7f98f646a740 1 -- 172.21.15.112:0/519824464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 msgr2=0x55f8a3bffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.324+0000 7f98f646a740 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bffec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55f8a3bee420 tx=0x7f98e401e490 comp rx=0 tx=0).stop 2024-09-21T03:43:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.324+0000 7f98f646a740 1 -- 172.21.15.112:0/519824464 shutdown_connections 2024-09-21T03:43:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.324+0000 7f98f646a740 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55f8a37d3da0 0x55f8a3c7aa80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.324+0000 7f98f646a740 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8a3840070 0x55f8a3c00400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.324+0000 7f98f646a740 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8a381f390 0x55f8a3bffec0 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55f8a3bee420 tx=0x7f98e401e490 comp rx=0 tx=0).stop 2024-09-21T03:43:38.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.324+0000 7f98f646a740 1 --2- 172.21.15.112:0/519824464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f8a38171b0 0x55f8a3bff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.324+0000 7f98f646a740 1 -- 172.21.15.112:0/519824464 >> 172.21.15.112:0/519824464 conn(0x55f8a3834460 msgr2=0x55f8a3bf0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.325+0000 7f98d17fa640 0 client.27445 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.325+0000 7f98f646a740 1 -- 172.21.15.112:0/519824464 shutdown_connections 2024-09-21T03:43:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.325+0000 7f98f646a740 1 -- 172.21.15.112:0/519824464 wait complete. 2024-09-21T03:43:38.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.325+0000 7f98f646a740 20 client.27445 trim_cache size 0 max 16384 2024-09-21T03:43:38.328 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:43:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.326+0000 7f98efef6640 1 -- 172.21.15.112:0/749618116 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98d40b6780 msgr2=0x7f98d40b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.326+0000 7f98efef6640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98d40b6780 0x7f98d40b8c90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f98e0006010 tx=0x7f98e00095c0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.326+0000 7f98efef6640 1 -- 172.21.15.112:0/749618116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8106bd0 msgr2=0x7f98e8103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.326+0000 7f98efef6640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8106bd0 0x7f98e8103a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f98e0005e30 tx=0x7f98e0031840 comp rx=0 tx=0).stop 2024-09-21T03:43:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.326+0000 7f98efef6640 1 -- 172.21.15.112:0/749618116 shutdown_connections 2024-09-21T03:43:38.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.326+0000 7f98efef6640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f98d40b6780 0x7f98d40b8c90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.326+0000 7f98efef6640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8106bd0 0x7f98e8103a50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.326+0000 7f98efef6640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8106280 0x7f98e8103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.326+0000 7f98efef6640 1 --2- 172.21.15.112:0/749618116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98e8102030 0x7f98e8102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.326+0000 7f98efef6640 1 -- 172.21.15.112:0/749618116 >> 172.21.15.112:0/749618116 conn(0x7f98e8077c60 msgr2=0x7f98e81074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.327+0000 7f98efef6640 1 -- 172.21.15.112:0/749618116 shutdown_connections 2024-09-21T03:43:38.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:38.327+0000 7f98efef6640 1 -- 172.21.15.112:0/749618116 wait complete. 2024-09-21T03:43:38.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 67699 38.99 MB/sec warmup 98 sec latency 182.645 ms 2024-09-21T03:43:38.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:38 smithi187 ceph-mon[34603]: pgmap v774: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 796 op/s 2024-09-21T03:43:38.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:38 smithi112 ceph-mon[31442]: pgmap v774: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 796 op/s 2024-09-21T03:43:38.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:38 smithi203 ceph-mon[34723]: pgmap v774: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 796 op/s 2024-09-21T03:43:39.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 68333 38.94 MB/sec warmup 99 sec latency 226.282 ms 2024-09-21T03:43:40.265 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:43:40.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:43:40.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 68854 38.72 MB/sec warmup 100 sec latency 292.790 ms 2024-09-21T03:43:40.559 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:40.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:40 smithi187 ceph-mon[34603]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 613 op/s 2024-09-21T03:43:40.680 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:40 smithi112 ceph-mon[31442]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 613 op/s 2024-09-21T03:43:40.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:40 smithi203 ceph-mon[34723]: pgmap v775: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 613 op/s 2024-09-21T03:43:41.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.113+0000 7fb86caa8640 1 -- 172.21.15.112:0/3097339353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb868103c30 msgr2=0x7fb868104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.113+0000 7fb86caa8640 1 --2- 172.21.15.112:0/3097339353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb868103c30 0x7fb868104040 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb858009900 tx=0x7fb85802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.113+0000 7fb86caa8640 1 -- 172.21.15.112:0/3097339353 shutdown_connections 2024-09-21T03:43:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.113+0000 7fb86caa8640 1 --2- 172.21.15.112:0/3097339353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868109ec0 0x7fb8681106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.113+0000 7fb86caa8640 1 --2- 172.21.15.112:0/3097339353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb868109570 0x7fb868109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.113+0000 7fb86caa8640 1 --2- 172.21.15.112:0/3097339353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb868103c30 0x7fb868104040 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.113+0000 7fb86caa8640 1 -- 172.21.15.112:0/3097339353 >> 172.21.15.112:0/3097339353 conn(0x7fb86807b8d0 msgr2=0x7fb86807bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.114+0000 7fb86caa8640 1 -- 172.21.15.112:0/3097339353 shutdown_connections 2024-09-21T03:43:41.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.114+0000 7fb86caa8640 1 -- 172.21.15.112:0/3097339353 wait complete. 2024-09-21T03:43:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.114+0000 7fb86caa8640 1 Processor -- start 2024-09-21T03:43:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb86caa8640 1 -- start start 2024-09-21T03:43:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb86caa8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868103c30 0x7fb86819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb86caa8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb868109570 0x7fb86819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb86caa8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb868109ec0 0x7fb8681a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb86caa8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb868075f60 con 0x7fb868109570 2024-09-21T03:43:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb86caa8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb868075de0 con 0x7fb868103c30 2024-09-21T03:43:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb86caa8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb8680760e0 con 0x7fb868109ec0 2024-09-21T03:43:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb865d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb868109570 0x7fb86819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb86659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868103c30 0x7fb86819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb865d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb868109570 0x7fb86819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42448/0 (socket says 172.21.15.112:42448) 2024-09-21T03:43:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb86659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868103c30 0x7fb86819c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34184/0 (socket says 172.21.15.112:34184) 2024-09-21T03:43:41.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.115+0000 7fb865d9b640 1 -- 172.21.15.112:0/1082556801 learned_addr learned my addr 172.21.15.112:0/1082556801 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.116+0000 7fb866d9d640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb868109ec0 0x7fb8681a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.116+0000 7fb865d9b640 1 -- 172.21.15.112:0/1082556801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb868109ec0 msgr2=0x7fb8681a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.116+0000 7fb865d9b640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb868109ec0 0x7fb8681a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.116+0000 7fb865d9b640 1 -- 172.21.15.112:0/1082556801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868103c30 msgr2=0x7fb86819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:41.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.116+0000 7fb865d9b640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868103c30 0x7fb86819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.116+0000 7fb865d9b640 1 -- 172.21.15.112:0/1082556801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8681a25a0 con 0x7fb868109570 2024-09-21T03:43:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.116+0000 7fb86659c640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868103c30 0x7fb86819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.116+0000 7fb866d9d640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb868109ec0 0x7fb8681a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.116+0000 7fb865d9b640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb868109570 0x7fb86819cca0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9c0 tx=0x7fb85c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.116+0000 7fb8577fe640 1 -- 172.21.15.112:0/1082556801 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00ccc0 con 0x7fb868109570 2024-09-21T03:43:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.117+0000 7fb86caa8640 1 -- 172.21.15.112:0/1082556801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8681a2890 con 0x7fb868109570 2024-09-21T03:43:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.117+0000 7fb86caa8640 1 -- 172.21.15.112:0/1082556801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8681a90b0 con 0x7fb868109570 2024-09-21T03:43:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.117+0000 7fb8577fe640 1 -- 172.21.15.112:0/1082556801 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb85c0054d0 con 0x7fb868109570 2024-09-21T03:43:41.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.117+0000 7fb8577fe640 1 -- 172.21.15.112:0/1082556801 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c010430 con 0x7fb868109570 2024-09-21T03:43:41.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.121+0000 7fb87301c740 1 Processor -- start 2024-09-21T03:43:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.121+0000 7fb87301c740 1 -- start start 2024-09-21T03:43:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.121+0000 7fb87301c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 0x555ea5bfe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.121+0000 7fb87301c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea582e490 0x555ea5bfe970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.121+0000 7fb87301c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 0x555ea5bfeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.121+0000 7fb87301c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ea5b74e60 con 0x555ea5836670 2024-09-21T03:43:41.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.121+0000 7fb87301c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555ea5b64290 con 0x555ea5857350 2024-09-21T03:43:41.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.121+0000 7fb87301c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555ea5b680a0 con 0x555ea582e490 2024-09-21T03:43:41.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb8577fe640 1 -- 172.21.15.112:0/1082556801 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb85c0046c0 con 0x7fb868109570 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb866d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 0x555ea5bfe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb866d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 0x555ea5bfe430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34188/0 (socket says 172.21.15.112:34188) 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb866d9d640 1 -- 172.21.15.112:0/4013532886 learned_addr learned my addr 172.21.15.112:0/4013532886 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb86659c640 1 --2- 172.21.15.112:0/4013532886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea582e490 0x555ea5bfe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb865d9b640 1 --2- 172.21.15.112:0/4013532886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 0x555ea5bfeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb866d9d640 1 -- 172.21.15.112:0/4013532886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea582e490 msgr2=0x555ea5bfe970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb866d9d640 1 --2- 172.21.15.112:0/4013532886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea582e490 0x555ea5bfe970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb866d9d640 1 -- 172.21.15.112:0/4013532886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 msgr2=0x555ea5bfeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb866d9d640 1 --2- 172.21.15.112:0/4013532886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 0x555ea5bfeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb866d9d640 1 -- 172.21.15.112:0/4013532886 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea58c2930 con 0x555ea5857350 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.122+0000 7fb866d9d640 1 --2- 172.21.15.112:0/4013532886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 0x555ea5bfe430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb860008e60 tx=0x7fb86000d710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de51549fe1d3cac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:41.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.123+0000 7fb865d9b640 1 --2- 172.21.15.112:0/4013532886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 0x555ea5bfeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:41.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.123+0000 7fb854ff9640 1 -- 172.21.15.112:0/4013532886 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86000f830 con 0x555ea5857350 2024-09-21T03:43:41.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.123+0000 7fb854ff9640 1 -- 172.21.15.112:0/4013532886 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb860018480 con 0x555ea5857350 2024-09-21T03:43:41.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.123+0000 7fb854ff9640 1 -- 172.21.15.112:0/4013532886 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86000e610 con 0x555ea5857350 2024-09-21T03:43:41.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.124+0000 7fb8577fe640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8500b6730 0x7fb8500b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.124+0000 7fb86659c640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8500b6730 0x7fb8500b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.124+0000 7fb8577fe640 1 -- 172.21.15.112:0/1082556801 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fb85c0e7890 con 0x7fb868109570 2024-09-21T03:43:41.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.124+0000 7fb87301c740 1 -- 172.21.15.112:0/4013532886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 msgr2=0x555ea5bfe430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.124+0000 7fb87301c740 1 --2- 172.21.15.112:0/4013532886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 0x555ea5bfe430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb860008e60 tx=0x7fb86000d710 comp rx=0 tx=0).stop 2024-09-21T03:43:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.124+0000 7fb87301c740 1 -- 172.21.15.112:0/4013532886 shutdown_connections 2024-09-21T03:43:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.124+0000 7fb87301c740 1 --2- 172.21.15.112:0/4013532886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 0x555ea5bfeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.124+0000 7fb87301c740 1 --2- 172.21.15.112:0/4013532886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea582e490 0x555ea5bfe970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.124+0000 7fb87301c740 1 --2- 172.21.15.112:0/4013532886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 0x555ea5bfe430 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.124+0000 7fb87301c740 1 -- 172.21.15.112:0/4013532886 >> 172.21.15.112:0/4013532886 conn(0x555ea584b740 msgr2=0x555ea5bfc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.125+0000 7fb87301c740 1 -- 172.21.15.112:0/4013532886 shutdown_connections 2024-09-21T03:43:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.125+0000 7fb87301c740 1 -- 172.21.15.112:0/4013532886 wait complete. 2024-09-21T03:43:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.125+0000 7fb86659c640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8500b6730 0x7fb8500b8c40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb858031440 tx=0x7fb858009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:41.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.125+0000 7fb87301c740 1 -- start start 2024-09-21T03:43:41.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.126+0000 7fb87301c740 1 Processor -- start 2024-09-21T03:43:41.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb87301c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea582e490 0x555ea5c0a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb87301c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 0x555ea5c0a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb87301c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 0x555ea5c15f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb86659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea582e490 0x555ea5c0a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb87301c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ea59c5890 con 0x555ea5836670 2024-09-21T03:43:41.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb87301c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555ea5afed40 con 0x555ea5857350 2024-09-21T03:43:41.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb87301c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555ea5c0cc10 con 0x555ea582e490 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb86659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea582e490 0x555ea5c0a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:47042/0 (socket says 172.21.15.112:47042) 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb86659c640 1 -- 172.21.15.112:0/428585344 learned_addr learned my addr 172.21.15.112:0/428585344 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb865d9b640 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 0x555ea5c0a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb87301c740 10 client.0 _ll_register_callbacks cb 0x555ea5b62c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb866d9d640 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 0x555ea5c15f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb865d9b640 1 -- 172.21.15.112:0/428585344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea582e490 msgr2=0x555ea5c0a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb865d9b640 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea582e490 0x555ea5c0a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb865d9b640 1 -- 172.21.15.112:0/428585344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 msgr2=0x555ea5c15f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb865d9b640 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 0x555ea5c15f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.127+0000 7fb865d9b640 1 -- 172.21.15.112:0/428585344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea5c164f0 con 0x555ea5836670 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.128+0000 7fb865d9b640 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 0x555ea5c0a560 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0e5010 tx=0x7fb85c0b7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9dad33d3360f1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.128+0000 7fb84dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.128+0000 7fb84dffb640 1 -- 172.21.15.112:0/428585344 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0ad060 con 0x555ea5836670 2024-09-21T03:43:41.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.128+0000 7fb84dffb640 1 -- 172.21.15.112:0/428585344 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fb85c0a9060 con 0x555ea5836670 2024-09-21T03:43:41.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.128+0000 7fb87301c740 1 -- 172.21.15.112:0/428585344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb860018480 con 0x555ea5836670 2024-09-21T03:43:41.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.128+0000 7fb84dffb640 1 -- 172.21.15.112:0/428585344 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb85c0ace80 con 0x555ea5836670 2024-09-21T03:43:41.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.128+0000 7fb87301c740 10 client.17904 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:41.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.128+0000 7fb87301c740 1 -- 172.21.15.112:0/428585344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555ea5c18dd0 con 0x555ea5836670 2024-09-21T03:43:41.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.128+0000 7fb84dffb640 1 -- 172.21.15.112:0/428585344 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb85c094e50 con 0x555ea5836670 2024-09-21T03:43:41.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.129+0000 7fb84dffb640 1 -- 172.21.15.112:0/428585344 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fb85c0a9260 con 0x555ea5836670 2024-09-21T03:43:41.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.129+0000 7fb87301c740 10 client.17904 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:41.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.129+0000 7fb87301c740 10 client.17904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:43:41.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.129+0000 7fb87301c740 20 client.17904 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:41.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.129+0000 7fb87301c740 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x555ea5815cd0 0x555ea5c1c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.129+0000 7fb87301c740 4 client.17904 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:41.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.129+0000 7fb87301c740 1 -- 172.21.15.112:0/428585344 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ea5b76db0 con 0x555ea5815cd0 2024-09-21T03:43:41.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.129+0000 7fb86659c640 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x555ea5815cd0 0x555ea5c1c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.129+0000 7fb84dffb640 10 client.17904 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.130+0000 7fb86659c640 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x555ea5815cd0 0x555ea5c1c430 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7da7a80d9d77872f server_cookie=48685afbd4ba5011 in_seq=0 out_seq=0 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb84dffb640 1 -- 172.21.15.112:0/428585344 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555ea5b76db0 con 0x555ea5815cd0 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb84dffb640 10 client.17904 handle_command_reply: tid=0 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 1 client.17904 shutdown 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 1 -- 172.21.15.112:0/428585344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 msgr2=0x555ea5c0a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 0x555ea5c0a560 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0e5010 tx=0x7fb85c0b7040 comp rx=0 tx=0).stop 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 1 -- 172.21.15.112:0/428585344 shutdown_connections 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x555ea5815cd0 0x555ea5c1c430 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea5857350 0x555ea5c15f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5836670 0x555ea5c0a560 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 1 --2- 172.21.15.112:0/428585344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea582e490 0x555ea5c0a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 1 -- 172.21.15.112:0/428585344 >> 172.21.15.112:0/428585344 conn(0x555ea584b740 msgr2=0x555ea5c015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:41.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb84dffb640 0 client.17904 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:41.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 1 -- 172.21.15.112:0/428585344 shutdown_connections 2024-09-21T03:43:41.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 1 -- 172.21.15.112:0/428585344 wait complete. 2024-09-21T03:43:41.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.131+0000 7fb87301c740 20 client.17904 trim_cache size 0 max 16384 2024-09-21T03:43:41.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.153+0000 7fb87301c740 1 Processor -- start 2024-09-21T03:43:41.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.153+0000 7fb87301c740 1 -- start start 2024-09-21T03:43:41.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.153+0000 7fb87301c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 0x555ea5c034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.153+0000 7fb87301c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea5836670 0x555ea5c03a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.153+0000 7fb87301c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 0x555ea5c01530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.153+0000 7fb87301c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb848011040 con 0x555ea5857350 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.153+0000 7fb87301c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb84800e880 con 0x555ea582e490 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.153+0000 7fb87301c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb84800c6f0 con 0x555ea5836670 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb866d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 0x555ea5c01530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb866d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 0x555ea5c01530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42470/0 (socket says 172.21.15.112:42470) 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb86659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 0x555ea5c034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb866d9d640 1 -- 172.21.15.112:0/557235275 learned_addr learned my addr 172.21.15.112:0/557235275 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb865d9b640 1 --2- 172.21.15.112:0/557235275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea5836670 0x555ea5c03a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb86659c640 1 -- 172.21.15.112:0/557235275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea5836670 msgr2=0x555ea5c03a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb86659c640 1 --2- 172.21.15.112:0/557235275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea5836670 0x555ea5c03a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb86659c640 1 -- 172.21.15.112:0/557235275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 msgr2=0x555ea5c01530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:41.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb86659c640 1 --2- 172.21.15.112:0/557235275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 0x555ea5c01530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb86659c640 1 -- 172.21.15.112:0/557235275 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea5c0a6e0 con 0x555ea582e490 2024-09-21T03:43:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb866d9d640 1 --2- 172.21.15.112:0/557235275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 0x555ea5c01530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb86659c640 1 --2- 172.21.15.112:0/557235275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 0x555ea5c034c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x555ea5c1cb10 tx=0x7fb858046000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7592b03b8c97248b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.154+0000 7fb84ffff640 1 -- 172.21.15.112:0/557235275 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858035040 con 0x555ea582e490 2024-09-21T03:43:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.155+0000 7fb84ffff640 1 -- 172.21.15.112:0/557235275 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb858005040 con 0x555ea582e490 2024-09-21T03:43:41.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.155+0000 7fb84ffff640 1 -- 172.21.15.112:0/557235275 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858004770 con 0x555ea582e490 2024-09-21T03:43:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.155+0000 7fb87301c740 1 -- 172.21.15.112:0/557235275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 msgr2=0x555ea5c034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.155+0000 7fb87301c740 1 --2- 172.21.15.112:0/557235275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 0x555ea5c034c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x555ea5c1cb10 tx=0x7fb858046000 comp rx=0 tx=0).stop 2024-09-21T03:43:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.155+0000 7fb87301c740 1 -- 172.21.15.112:0/557235275 shutdown_connections 2024-09-21T03:43:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.155+0000 7fb87301c740 1 --2- 172.21.15.112:0/557235275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 0x555ea5c01530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.155+0000 7fb87301c740 1 --2- 172.21.15.112:0/557235275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea5836670 0x555ea5c03a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.155+0000 7fb87301c740 1 --2- 172.21.15.112:0/557235275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 0x555ea5c034c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.155+0000 7fb87301c740 1 -- 172.21.15.112:0/557235275 >> 172.21.15.112:0/557235275 conn(0x555ea584b740 msgr2=0x555ea5c04fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.156+0000 7fb87301c740 1 -- 172.21.15.112:0/557235275 shutdown_connections 2024-09-21T03:43:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.156+0000 7fb87301c740 1 -- 172.21.15.112:0/557235275 wait complete. 2024-09-21T03:43:41.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.156+0000 7fb87301c740 1 -- start start 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.156+0000 7fb87301c740 1 Processor -- start 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.156+0000 7fb87301c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 0x555ea5c0e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.156+0000 7fb87301c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea5836670 0x555ea5c0ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.156+0000 7fb87301c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 0x555ea5c7fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.156+0000 7fb87301c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb84800cd40 con 0x555ea5857350 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.156+0000 7fb87301c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb84800e1b0 con 0x555ea582e490 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.156+0000 7fb87301c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb84800d410 con 0x555ea5836670 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.156+0000 7fb87301c740 10 client.0 _ll_register_callbacks cb 0x7fb85c0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb86659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 0x555ea5c0e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb86659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 0x555ea5c0e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:34216/0 (socket says 172.21.15.112:34216) 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb86659c640 1 -- 172.21.15.112:0/1512612416 learned_addr learned my addr 172.21.15.112:0/1512612416 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb866d9d640 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 0x555ea5c7fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb865d9b640 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea5836670 0x555ea5c0ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb86659c640 1 -- 172.21.15.112:0/1512612416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea5836670 msgr2=0x555ea5c0ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb86659c640 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea5836670 0x555ea5c0ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb86659c640 1 -- 172.21.15.112:0/1512612416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 msgr2=0x555ea5c7fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb86659c640 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 0x555ea5c7fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb86659c640 1 -- 172.21.15.112:0/1512612416 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea5c803e0 con 0x555ea582e490 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb865d9b640 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea5836670 0x555ea5c0ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb866d9d640 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 0x555ea5c7fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb86659c640 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 0x555ea5c0e750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb858048040 tx=0x7fb858000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e0ded35906abb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb84dffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.157+0000 7fb84dffb640 1 -- 172.21.15.112:0/1512612416 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858005330 con 0x555ea582e490 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb84dffb640 1 -- 172.21.15.112:0/1512612416 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fb8580054d0 con 0x555ea582e490 2024-09-21T03:43:41.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb87301c740 1 -- 172.21.15.112:0/1512612416 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb858005040 con 0x555ea582e490 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb84dffb640 1 -- 172.21.15.112:0/1512612416 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb858004030 con 0x555ea582e490 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb87301c740 10 client.27443 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb87301c740 1 -- 172.21.15.112:0/1512612416 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555ea5c87720 con 0x555ea582e490 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb84dffb640 1 -- 172.21.15.112:0/1512612416 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb858005040 con 0x555ea582e490 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb84dffb640 1 -- 172.21.15.112:0/1512612416 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fb8580035f0 con 0x555ea582e490 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb87301c740 10 client.27443 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb87301c740 10 client.27443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb87301c740 20 client.27443 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb87301c740 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x555ea57eb080 0x555ea5c89eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb87301c740 4 client.27443 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb87301c740 1 -- 172.21.15.112:0/1512612416 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555ea5b72860 con 0x555ea57eb080 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.158+0000 7fb865d9b640 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x555ea57eb080 0x555ea5c89eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:41.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.159+0000 7fb84dffb640 10 client.27443 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:41.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.159+0000 7fb865d9b640 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x555ea57eb080 0x555ea5c89eb0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccd5354ecfec386f server_cookie=f841e1897978e26 in_seq=0 out_seq=0 2024-09-21T03:43:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb84dffb640 1 -- 172.21.15.112:0/1512612416 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555ea5b72860 con 0x555ea57eb080 2024-09-21T03:43:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb84dffb640 10 client.27443 handle_command_reply: tid=0 2024-09-21T03:43:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 1 client.27443 shutdown 2024-09-21T03:43:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 1 -- 172.21.15.112:0/1512612416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 msgr2=0x555ea5c0e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 0x555ea5c0e750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb858048040 tx=0x7fb858000bc0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 1 -- 172.21.15.112:0/1512612416 shutdown_connections 2024-09-21T03:43:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x555ea57eb080 0x555ea5c89eb0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ea5857350 0x555ea5c7fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ea5836670 0x555ea5c0ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb84dffb640 0 client.27443 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:41.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 1 --2- 172.21.15.112:0/1512612416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ea582e490 0x555ea5c0e750 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 1 -- 172.21.15.112:0/1512612416 >> 172.21.15.112:0/1512612416 conn(0x555ea584b740 msgr2=0x555ea5c01530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:41.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 1 -- 172.21.15.112:0/1512612416 shutdown_connections 2024-09-21T03:43:41.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 1 -- 172.21.15.112:0/1512612416 wait complete. 2024-09-21T03:43:41.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.165+0000 7fb87301c740 20 client.27443 trim_cache size 0 max 16384 2024-09-21T03:43:41.168 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:43:41.168 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-09-21T03:43:41.168 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "5e9f01a6-49ed-4393-b672-9a31f31c59fb", 2024-09-21T03:43:41.169 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-09-21T03:43:41.169 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:43:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 -- 172.21.15.112:0/1082556801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8500b6730 msgr2=0x7fb8500b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8500b6730 0x7fb8500b8c40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb858031440 tx=0x7fb858009560 comp rx=0 tx=0).stop 2024-09-21T03:43:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 -- 172.21.15.112:0/1082556801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb868109570 msgr2=0x7fb86819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb868109570 0x7fb86819cca0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9c0 tx=0x7fb85c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:43:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 -- 172.21.15.112:0/1082556801 shutdown_connections 2024-09-21T03:43:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fb8500b6730 0x7fb8500b8c40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb868109ec0 0x7fb8681a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb868109570 0x7fb86819cca0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 --2- 172.21.15.112:0/1082556801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868103c30 0x7fb86819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 -- 172.21.15.112:0/1082556801 >> 172.21.15.112:0/1082556801 conn(0x7fb86807b8d0 msgr2=0x7fb868107810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 -- 172.21.15.112:0/1082556801 shutdown_connections 2024-09-21T03:43:41.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:41.166+0000 7fb86caa8640 1 -- 172.21.15.112:0/1082556801 wait complete. 2024-09-21T03:43:41.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 69442 38.82 MB/sec warmup 101 sec latency 164.740 ms 2024-09-21T03:43:41.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:41 smithi187 ceph-mon[34603]: scrub queued for path: / 2024-09-21T03:43:41.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:41 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:41.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:41 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:43:41.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:41 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:43:41.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:41 smithi112 ceph-mon[31442]: scrub queued for path: / 2024-09-21T03:43:41.751 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:41 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:41.752 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:41 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:43:41.752 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:41 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:43:41.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:41 smithi203 ceph-mon[34723]: scrub queued for path: / 2024-09-21T03:43:41.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:41 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:41.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:41 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:43:41.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:41 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:43:41.997 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:43:42.296 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:43:42.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 70056 38.81 MB/sec warmup 102 sec latency 278.244 ms 2024-09-21T03:43:42.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:42 smithi187 ceph-mon[34603]: pgmap v776: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 745 op/s 2024-09-21T03:43:42.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:42 smithi187 ceph-mon[34603]: scrub summary: active paths [/] 2024-09-21T03:43:42.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:42 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:42.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:42 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:42.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:42 smithi187 ceph-mon[34603]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:42.689 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:42 smithi112 ceph-mon[31442]: pgmap v776: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 745 op/s 2024-09-21T03:43:42.689 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:42 smithi112 ceph-mon[31442]: scrub summary: active paths [/] 2024-09-21T03:43:42.689 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:42 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:42.689 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:42 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:42.689 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:42 smithi112 ceph-mon[31442]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:42.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.794+0000 7fdee6e90640 1 -- 172.21.15.112:0/1132331279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee01094b0 msgr2=0x7fdee01098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.794+0000 7fdee6e90640 1 --2- 172.21.15.112:0/1132331279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee01094b0 0x7fdee01098c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009900 tx=0x7fdecc02a410 comp rx=0 tx=0).stop 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.794+0000 7fdee6e90640 1 -- 172.21.15.112:0/1132331279 shutdown_connections 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.794+0000 7fdee6e90640 1 --2- 172.21.15.112:0/1132331279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdee0109e00 0x7fdee0110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.794+0000 7fdee6e90640 1 --2- 172.21.15.112:0/1132331279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee01094b0 0x7fdee01098c0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.794+0000 7fdee6e90640 1 --2- 172.21.15.112:0/1132331279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee0103b70 0x7fdee0103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.794+0000 7fdee6e90640 1 -- 172.21.15.112:0/1132331279 >> 172.21.15.112:0/1132331279 conn(0x7fdee00ff5e0 msgr2=0x7fdee0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.794+0000 7fdee6e90640 1 -- 172.21.15.112:0/1132331279 shutdown_connections 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.794+0000 7fdee6e90640 1 -- 172.21.15.112:0/1132331279 wait complete. 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.795+0000 7fdee6e90640 1 Processor -- start 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.795+0000 7fdee6e90640 1 -- start start 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee6e90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdee0103b70 0x7fdee019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee6e90640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee01094b0 0x7fdee019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee6e90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee0109e00 0x7fdee019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee6e90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdee0113ac0 con 0x7fdee0109e00 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee6e90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdee0113940 con 0x7fdee0103b70 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee6e90640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdee0113c40 con 0x7fdee01094b0 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee542d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee0109e00 0x7fdee019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee4c2c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdee0103b70 0x7fdee019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdedffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee01094b0 0x7fdee019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee542d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee0109e00 0x7fdee019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42492/0 (socket says 172.21.15.112:42492) 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee542d640 1 -- 172.21.15.112:0/691134449 learned_addr learned my addr 172.21.15.112:0/691134449 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee542d640 1 -- 172.21.15.112:0/691134449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee01094b0 msgr2=0x7fdee019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee542d640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee01094b0 0x7fdee019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee542d640 1 -- 172.21.15.112:0/691134449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdee0103b70 msgr2=0x7fdee019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee542d640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdee0103b70 0x7fdee019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee542d640 1 -- 172.21.15.112:0/691134449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee01a0420 con 0x7fdee0109e00 2024-09-21T03:43:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.796+0000 7fdee4c2c640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdee0103b70 0x7fdee019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.797+0000 7fdee542d640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee0109e00 0x7fdee019fcf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fded800eb10 tx=0x7fded800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.797+0000 7fdeddffb640 1 -- 172.21.15.112:0/691134449 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8004b20 con 0x7fdee0109e00 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.797+0000 7fdee6e90640 1 -- 172.21.15.112:0/691134449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee01a0710 con 0x7fdee0109e00 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.797+0000 7fdeddffb640 1 -- 172.21.15.112:0/691134449 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded8004cc0 con 0x7fdee0109e00 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.797+0000 7fdeddffb640 1 -- 172.21.15.112:0/691134449 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8018630 con 0x7fdee0109e00 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.797+0000 7fdee6e90640 1 -- 172.21.15.112:0/691134449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee01a6e70 con 0x7fdee0109e00 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.800+0000 7fdeddffb640 1 -- 172.21.15.112:0/691134449 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fded8005000 con 0x7fdee0109e00 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.800+0000 7fdeed404740 1 Processor -- start 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.800+0000 7fdeed404740 1 -- start start 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdeed404740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8e8070 0x561cebc8f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdeed404740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc8fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdeed404740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdeed404740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561cebc05e50 con 0x561ceb8c7390 2024-09-21T03:43:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdeed404740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561cebbf5280 con 0x561ceb8e8070 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdeed404740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561cebbf9090 con 0x561ceb8bf1b0 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdedffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdedffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc900d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42504/0 (socket says 172.21.15.112:42504) 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdedffff640 1 -- 172.21.15.112:0/144609958 learned_addr learned my addr 172.21.15.112:0/144609958 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdedffff640 1 --2- 172.21.15.112:0/144609958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8e8070 0x561cebc8f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdeddffb640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fded00b6680 0x7fded00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdee4c2c640 1 --2- 172.21.15.112:0/144609958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc8fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdedffff640 1 -- 172.21.15.112:0/144609958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 msgr2=0x561cebc8fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdedffff640 1 --2- 172.21.15.112:0/144609958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc8fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdedffff640 1 -- 172.21.15.112:0/144609958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8e8070 msgr2=0x561cebc8f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.801+0000 7fdedffff640 1 --2- 172.21.15.112:0/144609958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8e8070 0x561cebc8f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.802+0000 7fdedffff640 1 -- 172.21.15.112:0/144609958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ceb953650 con 0x561ceb8c7390 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.802+0000 7fdeddffb640 1 -- 172.21.15.112:0/691134449 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fded80e7df0 con 0x7fdee0109e00 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.802+0000 7fdee542d640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fded00b6680 0x7fded00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.802+0000 7fdedffff640 1 --2- 172.21.15.112:0/144609958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc900d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0057c0 tx=0x7fdecc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07f7b6e4105fbad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.802+0000 7fdecaffd640 1 -- 172.21.15.112:0/144609958 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc035070 con 0x561ceb8c7390 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.802+0000 7fdecaffd640 1 -- 172.21.15.112:0/144609958 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdecc005030 con 0x561ceb8c7390 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.802+0000 7fdecaffd640 1 -- 172.21.15.112:0/144609958 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc032e60 con 0x561ceb8c7390 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.803+0000 7fdee542d640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fded00b6680 0x7fded00b8b90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fded80049d0 tx=0x7fded800efd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.803+0000 7fdeed404740 1 -- 172.21.15.112:0/144609958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 msgr2=0x561cebc900d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.804+0000 7fdeed404740 1 --2- 172.21.15.112:0/144609958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc900d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0057c0 tx=0x7fdecc000bc0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.804+0000 7fdeed404740 1 -- 172.21.15.112:0/144609958 shutdown_connections 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.804+0000 7fdeed404740 1 --2- 172.21.15.112:0/144609958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc900d0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.804+0000 7fdeed404740 1 --2- 172.21.15.112:0/144609958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc8fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.804+0000 7fdeed404740 1 --2- 172.21.15.112:0/144609958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8e8070 0x561cebc8f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.804+0000 7fdeed404740 1 -- 172.21.15.112:0/144609958 >> 172.21.15.112:0/144609958 conn(0x561ceb8dc460 msgr2=0x561cebc8d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.804+0000 7fdeed404740 1 -- 172.21.15.112:0/144609958 shutdown_connections 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.804+0000 7fdeed404740 1 -- 172.21.15.112:0/144609958 wait complete. 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.804+0000 7fdeed404740 1 -- start start 2024-09-21T03:43:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.805+0000 7fdeed404740 1 Processor -- start 2024-09-21T03:43:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.805+0000 7fdeed404740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8bf1b0 0x561cebc9acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.805+0000 7fdeed404740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8c7390 0x561cebc9b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.805+0000 7fdedffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8bf1b0 0x561cebc9acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.805+0000 7fdedffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8bf1b0 0x561cebc9acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42514/0 (socket says 172.21.15.112:42514) 2024-09-21T03:43:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.805+0000 7fdedffff640 1 -- 172.21.15.112:0/802231807 learned_addr learned my addr 172.21.15.112:0/802231807 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdeed404740 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8e8070 0x561cebca70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdeed404740 1 -- 172.21.15.112:0/802231807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561ceba56700 con 0x561ceb8bf1b0 2024-09-21T03:43:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdee4c2c640 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8c7390 0x561cebc9b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdeed404740 1 -- 172.21.15.112:0/802231807 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561cebb90110 con 0x561ceb8e8070 2024-09-21T03:43:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdeed404740 1 -- 172.21.15.112:0/802231807 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561cebc9ddd0 con 0x561ceb8c7390 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdedffff640 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8e8070 0x561cebca70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdedffff640 1 -- 172.21.15.112:0/802231807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8c7390 msgr2=0x561cebc9b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdedffff640 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8c7390 0x561cebc9b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdedffff640 1 -- 172.21.15.112:0/802231807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8e8070 msgr2=0x561cebca70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdedffff640 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8e8070 0x561cebca70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdedffff640 1 -- 172.21.15.112:0/802231807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cebbbdc40 con 0x561ceb8bf1b0 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdee4c2c640 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8c7390 0x561cebc9b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdedffff640 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8bf1b0 0x561cebc9acf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fdecc02a450 tx=0x7fdecc005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2765dd5b83b87aa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.806+0000 7fdebffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.807+0000 7fdeed404740 10 client.0 _ll_register_callbacks cb 0x561cebba9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.807+0000 7fdebffff640 1 -- 172.21.15.112:0/802231807 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc058060 con 0x561ceb8bf1b0 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.807+0000 7fdebffff640 1 -- 172.21.15.112:0/802231807 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fdecc0051e0 con 0x561ceb8bf1b0 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.807+0000 7fdeed404740 1 -- 172.21.15.112:0/802231807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdecc005030 con 0x561ceb8bf1b0 2024-09-21T03:43:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.807+0000 7fdebffff640 1 -- 172.21.15.112:0/802231807 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdecc035050 con 0x561ceb8bf1b0 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.807+0000 7fdeed404740 10 client.17922 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.807+0000 7fdeed404740 1 -- 172.21.15.112:0/802231807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({fsmap=35}) -- 0x561cebca9400 con 0x561ceb8bf1b0 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.807+0000 7fdebffff640 1 -- 172.21.15.112:0/802231807 <== mon.0 v2:172.21.15.112:3300/0 4 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fdecc04a040 con 0x561ceb8bf1b0 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.807+0000 7fdeed404740 10 client.17922 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.807+0000 7fdeed404740 10 client.17922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.808+0000 7fdeed404740 20 client.17922 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.808+0000 7fdeed404740 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561ceb8a69f0 0x561cebcac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.808+0000 7fdeed404740 4 client.17922 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.808+0000 7fdeed404740 1 -- 172.21.15.112:0/802231807 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdecc005030 con 0x561ceb8a69f0 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.808+0000 7fdee4c2c640 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561ceb8a69f0 0x561cebcac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.808+0000 7fdee4c2c640 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561ceb8a69f0 0x561cebcac7c0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb87b63887aa7bb server_cookie=1acea99a008731aa in_seq=0 out_seq=0 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.808+0000 7fdebffff640 10 client.17922 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdebffff640 1 -- 172.21.15.112:0/802231807 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fdecc005030 con 0x561ceb8a69f0 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdebffff640 10 client.17922 handle_command_reply: tid=0 2024-09-21T03:43:42.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 1 client.17922 shutdown 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 1 -- 172.21.15.112:0/802231807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8bf1b0 msgr2=0x561cebc9acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8bf1b0 0x561cebc9acf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fdecc02a450 tx=0x7fdecc005ea0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 1 -- 172.21.15.112:0/802231807 shutdown_connections 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561ceb8a69f0 0x561cebcac7c0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8e8070 0x561cebca70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdebffff640 0 client.17922 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8c7390 0x561cebc9b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 1 --2- 172.21.15.112:0/802231807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8bf1b0 0x561cebc9acf0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 1 -- 172.21.15.112:0/802231807 >> 172.21.15.112:0/802231807 conn(0x561ceb8dc460 msgr2=0x561cebc927e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 1 -- 172.21.15.112:0/802231807 shutdown_connections 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 1 -- 172.21.15.112:0/802231807 wait complete. 2024-09-21T03:43:42.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.812+0000 7fdeed404740 20 client.17922 trim_cache size 0 max 16384 2024-09-21T03:43:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdeed404740 1 Processor -- start 2024-09-21T03:43:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdeed404740 1 -- start start 2024-09-21T03:43:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdeed404740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8a69f0 0x561cebc9cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdeed404740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdeed404740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc8cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdeed404740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01afd80 con 0x561ceb8c7390 2024-09-21T03:43:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdeed404740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01ad5c0 con 0x561ceb8a69f0 2024-09-21T03:43:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdeed404740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01ab430 con 0x561ceb8bf1b0 2024-09-21T03:43:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdedffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc8cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdedffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc8cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42518/0 (socket says 172.21.15.112:42518) 2024-09-21T03:43:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdee4c2c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdedffff640 1 -- 172.21.15.112:0/912388940 learned_addr learned my addr 172.21.15.112:0/912388940 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.832+0000 7fdedffff640 1 --2- 172.21.15.112:0/912388940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8a69f0 0x561cebc9cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.833+0000 7fdedffff640 1 -- 172.21.15.112:0/912388940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 msgr2=0x561cebc954d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.833+0000 7fdedffff640 1 --2- 172.21.15.112:0/912388940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc954d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.833+0000 7fdedffff640 1 -- 172.21.15.112:0/912388940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8a69f0 msgr2=0x561cebc9cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.833+0000 7fdedffff640 1 --2- 172.21.15.112:0/912388940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8a69f0 0x561cebc9cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.833+0000 7fdedffff640 1 -- 172.21.15.112:0/912388940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cebc9bfb0 con 0x561ceb8c7390 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.833+0000 7fdedffff640 1 --2- 172.21.15.112:0/912388940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc8cb70 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fdecc005a90 tx=0x7fdecc05e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce119cb0945bb8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.833+0000 7fdeca7fc640 1 -- 172.21.15.112:0/912388940 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc058040 con 0x561ceb8c7390 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.833+0000 7fdeca7fc640 1 -- 172.21.15.112:0/912388940 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdecc005500 con 0x561ceb8c7390 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.833+0000 7fdeca7fc640 1 -- 172.21.15.112:0/912388940 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0044e0 con 0x561ceb8c7390 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.834+0000 7fdeed404740 1 -- 172.21.15.112:0/912388940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 msgr2=0x561cebc8cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.834+0000 7fdeed404740 1 --2- 172.21.15.112:0/912388940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc8cb70 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fdecc005a90 tx=0x7fdecc05e040 comp rx=0 tx=0).stop 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.834+0000 7fdeed404740 1 -- 172.21.15.112:0/912388940 shutdown_connections 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.834+0000 7fdeed404740 1 --2- 172.21.15.112:0/912388940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8c7390 0x561cebc8cb70 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.834+0000 7fdeed404740 1 --2- 172.21.15.112:0/912388940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc954d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.834+0000 7fdeed404740 1 --2- 172.21.15.112:0/912388940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8a69f0 0x561cebc9cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.834+0000 7fdeed404740 1 -- 172.21.15.112:0/912388940 >> 172.21.15.112:0/912388940 conn(0x561ceb8dc460 msgr2=0x561cebc99f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.834+0000 7fdeed404740 1 -- 172.21.15.112:0/912388940 shutdown_connections 2024-09-21T03:43:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.834+0000 7fdeed404740 1 -- 172.21.15.112:0/912388940 wait complete. 2024-09-21T03:43:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdeed404740 1 -- start start 2024-09-21T03:43:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdeed404740 1 Processor -- start 2024-09-21T03:43:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdeed404740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8a69f0 0x561cebc9f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdeed404740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc9f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdeed404740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8c7390 0x561cebc9fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdeed404740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01aba80 con 0x561ceb8a69f0 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdeed404740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01acef0 con 0x561ceb8c7390 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdeed404740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01ac150 con 0x561ceb8bf1b0 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdeed404740 10 client.0 _ll_register_callbacks cb 0x7fdecc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdedffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8a69f0 0x561cebc9f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdee4c2c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc9f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdedffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8a69f0 0x561cebc9f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42532/0 (socket says 172.21.15.112:42532) 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdedffff640 1 -- 172.21.15.112:0/2185513139 learned_addr learned my addr 172.21.15.112:0/2185513139 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.835+0000 7fdee4c2c640 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc9f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.112:47112/0 (socket says 172.21.15.112:47112) 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.836+0000 7fdedffff640 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8c7390 0x561cebc9fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.836+0000 7fdee4c2c640 1 -- 172.21.15.112:0/2185513139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8c7390 msgr2=0x561cebc9fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.836+0000 7fdee4c2c640 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8c7390 0x561cebc9fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.836+0000 7fdee4c2c640 1 -- 172.21.15.112:0/2185513139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8a69f0 msgr2=0x561cebc9f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.836+0000 7fdee4c2c640 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8a69f0 0x561cebc9f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.836+0000 7fdee4c2c640 1 -- 172.21.15.112:0/2185513139 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cebd162a0 con 0x561ceb8bf1b0 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.836+0000 7fdee4c2c640 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc9f6b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fded4000c00 tx=0x7fded4010f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e36afc5b7c28879 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.836+0000 7fdebffff640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.836+0000 7fdebffff640 1 -- 172.21.15.112:0/2185513139 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4020040 con 0x561ceb8bf1b0 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.837+0000 7fdeed404740 1 -- 172.21.15.112:0/2185513139 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdecc005500 con 0x561ceb8bf1b0 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.837+0000 7fdebffff640 1 -- 172.21.15.112:0/2185513139 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8470+0+0 (secure 0 0 0) 0x7fded400fc30 con 0x561ceb8bf1b0 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.837+0000 7fdebffff640 1 -- 172.21.15.112:0/2185513139 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fded402c050 con 0x561ceb8bf1b0 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.837+0000 7fdeed404740 10 client.27475 fetch_fsmap learned FSMap version 35 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.837+0000 7fdeed404740 1 -- 172.21.15.112:0/2185513139 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561cebd175c0 con 0x561ceb8bf1b0 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.837+0000 7fdebffff640 1 -- 172.21.15.112:0/2185513139 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdecc005500 con 0x561ceb8bf1b0 2024-09-21T03:43:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.837+0000 7fdebffff640 1 -- 172.21.15.112:0/2185513139 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fded40039d0 con 0x561ceb8bf1b0 2024-09-21T03:43:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.838+0000 7fdeed404740 10 client.27475 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:43:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.838+0000 7fdeed404740 10 client.27475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:43:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.838+0000 7fdeed404740 20 client.27475 populate_metadata read hostname 'smithi112' 2024-09-21T03:43:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.838+0000 7fdeed404740 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561ceb8e8070 0x561cebd19d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:43:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.838+0000 7fdeed404740 4 client.27475 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:43:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.838+0000 7fdeed404740 1 -- 172.21.15.112:0/2185513139 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561cebc07da0 con 0x561ceb8e8070 2024-09-21T03:43:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.838+0000 7fdedffff640 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561ceb8e8070 0x561cebd19d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:43:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.838+0000 7fdedffff640 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561ceb8e8070 0x561cebd19d80 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d55169dc52c25e2 server_cookie=14d488d347741ea8 in_seq=0 out_seq=0 2024-09-21T03:43:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.838+0000 7fdebffff640 10 client.27475 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdebffff640 1 -- 172.21.15.112:0/2185513139 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6818 (crc 0 0 0) 0x561cebc07da0 con 0x561ceb8e8070 2024-09-21T03:43:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdebffff640 10 client.27475 handle_command_reply: tid=0 2024-09-21T03:43:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 1 client.27475 shutdown 2024-09-21T03:43:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 1 -- 172.21.15.112:0/2185513139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 msgr2=0x561cebc9f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc9f6b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fded4000c00 tx=0x7fded4010f00 comp rx=0 tx=0).stop 2024-09-21T03:43:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 1 -- 172.21.15.112:0/2185513139 shutdown_connections 2024-09-21T03:43:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x561ceb8e8070 0x561cebd19d80 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ceb8c7390 0x561cebc9fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561ceb8bf1b0 0x561cebc9f6b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdebffff640 0 client.27475 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:43:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 1 --2- 172.21.15.112:0/2185513139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ceb8a69f0 0x561cebc9f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 1 -- 172.21.15.112:0/2185513139 >> 172.21.15.112:0/2185513139 conn(0x561ceb8dc460 msgr2=0x561cebc944c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 1 -- 172.21.15.112:0/2185513139 shutdown_connections 2024-09-21T03:43:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 1 -- 172.21.15.112:0/2185513139 wait complete. 2024-09-21T03:43:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.841+0000 7fdeed404740 20 client.27475 trim_cache size 0 max 16384 2024-09-21T03:43:42.845 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:43:42.845 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:43:42.845 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-09-21T03:43:42.845 INFO:teuthology.orchestra.run.smithi112.stdout: "5e9f01a6-49ed-4393-b672-9a31f31c59fb": { 2024-09-21T03:43:42.845 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "5e9f01a6-49ed-4393-b672-9a31f31c59fb", 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: "5e9f01a6-49ed-4393-b672-9a31f31c59fb": { 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:43:41.163251+0000", 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.846 INFO:teuthology.orchestra.run.smithi112.stdout: "33db2ca2-efca-4efa-81a7-253c9075949e": { 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:42:56.630350+0000", 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "4591c632-505e-4006-8d95-6080e5d063e2": { 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:42:11.919443+0000", 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.847 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c": { 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:45.957537+0000", 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "3d0e4442-3ee3-44bd-99e2-f0a81392c8c4": { 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:41:28.881491+0000", 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.848 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.849 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:43:42.850 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:43:42.851 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:43:42.852 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:43:42.853 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:43:42.854 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:43:42.855 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:43:42.856 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:43:42.857 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:43:42.858 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:43:42.858 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:43:42.858 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:43:42.858 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:43:42.858 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:43:42.858 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:43:42.858 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:43:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 -- 172.21.15.112:0/691134449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fded00b6680 msgr2=0x7fded00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fded00b6680 0x7fded00b8b90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fded80049d0 tx=0x7fded800efd0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 -- 172.21.15.112:0/691134449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee0109e00 msgr2=0x7fdee019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:43:42.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee0109e00 0x7fdee019fcf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fded800eb10 tx=0x7fded800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 -- 172.21.15.112:0/691134449 shutdown_connections 2024-09-21T03:43:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fded00b6680 0x7fded00b8b90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdee0109e00 0x7fdee019fcf0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee01094b0 0x7fdee019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 --2- 172.21.15.112:0/691134449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdee0103b70 0x7fdee019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:43:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 -- 172.21.15.112:0/691134449 >> 172.21.15.112:0/691134449 conn(0x7fdee00ff5e0 msgr2=0x7fdee0101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:43:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 -- 172.21.15.112:0/691134449 shutdown_connections 2024-09-21T03:43:42.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:43:42.842+0000 7fdee6e90640 1 -- 172.21.15.112:0/691134449 wait complete. 2024-09-21T03:43:42.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:42 smithi203 ceph-mon[34723]: pgmap v776: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 745 op/s 2024-09-21T03:43:42.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:42 smithi203 ceph-mon[34723]: scrub summary: active paths [/] 2024-09-21T03:43:42.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:42 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:42.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:42 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:42.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:42 smithi203 ceph-mon[34723]: scrub summary: idle+waiting paths [/] 2024-09-21T03:43:43.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 70643 38.68 MB/sec warmup 103 sec latency 45.600 ms 2024-09-21T03:43:43.617 INFO:tasks.cephfs.filesystem:scrub status for tag:5e9f01a6-49ed-4393-b672-9a31f31c59fb - {'path': '/', 'tag': '5e9f01a6-49ed-4393-b672-9a31f31c59fb', 'options': 'recursive,force'} 2024-09-21T03:43:44.545 INFO:tasks.workunit.client.0.smithi112.stdout: 10 71505 38.65 MB/sec warmup 104 sec latency 76.351 ms 2024-09-21T03:43:44.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:44 smithi112 ceph-mon[31442]: pgmap v777: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-09-21T03:43:44.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:44 smithi203 ceph-mon[34723]: pgmap v777: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-09-21T03:43:44.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:44 smithi187 ceph-mon[34603]: pgmap v777: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-09-21T03:43:45.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 72164 38.71 MB/sec warmup 105 sec latency 295.433 ms 2024-09-21T03:43:46.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 73077 38.81 MB/sec warmup 106 sec latency 56.822 ms 2024-09-21T03:43:46.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:46 smithi112 ceph-mon[31442]: pgmap v778: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 582 op/s 2024-09-21T03:43:46.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:46 smithi203 ceph-mon[34723]: pgmap v778: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 582 op/s 2024-09-21T03:43:46.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:46 smithi187 ceph-mon[34603]: pgmap v778: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 582 op/s 2024-09-21T03:43:47.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 73808 38.84 MB/sec warmup 107 sec latency 70.134 ms 2024-09-21T03:43:48.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 74460 38.76 MB/sec warmup 108 sec latency 341.076 ms 2024-09-21T03:43:48.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:48 smithi112 ceph-mon[31442]: pgmap v779: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 816 op/s 2024-09-21T03:43:48.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:48 smithi203 ceph-mon[34723]: pgmap v779: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 816 op/s 2024-09-21T03:43:48.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:48 smithi187 ceph-mon[34603]: pgmap v779: 97 pgs: 97 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 816 op/s 2024-09-21T03:43:49.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 75076 38.68 MB/sec warmup 109 sec latency 305.720 ms 2024-09-21T03:43:50.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 75592 38.61 MB/sec warmup 110 sec latency 351.455 ms 2024-09-21T03:43:50.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:50 smithi112 ceph-mon[31442]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 642 op/s 2024-09-21T03:43:50.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:50 smithi203 ceph-mon[34723]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 642 op/s 2024-09-21T03:43:50.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:50 smithi187 ceph-mon[34603]: pgmap v780: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 642 op/s 2024-09-21T03:43:51.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 76436 38.64 MB/sec warmup 111 sec latency 119.431 ms 2024-09-21T03:43:52.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 77407 38.81 MB/sec warmup 112 sec latency 36.345 ms 2024-09-21T03:43:52.713 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:52 smithi203 ceph-mon[34723]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 726 op/s 2024-09-21T03:43:52.731 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:52 smithi187 ceph-mon[34603]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 726 op/s 2024-09-21T03:43:52.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:52 smithi112 ceph-mon[31442]: pgmap v781: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 726 op/s 2024-09-21T03:43:53.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 78138 38.87 MB/sec warmup 113 sec latency 63.655 ms 2024-09-21T03:43:54.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 78785 38.77 MB/sec warmup 114 sec latency 70.943 ms 2024-09-21T03:43:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:54 smithi112 ceph-mon[31442]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 649 op/s 2024-09-21T03:43:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:54 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:43:54.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:54 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:43:54.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:54 smithi203 ceph-mon[34723]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 649 op/s 2024-09-21T03:43:54.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:54 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:43:54.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:54 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:43:54.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:54 smithi187 ceph-mon[34603]: pgmap v782: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 649 op/s 2024-09-21T03:43:54.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:54 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:43:54.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:54 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:43:55.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 79215 38.64 MB/sec warmup 115 sec latency 340.227 ms 2024-09-21T03:43:55.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:55 smithi112 ceph-mon[31442]: scrub summary: idle 2024-09-21T03:43:55.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:55 smithi203 ceph-mon[34723]: scrub summary: idle 2024-09-21T03:43:55.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:55 smithi187 ceph-mon[34603]: scrub summary: idle 2024-09-21T03:43:56.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 79929 38.59 MB/sec warmup 116 sec latency 162.003 ms 2024-09-21T03:43:56.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:56 smithi112 ceph-mon[31442]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 666 op/s 2024-09-21T03:43:56.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:56 smithi203 ceph-mon[34723]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 666 op/s 2024-09-21T03:43:56.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:56 smithi187 ceph-mon[34603]: pgmap v783: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 666 op/s 2024-09-21T03:43:57.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 80891 38.71 MB/sec warmup 117 sec latency 57.074 ms 2024-09-21T03:43:58.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 81729 38.80 MB/sec warmup 118 sec latency 50.532 ms 2024-09-21T03:43:58.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:43:58 smithi112 ceph-mon[31442]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 817 op/s 2024-09-21T03:43:58.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:43:58 smithi203 ceph-mon[34723]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 817 op/s 2024-09-21T03:43:58.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:43:58 smithi187 ceph-mon[34603]: pgmap v784: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 817 op/s 2024-09-21T03:43:59.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 82346 38.72 MB/sec warmup 119 sec latency 74.813 ms 2024-09-21T03:44:00.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:00 smithi112 ceph-mon[31442]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 619 op/s 2024-09-21T03:44:00.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:00 smithi203 ceph-mon[34723]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 619 op/s 2024-09-21T03:44:00.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:00 smithi187 ceph-mon[34603]: pgmap v785: 97 pgs: 97 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 619 op/s 2024-09-21T03:44:01.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 83304 14.69 MB/sec execute 1 sec latency 324.028 ms 2024-09-21T03:44:02.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 83702 19.00 MB/sec execute 2 sec latency 108.480 ms 2024-09-21T03:44:02.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:02 smithi112 ceph-mon[31442]: pgmap v786: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 752 op/s 2024-09-21T03:44:02.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:02 smithi203 ceph-mon[34723]: pgmap v786: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 752 op/s 2024-09-21T03:44:02.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:02 smithi187 ceph-mon[34603]: pgmap v786: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 752 op/s 2024-09-21T03:44:03.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 84374 26.11 MB/sec execute 3 sec latency 67.290 ms 2024-09-21T03:44:04.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 85169 28.88 MB/sec execute 4 sec latency 73.057 ms 2024-09-21T03:44:04.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:04 smithi112 ceph-mon[31442]: pgmap v787: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-09-21T03:44:04.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:04 smithi203 ceph-mon[34723]: pgmap v787: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-09-21T03:44:04.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:04 smithi187 ceph-mon[34603]: pgmap v787: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-09-21T03:44:05.546 INFO:tasks.workunit.client.0.smithi112.stdout: 10 85765 29.90 MB/sec execute 5 sec latency 265.585 ms 2024-09-21T03:44:05.593 INFO:tasks.workunit.client.0.smithi112.stdout:[107975] write failed on handle 11615 (Resource temporarily unavailable) 2024-09-21T03:44:05.593 INFO:tasks.workunit.client.0.smithi112.stdout:Child failed with status 1 2024-09-21T03:44:05.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:44:05.602 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-21T03:44:05.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T03:44:05.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:05 smithi112 ceph-mon[31442]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T03:44:05.882 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi112 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-21T03:44:12.845 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=2d4b4ffd0efb4be58a5a7ea219dab69f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi112 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-21T03:44:12.847 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T03:44:12.856 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T03:44:12.857 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:44:12.857 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T03:44:12.857 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T03:44:12.862 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:05 smithi187 ceph-mon[34603]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T03:44:12.862 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:06 smithi187 ceph-mon[34603]: pgmap v789: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 758 op/s 2024-09-21T03:44:12.862 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:08 smithi187 ceph-mon[34603]: pgmap v790: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 930 op/s 2024-09-21T03:44:12.862 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:10 smithi187 ceph-mon[34603]: pgmap v791: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 910 op/s 2024-09-21T03:44:12.862 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:05 smithi203 ceph-mon[34723]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T03:44:12.862 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:06 smithi203 ceph-mon[34723]: pgmap v789: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 758 op/s 2024-09-21T03:44:12.862 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:08 smithi203 ceph-mon[34723]: pgmap v790: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 930 op/s 2024-09-21T03:44:12.863 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:10 smithi203 ceph-mon[34723]: pgmap v791: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 910 op/s 2024-09-21T03:44:12.863 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:06 smithi112 ceph-mon[31442]: pgmap v789: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 758 op/s 2024-09-21T03:44:12.863 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:08 smithi112 ceph-mon[31442]: pgmap v790: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 930 op/s 2024-09-21T03:44:12.863 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:10 smithi112 ceph-mon[31442]: pgmap v791: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 910 op/s 2024-09-21T03:44:12.863 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:12 smithi112 ceph-mon[31442]: pgmap v792: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 777 op/s 2024-09-21T03:44:12.898 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T03:44:12.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:12 smithi203 ceph-mon[34723]: pgmap v792: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 777 op/s 2024-09-21T03:44:12.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:12 smithi187 ceph-mon[34603]: pgmap v792: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 777 op/s 2024-09-21T03:44:12.968 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T03:44:13.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.123+0000 7efddfbc3640 1 -- 172.21.15.112:0/4229913183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdd8102a10 msgr2=0x7efdd8102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.123+0000 7efddfbc3640 1 --2- 172.21.15.112:0/4229913183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdd8102a10 0x7efdd8102e20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009900 tx=0x7efdd402a410 comp rx=0 tx=0).stop 2024-09-21T03:44:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.123+0000 7efddfbc3640 1 -- 172.21.15.112:0/4229913183 shutdown_connections 2024-09-21T03:44:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.123+0000 7efddfbc3640 1 --2- 172.21.15.112:0/4229913183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdd8108c90 0x7efdd810f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.123+0000 7efddfbc3640 1 --2- 172.21.15.112:0/4229913183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdd8108340 0x7efdd8108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.123+0000 7efddfbc3640 1 --2- 172.21.15.112:0/4229913183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdd8102a10 0x7efdd8102e20 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.123+0000 7efddfbc3640 1 -- 172.21.15.112:0/4229913183 >> 172.21.15.112:0/4229913183 conn(0x7efdd80fe410 msgr2=0x7efdd8100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.123+0000 7efddfbc3640 1 -- 172.21.15.112:0/4229913183 shutdown_connections 2024-09-21T03:44:13.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.123+0000 7efddfbc3640 1 -- 172.21.15.112:0/4229913183 wait complete. 2024-09-21T03:44:13.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddfbc3640 1 Processor -- start 2024-09-21T03:44:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddfbc3640 1 -- start start 2024-09-21T03:44:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddfbc3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdd8102a10 0x7efdd819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddfbc3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdd8108340 0x7efdd819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddfbc3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdd8108c90 0x7efdd819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddfbc3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efdd8113a70 con 0x7efdd8102a10 2024-09-21T03:44:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddfbc3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efdd81138f0 con 0x7efdd8108c90 2024-09-21T03:44:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddfbc3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efdd8113bf0 con 0x7efdd8108340 2024-09-21T03:44:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddd95f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdd8102a10 0x7efdd819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:13.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddd95f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdd8102a10 0x7efdd819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42898/0 (socket says 172.21.15.112:42898) 2024-09-21T03:44:13.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddd95f640 1 -- 172.21.15.112:0/351443672 learned_addr learned my addr 172.21.15.112:0/351443672 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:13.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.124+0000 7efddd15e640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdd8108340 0x7efdd819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:13.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efdde160640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdd8108c90 0x7efdd819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:13.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efddd95f640 1 -- 172.21.15.112:0/351443672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdd8108340 msgr2=0x7efdd819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efddd95f640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdd8108340 0x7efdd819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efddd95f640 1 -- 172.21.15.112:0/351443672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdd8108c90 msgr2=0x7efdd819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efddd95f640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdd8108c90 0x7efdd819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efddd95f640 1 -- 172.21.15.112:0/351443672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd81a0420 con 0x7efdd8102a10 2024-09-21T03:44:13.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efddd15e640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdd8108340 0x7efdd819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efddd95f640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdd8102a10 0x7efdd819a4f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efdd4005940 tx=0x7efdd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efdceffd640 1 -- 172.21.15.112:0/351443672 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd403f030 con 0x7efdd8102a10 2024-09-21T03:44:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efdceffd640 1 -- 172.21.15.112:0/351443672 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdd4003cb0 con 0x7efdd8102a10 2024-09-21T03:44:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efddfbc3640 1 -- 172.21.15.112:0/351443672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd81a06b0 con 0x7efdd8102a10 2024-09-21T03:44:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efdceffd640 1 -- 172.21.15.112:0/351443672 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4035070 con 0x7efdd8102a10 2024-09-21T03:44:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.125+0000 7efddfbc3640 1 -- 172.21.15.112:0/351443672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd81a6ea0 con 0x7efdd8102a10 2024-09-21T03:44:13.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.126+0000 7efddfbc3640 1 -- 172.21.15.112:0/351443672 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd8102e20 con 0x7efdd8102a10 2024-09-21T03:44:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.128+0000 7efdceffd640 1 -- 172.21.15.112:0/351443672 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7efdd4042030 con 0x7efdd8102a10 2024-09-21T03:44:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.129+0000 7efdceffd640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efdc40b6700 0x7efdc40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.129+0000 7efddd15e640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efdc40b6700 0x7efdc40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:13.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.129+0000 7efdceffd640 1 -- 172.21.15.112:0/351443672 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8514+0+0 (secure 0 0 0) 0x7efdd41082a0 con 0x7efdd8102a10 2024-09-21T03:44:13.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.129+0000 7efddd15e640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efdc40b6700 0x7efdc40b8c10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efdd8101c80 tx=0x7efdc800b5f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:13.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.132+0000 7efdceffd640 1 -- 172.21.15.112:0/351443672 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efdd40ce7f0 con 0x7efdd8102a10 2024-09-21T03:44:13.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.331+0000 7efddfbc3640 1 -- 172.21.15.112:0/351443672 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7efdd8063800 con 0x7efdc40b6700 2024-09-21T03:44:13.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.345+0000 7efdceffd640 1 -- 172.21.15.112:0/351443672 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7efdd8063800 con 0x7efdc40b6700 2024-09-21T03:44:13.347 INFO:teuthology.orchestra.run.smithi112.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T03:28:54", "first": "2024-09-21T03:29:00", "last": "2024-09-21T03:44:00", "last_pruned": "2024-09-21T03:44:00", "created_count": 16, "pruned_count": 10, "active": true} 2024-09-21T03:44:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 -- 172.21.15.112:0/351443672 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efdc40b6700 msgr2=0x7efdc40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efdc40b6700 0x7efdc40b8c10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efdd8101c80 tx=0x7efdc800b5f0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 -- 172.21.15.112:0/351443672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdd8102a10 msgr2=0x7efdd819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdd8102a10 0x7efdd819a4f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efdd4005940 tx=0x7efdd40042f0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 -- 172.21.15.112:0/351443672 shutdown_connections 2024-09-21T03:44:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efdc40b6700 0x7efdc40b8c10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdd8108c90 0x7efdd819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdd8108340 0x7efdd819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 --2- 172.21.15.112:0/351443672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdd8102a10 0x7efdd819a4f0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 -- 172.21.15.112:0/351443672 >> 172.21.15.112:0/351443672 conn(0x7efdd80fe410 msgr2=0x7efdd8107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 -- 172.21.15.112:0/351443672 shutdown_connections 2024-09-21T03:44:13.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.347+0000 7efddfbc3640 1 -- 172.21.15.112:0/351443672 wait complete. 2024-09-21T03:44:13.365 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T03:44:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.496+0000 7f73af798640 1 -- 172.21.15.112:0/2403562373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a8108c60 msgr2=0x7f73a810f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.496+0000 7f73af798640 1 --2- 172.21.15.112:0/2403562373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a8108c60 0x7f73a810f490 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009ca0 tx=0x7f73a002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.496+0000 7f73af798640 1 -- 172.21.15.112:0/2403562373 shutdown_connections 2024-09-21T03:44:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.496+0000 7f73af798640 1 --2- 172.21.15.112:0/2403562373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a8108c60 0x7f73a810f490 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.496+0000 7f73af798640 1 --2- 172.21.15.112:0/2403562373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a8108310 0x7f73a8108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.496+0000 7f73af798640 1 --2- 172.21.15.112:0/2403562373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73a81029e0 0x7f73a8102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.496+0000 7f73af798640 1 -- 172.21.15.112:0/2403562373 >> 172.21.15.112:0/2403562373 conn(0x7f73a80fe3e0 msgr2=0x7f73a8100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:13.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.497+0000 7f73af798640 1 -- 172.21.15.112:0/2403562373 shutdown_connections 2024-09-21T03:44:13.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.497+0000 7f73af798640 1 -- 172.21.15.112:0/2403562373 wait complete. 2024-09-21T03:44:13.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.497+0000 7f73af798640 1 Processor -- start 2024-09-21T03:44:13.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.497+0000 7f73af798640 1 -- start start 2024-09-21T03:44:13.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.497+0000 7f73af798640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a81029e0 0x7f73a81990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:13.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.497+0000 7f73af798640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a8108310 0x7f73a8199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:13.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.497+0000 7f73af798640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73a8108c60 0x7f73a8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:13.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.497+0000 7f73af798640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8112920 con 0x7f73a81029e0 2024-09-21T03:44:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.497+0000 7f73af798640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81127a0 con 0x7f73a8108310 2024-09-21T03:44:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.497+0000 7f73af798640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8112aa0 con 0x7f73a8108c60 2024-09-21T03:44:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73ad534640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a81029e0 0x7f73a81990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73ad534640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a81029e0 0x7f73a81990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42916/0 (socket says 172.21.15.112:42916) 2024-09-21T03:44:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73ad534640 1 -- 172.21.15.112:0/4149891466 learned_addr learned my addr 172.21.15.112:0/4149891466 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73acd33640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a8108310 0x7f73a8199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73add35640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73a8108c60 0x7f73a8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73ad534640 1 -- 172.21.15.112:0/4149891466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73a8108c60 msgr2=0x7f73a8199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73ad534640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73a8108c60 0x7f73a8199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73ad534640 1 -- 172.21.15.112:0/4149891466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a8108310 msgr2=0x7f73a8199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73ad534640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a8108310 0x7f73a8199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73ad534640 1 -- 172.21.15.112:0/4149891466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a81a8770 con 0x7f73a81029e0 2024-09-21T03:44:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73add35640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73a8108c60 0x7f73a8199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73ad534640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a81029e0 0x7f73a81990c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f73a400e9f0 tx=0x7f73a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f739e7fc640 1 -- 172.21.15.112:0/4149891466 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a400cd20 con 0x7f73a81029e0 2024-09-21T03:44:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f739e7fc640 1 -- 172.21.15.112:0/4149891466 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73a40054d0 con 0x7f73a81029e0 2024-09-21T03:44:13.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73af798640 1 -- 172.21.15.112:0/4149891466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a81a8a60 con 0x7f73a81029e0 2024-09-21T03:44:13.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f739e7fc640 1 -- 172.21.15.112:0/4149891466 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4010690 con 0x7f73a81029e0 2024-09-21T03:44:13.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.498+0000 7f73af798640 1 -- 172.21.15.112:0/4149891466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a81a8f60 con 0x7f73a81029e0 2024-09-21T03:44:13.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.499+0000 7f73af798640 1 -- 172.21.15.112:0/4149891466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a8102df0 con 0x7f73a81029e0 2024-09-21T03:44:13.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.501+0000 7f739e7fc640 1 -- 172.21.15.112:0/4149891466 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f73a4010940 con 0x7f73a81029e0 2024-09-21T03:44:13.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.501+0000 7f739e7fc640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73940b6680 0x7f73940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:13.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.502+0000 7f73acd33640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73940b6680 0x7f73940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:13.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.502+0000 7f739e7fc640 1 -- 172.21.15.112:0/4149891466 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8514+0+0 (secure 0 0 0) 0x7f73a40e7d60 con 0x7f73a81029e0 2024-09-21T03:44:13.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.502+0000 7f73acd33640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73940b6680 0x7f73940b8b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f73a8101ae0 tx=0x7f7398005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:13.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.505+0000 7f739e7fc640 1 -- 172.21.15.112:0/4149891466 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f73a4056710 con 0x7f73a81029e0 2024-09-21T03:44:13.619 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:44:13.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.712+0000 7f73af798640 1 -- 172.21.15.112:0/4149891466 --> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f73a8063800 con 0x7f73940b6680 2024-09-21T03:44:13.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.724+0000 7f739e7fc640 1 -- 172.21.15.112:0/4149891466 <== mgr.14226 v2:172.21.15.112:6800/4024464582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f73a8063800 con 0x7f73940b6680 2024-09-21T03:44:13.726 INFO:teuthology.orchestra.run.smithi112.stdout:/ 1m 6m3h 2024-09-21T03:44:13.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 -- 172.21.15.112:0/4149891466 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73940b6680 msgr2=0x7f73940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73940b6680 0x7f73940b8b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f73a8101ae0 tx=0x7f7398005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:13.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 -- 172.21.15.112:0/4149891466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a81029e0 msgr2=0x7f73a81990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:13.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a81029e0 0x7f73a81990c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f73a400e9f0 tx=0x7f73a400edf0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 -- 172.21.15.112:0/4149891466 shutdown_connections 2024-09-21T03:44:13.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f73940b6680 0x7f73940b8b90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73a8108c60 0x7f73a8199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a8108310 0x7f73a8199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 --2- 172.21.15.112:0/4149891466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73a81029e0 0x7f73a81990c0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:13.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 -- 172.21.15.112:0/4149891466 >> 172.21.15.112:0/4149891466 conn(0x7f73a80fe3e0 msgr2=0x7f73a81058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:13.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 -- 172.21.15.112:0/4149891466 shutdown_connections 2024-09-21T03:44:13.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:13.726+0000 7f73af798640 1 -- 172.21.15.112:0/4149891466 wait complete. 2024-09-21T03:44:13.740 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T03:44:13.751 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T03:44:13.875 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:14.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.393+0000 7f60ae2a3640 1 -- 172.21.15.112:0/3356419730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a81083a0 msgr2=0x7f60a81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.393+0000 7f60ae2a3640 1 --2- 172.21.15.112:0/3356419730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a81083a0 0x7f60a81087b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f609c009900 tx=0x7f609c02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:14.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.393+0000 7f60ae2a3640 1 -- 172.21.15.112:0/3356419730 shutdown_connections 2024-09-21T03:44:14.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.393+0000 7f60ae2a3640 1 --2- 172.21.15.112:0/3356419730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60a8108cf0 0x7f60a810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.393+0000 7f60ae2a3640 1 --2- 172.21.15.112:0/3356419730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a81083a0 0x7f60a81087b0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.393+0000 7f60ae2a3640 1 --2- 172.21.15.112:0/3356419730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60a8102a60 0x7f60a8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.393+0000 7f60ae2a3640 1 -- 172.21.15.112:0/3356419730 >> 172.21.15.112:0/3356419730 conn(0x7f60a8079730 msgr2=0x7f60a8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:14.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.393+0000 7f60ae2a3640 1 -- 172.21.15.112:0/3356419730 shutdown_connections 2024-09-21T03:44:14.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.394+0000 7f60ae2a3640 1 -- 172.21.15.112:0/3356419730 wait complete. 2024-09-21T03:44:14.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.394+0000 7f60ae2a3640 1 Processor -- start 2024-09-21T03:44:14.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.394+0000 7f60ae2a3640 1 -- start start 2024-09-21T03:44:14.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ae2a3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60a8102a60 0x7f60a819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ae2a3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60a81083a0 0x7f60a819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ae2a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8108cf0 0x7f60a819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ae2a3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8113a80 con 0x7f60a8108cf0 2024-09-21T03:44:14.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ae2a3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8113900 con 0x7f60a8102a60 2024-09-21T03:44:14.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ae2a3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8113c00 con 0x7f60a81083a0 2024-09-21T03:44:14.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ac840640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8108cf0 0x7f60a819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ac840640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8108cf0 0x7f60a819fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42942/0 (socket says 172.21.15.112:42942) 2024-09-21T03:44:14.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ac840640 1 -- 172.21.15.112:0/2570018007 learned_addr learned my addr 172.21.15.112:0/2570018007 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:14.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60a77fe640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60a81083a0 0x7f60a819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60a7fff640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60a8102a60 0x7f60a819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ac840640 1 -- 172.21.15.112:0/2570018007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60a81083a0 msgr2=0x7f60a819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ac840640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60a81083a0 0x7f60a819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ac840640 1 -- 172.21.15.112:0/2570018007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60a8102a60 msgr2=0x7f60a819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ac840640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60a8102a60 0x7f60a819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.395+0000 7f60ac840640 1 -- 172.21.15.112:0/2570018007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a81a0420 con 0x7f60a8108cf0 2024-09-21T03:44:14.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.396+0000 7f60ac840640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8108cf0 0x7f60a819fcf0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f60a000eaa0 tx=0x7f60a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:14.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.396+0000 7f60a57fa640 1 -- 172.21.15.112:0/2570018007 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a000cdb0 con 0x7f60a8108cf0 2024-09-21T03:44:14.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.396+0000 7f60ae2a3640 1 -- 172.21.15.112:0/2570018007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a81a0710 con 0x7f60a8108cf0 2024-09-21T03:44:14.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.396+0000 7f60a57fa640 1 -- 172.21.15.112:0/2570018007 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a0005020 con 0x7f60a8108cf0 2024-09-21T03:44:14.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.396+0000 7f60a57fa640 1 -- 172.21.15.112:0/2570018007 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00116d0 con 0x7f60a8108cf0 2024-09-21T03:44:14.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.396+0000 7f60ae2a3640 1 -- 172.21.15.112:0/2570018007 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a81a6e70 con 0x7f60a8108cf0 2024-09-21T03:44:14.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.398+0000 7f60b4817740 1 Processor -- start 2024-09-21T03:44:14.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60a57fa640 1 -- 172.21.15.112:0/2570018007 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f60a00054a0 con 0x7f60a8108cf0 2024-09-21T03:44:14.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60b4817740 1 -- start start 2024-09-21T03:44:14.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60b4817740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae24070 0x55e6db1cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60b4817740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 0x55e6db1cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60b4817740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae03390 0x55e6db1cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60b4817740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e6db141e50 con 0x55e6dae03390 2024-09-21T03:44:14.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60b4817740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e6db131280 con 0x55e6dadfb1b0 2024-09-21T03:44:14.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60b4817740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e6db135090 con 0x55e6dae24070 2024-09-21T03:44:14.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae03390 0x55e6db1cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 0x55e6db1cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae03390 0x55e6db1cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42950/0 (socket says 172.21.15.112:42950) 2024-09-21T03:44:14.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60a77fe640 1 -- 172.21.15.112:0/3614768958 learned_addr learned my addr 172.21.15.112:0/3614768958 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:14.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.399+0000 7f60a77fe640 1 --2- 172.21.15.112:0/3614768958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae24070 0x55e6db1cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f60a7fff640 1 -- 172.21.15.112:0/3614768958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae24070 msgr2=0x55e6db1cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:14.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f60a7fff640 1 --2- 172.21.15.112:0/3614768958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae24070 0x55e6db1cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f60a7fff640 1 -- 172.21.15.112:0/3614768958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae03390 msgr2=0x55e6db1cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:14.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f60a7fff640 1 --2- 172.21.15.112:0/3614768958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae03390 0x55e6db1cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f60a7fff640 1 -- 172.21.15.112:0/3614768958 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dae8f650 con 0x55e6dadfb1b0 2024-09-21T03:44:14.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f60a57fa640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f60900b6680 0x7f60900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f60a77fe640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f60900b6680 0x7f60900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f60a7fff640 1 --2- 172.21.15.112:0/3614768958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 0x55e6db1cb7b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f609800bbf0 tx=0x7f60980107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62b4e531199400a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:14.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f60a57fa640 1 -- 172.21.15.112:0/2570018007 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8514+0+0 (secure 0 0 0) 0x7f60a00e77c0 con 0x7f60a8108cf0 2024-09-21T03:44:14.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f608e7fc640 1 -- 172.21.15.112:0/3614768958 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098004030 con 0x55e6dadfb1b0 2024-09-21T03:44:14.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f608e7fc640 1 -- 172.21.15.112:0/3614768958 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60980035f0 con 0x55e6dadfb1b0 2024-09-21T03:44:14.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.400+0000 7f608e7fc640 1 -- 172.21.15.112:0/3614768958 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60980116a0 con 0x55e6dadfb1b0 2024-09-21T03:44:14.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.401+0000 7f60b4817740 1 -- 172.21.15.112:0/3614768958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 msgr2=0x55e6db1cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.401+0000 7f60a77fe640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f60900b6680 0x7f60900b8b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f60a806aaa0 tx=0x7f609c005780 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:14.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.401+0000 7f60b4817740 1 --2- 172.21.15.112:0/3614768958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 0x55e6db1cb7b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f609800bbf0 tx=0x7f60980107a0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.401+0000 7f60b4817740 1 -- 172.21.15.112:0/3614768958 shutdown_connections 2024-09-21T03:44:14.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.401+0000 7f60b4817740 1 --2- 172.21.15.112:0/3614768958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae03390 0x55e6db1cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.401+0000 7f60b4817740 1 --2- 172.21.15.112:0/3614768958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 0x55e6db1cb7b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.401+0000 7f60b4817740 1 --2- 172.21.15.112:0/3614768958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae24070 0x55e6db1cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.401+0000 7f60b4817740 1 -- 172.21.15.112:0/3614768958 >> 172.21.15.112:0/3614768958 conn(0x55e6dae18460 msgr2=0x55e6db1c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:14.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.401+0000 7f60b4817740 1 -- 172.21.15.112:0/3614768958 shutdown_connections 2024-09-21T03:44:14.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.401+0000 7f60b4817740 1 -- 172.21.15.112:0/3614768958 wait complete. 2024-09-21T03:44:14.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.401+0000 7f60b4817740 1 -- start start 2024-09-21T03:44:14.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60b4817740 1 Processor -- start 2024-09-21T03:44:14.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60b4817740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dadfb1b0 0x55e6db1d6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60b4817740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae03390 0x55e6db1d6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60b4817740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dae24070 0x55e6db1e2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60b4817740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e6daf92700 con 0x55e6dadfb1b0 2024-09-21T03:44:14.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60b4817740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e6db0cc110 con 0x55e6dae24070 2024-09-21T03:44:14.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60b4817740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e6db1d9a50 con 0x55e6dae03390 2024-09-21T03:44:14.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dadfb1b0 0x55e6db1d6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60b4817740 10 client.0 _ll_register_callbacks cb 0x55e6db0e5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:14.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dadfb1b0 0x55e6db1d6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42966/0 (socket says 172.21.15.112:42966) 2024-09-21T03:44:14.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60a7fff640 1 -- 172.21.15.112:0/1948959296 learned_addr learned my addr 172.21.15.112:0/1948959296 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:14.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60ac840640 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dae24070 0x55e6db1e2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60a77fe640 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae03390 0x55e6db1d6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60a7fff640 1 -- 172.21.15.112:0/1948959296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae03390 msgr2=0x55e6db1d6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:14.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60a7fff640 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae03390 0x55e6db1d6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60a7fff640 1 -- 172.21.15.112:0/1948959296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dae24070 msgr2=0x55e6db1e2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:14.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.402+0000 7f60a7fff640 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dae24070 0x55e6db1e2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.403+0000 7f60a7fff640 1 -- 172.21.15.112:0/1948959296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6db0f9c40 con 0x55e6dadfb1b0 2024-09-21T03:44:14.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.403+0000 7f60a7fff640 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dadfb1b0 0x55e6db1d6900 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6098011010 tx=0x7f609800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff52c688a98956a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:14.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.403+0000 7f60877fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:44:14.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.403+0000 7f60877fe640 1 -- 172.21.15.112:0/1948959296 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60980039b0 con 0x55e6dadfb1b0 2024-09-21T03:44:14.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.403+0000 7f60877fe640 1 -- 172.21.15.112:0/1948959296 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8514+0+0 (secure 0 0 0) 0x7f6098004000 con 0x55e6dadfb1b0 2024-09-21T03:44:14.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.403+0000 7f60b4817740 1 -- 172.21.15.112:0/1948959296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60980035f0 con 0x55e6dadfb1b0 2024-09-21T03:44:14.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.403+0000 7f60877fe640 1 -- 172.21.15.112:0/1948959296 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6098004200 con 0x55e6dadfb1b0 2024-09-21T03:44:14.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.403+0000 7f60b4817740 10 client.17952 fetch_fsmap learned FSMap version 35 2024-09-21T03:44:14.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.403+0000 7f60b4817740 1 -- 172.21.15.112:0/1948959296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e6db1e5760 con 0x55e6dadfb1b0 2024-09-21T03:44:14.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.403+0000 7f60877fe640 1 -- 172.21.15.112:0/1948959296 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60980035f0 con 0x55e6dadfb1b0 2024-09-21T03:44:14.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.403+0000 7f60877fe640 1 -- 172.21.15.112:0/1948959296 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f6098004930 con 0x55e6dadfb1b0 2024-09-21T03:44:14.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.404+0000 7f60b4817740 10 client.17952 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:44:14.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.404+0000 7f60b4817740 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:44:14.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.404+0000 7f60b4817740 20 client.17952 populate_metadata read hostname 'smithi112' 2024-09-21T03:44:14.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.404+0000 7f60b4817740 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e6dade29f0 0x55e6db1e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.404+0000 7f60b4817740 4 client.17952 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:14.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.404+0000 7f60b4817740 1 -- 172.21.15.112:0/1948959296 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6db143da0 con 0x55e6dade29f0 2024-09-21T03:44:14.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.404+0000 7f60a77fe640 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e6dade29f0 0x55e6db1e8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.404+0000 7f60877fe640 10 client.17952 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:44:14.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.404+0000 7f60a77fe640 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e6dade29f0 0x55e6db1e8b70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c915bd4641e3a server_cookie=d35fb64913f57568 in_seq=0 out_seq=0 2024-09-21T03:44:14.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60877fe640 1 -- 172.21.15.112:0/1948959296 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e6db143da0 con 0x55e6dade29f0 2024-09-21T03:44:14.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60877fe640 10 client.17952 handle_command_reply: tid=0 2024-09-21T03:44:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 1 client.17952 shutdown 2024-09-21T03:44:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 1 -- 172.21.15.112:0/1948959296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dadfb1b0 msgr2=0x55e6db1d6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dadfb1b0 0x55e6db1d6900 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6098011010 tx=0x7f609800bfd0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 1 -- 172.21.15.112:0/1948959296 shutdown_connections 2024-09-21T03:44:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e6dade29f0 0x55e6db1e8b70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dae24070 0x55e6db1e2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae03390 0x55e6db1d6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60877fe640 0 client.17952 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:44:14.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 1 --2- 172.21.15.112:0/1948959296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dadfb1b0 0x55e6db1d6900 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 1 -- 172.21.15.112:0/1948959296 >> 172.21.15.112:0/1948959296 conn(0x55e6dae18460 msgr2=0x55e6db1ce400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:14.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 1 -- 172.21.15.112:0/1948959296 shutdown_connections 2024-09-21T03:44:14.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 1 -- 172.21.15.112:0/1948959296 wait complete. 2024-09-21T03:44:14.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.405+0000 7f60b4817740 20 client.17952 trim_cache size 0 max 16384 2024-09-21T03:44:14.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.422+0000 7f60b4817740 1 Processor -- start 2024-09-21T03:44:14.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.422+0000 7f60b4817740 1 -- start start 2024-09-21T03:44:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60b4817740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dadfb1b0 0x55e6db1cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60b4817740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dae03390 0x55e6db1c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60b4817740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60b4817740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81afea0 con 0x55e6dae24070 2024-09-21T03:44:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60b4817740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81ad6e0 con 0x55e6dae03390 2024-09-21T03:44:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60b4817740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81ab550 con 0x55e6dadfb1b0 2024-09-21T03:44:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60ac840640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60ac840640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42982/0 (socket says 172.21.15.112:42982) 2024-09-21T03:44:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60ac840640 1 -- 172.21.15.112:0/998591284 learned_addr learned my addr 172.21.15.112:0/998591284 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60a7fff640 1 --2- 172.21.15.112:0/998591284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dadfb1b0 0x55e6db1cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60a77fe640 1 --2- 172.21.15.112:0/998591284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dae03390 0x55e6db1c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60ac840640 1 -- 172.21.15.112:0/998591284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dadfb1b0 msgr2=0x55e6db1cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60ac840640 1 --2- 172.21.15.112:0/998591284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dadfb1b0 0x55e6db1cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60ac840640 1 -- 172.21.15.112:0/998591284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dae03390 msgr2=0x55e6db1c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60ac840640 1 --2- 172.21.15.112:0/998591284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dae03390 0x55e6db1c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60ac840640 1 -- 172.21.15.112:0/998591284 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6db1cd790 con 0x55e6dae24070 2024-09-21T03:44:14.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60a77fe640 1 --2- 172.21.15.112:0/998591284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dae03390 0x55e6db1c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:14.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f60ac840640 1 --2- 172.21.15.112:0/998591284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1c8c50 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f60a00e5510 tx=0x7f60a00ad290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be16851078a4ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:14.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f608dffb640 1 -- 172.21.15.112:0/998591284 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a001f040 con 0x55e6dae24070 2024-09-21T03:44:14.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f608dffb640 1 -- 172.21.15.112:0/998591284 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a00e5190 con 0x55e6dae24070 2024-09-21T03:44:14.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.423+0000 7f608dffb640 1 -- 172.21.15.112:0/998591284 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00a9370 con 0x55e6dae24070 2024-09-21T03:44:14.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.424+0000 7f60b4817740 1 -- 172.21.15.112:0/998591284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 msgr2=0x55e6db1c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.424+0000 7f60b4817740 1 --2- 172.21.15.112:0/998591284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1c8c50 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f60a00e5510 tx=0x7f60a00ad290 comp rx=0 tx=0).stop 2024-09-21T03:44:14.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.424+0000 7f60b4817740 1 -- 172.21.15.112:0/998591284 shutdown_connections 2024-09-21T03:44:14.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.424+0000 7f60b4817740 1 --2- 172.21.15.112:0/998591284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1c8c50 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.424+0000 7f60b4817740 1 --2- 172.21.15.112:0/998591284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dae03390 0x55e6db1c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.424+0000 7f60b4817740 1 --2- 172.21.15.112:0/998591284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dadfb1b0 0x55e6db1cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.424+0000 7f60b4817740 1 -- 172.21.15.112:0/998591284 >> 172.21.15.112:0/998591284 conn(0x55e6dae18460 msgr2=0x55e6db1cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:14.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.424+0000 7f60b4817740 1 -- 172.21.15.112:0/998591284 shutdown_connections 2024-09-21T03:44:14.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.424+0000 7f60b4817740 1 -- 172.21.15.112:0/998591284 wait complete. 2024-09-21T03:44:14.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60b4817740 1 -- start start 2024-09-21T03:44:14.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60b4817740 1 Processor -- start 2024-09-21T03:44:14.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60b4817740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 0x55e6db1df620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60b4817740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae03390 0x55e6db1daad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60b4817740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1db170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60b4817740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81abba0 con 0x55e6dae24070 2024-09-21T03:44:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60b4817740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81ad010 con 0x55e6dadfb1b0 2024-09-21T03:44:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60b4817740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81ac270 con 0x55e6dae03390 2024-09-21T03:44:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60b4817740 10 client.0 _ll_register_callbacks cb 0x7f609c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60ac840640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1db170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60ac840640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1db170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42990/0 (socket says 172.21.15.112:42990) 2024-09-21T03:44:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60ac840640 1 -- 172.21.15.112:0/760939352 learned_addr learned my addr 172.21.15.112:0/760939352 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60a7fff640 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 0x55e6db1df620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.425+0000 7f60a77fe640 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae03390 0x55e6db1daad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60ac840640 1 -- 172.21.15.112:0/760939352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae03390 msgr2=0x55e6db1daad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60ac840640 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae03390 0x55e6db1daad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60ac840640 1 -- 172.21.15.112:0/760939352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 msgr2=0x55e6db1df620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60ac840640 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 0x55e6db1df620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60ac840640 1 -- 172.21.15.112:0/760939352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6db1db900 con 0x55e6dae24070 2024-09-21T03:44:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60a77fe640 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae03390 0x55e6db1daad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:14.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60a7fff640 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 0x55e6db1df620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60ac840640 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1db170 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f60a00b22b0 tx=0x7f60a00a9690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31745a2355c2eeca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60877fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:44:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60877fe640 1 -- 172.21.15.112:0/760939352 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a001f070 con 0x55e6dae24070 2024-09-21T03:44:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60877fe640 1 -- 172.21.15.112:0/760939352 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8514+0+0 (secure 0 0 0) 0x7f60a00acb30 con 0x55e6dae24070 2024-09-21T03:44:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60b4817740 1 -- 172.21.15.112:0/760939352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a00e5190 con 0x55e6dae24070 2024-09-21T03:44:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60877fe640 1 -- 172.21.15.112:0/760939352 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f60a00ac390 con 0x55e6dae24070 2024-09-21T03:44:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60b4817740 10 client.17958 fetch_fsmap learned FSMap version 35 2024-09-21T03:44:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60b4817740 1 -- 172.21.15.112:0/760939352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e6db253610 con 0x55e6dae24070 2024-09-21T03:44:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.426+0000 7f60877fe640 1 -- 172.21.15.112:0/760939352 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a00e5190 con 0x55e6dae24070 2024-09-21T03:44:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.427+0000 7f60877fe640 1 -- 172.21.15.112:0/760939352 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f60a000f040 con 0x55e6dae24070 2024-09-21T03:44:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.427+0000 7f60b4817740 10 client.17958 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:44:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.427+0000 7f60b4817740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:44:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.427+0000 7f60b4817740 20 client.17958 populate_metadata read hostname 'smithi112' 2024-09-21T03:44:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.427+0000 7f60b4817740 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e6dadb7da0 0x55e6db255d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.427+0000 7f60b4817740 4 client.17958 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:44:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.427+0000 7f60b4817740 1 -- 172.21.15.112:0/760939352 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e6db13f850 con 0x55e6dadb7da0 2024-09-21T03:44:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.427+0000 7f60a7fff640 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e6dadb7da0 0x55e6db255d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.427+0000 7f60877fe640 10 client.17958 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:44:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.427+0000 7f60a7fff640 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e6dadb7da0 0x55e6db255d80 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa294e7bd31c9dee server_cookie=6c17091c0853ad36 in_seq=0 out_seq=0 2024-09-21T03:44:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60877fe640 1 -- 172.21.15.112:0/760939352 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6622 (crc 0 0 0) 0x55e6db13f850 con 0x55e6dadb7da0 2024-09-21T03:44:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60877fe640 10 client.17958 handle_command_reply: tid=0 2024-09-21T03:44:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 1 client.17958 shutdown 2024-09-21T03:44:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 1 -- 172.21.15.112:0/760939352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 msgr2=0x55e6db1db170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1db170 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f60a00b22b0 tx=0x7f60a00a9690 comp rx=0 tx=0).stop 2024-09-21T03:44:14.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 1 -- 172.21.15.112:0/760939352 shutdown_connections 2024-09-21T03:44:14.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x55e6dadb7da0 0x55e6db255d80 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e6dae24070 0x55e6db1db170 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e6dae03390 0x55e6db1daad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 1 --2- 172.21.15.112:0/760939352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6dadfb1b0 0x55e6db1df620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60877fe640 0 client.17958 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:44:14.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 1 -- 172.21.15.112:0/760939352 >> 172.21.15.112:0/760939352 conn(0x55e6dae18460 msgr2=0x55e6db1ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:14.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 1 -- 172.21.15.112:0/760939352 shutdown_connections 2024-09-21T03:44:14.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 1 -- 172.21.15.112:0/760939352 wait complete. 2024-09-21T03:44:14.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.428+0000 7f60b4817740 20 client.17958 trim_cache size 0 max 16384 2024-09-21T03:44:14.440 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:44:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-09-21T03:44:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {}, 2024-09-21T03:44:14.440 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_stats": { 2024-09-21T03:44:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "5e9f01a6-49ed-4393-b672-9a31f31c59fb": { 2024-09-21T03:44:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:43:41.163251+0000", 2024-09-21T03:44:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "33db2ca2-efca-4efa-81a7-253c9075949e": { 2024-09-21T03:44:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:42:56.630350+0000", 2024-09-21T03:44:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.442 INFO:teuthology.orchestra.run.smithi112.stdout: "4591c632-505e-4006-8d95-6080e5d063e2": { 2024-09-21T03:44:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:42:11.919443+0000", 2024-09-21T03:44:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.443 INFO:teuthology.orchestra.run.smithi112.stdout: "d0d08a40-4f31-4299-9cb9-9dfae9d0f21c": { 2024-09-21T03:44:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:45.957537+0000", 2024-09-21T03:44:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "3d0e4442-3ee3-44bd-99e2-f0a81392c8c4": { 2024-09-21T03:44:14.444 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:41:28.881491+0000", 2024-09-21T03:44:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "c37fa7ed-cde7-4b2c-aba7-8fd48b1cdf5e": { 2024-09-21T03:44:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:40:02.851596+0000", 2024-09-21T03:44:14.445 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "730258e8-6c54-4776-b8f7-7ce3b100573d": { 2024-09-21T03:44:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:38:36.551738+0000", 2024-09-21T03:44:14.446 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "790a90df-67c8-4c06-8295-aa8220726acb": { 2024-09-21T03:44:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:28:54.759691+0000", 2024-09-21T03:44:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.447 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "7227cba3-c722-4732-8f72-ff30c3ee5d5a": { 2024-09-21T03:44:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:39:20.179650+0000", 2024-09-21T03:44:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "575b64ed-beba-489e-b2ec-f4aac3fe0d1c": { 2024-09-21T03:44:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:09.690923+0000", 2024-09-21T03:44:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "1e6e5cf7-2802-4874-abd6-2efed3be7f30": { 2024-09-21T03:44:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:00.377992+0000", 2024-09-21T03:44:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.450 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "82573db5-77b6-4d68-9155-414eebbfe779": { 2024-09-21T03:44:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:37.418586+0000", 2024-09-21T03:44:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.451 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "148c9e2b-f634-4315-ad67-9877174aed84": { 2024-09-21T03:44:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:00.002245+0000", 2024-09-21T03:44:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.452 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "576b14cd-2e49-4418-842f-8aaa89f9d08b": { 2024-09-21T03:44:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:30:42.273910+0000", 2024-09-21T03:44:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.453 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "cd748fd9-1e14-47f0-aa33-872010daf567": { 2024-09-21T03:44:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:51.547566+0000", 2024-09-21T03:44:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.454 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "d5e6af18-7419-4d89-a8cb-849ebeffa647": { 2024-09-21T03:44:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:31:24.998804+0000", 2024-09-21T03:44:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.455 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "0a173e6c-39aa-46f9-8fd2-28b926e70bb6": { 2024-09-21T03:44:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:37:52.859834+0000", 2024-09-21T03:44:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "b1f22425-3f0c-43b0-aebe-8cd8df2d8753": { 2024-09-21T03:44:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:33:34.348457+0000", 2024-09-21T03:44:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "70d7715c-658d-43b2-ada8-48508980cc8e": { 2024-09-21T03:44:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:32:08.060690+0000", 2024-09-21T03:44:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.458 INFO:teuthology.orchestra.run.smithi112.stdout: "11f692fb-1419-449e-b703-28887d34acb4": { 2024-09-21T03:44:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:29:48.576297+0000", 2024-09-21T03:44:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.459 INFO:teuthology.orchestra.run.smithi112.stdout: "7f611a1d-7b3b-4b63-add4-85afb713d24a": { 2024-09-21T03:44:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:34:17.086351+0000", 2024-09-21T03:44:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "825fdaa4-4c40-45c9-ab9d-f8e9a0da0d27": { 2024-09-21T03:44:14.460 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:35:43.281130+0000", 2024-09-21T03:44:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "df8c91a7-299b-410b-babe-4f4ca1487b26": { 2024-09-21T03:44:14.461 INFO:teuthology.orchestra.run.smithi112.stdout: "start_time": "2024-09-21T03:36:26.823013+0000", 2024-09-21T03:44:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-09-21T03:44:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_started": 0, 2024-09-21T03:44:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_passed": 0, 2024-09-21T03:44:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_failed": 0, 2024-09-21T03:44:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: "uninline_skipped": 0 2024-09-21T03:44:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:14.462 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:44:14.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 -- 172.21.15.112:0/2570018007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f60900b6680 msgr2=0x7f60900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f60900b6680 0x7f60900b8b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f60a806aaa0 tx=0x7f609c005780 comp rx=0 tx=0).stop 2024-09-21T03:44:14.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 -- 172.21.15.112:0/2570018007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8108cf0 msgr2=0x7f60a819fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:14.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8108cf0 0x7f60a819fcf0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f60a000eaa0 tx=0x7f60a000ef60 comp rx=0 tx=0).stop 2024-09-21T03:44:14.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 -- 172.21.15.112:0/2570018007 shutdown_connections 2024-09-21T03:44:14.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f60900b6680 0x7f60900b8b90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60a8108cf0 0x7f60a819fcf0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60a81083a0 0x7f60a819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 --2- 172.21.15.112:0/2570018007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60a8102a60 0x7f60a819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:14.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 -- 172.21.15.112:0/2570018007 >> 172.21.15.112:0/2570018007 conn(0x7f60a8079730 msgr2=0x7f60a81044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:14.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 -- 172.21.15.112:0/2570018007 shutdown_connections 2024-09-21T03:44:14.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:14.429+0000 7f60ae2a3640 1 -- 172.21.15.112:0/2570018007 wait complete. 2024-09-21T03:44:14.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:14 smithi112 ceph-mon[31442]: pgmap v793: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 685 op/s 2024-09-21T03:44:14.788 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:14 smithi112 ceph-mon[31442]: from='client.17934 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:14.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:14 smithi203 ceph-mon[34723]: pgmap v793: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 685 op/s 2024-09-21T03:44:14.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:14 smithi203 ceph-mon[34723]: from='client.17934 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:14 smithi187 ceph-mon[34603]: pgmap v793: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 685 op/s 2024-09-21T03:44:14.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:14 smithi187 ceph-mon[34603]: from='client.17934 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:15.341 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:44:15.341 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:44:15.683 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:15.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:15 smithi112 ceph-mon[31442]: from='client.17940 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:15.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:15 smithi112 ceph-mon[31442]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T03:44:15.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:15 smithi203 ceph-mon[34723]: from='client.17940 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:15.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:15 smithi203 ceph-mon[34723]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T03:44:15.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:15 smithi187 ceph-mon[34603]: from='client.17940 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:44:15.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:15 smithi187 ceph-mon[34603]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T03:44:16.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.142+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4245889477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1ec104860 msgr2=0x7ff1ec104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.142+0000 7ff1f33a1640 1 --2- 172.21.15.112:0/4245889477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1ec104860 0x7ff1ec104c70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009900 tx=0x7ff1e802a3f0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.142+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4245889477 shutdown_connections 2024-09-21T03:44:16.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.142+0000 7ff1f33a1640 1 --2- 172.21.15.112:0/4245889477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec10de10 0x7ff1ec110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.142+0000 7ff1f33a1640 1 --2- 172.21.15.112:0/4245889477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1ec105290 0x7ff1ec10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.142+0000 7ff1f33a1640 1 --2- 172.21.15.112:0/4245889477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1ec104860 0x7ff1ec104c70 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.142+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4245889477 >> 172.21.15.112:0/4245889477 conn(0x7ff1ec079740 msgr2=0x7ff1ec079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:16.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.142+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4245889477 shutdown_connections 2024-09-21T03:44:16.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.142+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4245889477 wait complete. 2024-09-21T03:44:16.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f33a1640 1 Processor -- start 2024-09-21T03:44:16.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f33a1640 1 -- start start 2024-09-21T03:44:16.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f33a1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec104860 0x7ff1ec19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f33a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1ec105290 0x7ff1ec19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f33a1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1ec10de10 0x7ff1ec19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f33a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec113a90 con 0x7ff1ec105290 2024-09-21T03:44:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f33a1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec113910 con 0x7ff1ec104860 2024-09-21T03:44:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f33a1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec113c10 con 0x7ff1ec10de10 2024-09-21T03:44:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f093c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1ec105290 0x7ff1ec19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f113d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec104860 0x7ff1ec19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f113d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec104860 0x7ff1ec19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:55542/0 (socket says 172.21.15.112:55542) 2024-09-21T03:44:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f113d640 1 -- 172.21.15.112:0/4163984302 learned_addr learned my addr 172.21.15.112:0/4163984302 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:16.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f093c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1ec105290 0x7ff1ec19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45516/0 (socket says 172.21.15.112:45516) 2024-09-21T03:44:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f193e640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1ec10de10 0x7ff1ec19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f093c640 1 -- 172.21.15.112:0/4163984302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1ec10de10 msgr2=0x7ff1ec19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f093c640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1ec10de10 0x7ff1ec19fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f093c640 1 -- 172.21.15.112:0/4163984302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec104860 msgr2=0x7ff1ec19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f093c640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec104860 0x7ff1ec19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f093c640 1 -- 172.21.15.112:0/4163984302 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec1a0470 con 0x7ff1ec105290 2024-09-21T03:44:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.143+0000 7ff1f113d640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec104860 0x7ff1ec19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:16.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.144+0000 7ff1f093c640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1ec105290 0x7ff1ec19ab60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec06b950 tx=0x7ff1dc00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.144+0000 7ff1e27fc640 1 -- 172.21.15.112:0/4163984302 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc007bb0 con 0x7ff1ec105290 2024-09-21T03:44:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.144+0000 7ff1e27fc640 1 -- 172.21.15.112:0/4163984302 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc007d50 con 0x7ff1ec105290 2024-09-21T03:44:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.144+0000 7ff1e27fc640 1 -- 172.21.15.112:0/4163984302 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0186f0 con 0x7ff1ec105290 2024-09-21T03:44:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.144+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4163984302 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec1a0760 con 0x7ff1ec105290 2024-09-21T03:44:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.144+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4163984302 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec1a6f80 con 0x7ff1ec105290 2024-09-21T03:44:16.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.145+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4163984302 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ec06c1e0 con 0x7ff1ec105290 2024-09-21T03:44:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.146+0000 7ff1e27fc640 1 -- 172.21.15.112:0/4163984302 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff1dc00f450 con 0x7ff1ec105290 2024-09-21T03:44:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.147+0000 7ff1e27fc640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1d80b66b0 0x7ff1d80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.147+0000 7ff1e27fc640 1 -- 172.21.15.112:0/4163984302 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7ff1dc0e7970 con 0x7ff1ec105290 2024-09-21T03:44:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.147+0000 7ff1f113d640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1d80b66b0 0x7ff1d80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:16.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.148+0000 7ff1f113d640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1d80b66b0 0x7ff1d80b8bc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80096c0 tx=0x7ff1e8032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:16.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.150+0000 7ff1e27fc640 1 -- 172.21.15.112:0/4163984302 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff1dc014030 con 0x7ff1ec105290 2024-09-21T03:44:16.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.406+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4163984302 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1ec104c70 con 0x7ff1ec105290 2024-09-21T03:44:16.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.407+0000 7ff1e27fc640 1 -- 172.21.15.112:0/4163984302 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11936 (secure 0 0 0) 0x7ff1dc0adf80 con 0x7ff1ec105290 2024-09-21T03:44:16.409 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:44:16.409 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":35,"btime":"2024-09-21T03:42:33:280889+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:42:32.317716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24685,24674,24667]},"id":1}]} 2024-09-21T03:44:16.410 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 35 2024-09-21T03:44:16.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4163984302 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1d80b66b0 msgr2=0x7ff1d80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1d80b66b0 0x7ff1d80b8bc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80096c0 tx=0x7ff1e8032ed0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4163984302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1ec105290 msgr2=0x7ff1ec19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:16.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1ec105290 0x7ff1ec19ab60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec06b950 tx=0x7ff1dc00cdd0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4163984302 shutdown_connections 2024-09-21T03:44:16.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1d80b66b0 0x7ff1d80b8bc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1ec10de10 0x7ff1ec19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1ec105290 0x7ff1ec19ab60 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 --2- 172.21.15.112:0/4163984302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1ec104860 0x7ff1ec19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:16.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4163984302 >> 172.21.15.112:0/4163984302 conn(0x7ff1ec079740 msgr2=0x7ff1ec0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:16.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4163984302 shutdown_connections 2024-09-21T03:44:16.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:16.409+0000 7ff1f33a1640 1 -- 172.21.15.112:0/4163984302 wait complete. 2024-09-21T03:44:16.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:16 smithi112 ceph-mon[31442]: pgmap v794: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.2 MiB/s wr, 589 op/s 2024-09-21T03:44:16.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:16 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/4163984302' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:16.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:16 smithi203 ceph-mon[34723]: pgmap v794: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.2 MiB/s wr, 589 op/s 2024-09-21T03:44:16.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:16 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/4163984302' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:16.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:16 smithi187 ceph-mon[34603]: pgmap v794: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.2 MiB/s wr, 589 op/s 2024-09-21T03:44:16.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:16 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/4163984302' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:17.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:44:17.529 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.996+0000 7f25e6ca3640 1 -- 172.21.15.112:0/921151081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e01084c0 msgr2=0x7f25e010ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.996+0000 7f25e6ca3640 1 --2- 172.21.15.112:0/921151081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e01084c0 0x7f25e010ecf0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009ca0 tx=0x7f25d802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.996+0000 7f25e6ca3640 1 -- 172.21.15.112:0/921151081 shutdown_connections 2024-09-21T03:44:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.996+0000 7f25e6ca3640 1 --2- 172.21.15.112:0/921151081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e01084c0 0x7f25e010ecf0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.996+0000 7f25e6ca3640 1 --2- 172.21.15.112:0/921151081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e0107b70 0x7f25e0107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.996+0000 7f25e6ca3640 1 --2- 172.21.15.112:0/921151081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25e0102230 0x7f25e0102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.996+0000 7f25e6ca3640 1 -- 172.21.15.112:0/921151081 >> 172.21.15.112:0/921151081 conn(0x7f25e0079e40 msgr2=0x7f25e007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.996+0000 7f25e6ca3640 1 -- 172.21.15.112:0/921151081 shutdown_connections 2024-09-21T03:44:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.996+0000 7f25e6ca3640 1 -- 172.21.15.112:0/921151081 wait complete. 2024-09-21T03:44:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.997+0000 7f25e6ca3640 1 Processor -- start 2024-09-21T03:44:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.997+0000 7f25e6ca3640 1 -- start start 2024-09-21T03:44:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.997+0000 7f25e6ca3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25e0102230 0x7f25e01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.997+0000 7f25e6ca3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e0107b70 0x7f25e0199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.997+0000 7f25e6ca3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e01084c0 0x7f25e019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.997+0000 7f25e6ca3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25e0112a00 con 0x7f25e0107b70 2024-09-21T03:44:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.997+0000 7f25e6ca3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25e0112880 con 0x7f25e01084c0 2024-09-21T03:44:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.997+0000 7f25e6ca3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f25e0112b80 con 0x7f25e0102230 2024-09-21T03:44:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.997+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e0107b70 0x7f25e0199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e0107b70 0x7f25e0199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45530/0 (socket says 172.21.15.112:45530) 2024-09-21T03:44:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25d7fff640 1 -- 172.21.15.112:0/129001266 learned_addr learned my addr 172.21.15.112:0/129001266 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25e4a3f640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25e0102230 0x7f25e01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25e5240640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e01084c0 0x7f25e019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25d7fff640 1 -- 172.21.15.112:0/129001266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25e0102230 msgr2=0x7f25e01993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25d7fff640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25e0102230 0x7f25e01993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25d7fff640 1 -- 172.21.15.112:0/129001266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e01084c0 msgr2=0x7f25e019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25d7fff640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e01084c0 0x7f25e019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25d7fff640 1 -- 172.21.15.112:0/129001266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e019f340 con 0x7f25e0107b70 2024-09-21T03:44:18.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25d7fff640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e0107b70 0x7f25e0199970 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f25d000d740 tx=0x7f25d000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25d5ffb640 1 -- 172.21.15.112:0/129001266 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0017070 con 0x7f25e0107b70 2024-09-21T03:44:18.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25d5ffb640 1 -- 172.21.15.112:0/129001266 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d000de50 con 0x7f25e0107b70 2024-09-21T03:44:18.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25e6ca3640 1 -- 172.21.15.112:0/129001266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e019f630 con 0x7f25e0107b70 2024-09-21T03:44:18.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.998+0000 7f25d5ffb640 1 -- 172.21.15.112:0/129001266 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d000f620 con 0x7f25e0107b70 2024-09-21T03:44:18.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:17.999+0000 7f25e6ca3640 1 -- 172.21.15.112:0/129001266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e01a5d90 con 0x7f25e0107b70 2024-09-21T03:44:18.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25ed217740 1 Processor -- start 2024-09-21T03:44:18.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25ed217740 1 -- start start 2024-09-21T03:44:18.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25d5ffb640 1 -- 172.21.15.112:0/129001266 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f25d001a070 con 0x7f25e0107b70 2024-09-21T03:44:18.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25ed217740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f860766a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25ed217740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f86076be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25ed217740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f86077120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25ed217740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f85fecea0 con 0x559f85cae390 2024-09-21T03:44:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25ed217740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559f85fdc2d0 con 0x559f85ca61b0 2024-09-21T03:44:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25ed217740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559f85fe00e0 con 0x559f85ccf070 2024-09-21T03:44:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f86077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f86077120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45532/0 (socket says 172.21.15.112:45532) 2024-09-21T03:44:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.001+0000 7f25d7fff640 1 -- 172.21.15.112:0/2773893381 learned_addr learned my addr 172.21.15.112:0/2773893381 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25e4a3f640 1 --2- 172.21.15.112:0/2773893381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f86076be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25e5240640 1 --2- 172.21.15.112:0/2773893381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f860766a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25d7fff640 1 -- 172.21.15.112:0/2773893381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 msgr2=0x559f860766a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25d7fff640 1 --2- 172.21.15.112:0/2773893381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f860766a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25d7fff640 1 -- 172.21.15.112:0/2773893381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 msgr2=0x559f86076be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25d7fff640 1 --2- 172.21.15.112:0/2773893381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f86076be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25d7fff640 1 -- 172.21.15.112:0/2773893381 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f85d3a650 con 0x559f85cae390 2024-09-21T03:44:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25e5240640 1 --2- 172.21.15.112:0/2773893381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f860766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25d5ffb640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c80b6680 0x7f25c80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25e5240640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c80b6680 0x7f25c80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25d7fff640 1 --2- 172.21.15.112:0/2773893381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f86077120 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f25d00e4770 tx=0x7f25d00ef780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7b966c77b3b4de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25d5ffb640 1 -- 172.21.15.112:0/129001266 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f25d00e7080 con 0x7f25e0107b70 2024-09-21T03:44:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25c6ffd640 1 -- 172.21.15.112:0/2773893381 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00a9550 con 0x559f85cae390 2024-09-21T03:44:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25c6ffd640 1 -- 172.21.15.112:0/2773893381 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d00aca10 con 0x559f85cae390 2024-09-21T03:44:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.002+0000 7f25c6ffd640 1 -- 172.21.15.112:0/2773893381 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00ac670 con 0x559f85cae390 2024-09-21T03:44:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.003+0000 7f25ed217740 1 -- 172.21.15.112:0/2773893381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 msgr2=0x559f86077120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.003+0000 7f25ed217740 1 --2- 172.21.15.112:0/2773893381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f86077120 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f25d00e4770 tx=0x7f25d00ef780 comp rx=0 tx=0).stop 2024-09-21T03:44:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.003+0000 7f25e5240640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c80b6680 0x7f25c80b8b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f25d8005cc0 tx=0x7f25d80099c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.003+0000 7f25ed217740 1 -- 172.21.15.112:0/2773893381 shutdown_connections 2024-09-21T03:44:18.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.003+0000 7f25ed217740 1 --2- 172.21.15.112:0/2773893381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f86077120 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.003+0000 7f25ed217740 1 --2- 172.21.15.112:0/2773893381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f86076be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.003+0000 7f25ed217740 1 --2- 172.21.15.112:0/2773893381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f860766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.003+0000 7f25ed217740 1 -- 172.21.15.112:0/2773893381 >> 172.21.15.112:0/2773893381 conn(0x559f85cc3460 msgr2=0x559f86074590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.003+0000 7f25ed217740 1 -- 172.21.15.112:0/2773893381 shutdown_connections 2024-09-21T03:44:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.003+0000 7f25ed217740 1 -- 172.21.15.112:0/2773893381 wait complete. 2024-09-21T03:44:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25ed217740 1 -- start start 2024-09-21T03:44:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25ed217740 1 Processor -- start 2024-09-21T03:44:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25ed217740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ca61b0 0x559f86080380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25ed217740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85cae390 0x559f86080960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25ed217740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f8608c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25ed217740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f85e3d700 con 0x559f85ca61b0 2024-09-21T03:44:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25ed217740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559f85f77160 con 0x559f85cae390 2024-09-21T03:44:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25ed217740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559f86084dd0 con 0x559f85ccf070 2024-09-21T03:44:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25ed217740 10 client.0 _ll_register_callbacks cb 0x559f85f909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25e4a3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ca61b0 0x559f86080380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25e4a3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ca61b0 0x559f86080380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45538/0 (socket says 172.21.15.112:45538) 2024-09-21T03:44:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25e4a3f640 1 -- 172.21.15.112:0/1327916051 learned_addr learned my addr 172.21.15.112:0/1327916051 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.004+0000 7f25d7fff640 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85cae390 0x559f86080960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25e5240640 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f8608c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25e4a3f640 1 -- 172.21.15.112:0/1327916051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 msgr2=0x559f8608c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25e4a3f640 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f8608c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25e4a3f640 1 -- 172.21.15.112:0/1327916051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85cae390 msgr2=0x559f86080960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25e4a3f640 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85cae390 0x559f86080960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25e4a3f640 1 -- 172.21.15.112:0/1327916051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f85fa4c90 con 0x559f85ca61b0 2024-09-21T03:44:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25e4a3f640 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ca61b0 0x559f86080380 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f25dc010820 tx=0x7f25dc010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a875f4190e4952 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25bffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:44:18.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25bffff640 1 -- 172.21.15.112:0/1327916051 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc01e060 con 0x559f85ca61b0 2024-09-21T03:44:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25bffff640 1 -- 172.21.15.112:0/1327916051 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f25dc00fb90 con 0x559f85ca61b0 2024-09-21T03:44:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25ed217740 1 -- 172.21.15.112:0/1327916051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d00aca10 con 0x559f85ca61b0 2024-09-21T03:44:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25bffff640 1 -- 172.21.15.112:0/1327916051 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f25dc00fdd0 con 0x559f85ca61b0 2024-09-21T03:44:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25ed217740 10 client.17976 fetch_fsmap learned FSMap version 35 2024-09-21T03:44:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25ed217740 1 -- 172.21.15.112:0/1327916051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559f86094fa0 con 0x559f85ca61b0 2024-09-21T03:44:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.005+0000 7f25bffff640 1 -- 172.21.15.112:0/1327916051 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d00aca10 con 0x559f85ca61b0 2024-09-21T03:44:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.006+0000 7f25bffff640 1 -- 172.21.15.112:0/1327916051 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f25dc0214a0 con 0x559f85ca61b0 2024-09-21T03:44:18.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.006+0000 7f25ed217740 10 client.17976 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:44:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.006+0000 7f25ed217740 10 client.17976 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:44:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.006+0000 7f25ed217740 20 client.17976 populate_metadata read hostname 'smithi112' 2024-09-21T03:44:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.006+0000 7f25ed217740 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x559f85c8d9f0 0x559f860984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.006+0000 7f25ed217740 4 client.17976 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.006+0000 7f25ed217740 1 -- 172.21.15.112:0/1327916051 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f85feedf0 con 0x559f85c8d9f0 2024-09-21T03:44:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.006+0000 7f25d7fff640 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x559f85c8d9f0 0x559f860984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.006+0000 7f25d7fff640 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x559f85c8d9f0 0x559f860984d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e19d5860bbb8a6f4 server_cookie=8c73283d06e3343e in_seq=0 out_seq=0 2024-09-21T03:44:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.006+0000 7f25bffff640 10 client.17976 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:44:18.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25bffff640 1 -- 172.21.15.112:0/1327916051 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f85feedf0 con 0x559f85c8d9f0 2024-09-21T03:44:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25bffff640 10 client.17976 handle_command_reply: tid=0 2024-09-21T03:44:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 1 client.17976 shutdown 2024-09-21T03:44:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 1 -- 172.21.15.112:0/1327916051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ca61b0 msgr2=0x559f86080380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ca61b0 0x559f86080380 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f25dc010820 tx=0x7f25dc010ce0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 1 -- 172.21.15.112:0/1327916051 shutdown_connections 2024-09-21T03:44:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x559f85c8d9f0 0x559f860984d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f8608c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85cae390 0x559f86080960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25bffff640 0 client.17976 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:44:18.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 1 --2- 172.21.15.112:0/1327916051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ca61b0 0x559f86080380 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 1 -- 172.21.15.112:0/1327916051 >> 172.21.15.112:0/1327916051 conn(0x559f85cc3460 msgr2=0x559f86079830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 1 -- 172.21.15.112:0/1327916051 shutdown_connections 2024-09-21T03:44:18.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 1 -- 172.21.15.112:0/1327916051 wait complete. 2024-09-21T03:44:18.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.007+0000 7f25ed217740 20 client.17976 trim_cache size 0 max 16384 2024-09-21T03:44:18.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.020+0000 7f25ed217740 1 Processor -- start 2024-09-21T03:44:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.020+0000 7f25ed217740 1 -- start start 2024-09-21T03:44:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25ed217740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f86077de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25ed217740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85cae390 0x559f86092f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25ed217740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ccf070 0x559f86073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25ed217740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01afe50 con 0x559f85ccf070 2024-09-21T03:44:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25ed217740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01ad690 con 0x559f85ca61b0 2024-09-21T03:44:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25ed217740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01ab500 con 0x559f85cae390 2024-09-21T03:44:18.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25e5240640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ccf070 0x559f86073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25e5240640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ccf070 0x559f86073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45546/0 (socket says 172.21.15.112:45546) 2024-09-21T03:44:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25e5240640 1 -- 172.21.15.112:0/16660090 learned_addr learned my addr 172.21.15.112:0/16660090 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25d7fff640 1 --2- 172.21.15.112:0/16660090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85cae390 0x559f86092f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25e4a3f640 1 --2- 172.21.15.112:0/16660090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f86077de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25e5240640 1 -- 172.21.15.112:0/16660090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85cae390 msgr2=0x559f86092f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25e5240640 1 --2- 172.21.15.112:0/16660090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85cae390 0x559f86092f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25e5240640 1 -- 172.21.15.112:0/16660090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 msgr2=0x559f86077de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25e5240640 1 --2- 172.21.15.112:0/16660090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f86077de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25e5240640 1 -- 172.21.15.112:0/16660090 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f8607e680 con 0x559f85ccf070 2024-09-21T03:44:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25d7fff640 1 --2- 172.21.15.112:0/16660090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85cae390 0x559f86092f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25e4a3f640 1 --2- 172.21.15.112:0/16660090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f86077de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25e5240640 1 --2- 172.21.15.112:0/16660090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ccf070 0x559f86073bc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f25d8031ff0 tx=0x7f25d803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c22a2125af4bc8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.021+0000 7f25c67fc640 1 -- 172.21.15.112:0/16660090 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80318e0 con 0x559f85ccf070 2024-09-21T03:44:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.022+0000 7f25c67fc640 1 -- 172.21.15.112:0/16660090 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d8036070 con 0x559f85ccf070 2024-09-21T03:44:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.022+0000 7f25c67fc640 1 -- 172.21.15.112:0/16660090 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d802fbc0 con 0x559f85ccf070 2024-09-21T03:44:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.022+0000 7f25ed217740 1 -- 172.21.15.112:0/16660090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ccf070 msgr2=0x559f86073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.022+0000 7f25ed217740 1 --2- 172.21.15.112:0/16660090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ccf070 0x559f86073bc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f25d8031ff0 tx=0x7f25d803f000 comp rx=0 tx=0).stop 2024-09-21T03:44:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.022+0000 7f25ed217740 1 -- 172.21.15.112:0/16660090 shutdown_connections 2024-09-21T03:44:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.022+0000 7f25ed217740 1 --2- 172.21.15.112:0/16660090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85ccf070 0x559f86073bc0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.022+0000 7f25ed217740 1 --2- 172.21.15.112:0/16660090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85cae390 0x559f86092f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.022+0000 7f25ed217740 1 --2- 172.21.15.112:0/16660090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f86077de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.022+0000 7f25ed217740 1 -- 172.21.15.112:0/16660090 >> 172.21.15.112:0/16660090 conn(0x559f85cc3460 msgr2=0x559f860776d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.022+0000 7f25ed217740 1 -- 172.21.15.112:0/16660090 shutdown_connections 2024-09-21T03:44:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.022+0000 7f25ed217740 1 -- 172.21.15.112:0/16660090 wait complete. 2024-09-21T03:44:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.023+0000 7f25ed217740 1 -- start start 2024-09-21T03:44:18.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.023+0000 7f25ed217740 1 Processor -- start 2024-09-21T03:44:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.023+0000 7f25ed217740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f8608c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.023+0000 7f25ed217740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f8608cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.023+0000 7f25ed217740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f86091fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.023+0000 7f25ed217740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01abb50 con 0x559f85cae390 2024-09-21T03:44:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.023+0000 7f25ed217740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01acfc0 con 0x559f85ca61b0 2024-09-21T03:44:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.023+0000 7f25ed217740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01ac220 con 0x559f85ccf070 2024-09-21T03:44:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.023+0000 7f25ed217740 10 client.0 _ll_register_callbacks cb 0x7f25dc00a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:18.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.023+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f8608cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f8608cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45548/0 (socket says 172.21.15.112:45548) 2024-09-21T03:44:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25d7fff640 1 -- 172.21.15.112:0/3971511298 learned_addr learned my addr 172.21.15.112:0/3971511298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25e5240640 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f86091fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25e4a3f640 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f8608c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25d7fff640 1 -- 172.21.15.112:0/3971511298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 msgr2=0x559f86091fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25d7fff640 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f86091fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25d7fff640 1 -- 172.21.15.112:0/3971511298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 msgr2=0x559f8608c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25d7fff640 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f8608c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25d7fff640 1 -- 172.21.15.112:0/3971511298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f86092510 con 0x559f85cae390 2024-09-21T03:44:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25e4a3f640 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f8608c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25d7fff640 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f8608cd80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f25d00aa170 tx=0x7f25d0010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757d381809e83fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25bffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:44:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25bffff640 1 -- 172.21.15.112:0/3971511298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0021040 con 0x559f85cae390 2024-09-21T03:44:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25bffff640 1 -- 172.21.15.112:0/3971511298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f25d00e4a80 con 0x559f85cae390 2024-09-21T03:44:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25ed217740 1 -- 172.21.15.112:0/3971511298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d8036070 con 0x559f85cae390 2024-09-21T03:44:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25bffff640 1 -- 172.21.15.112:0/3971511298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f25d00ac850 con 0x559f85cae390 2024-09-21T03:44:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25ed217740 10 client.17982 fetch_fsmap learned FSMap version 35 2024-09-21T03:44:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25ed217740 1 -- 172.21.15.112:0/3971511298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559f86103540 con 0x559f85cae390 2024-09-21T03:44:18.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.024+0000 7f25bffff640 1 -- 172.21.15.112:0/3971511298 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d8036070 con 0x559f85cae390 2024-09-21T03:44:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.025+0000 7f25bffff640 1 -- 172.21.15.112:0/3971511298 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f25d005c1b0 con 0x559f85cae390 2024-09-21T03:44:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.025+0000 7f25ed217740 10 client.17982 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:44:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.025+0000 7f25ed217740 10 client.17982 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T03:44:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.025+0000 7f25ed217740 20 client.17982 populate_metadata read hostname 'smithi112' 2024-09-21T03:44:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.025+0000 7f25ed217740 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x559f85c62da0 0x559f861059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.025+0000 7f25ed217740 4 client.17982 mds_command: new command op to 24667 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:44:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.025+0000 7f25ed217740 1 -- 172.21.15.112:0/3971511298 --> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f85fea8a0 con 0x559f85c62da0 2024-09-21T03:44:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.025+0000 7f25e4a3f640 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x559f85c62da0 0x559f861059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:18.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.025+0000 7f25bffff640 10 client.17982 ms_handle_connect on v2:172.21.15.203:6836/4062574800 2024-09-21T03:44:18.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.025+0000 7f25e4a3f640 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x559f85c62da0 0x559f861059a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fece0bdd17c9019 server_cookie=4ce3dd27540a7963 in_seq=0 out_seq=0 2024-09-21T03:44:18.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25bffff640 1 -- 172.21.15.112:0/3971511298 <== mds.2 v2:172.21.15.203:6836/4062574800 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f85fea8a0 con 0x559f85c62da0 2024-09-21T03:44:18.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25bffff640 10 client.17982 handle_command_reply: tid=0 2024-09-21T03:44:18.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 1 client.17982 shutdown 2024-09-21T03:44:18.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 1 -- 172.21.15.112:0/3971511298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 msgr2=0x559f8608cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f8608cd80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f25d00aa170 tx=0x7f25d0010f70 comp rx=0 tx=0).stop 2024-09-21T03:44:18.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 1 -- 172.21.15.112:0/3971511298 shutdown_connections 2024-09-21T03:44:18.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.203:6836/4062574800,v1:172.21.15.203:6837/4062574800] conn(0x559f85c62da0 0x559f861059a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f85ccf070 0x559f86091fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f85cae390 0x559f8608cd80 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 1 --2- 172.21.15.112:0/3971511298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f85ca61b0 0x559f8608c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 1 -- 172.21.15.112:0/3971511298 >> 172.21.15.112:0/3971511298 conn(0x559f85cc3460 msgr2=0x559f860836b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25bffff640 0 client.17982 ms_handle_reset on v2:172.21.15.203:6836/4062574800 2024-09-21T03:44:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 1 -- 172.21.15.112:0/3971511298 shutdown_connections 2024-09-21T03:44:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 1 -- 172.21.15.112:0/3971511298 wait complete. 2024-09-21T03:44:18.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.026+0000 7f25ed217740 20 client.17982 trim_cache size 0 max 16384 2024-09-21T03:44:18.044 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:44:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 -- 172.21.15.112:0/129001266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c80b6680 msgr2=0x7f25c80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c80b6680 0x7f25c80b8b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f25d8005cc0 tx=0x7f25d80099c0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 -- 172.21.15.112:0/129001266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e0107b70 msgr2=0x7f25e0199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:18.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e0107b70 0x7f25e0199970 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f25d000d740 tx=0x7f25d000dc00 comp rx=0 tx=0).stop 2024-09-21T03:44:18.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 -- 172.21.15.112:0/129001266 shutdown_connections 2024-09-21T03:44:18.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f25c80b6680 0x7f25c80b8b90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e01084c0 0x7f25e019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e0107b70 0x7f25e0199970 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 --2- 172.21.15.112:0/129001266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25e0102230 0x7f25e01993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:18.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 -- 172.21.15.112:0/129001266 >> 172.21.15.112:0/129001266 conn(0x7f25e0079e40 msgr2=0x7f25e0105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:18.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 -- 172.21.15.112:0/129001266 shutdown_connections 2024-09-21T03:44:18.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:18.027+0000 7f25e6ca3640 1 -- 172.21.15.112:0/129001266 wait complete. 2024-09-21T03:44:18.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:18 smithi112 ceph-mon[31442]: pgmap v796: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.3 MiB/s wr, 255 op/s 2024-09-21T03:44:18.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:44:18.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:18 smithi203 ceph-mon[34723]: pgmap v796: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.3 MiB/s wr, 255 op/s 2024-09-21T03:44:18.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:18 smithi187 ceph-mon[34603]: pgmap v796: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.3 MiB/s wr, 255 op/s 2024-09-21T03:44:19.132 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:19.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.606+0000 7ff1c4e34640 1 -- 172.21.15.112:0/1719270705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0103b60 msgr2=0x7ff1c0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.606+0000 7ff1c4e34640 1 --2- 172.21.15.112:0/1719270705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0103b60 0x7ff1c0103f70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009900 tx=0x7ff1b002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:19.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.606+0000 7ff1c4e34640 1 -- 172.21.15.112:0/1719270705 shutdown_connections 2024-09-21T03:44:19.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.606+0000 7ff1c4e34640 1 --2- 172.21.15.112:0/1719270705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1c0109df0 0x7ff1c0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.606+0000 7ff1c4e34640 1 --2- 172.21.15.112:0/1719270705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c01094a0 0x7ff1c01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.606+0000 7ff1c4e34640 1 --2- 172.21.15.112:0/1719270705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0103b60 0x7ff1c0103f70 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.606+0000 7ff1c4e34640 1 -- 172.21.15.112:0/1719270705 >> 172.21.15.112:0/1719270705 conn(0x7ff1c007b850 msgr2=0x7ff1c007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:19.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.606+0000 7ff1c4e34640 1 -- 172.21.15.112:0/1719270705 shutdown_connections 2024-09-21T03:44:19.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.606+0000 7ff1c4e34640 1 -- 172.21.15.112:0/1719270705 wait complete. 2024-09-21T03:44:19.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1c4e34640 1 Processor -- start 2024-09-21T03:44:19.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1c4e34640 1 -- start start 2024-09-21T03:44:19.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1c4e34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0103b60 0x7ff1c019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1c4e34640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c01094a0 0x7ff1c019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1c4e34640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1c0109df0 0x7ff1c019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1c4e34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0113ab0 con 0x7ff1c0103b60 2024-09-21T03:44:19.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1c4e34640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0113930 con 0x7ff1c01094a0 2024-09-21T03:44:19.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1c4e34640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0113c30 con 0x7ff1c0109df0 2024-09-21T03:44:19.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1be59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0103b60 0x7ff1c019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1be59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0103b60 0x7ff1c019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45560/0 (socket says 172.21.15.112:45560) 2024-09-21T03:44:19.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1be59c640 1 -- 172.21.15.112:0/3179985810 learned_addr learned my addr 172.21.15.112:0/3179985810 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:19.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1bdd9b640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c01094a0 0x7ff1c019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1bed9d640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1c0109df0 0x7ff1c019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1be59c640 1 -- 172.21.15.112:0/3179985810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1c0109df0 msgr2=0x7ff1c019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1be59c640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1c0109df0 0x7ff1c019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1be59c640 1 -- 172.21.15.112:0/3179985810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c01094a0 msgr2=0x7ff1c019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1be59c640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c01094a0 0x7ff1c019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.607+0000 7ff1be59c640 1 -- 172.21.15.112:0/3179985810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c01a03b0 con 0x7ff1c0103b60 2024-09-21T03:44:19.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.608+0000 7ff1be59c640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0103b60 0x7ff1c019a500 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0005940 tx=0x7ff1b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:19.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.608+0000 7ff1b77fe640 1 -- 172.21.15.112:0/3179985810 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b003f030 con 0x7ff1c0103b60 2024-09-21T03:44:19.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.608+0000 7ff1c4e34640 1 -- 172.21.15.112:0/3179985810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c01a0640 con 0x7ff1c0103b60 2024-09-21T03:44:19.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.608+0000 7ff1b77fe640 1 -- 172.21.15.112:0/3179985810 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b0003cb0 con 0x7ff1c0103b60 2024-09-21T03:44:19.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.608+0000 7ff1b77fe640 1 -- 172.21.15.112:0/3179985810 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0035070 con 0x7ff1c0103b60 2024-09-21T03:44:19.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.608+0000 7ff1c4e34640 1 -- 172.21.15.112:0/3179985810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c01a0b70 con 0x7ff1c0103b60 2024-09-21T03:44:19.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.610+0000 7ff1cb3a8740 1 Processor -- start 2024-09-21T03:44:19.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.610+0000 7ff1cb3a8740 1 -- start start 2024-09-21T03:44:19.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.610+0000 7ff1b77fe640 1 -- 172.21.15.112:0/3179985810 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff1b0042030 con 0x7ff1c0103b60 2024-09-21T03:44:19.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b685f070 0x5633b6c066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c06be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b683e390 0x5633b6c07120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5633b6b7cea0 con 0x5633b68361b0 2024-09-21T03:44:19.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5633b6b6c2d0 con 0x5633b685f070 2024-09-21T03:44:19.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5633b6b700e0 con 0x5633b683e390 2024-09-21T03:44:19.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1be59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c06be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1be59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c06be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45572/0 (socket says 172.21.15.112:45572) 2024-09-21T03:44:19.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1be59c640 1 -- 172.21.15.112:0/1007514021 learned_addr learned my addr 172.21.15.112:0/1007514021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:19.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1bdd9b640 1 --2- 172.21.15.112:0/1007514021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b683e390 0x5633b6c07120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1bed9d640 1 --2- 172.21.15.112:0/1007514021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b685f070 0x5633b6c066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1be59c640 1 -- 172.21.15.112:0/1007514021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b683e390 msgr2=0x5633b6c07120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:19.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1be59c640 1 --2- 172.21.15.112:0/1007514021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b683e390 0x5633b6c07120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1be59c640 1 -- 172.21.15.112:0/1007514021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b685f070 msgr2=0x5633b6c066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:19.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1be59c640 1 --2- 172.21.15.112:0/1007514021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b685f070 0x5633b6c066a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1be59c640 1 -- 172.21.15.112:0/1007514021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633b68ca650 con 0x5633b68361b0 2024-09-21T03:44:19.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1b77fe640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1ac0b66b0 0x7ff1ac0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.611+0000 7ff1be59c640 1 --2- 172.21.15.112:0/1007514021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c06be0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0105250 tx=0x7ff1b010be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a8fcf42a4d50a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:19.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.612+0000 7ff1bed9d640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1ac0b66b0 0x7ff1ac0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.612+0000 7ff1b77fe640 1 -- 172.21.15.112:0/3179985810 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7ff1b0107b60 con 0x7ff1c0103b60 2024-09-21T03:44:19.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.612+0000 7ff1b4ff9640 1 -- 172.21.15.112:0/1007514021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00ca0e0 con 0x5633b68361b0 2024-09-21T03:44:19.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.612+0000 7ff1b4ff9640 1 -- 172.21.15.112:0/1007514021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b00f1080 con 0x5633b68361b0 2024-09-21T03:44:19.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.612+0000 7ff1b4ff9640 1 -- 172.21.15.112:0/1007514021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00cd200 con 0x5633b68361b0 2024-09-21T03:44:19.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/1007514021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 msgr2=0x5633b6c06be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1bed9d640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1ac0b66b0 0x7ff1ac0b8bc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8007f00 tx=0x7ff1b8007520 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:19.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/1007514021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c06be0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0105250 tx=0x7ff1b010be30 comp rx=0 tx=0).stop 2024-09-21T03:44:19.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/1007514021 shutdown_connections 2024-09-21T03:44:19.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/1007514021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b683e390 0x5633b6c07120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/1007514021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c06be0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/1007514021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b685f070 0x5633b6c066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/1007514021 >> 172.21.15.112:0/1007514021 conn(0x5633b6853460 msgr2=0x5633b6c04590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/1007514021 shutdown_connections 2024-09-21T03:44:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/1007514021 wait complete. 2024-09-21T03:44:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1cb3a8740 1 -- start start 2024-09-21T03:44:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.613+0000 7ff1cb3a8740 1 Processor -- start 2024-09-21T03:44:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b68361b0 0x5633b6c11db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b683e390 0x5633b6c122f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b685f070 0x5633b6c1dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5633b69cd700 con 0x5633b685f070 2024-09-21T03:44:19.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5633b6b07160 con 0x5633b683e390 2024-09-21T03:44:19.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5633b6c14c40 con 0x5633b68361b0 2024-09-21T03:44:19.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1cb3a8740 10 client.0 _ll_register_callbacks cb 0x5633b6b209e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:19.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1bed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b685f070 0x5633b6c1dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1bed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b685f070 0x5633b6c1dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45584/0 (socket says 172.21.15.112:45584) 2024-09-21T03:44:19.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1bed9d640 1 -- 172.21.15.112:0/2452555036 learned_addr learned my addr 172.21.15.112:0/2452555036 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1be59c640 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b68361b0 0x5633b6c11db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1bdd9b640 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b683e390 0x5633b6c122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1be59c640 1 -- 172.21.15.112:0/2452555036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b683e390 msgr2=0x5633b6c122f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1be59c640 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b683e390 0x5633b6c122f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1be59c640 1 -- 172.21.15.112:0/2452555036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b685f070 msgr2=0x5633b6c1dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1be59c640 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b685f070 0x5633b6c1dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1be59c640 1 -- 172.21.15.112:0/2452555036 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633b6b34c90 con 0x5633b68361b0 2024-09-21T03:44:19.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1bed9d640 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b685f070 0x5633b6c1dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1be59c640 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b68361b0 0x5633b6c11db0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff1b003b660 tx=0x7ff1b002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f76f513b7cddec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.614+0000 7ff1a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-21T03:44:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2452555036 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0049030 con 0x5633b68361b0 2024-09-21T03:44:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2452555036 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7ff1b003b460 con 0x5633b68361b0 2024-09-21T03:44:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2452555036 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5633b6b7edf0 con 0x5633b68361b0 2024-09-21T03:44:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2452555036 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff1b00cd3e0 con 0x5633b68361b0 2024-09-21T03:44:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1cb3a8740 10 client.27529 fetch_fsmap learned FSMap version 35 2024-09-21T03:44:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2452555036 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5633b6c208c0 con 0x5633b68361b0 2024-09-21T03:44:19.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2452555036 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5633b6b7edf0 con 0x5633b68361b0 2024-09-21T03:44:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2452555036 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7ff1b0105560 con 0x5633b68361b0 2024-09-21T03:44:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1cb3a8740 10 client.27529 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:44:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1cb3a8740 10 client.27529 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:44:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1cb3a8740 20 client.27529 populate_metadata read hostname 'smithi112' 2024-09-21T03:44:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5633b681d9f0 0x5633b6c23c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1cb3a8740 4 client.27529 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.615+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2452555036 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633b6b7a8a0 con 0x5633b681d9f0 2024-09-21T03:44:19.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.616+0000 7ff1bdd9b640 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5633b681d9f0 0x5633b6c23c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.616+0000 7ff1a5ffb640 10 client.27529 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:44:19.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.616+0000 7ff1bdd9b640 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5633b681d9f0 0x5633b6c23c20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7b7136329db036f server_cookie=72131c67c1112b52 in_seq=0 out_seq=0 2024-09-21T03:44:19.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2452555036 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5633b6b7a8a0 con 0x5633b681d9f0 2024-09-21T03:44:19.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1a5ffb640 10 client.27529 handle_command_reply: tid=0 2024-09-21T03:44:19.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 1 client.27529 shutdown 2024-09-21T03:44:19.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2452555036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b68361b0 msgr2=0x5633b6c11db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b68361b0 0x5633b6c11db0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff1b003b660 tx=0x7ff1b002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2452555036 shutdown_connections 2024-09-21T03:44:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5633b681d9f0 0x5633b6c23c20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b685f070 0x5633b6c1dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b683e390 0x5633b6c122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1a5ffb640 0 client.27529 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:44:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2452555036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b68361b0 0x5633b6c11db0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2452555036 >> 172.21.15.112:0/2452555036 conn(0x5633b6853460 msgr2=0x5633b6c09880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2452555036 shutdown_connections 2024-09-21T03:44:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2452555036 wait complete. 2024-09-21T03:44:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.617+0000 7ff1cb3a8740 20 client.27529 trim_cache size 0 max 16384 2024-09-21T03:44:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1cb3a8740 1 Processor -- start 2024-09-21T03:44:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1cb3a8740 1 -- start start 2024-09-21T03:44:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b681d9f0 0x5633b6c04600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b68361b0 0x5633b6c1d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b683e390 0x5633b6c1de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01b2020 con 0x5633b683e390 2024-09-21T03:44:19.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01af860 con 0x5633b68361b0 2024-09-21T03:44:19.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01ad6d0 con 0x5633b681d9f0 2024-09-21T03:44:19.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1bed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b683e390 0x5633b6c1de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1bed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b683e390 0x5633b6c1de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45586/0 (socket says 172.21.15.112:45586) 2024-09-21T03:44:19.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1bed9d640 1 -- 172.21.15.112:0/3959426170 learned_addr learned my addr 172.21.15.112:0/3959426170 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:19.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.630+0000 7ff1bdd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b68361b0 0x5633b6c1d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.631+0000 7ff1be59c640 1 --2- 172.21.15.112:0/3959426170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b681d9f0 0x5633b6c04600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.631+0000 7ff1bed9d640 1 -- 172.21.15.112:0/3959426170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b681d9f0 msgr2=0x5633b6c04600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:19.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.631+0000 7ff1bed9d640 1 --2- 172.21.15.112:0/3959426170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b681d9f0 0x5633b6c04600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.631+0000 7ff1bed9d640 1 -- 172.21.15.112:0/3959426170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b68361b0 msgr2=0x5633b6c1d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:19.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.631+0000 7ff1bed9d640 1 --2- 172.21.15.112:0/3959426170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b68361b0 0x5633b6c1d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.631+0000 7ff1bed9d640 1 -- 172.21.15.112:0/3959426170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633b6c08990 con 0x5633b683e390 2024-09-21T03:44:19.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.631+0000 7ff1bdd9b640 1 --2- 172.21.15.112:0/3959426170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b68361b0 0x5633b6c1d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:19.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.631+0000 7ff1bed9d640 1 --2- 172.21.15.112:0/3959426170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b683e390 0x5633b6c1de80 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff1b801ec10 tx=0x7ff1b8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f7a67ac1ca049 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:19.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.631+0000 7ff1a7fff640 1 -- 172.21.15.112:0/3959426170 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8004b20 con 0x5633b683e390 2024-09-21T03:44:19.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.631+0000 7ff1a7fff640 1 -- 172.21.15.112:0/3959426170 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b8004cc0 con 0x5633b683e390 2024-09-21T03:44:19.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.631+0000 7ff1a7fff640 1 -- 172.21.15.112:0/3959426170 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8029650 con 0x5633b683e390 2024-09-21T03:44:19.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.632+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/3959426170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b683e390 msgr2=0x5633b6c1de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.632+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/3959426170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b683e390 0x5633b6c1de80 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff1b801ec10 tx=0x7ff1b8018000 comp rx=0 tx=0).stop 2024-09-21T03:44:19.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.632+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/3959426170 shutdown_connections 2024-09-21T03:44:19.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.632+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/3959426170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b683e390 0x5633b6c1de80 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.632+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/3959426170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b68361b0 0x5633b6c1d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.632+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/3959426170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b681d9f0 0x5633b6c04600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.632+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/3959426170 >> 172.21.15.112:0/3959426170 conn(0x5633b6853460 msgr2=0x5633b6c03ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.632+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/3959426170 shutdown_connections 2024-09-21T03:44:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.632+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/3959426170 wait complete. 2024-09-21T03:44:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.632+0000 7ff1cb3a8740 1 -- start start 2024-09-21T03:44:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.632+0000 7ff1cb3a8740 1 Processor -- start 2024-09-21T03:44:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b681d9f0 0x5633b6c15c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c16150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1cb3a8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b683e390 0x5633b6c8d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01add20 con 0x5633b68361b0 2024-09-21T03:44:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01af190 con 0x5633b681d9f0 2024-09-21T03:44:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1cb3a8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01ae3f0 con 0x5633b683e390 2024-09-21T03:44:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1cb3a8740 10 client.0 _ll_register_callbacks cb 0x7ff1b8007860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1bdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c16150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1bdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c16150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45588/0 (socket says 172.21.15.112:45588) 2024-09-21T03:44:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1bdd9b640 1 -- 172.21.15.112:0/2217722054 learned_addr learned my addr 172.21.15.112:0/2217722054 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1bed9d640 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b683e390 0x5633b6c8d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1be59c640 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b681d9f0 0x5633b6c15c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1bdd9b640 1 -- 172.21.15.112:0/2217722054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b683e390 msgr2=0x5633b6c8d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:19.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1bdd9b640 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b683e390 0x5633b6c8d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1bdd9b640 1 -- 172.21.15.112:0/2217722054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b681d9f0 msgr2=0x5633b6c15c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:19.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1bdd9b640 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b681d9f0 0x5633b6c15c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1bdd9b640 1 -- 172.21.15.112:0/2217722054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633b6c16970 con 0x5633b68361b0 2024-09-21T03:44:19.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1bdd9b640 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c16150 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x5633b6c241f0 tx=0x7ff1a80044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc918f281e583be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:19.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.633+0000 7ff1a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:44:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2217722054 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8029060 con 0x5633b68361b0 2024-09-21T03:44:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2217722054 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7ff1a8011640 con 0x5633b68361b0 2024-09-21T03:44:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2217722054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b8004cc0 con 0x5633b68361b0 2024-09-21T03:44:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2217722054 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff1a8011840 con 0x5633b68361b0 2024-09-21T03:44:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1cb3a8740 10 client.18000 fetch_fsmap learned FSMap version 35 2024-09-21T03:44:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2217722054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5633b6c8e9c0 con 0x5633b68361b0 2024-09-21T03:44:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2217722054 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b8004cc0 con 0x5633b68361b0 2024-09-21T03:44:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2217722054 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7ff1a802eb90 con 0x5633b68361b0 2024-09-21T03:44:19.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1cb3a8740 10 client.18000 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:44:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1cb3a8740 10 client.18000 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-21T03:44:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1cb3a8740 20 client.18000 populate_metadata read hostname 'smithi112' 2024-09-21T03:44:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5633b685f070 0x5633b6c91210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1cb3a8740 4 client.18000 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:44:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2217722054 --> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633b6b77cd0 con 0x5633b685f070 2024-09-21T03:44:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.634+0000 7ff1be59c640 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5633b685f070 0x5633b6c91210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1a5ffb640 10 client.18000 ms_handle_connect on v2:172.21.15.187:6838/910214120 2024-09-21T03:44:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1be59c640 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5633b685f070 0x5633b6c91210 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2147b9af52fd641 server_cookie=df9e2aed93b3a8aa in_seq=0 out_seq=0 2024-09-21T03:44:19.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1a5ffb640 1 -- 172.21.15.112:0/2217722054 <== mds.1 v2:172.21.15.187:6838/910214120 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633b6b77cd0 con 0x5633b685f070 2024-09-21T03:44:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1a5ffb640 10 client.18000 handle_command_reply: tid=0 2024-09-21T03:44:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 1 client.18000 shutdown 2024-09-21T03:44:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2217722054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 msgr2=0x5633b6c16150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c16150 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x5633b6c241f0 tx=0x7ff1a80044a0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2217722054 shutdown_connections 2024-09-21T03:44:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.187:6838/910214120,v1:172.21.15.187:6839/910214120] conn(0x5633b685f070 0x5633b6c91210 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633b683e390 0x5633b6c8d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633b68361b0 0x5633b6c16150 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 1 --2- 172.21.15.112:0/2217722054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633b681d9f0 0x5633b6c15c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1a5ffb640 0 client.18000 ms_handle_reset on v2:172.21.15.187:6838/910214120 2024-09-21T03:44:19.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2217722054 >> 172.21.15.112:0/2217722054 conn(0x5633b6853460 msgr2=0x5633b6c097a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:19.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2217722054 shutdown_connections 2024-09-21T03:44:19.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 1 -- 172.21.15.112:0/2217722054 wait complete. 2024-09-21T03:44:19.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.635+0000 7ff1cb3a8740 20 client.18000 trim_cache size 0 max 16384 2024-09-21T03:44:19.648 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:44:19.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 -- 172.21.15.112:0/3179985810 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1ac0b66b0 msgr2=0x7ff1ac0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1ac0b66b0 0x7ff1ac0b8bc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8007f00 tx=0x7ff1b8007520 comp rx=0 tx=0).stop 2024-09-21T03:44:19.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 -- 172.21.15.112:0/3179985810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0103b60 msgr2=0x7ff1c019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:19.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0103b60 0x7ff1c019a500 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0005940 tx=0x7ff1b00042f0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 -- 172.21.15.112:0/3179985810 shutdown_connections 2024-09-21T03:44:19.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff1ac0b66b0 0x7ff1ac0b8bc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1c0109df0 0x7ff1c019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1c01094a0 0x7ff1c019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 --2- 172.21.15.112:0/3179985810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1c0103b60 0x7ff1c019a500 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:19.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 -- 172.21.15.112:0/3179985810 >> 172.21.15.112:0/3179985810 conn(0x7ff1c007b850 msgr2=0x7ff1c01075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:19.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 -- 172.21.15.112:0/3179985810 shutdown_connections 2024-09-21T03:44:19.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:19.636+0000 7ff1c4e34640 1 -- 172.21.15.112:0/3179985810 wait complete. 2024-09-21T03:44:20.409 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:44:20.758 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:20.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:20 smithi112 ceph-mon[31442]: pgmap v797: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.5 MiB/s wr, 228 op/s 2024-09-21T03:44:20.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:20 smithi203 ceph-mon[34723]: pgmap v797: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.5 MiB/s wr, 228 op/s 2024-09-21T03:44:20.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:20 smithi187 ceph-mon[34603]: pgmap v797: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.5 MiB/s wr, 228 op/s 2024-09-21T03:44:21.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.246+0000 7f961af1d640 1 -- 172.21.15.112:0/4205120546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9614108cf0 msgr2=0x7f961410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.246+0000 7f961af1d640 1 --2- 172.21.15.112:0/4205120546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9614108cf0 0x7f961410f520 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f960c009ca0 tx=0x7f960c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.246+0000 7f961af1d640 1 -- 172.21.15.112:0/4205120546 shutdown_connections 2024-09-21T03:44:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.246+0000 7f961af1d640 1 --2- 172.21.15.112:0/4205120546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9614108cf0 0x7f961410f520 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.246+0000 7f961af1d640 1 --2- 172.21.15.112:0/4205120546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96141083a0 0x7f96141087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.246+0000 7f961af1d640 1 --2- 172.21.15.112:0/4205120546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9614102a60 0x7f9614102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.246+0000 7f961af1d640 1 -- 172.21.15.112:0/4205120546 >> 172.21.15.112:0/4205120546 conn(0x7f9614079730 msgr2=0x7f9614079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.246+0000 7f961af1d640 1 -- 172.21.15.112:0/4205120546 shutdown_connections 2024-09-21T03:44:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.246+0000 7f961af1d640 1 -- 172.21.15.112:0/4205120546 wait complete. 2024-09-21T03:44:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f961af1d640 1 Processor -- start 2024-09-21T03:44:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f961af1d640 1 -- start start 2024-09-21T03:44:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f961af1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9614102a60 0x7f961419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f961af1d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96141083a0 0x7f961419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f961af1d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9614108cf0 0x7f961419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f961af1d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9614112960 con 0x7f9614102a60 2024-09-21T03:44:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f961af1d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96141127e0 con 0x7f9614108cf0 2024-09-21T03:44:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f961af1d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9614112ae0 con 0x7f96141083a0 2024-09-21T03:44:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f9618cb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9614102a60 0x7f961419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f9618cb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9614102a60 0x7f961419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45604/0 (socket says 172.21.15.112:45604) 2024-09-21T03:44:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f9618cb9640 1 -- 172.21.15.112:0/3324738485 learned_addr learned my addr 172.21.15.112:0/3324738485 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f96194ba640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9614108cf0 0x7f961419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.247+0000 7f960bfff640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96141083a0 0x7f961419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.248+0000 7f9618cb9640 1 -- 172.21.15.112:0/3324738485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96141083a0 msgr2=0x7f961419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.248+0000 7f9618cb9640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96141083a0 0x7f961419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.248+0000 7f9618cb9640 1 -- 172.21.15.112:0/3324738485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9614108cf0 msgr2=0x7f961419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.248+0000 7f9618cb9640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9614108cf0 0x7f961419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.248+0000 7f9618cb9640 1 -- 172.21.15.112:0/3324738485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96141a03f0 con 0x7f9614102a60 2024-09-21T03:44:21.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.248+0000 7f9618cb9640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9614102a60 0x7f961419a480 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f961000d950 tx=0x7f961000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.248+0000 7f9609ffb640 1 -- 172.21.15.112:0/3324738485 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610017030 con 0x7f9614102a60 2024-09-21T03:44:21.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.248+0000 7f961af1d640 1 -- 172.21.15.112:0/3324738485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96141a06e0 con 0x7f9614102a60 2024-09-21T03:44:21.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.248+0000 7f9609ffb640 1 -- 172.21.15.112:0/3324738485 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9610004020 con 0x7f9614102a60 2024-09-21T03:44:21.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.248+0000 7f9609ffb640 1 -- 172.21.15.112:0/3324738485 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96100047a0 con 0x7f9614102a60 2024-09-21T03:44:21.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.248+0000 7f961af1d640 1 -- 172.21.15.112:0/3324738485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96141a6e40 con 0x7f9614102a60 2024-09-21T03:44:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9609ffb640 1 -- 172.21.15.112:0/3324738485 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9610009a30 con 0x7f9614102a60 2024-09-21T03:44:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9621491740 1 Processor -- start 2024-09-21T03:44:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9621491740 1 -- start start 2024-09-21T03:44:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9621491740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe4212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9621491740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe0511b0 0x5555fe421800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9621491740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 0x5555fe421d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9621491740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5555fe397ea0 con 0x5555fe0511b0 2024-09-21T03:44:21.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9621491740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5555fe3872d0 con 0x5555fe059390 2024-09-21T03:44:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9621491740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5555fe38b0e0 con 0x5555fe07a070 2024-09-21T03:44:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9618cb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe0511b0 0x5555fe421800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9618cb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe0511b0 0x5555fe421800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45620/0 (socket says 172.21.15.112:45620) 2024-09-21T03:44:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f9618cb9640 1 -- 172.21.15.112:0/3890753083 learned_addr learned my addr 172.21.15.112:0/3890753083 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.251+0000 7f960bfff640 1 --2- 172.21.15.112:0/3890753083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 0x5555fe421d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f96194ba640 1 --2- 172.21.15.112:0/3890753083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe4212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f9609ffb640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95fc0b6680 0x7f95fc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f9618cb9640 1 -- 172.21.15.112:0/3890753083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 msgr2=0x5555fe4212c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:21.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f9618cb9640 1 --2- 172.21.15.112:0/3890753083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe4212c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f9618cb9640 1 -- 172.21.15.112:0/3890753083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 msgr2=0x5555fe421d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:21.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f9618cb9640 1 --2- 172.21.15.112:0/3890753083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 0x5555fe421d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f9618cb9640 1 -- 172.21.15.112:0/3890753083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555fe0e5650 con 0x5555fe0511b0 2024-09-21T03:44:21.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f960bfff640 1 --2- 172.21.15.112:0/3890753083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 0x5555fe421d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:21.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f96194ba640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95fc0b6680 0x7f95fc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f9609ffb640 1 -- 172.21.15.112:0/3324738485 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f96100e70b0 con 0x7f9614102a60 2024-09-21T03:44:21.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f9618cb9640 1 --2- 172.21.15.112:0/3890753083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe0511b0 0x5555fe421800 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f96100e4840 tx=0x7f96100ad1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41ddf845a062fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f95faffd640 1 -- 172.21.15.112:0/3890753083 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96100ac920 con 0x5555fe0511b0 2024-09-21T03:44:21.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f95faffd640 1 -- 172.21.15.112:0/3890753083 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96100acae0 con 0x5555fe0511b0 2024-09-21T03:44:21.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.252+0000 7f95faffd640 1 -- 172.21.15.112:0/3890753083 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96100ab3a0 con 0x5555fe0511b0 2024-09-21T03:44:21.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.253+0000 7f9621491740 1 -- 172.21.15.112:0/3890753083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe0511b0 msgr2=0x5555fe421800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.253+0000 7f96194ba640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95fc0b6680 0x7f95fc0b8b90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f960c006010 tx=0x7f960c0095c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.253+0000 7f9621491740 1 --2- 172.21.15.112:0/3890753083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe0511b0 0x5555fe421800 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f96100e4840 tx=0x7f96100ad1c0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.253+0000 7f9621491740 1 -- 172.21.15.112:0/3890753083 shutdown_connections 2024-09-21T03:44:21.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.253+0000 7f9621491740 1 --2- 172.21.15.112:0/3890753083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 0x5555fe421d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.253+0000 7f9621491740 1 --2- 172.21.15.112:0/3890753083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe0511b0 0x5555fe421800 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.253+0000 7f9621491740 1 --2- 172.21.15.112:0/3890753083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe4212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.253+0000 7f9621491740 1 -- 172.21.15.112:0/3890753083 >> 172.21.15.112:0/3890753083 conn(0x5555fe06e460 msgr2=0x5555fe41f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:21.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.253+0000 7f9621491740 1 -- 172.21.15.112:0/3890753083 shutdown_connections 2024-09-21T03:44:21.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.253+0000 7f9621491740 1 -- 172.21.15.112:0/3890753083 wait complete. 2024-09-21T03:44:21.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f9621491740 1 -- start start 2024-09-21T03:44:21.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f9621491740 1 Processor -- start 2024-09-21T03:44:21.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f9621491740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe0511b0 0x5555fe42c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f9621491740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 0x5555fe42ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f9621491740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe07a070 0x5555fe42d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f9621491740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5555fe1e8700 con 0x5555fe07a070 2024-09-21T03:44:21.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f9621491740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5555fe322160 con 0x5555fe059390 2024-09-21T03:44:21.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f9621491740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5555fe42f9e0 con 0x5555fe0511b0 2024-09-21T03:44:21.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f9621491740 10 client.0 _ll_register_callbacks cb 0x5555fe33b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:21.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f96194ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe07a070 0x5555fe42d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f96194ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe07a070 0x5555fe42d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45624/0 (socket says 172.21.15.112:45624) 2024-09-21T03:44:21.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f96194ba640 1 -- 172.21.15.112:0/1657532473 learned_addr learned my addr 172.21.15.112:0/1657532473 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:21.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.254+0000 7f960bfff640 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 0x5555fe42ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f9618cb9640 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe0511b0 0x5555fe42c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f96194ba640 1 -- 172.21.15.112:0/1657532473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe0511b0 msgr2=0x5555fe42c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:21.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f96194ba640 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe0511b0 0x5555fe42c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f96194ba640 1 -- 172.21.15.112:0/1657532473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 msgr2=0x5555fe42ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:21.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f96194ba640 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 0x5555fe42ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f96194ba640 1 -- 172.21.15.112:0/1657532473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555fe34fc90 con 0x5555fe07a070 2024-09-21T03:44:21.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f960bfff640 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 0x5555fe42ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:21.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f96194ba640 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe07a070 0x5555fe42d390 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f960c031590 tx=0x7f960c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e737690c30c87be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f95f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:44:21.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f95f3fff640 1 -- 172.21.15.112:0/1657532473 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c003d00 con 0x5555fe07a070 2024-09-21T03:44:21.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f95f3fff640 1 -- 172.21.15.112:0/1657532473 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f960c004040 con 0x5555fe07a070 2024-09-21T03:44:21.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f9621491740 1 -- 172.21.15.112:0/1657532473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96100acae0 con 0x5555fe07a070 2024-09-21T03:44:21.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f95f3fff640 1 -- 172.21.15.112:0/1657532473 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f960c004240 con 0x5555fe07a070 2024-09-21T03:44:21.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f9621491740 10 client.18012 fetch_fsmap learned FSMap version 35 2024-09-21T03:44:21.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.255+0000 7f9621491740 1 -- 172.21.15.112:0/1657532473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5555fe43b690 con 0x5555fe07a070 2024-09-21T03:44:21.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.256+0000 7f95f3fff640 1 -- 172.21.15.112:0/1657532473 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96100acae0 con 0x5555fe07a070 2024-09-21T03:44:21.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.256+0000 7f95f3fff640 1 -- 172.21.15.112:0/1657532473 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f960c0305c0 con 0x5555fe07a070 2024-09-21T03:44:21.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.256+0000 7f9621491740 10 client.18012 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:44:21.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.256+0000 7f9621491740 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:44:21.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.256+0000 7f9621491740 20 client.18012 populate_metadata read hostname 'smithi112' 2024-09-21T03:44:21.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.256+0000 7f9621491740 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555fe0389f0 0x5555fe43eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.256+0000 7f9621491740 4 client.18012 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:44:21.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.256+0000 7f9621491740 1 -- 172.21.15.112:0/1657532473 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555fe399df0 con 0x5555fe0389f0 2024-09-21T03:44:21.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.256+0000 7f9618cb9640 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555fe0389f0 0x5555fe43eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.256+0000 7f95f3fff640 10 client.18012 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:44:21.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.256+0000 7f9618cb9640 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555fe0389f0 0x5555fe43eb50 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=422e8e48f44584f0 server_cookie=60211a943a2d15a1 in_seq=0 out_seq=0 2024-09-21T03:44:21.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.257+0000 7f95f3fff640 1 -- 172.21.15.112:0/1657532473 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5555fe399df0 con 0x5555fe0389f0 2024-09-21T03:44:21.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.257+0000 7f95f3fff640 10 client.18012 handle_command_reply: tid=0 2024-09-21T03:44:21.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.257+0000 7f9621491740 1 client.18012 shutdown 2024-09-21T03:44:21.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.257+0000 7f9621491740 1 -- 172.21.15.112:0/1657532473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe07a070 msgr2=0x5555fe42d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.257+0000 7f9621491740 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe07a070 0x5555fe42d390 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f960c031590 tx=0x7f960c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.257+0000 7f9621491740 1 -- 172.21.15.112:0/1657532473 shutdown_connections 2024-09-21T03:44:21.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.258+0000 7f9621491740 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555fe0389f0 0x5555fe43eb50 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.258+0000 7f9621491740 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe07a070 0x5555fe42d390 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.258+0000 7f9621491740 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe059390 0x5555fe42ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.258+0000 7f9621491740 1 --2- 172.21.15.112:0/1657532473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe0511b0 0x5555fe42c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.258+0000 7f9621491740 1 -- 172.21.15.112:0/1657532473 >> 172.21.15.112:0/1657532473 conn(0x5555fe06e460 msgr2=0x5555fe424450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:21.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.258+0000 7f95f3fff640 0 client.18012 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:44:21.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.258+0000 7f9621491740 1 -- 172.21.15.112:0/1657532473 shutdown_connections 2024-09-21T03:44:21.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.258+0000 7f9621491740 1 -- 172.21.15.112:0/1657532473 wait complete. 2024-09-21T03:44:21.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.258+0000 7f9621491740 20 client.18012 trim_cache size 0 max 16384 2024-09-21T03:44:21.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f9621491740 1 Processor -- start 2024-09-21T03:44:21.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f9621491740 1 -- start start 2024-09-21T03:44:21.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f9621491740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe0511b0 0x5555fe42de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f9621491740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe438c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f9621491740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe429b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f9621491740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96141b20c0 con 0x5555fe059390 2024-09-21T03:44:21.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f9621491740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96141af900 con 0x5555fe0511b0 2024-09-21T03:44:21.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f9621491740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96141ad770 con 0x5555fe07a070 2024-09-21T03:44:21.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f960bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe438c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f960bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe438c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45638/0 (socket says 172.21.15.112:45638) 2024-09-21T03:44:21.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f960bfff640 1 -- 172.21.15.112:0/1618863360 learned_addr learned my addr 172.21.15.112:0/1618863360 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:21.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f96194ba640 1 --2- 172.21.15.112:0/1618863360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe429b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f9618cb9640 1 --2- 172.21.15.112:0/1618863360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe0511b0 0x5555fe42de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f960bfff640 1 -- 172.21.15.112:0/1618863360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 msgr2=0x5555fe429b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:21.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f960bfff640 1 --2- 172.21.15.112:0/1618863360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe429b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f960bfff640 1 -- 172.21.15.112:0/1618863360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe0511b0 msgr2=0x5555fe42de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:21.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f960bfff640 1 --2- 172.21.15.112:0/1618863360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe0511b0 0x5555fe42de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.271+0000 7f960bfff640 1 -- 172.21.15.112:0/1618863360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555fe42ece0 con 0x5555fe059390 2024-09-21T03:44:21.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.272+0000 7f960bfff640 1 --2- 172.21.15.112:0/1618863360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe438c30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f960400bd90 tx=0x7f9604012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf8c9f78546c5a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.272+0000 7f95fa7fc640 1 -- 172.21.15.112:0/1618863360 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604004030 con 0x5555fe059390 2024-09-21T03:44:21.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.272+0000 7f95fa7fc640 1 -- 172.21.15.112:0/1618863360 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96040041d0 con 0x5555fe059390 2024-09-21T03:44:21.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.272+0000 7f95fa7fc640 1 -- 172.21.15.112:0/1618863360 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960401b800 con 0x5555fe059390 2024-09-21T03:44:21.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.272+0000 7f9621491740 1 -- 172.21.15.112:0/1618863360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 msgr2=0x5555fe438c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.272+0000 7f9621491740 1 --2- 172.21.15.112:0/1618863360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe438c30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f960400bd90 tx=0x7f9604012820 comp rx=0 tx=0).stop 2024-09-21T03:44:21.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.272+0000 7f9621491740 1 -- 172.21.15.112:0/1618863360 shutdown_connections 2024-09-21T03:44:21.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.272+0000 7f9621491740 1 --2- 172.21.15.112:0/1618863360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe429b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 --2- 172.21.15.112:0/1618863360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe438c30 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 --2- 172.21.15.112:0/1618863360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe0511b0 0x5555fe42de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 -- 172.21.15.112:0/1618863360 >> 172.21.15.112:0/1618863360 conn(0x5555fe06e460 msgr2=0x5555fe422cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:21.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 -- 172.21.15.112:0/1618863360 shutdown_connections 2024-09-21T03:44:21.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 -- 172.21.15.112:0/1618863360 wait complete. 2024-09-21T03:44:21.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 -- start start 2024-09-21T03:44:21.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 Processor -- start 2024-09-21T03:44:21.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe0511b0 0x5555fe4a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe434c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe4351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96141addc0 con 0x5555fe059390 2024-09-21T03:44:21.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96141af230 con 0x5555fe0511b0 2024-09-21T03:44:21.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96141ae490 con 0x5555fe07a070 2024-09-21T03:44:21.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.273+0000 7f9621491740 10 client.0 _ll_register_callbacks cb 0x7f960c031b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:44:21.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f960bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe434c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f960bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe434c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45646/0 (socket says 172.21.15.112:45646) 2024-09-21T03:44:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f960bfff640 1 -- 172.21.15.112:0/3029832120 learned_addr learned my addr 172.21.15.112:0/3029832120 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f9618cb9640 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe0511b0 0x5555fe4a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f96194ba640 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe4351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f960bfff640 1 -- 172.21.15.112:0/3029832120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 msgr2=0x5555fe4351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f960bfff640 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe4351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f960bfff640 1 -- 172.21.15.112:0/3029832120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe0511b0 msgr2=0x5555fe4a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:44:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f960bfff640 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe0511b0 0x5555fe4a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f960bfff640 1 -- 172.21.15.112:0/3029832120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555fe435760 con 0x5555fe059390 2024-09-21T03:44:21.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f960bfff640 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe434c90 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9604005780 tx=0x7f9604003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5856f3fc0a5902 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f95f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-09-21T03:44:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f95f3fff640 1 -- 172.21.15.112:0/3029832120 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604029020 con 0x5555fe059390 2024-09-21T03:44:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f95f3fff640 1 -- 172.21.15.112:0/3029832120 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f9604024440 con 0x5555fe059390 2024-09-21T03:44:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f9621491740 1 -- 172.21.15.112:0/3029832120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96040041d0 con 0x5555fe059390 2024-09-21T03:44:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f95f3fff640 1 -- 172.21.15.112:0/3029832120 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9604034050 con 0x5555fe059390 2024-09-21T03:44:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f9621491740 10 client.18018 fetch_fsmap learned FSMap version 35 2024-09-21T03:44:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.274+0000 7f9621491740 1 -- 172.21.15.112:0/3029832120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5555fe4a2e10 con 0x5555fe059390 2024-09-21T03:44:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.275+0000 7f95f3fff640 1 -- 172.21.15.112:0/3029832120 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96040041d0 con 0x5555fe059390 2024-09-21T03:44:21.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.275+0000 7f95f3fff640 1 -- 172.21.15.112:0/3029832120 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f960401b490 con 0x5555fe059390 2024-09-21T03:44:21.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.275+0000 7f9621491740 10 client.18018 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T03:44:21.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.275+0000 7f9621491740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-21T03:44:21.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.275+0000 7f9621491740 20 client.18018 populate_metadata read hostname 'smithi112' 2024-09-21T03:44:21.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.275+0000 7f9621491740 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555fe00dda0 0x5555fe4afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:21.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.275+0000 7f9621491740 4 client.18018 mds_command: new command op to 24685 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:44:21.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.275+0000 7f9621491740 1 -- 172.21.15.112:0/3029832120 --> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555fe3958a0 con 0x5555fe00dda0 2024-09-21T03:44:21.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.275+0000 7f9618cb9640 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555fe00dda0 0x5555fe4afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:21.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f95f3fff640 10 client.18018 ms_handle_connect on v2:172.21.15.203:6838/482815172 2024-09-21T03:44:21.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9618cb9640 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555fe00dda0 0x5555fe4afe80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9500c8c679240fa server_cookie=f424ec9ff7c612d1 in_seq=0 out_seq=0 2024-09-21T03:44:21.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f95f3fff640 1 -- 172.21.15.112:0/3029832120 <== mds.0 v2:172.21.15.203:6838/482815172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555fe3958a0 con 0x5555fe00dda0 2024-09-21T03:44:21.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f95f3fff640 10 client.18018 handle_command_reply: tid=0 2024-09-21T03:44:21.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 1 client.18018 shutdown 2024-09-21T03:44:21.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 1 -- 172.21.15.112:0/3029832120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 msgr2=0x5555fe434c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe434c90 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9604005780 tx=0x7f9604003d40 comp rx=0 tx=0).stop 2024-09-21T03:44:21.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 1 -- 172.21.15.112:0/3029832120 shutdown_connections 2024-09-21T03:44:21.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.203:6838/482815172,v1:172.21.15.203:6839/482815172] conn(0x5555fe00dda0 0x5555fe4afe80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f95f3fff640 0 client.18018 ms_handle_reset on v2:172.21.15.203:6838/482815172 2024-09-21T03:44:21.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555fe07a070 0x5555fe4351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555fe059390 0x5555fe434c90 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 1 --2- 172.21.15.112:0/3029832120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555fe0511b0 0x5555fe4a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 1 -- 172.21.15.112:0/3029832120 >> 172.21.15.112:0/3029832120 conn(0x5555fe06e460 msgr2=0x5555fe422cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:21.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 1 -- 172.21.15.112:0/3029832120 shutdown_connections 2024-09-21T03:44:21.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 1 -- 172.21.15.112:0/3029832120 wait complete. 2024-09-21T03:44:21.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.276+0000 7f9621491740 20 client.18018 trim_cache size 0 max 16384 2024-09-21T03:44:21.294 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-09-21T03:44:21.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 -- 172.21.15.112:0/3324738485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95fc0b6680 msgr2=0x7f95fc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95fc0b6680 0x7f95fc0b8b90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f960c006010 tx=0x7f960c0095c0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 -- 172.21.15.112:0/3324738485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9614102a60 msgr2=0x7f961419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:21.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9614102a60 0x7f961419a480 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f961000d950 tx=0x7f961000de10 comp rx=0 tx=0).stop 2024-09-21T03:44:21.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 -- 172.21.15.112:0/3324738485 shutdown_connections 2024-09-21T03:44:21.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f95fc0b6680 0x7f95fc0b8b90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9614108cf0 0x7f961419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96141083a0 0x7f961419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 --2- 172.21.15.112:0/3324738485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9614102a60 0x7f961419a480 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:21.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 -- 172.21.15.112:0/3324738485 >> 172.21.15.112:0/3324738485 conn(0x7f9614079730 msgr2=0x7f961406b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:21.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 -- 172.21.15.112:0/3324738485 shutdown_connections 2024-09-21T03:44:21.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:21.277+0000 7f961af1d640 1 -- 172.21.15.112:0/3324738485 wait complete. 2024-09-21T03:44:21.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:21 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:21.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:21 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:21.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:21 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:44:22.297 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T03:44:22.301 INFO:tasks.fwd_scrub:done joining 2024-09-21T03:44:22.301 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T03:44:22.311 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:44:22.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.428+0000 7ff65612b640 1 -- 172.21.15.112:0/1837976733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650102980 msgr2=0x7ff650102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.428+0000 7ff65612b640 1 --2- 172.21.15.112:0/1837976733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650102980 0x7ff650102d90 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff640009900 tx=0x7ff64002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.428+0000 7ff65612b640 1 -- 172.21.15.112:0/1837976733 shutdown_connections 2024-09-21T03:44:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.428+0000 7ff65612b640 1 --2- 172.21.15.112:0/1837976733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff650108c00 0x7ff65010f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.428+0000 7ff65612b640 1 --2- 172.21.15.112:0/1837976733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6501082b0 0x7ff6501086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.428+0000 7ff65612b640 1 --2- 172.21.15.112:0/1837976733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650102980 0x7ff650102d90 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.428+0000 7ff65612b640 1 -- 172.21.15.112:0/1837976733 >> 172.21.15.112:0/1837976733 conn(0x7ff6500fe380 msgr2=0x7ff6501007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.428+0000 7ff65612b640 1 -- 172.21.15.112:0/1837976733 shutdown_connections 2024-09-21T03:44:22.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.428+0000 7ff65612b640 1 -- 172.21.15.112:0/1837976733 wait complete. 2024-09-21T03:44:22.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff65612b640 1 Processor -- start 2024-09-21T03:44:22.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff65612b640 1 -- start start 2024-09-21T03:44:22.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff65612b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff650102980 0x7ff65019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff65612b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6501082b0 0x7ff65019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff65612b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650108c00 0x7ff65019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff65612b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff650112920 con 0x7ff650108c00 2024-09-21T03:44:22.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff65612b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6501127a0 con 0x7ff6501082b0 2024-09-21T03:44:22.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff65612b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff650112aa0 con 0x7ff650102980 2024-09-21T03:44:22.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff64ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650108c00 0x7ff65019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff64ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650108c00 0x7ff65019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45668/0 (socket says 172.21.15.112:45668) 2024-09-21T03:44:22.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff64ffff640 1 -- 172.21.15.112:0/2654724236 learned_addr learned my addr 172.21.15.112:0/2654724236 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:22.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.429+0000 7ff64effd640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6501082b0 0x7ff65019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff64f7fe640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff650102980 0x7ff65019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff64ffff640 1 -- 172.21.15.112:0/2654724236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff650102980 msgr2=0x7ff65019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff64ffff640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff650102980 0x7ff65019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff64ffff640 1 -- 172.21.15.112:0/2654724236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6501082b0 msgr2=0x7ff65019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff64ffff640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6501082b0 0x7ff65019b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff64ffff640 1 -- 172.21.15.112:0/2654724236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6501a6520 con 0x7ff650108c00 2024-09-21T03:44:22.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff64f7fe640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff650102980 0x7ff65019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:22.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff64ffff640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650108c00 0x7ff65019bc90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff64800b970 tx=0x7ff64800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff64cff9640 1 -- 172.21.15.112:0/2654724236 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648019070 con 0x7ff650108c00 2024-09-21T03:44:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff64cff9640 1 -- 172.21.15.112:0/2654724236 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff648014440 con 0x7ff650108c00 2024-09-21T03:44:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff65612b640 1 -- 172.21.15.112:0/2654724236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6501a6810 con 0x7ff650108c00 2024-09-21T03:44:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff64cff9640 1 -- 172.21.15.112:0/2654724236 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6480075d0 con 0x7ff650108c00 2024-09-21T03:44:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.430+0000 7ff65612b640 1 -- 172.21.15.112:0/2654724236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6501a6d10 con 0x7ff650108c00 2024-09-21T03:44:22.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.431+0000 7ff65612b640 1 -- 172.21.15.112:0/2654724236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff650102d90 con 0x7ff650108c00 2024-09-21T03:44:22.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.433+0000 7ff64cff9640 1 -- 172.21.15.112:0/2654724236 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff64801c070 con 0x7ff650108c00 2024-09-21T03:44:22.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.434+0000 7ff64cff9640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6380b6680 0x7ff6380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.434+0000 7ff64f7fe640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6380b6680 0x7ff6380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.434+0000 7ff64cff9640 1 -- 172.21.15.112:0/2654724236 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7ff6480e7040 con 0x7ff650108c00 2024-09-21T03:44:22.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.434+0000 7ff64f7fe640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6380b6680 0x7ff6380b8b90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff640000c00 tx=0x7ff640009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.437+0000 7ff64cff9640 1 -- 172.21.15.112:0/2654724236 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff6480b19a0 con 0x7ff650108c00 2024-09-21T03:44:22.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.620+0000 7ff65612b640 1 -- 172.21.15.112:0/2654724236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff65002e640 con 0x7ff650108c00 2024-09-21T03:44:22.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.622+0000 7ff64cff9640 1 -- 172.21.15.112:0/2654724236 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23156 (secure 0 0 0) 0x7ff6480b68e0 con 0x7ff650108c00 2024-09-21T03:44:22.624 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:44:22.625 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":96,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:44:14.915070+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:26:58.087349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:26:59.970444+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:22.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 -- 172.21.15.112:0/2654724236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6380b6680 msgr2=0x7ff6380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6380b6680 0x7ff6380b8b90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff640000c00 tx=0x7ff640009680 comp rx=0 tx=0).stop 2024-09-21T03:44:22.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 -- 172.21.15.112:0/2654724236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650108c00 msgr2=0x7ff65019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650108c00 0x7ff65019bc90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff64800b970 tx=0x7ff64800be30 comp rx=0 tx=0).stop 2024-09-21T03:44:22.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 -- 172.21.15.112:0/2654724236 shutdown_connections 2024-09-21T03:44:22.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff6380b6680 0x7ff6380b8b90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650108c00 0x7ff65019bc90 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6501082b0 0x7ff65019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 --2- 172.21.15.112:0/2654724236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff650102980 0x7ff65019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 -- 172.21.15.112:0/2654724236 >> 172.21.15.112:0/2654724236 conn(0x7ff6500fe380 msgr2=0x7ff650107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:22.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 -- 172.21.15.112:0/2654724236 shutdown_connections 2024-09-21T03:44:22.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.624+0000 7ff65612b640 1 -- 172.21.15.112:0/2654724236 wait complete. 2024-09-21T03:44:22.638 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:22:30.533177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T03:26:58.087349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T03:26:59.970444+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:44:22.639 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:44:22.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.749+0000 7ff6767a8640 1 -- 172.21.15.112:0/56401806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff670101fd0 msgr2=0x7ff6701023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.749+0000 7ff6767a8640 1 --2- 172.21.15.112:0/56401806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff670101fd0 0x7ff6701023e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff660009900 tx=0x7ff66002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:22.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.749+0000 7ff6767a8640 1 -- 172.21.15.112:0/56401806 shutdown_connections 2024-09-21T03:44:22.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.749+0000 7ff6767a8640 1 --2- 172.21.15.112:0/56401806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff670106b00 0x7ff670110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.749+0000 7ff6767a8640 1 --2- 172.21.15.112:0/56401806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6701061b0 0x7ff6701065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.749+0000 7ff6767a8640 1 --2- 172.21.15.112:0/56401806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff670101fd0 0x7ff6701023e0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.749+0000 7ff6767a8640 1 -- 172.21.15.112:0/56401806 >> 172.21.15.112:0/56401806 conn(0x7ff670079e00 msgr2=0x7ff67007a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:22.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.749+0000 7ff6767a8640 1 -- 172.21.15.112:0/56401806 shutdown_connections 2024-09-21T03:44:22.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.749+0000 7ff6767a8640 1 -- 172.21.15.112:0/56401806 wait complete. 2024-09-21T03:44:22.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff6767a8640 1 Processor -- start 2024-09-21T03:44:22.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff6767a8640 1 -- start start 2024-09-21T03:44:22.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff6767a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff670101fd0 0x7ff670102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff6767a8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6701061b0 0x7ff6701034a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff6767a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff670106b00 0x7ff6701039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff6767a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701139f0 con 0x7ff670106b00 2024-09-21T03:44:22.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff6767a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff670113870 con 0x7ff670101fd0 2024-09-21T03:44:22.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff6767a8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff670113b70 con 0x7ff6701061b0 2024-09-21T03:44:22.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff674d45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff670106b00 0x7ff6701039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff674d45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff670106b00 0x7ff6701039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45690/0 (socket says 172.21.15.112:45690) 2024-09-21T03:44:22.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff66ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff670101fd0 0x7ff670102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff674d45640 1 -- 172.21.15.112:0/3096033680 learned_addr learned my addr 172.21.15.112:0/3096033680 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:22.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff667fff640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6701061b0 0x7ff6701034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff674d45640 1 -- 172.21.15.112:0/3096033680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6701061b0 msgr2=0x7ff6701034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff674d45640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6701061b0 0x7ff6701034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff674d45640 1 -- 172.21.15.112:0/3096033680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff670101fd0 msgr2=0x7ff670102f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff674d45640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff670101fd0 0x7ff670102f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.750+0000 7ff674d45640 1 -- 172.21.15.112:0/3096033680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6701a8720 con 0x7ff670106b00 2024-09-21T03:44:22.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.751+0000 7ff674d45640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff670106b00 0x7ff6701039e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff66800eb10 tx=0x7ff66800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.751+0000 7ff66dffb640 1 -- 172.21.15.112:0/3096033680 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668004b20 con 0x7ff670106b00 2024-09-21T03:44:22.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.751+0000 7ff66dffb640 1 -- 172.21.15.112:0/3096033680 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff668004cc0 con 0x7ff670106b00 2024-09-21T03:44:22.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.751+0000 7ff6767a8640 1 -- 172.21.15.112:0/3096033680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6701a8a10 con 0x7ff670106b00 2024-09-21T03:44:22.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.751+0000 7ff66dffb640 1 -- 172.21.15.112:0/3096033680 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668010670 con 0x7ff670106b00 2024-09-21T03:44:22.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.751+0000 7ff6767a8640 1 -- 172.21.15.112:0/3096033680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6701a8f10 con 0x7ff670106b00 2024-09-21T03:44:22.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.752+0000 7ff6767a8640 1 -- 172.21.15.112:0/3096033680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6701023e0 con 0x7ff670106b00 2024-09-21T03:44:22.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.753+0000 7ff66dffb640 1 -- 172.21.15.112:0/3096033680 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff668010810 con 0x7ff670106b00 2024-09-21T03:44:22.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.754+0000 7ff66dffb640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff65c0b66b0 0x7ff65c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:22.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.754+0000 7ff66ffff640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff65c0b66b0 0x7ff65c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:22.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.754+0000 7ff66dffb640 1 -- 172.21.15.112:0/3096033680 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7ff6680e7e00 con 0x7ff670106b00 2024-09-21T03:44:22.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.755+0000 7ff66ffff640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff65c0b66b0 0x7ff65c0b8bc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff660009230 tx=0x7ff660031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:22.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.757+0000 7ff66dffb640 1 -- 172.21.15.112:0/3096033680 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff668056710 con 0x7ff670106b00 2024-09-21T03:44:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:22 smithi112 ceph-mon[31442]: pgmap v798: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.6 MiB/s wr, 189 op/s 2024-09-21T03:44:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:22 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:22 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:22.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:22 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:44:22.812 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:22 smithi203 ceph-mon[34723]: pgmap v798: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.6 MiB/s wr, 189 op/s 2024-09-21T03:44:22.812 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:22 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:22.812 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:22 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:22.813 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:22 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:44:22.831 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:22 smithi187 ceph-mon[34603]: pgmap v798: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.6 MiB/s wr, 189 op/s 2024-09-21T03:44:22.831 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:22 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:44:22.831 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:22 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:44:22.831 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:22 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' 2024-09-21T03:44:22.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.933+0000 7ff6767a8640 1 -- 172.21.15.112:0/3096033680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff67002e640 con 0x7ff670106b00 2024-09-21T03:44:22.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.934+0000 7ff66dffb640 1 -- 172.21.15.112:0/3096033680 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7ff6680ae380 con 0x7ff670106b00 2024-09-21T03:44:22.936 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 -- 172.21.15.112:0/3096033680 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff65c0b66b0 msgr2=0x7ff65c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff65c0b66b0 0x7ff65c0b8bc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff660009230 tx=0x7ff660031400 comp rx=0 tx=0).stop 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 -- 172.21.15.112:0/3096033680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff670106b00 msgr2=0x7ff6701039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:22.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff670106b00 0x7ff6701039e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff66800eb10 tx=0x7ff66800c2d0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 -- 172.21.15.112:0/3096033680 shutdown_connections 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7ff65c0b66b0 0x7ff65c0b8bc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff670106b00 0x7ff6701039e0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6701061b0 0x7ff6701034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 --2- 172.21.15.112:0/3096033680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff670101fd0 0x7ff670102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 -- 172.21.15.112:0/3096033680 >> 172.21.15.112:0/3096033680 conn(0x7ff670079e00 msgr2=0x7ff6701074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 -- 172.21.15.112:0/3096033680 shutdown_connections 2024-09-21T03:44:22.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:22.936+0000 7ff6767a8640 1 -- 172.21.15.112:0/3096033680 wait complete. 2024-09-21T03:44:22.949 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:44:23.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.059+0000 7f977bf57640 1 -- 172.21.15.112:0/1708728874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9774103ab0 msgr2=0x7f9774103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.059+0000 7f977bf57640 1 --2- 172.21.15.112:0/1708728874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9774103ab0 0x7f9774103ec0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9770009900 tx=0x7f977002a410 comp rx=0 tx=0).stop 2024-09-21T03:44:23.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.059+0000 7f977bf57640 1 -- 172.21.15.112:0/1708728874 shutdown_connections 2024-09-21T03:44:23.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.059+0000 7f977bf57640 1 --2- 172.21.15.112:0/1708728874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9774109d30 0x7f9774110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.059+0000 7f977bf57640 1 --2- 172.21.15.112:0/1708728874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97741093e0 0x7f97741097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.059+0000 7f977bf57640 1 --2- 172.21.15.112:0/1708728874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9774103ab0 0x7f9774103ec0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.059+0000 7f977bf57640 1 -- 172.21.15.112:0/1708728874 >> 172.21.15.112:0/1708728874 conn(0x7f97740ff4d0 msgr2=0x7f9774101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:23.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.060+0000 7f977bf57640 1 -- 172.21.15.112:0/1708728874 shutdown_connections 2024-09-21T03:44:23.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.060+0000 7f977bf57640 1 -- 172.21.15.112:0/1708728874 wait complete. 2024-09-21T03:44:23.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.060+0000 7f977bf57640 1 Processor -- start 2024-09-21T03:44:23.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.060+0000 7f977bf57640 1 -- start start 2024-09-21T03:44:23.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.060+0000 7f977bf57640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9774103ab0 0x7f977419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.060+0000 7f977bf57640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97741093e0 0x7f977419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f977bf57640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9774109d30 0x7f977419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f977bf57640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97741139f0 con 0x7f9774103ab0 2024-09-21T03:44:23.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f977bf57640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9774113870 con 0x7f9774109d30 2024-09-21T03:44:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f977bf57640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9774113b70 con 0x7f97741093e0 2024-09-21T03:44:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f9779cf3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9774103ab0 0x7f977419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f9779cf3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9774103ab0 0x7f977419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45698/0 (socket says 172.21.15.112:45698) 2024-09-21T03:44:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f9779cf3640 1 -- 172.21.15.112:0/152616924 learned_addr learned my addr 172.21.15.112:0/152616924 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f97794f2640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97741093e0 0x7f977419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f977a4f4640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9774109d30 0x7f977419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f9779cf3640 1 -- 172.21.15.112:0/152616924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97741093e0 msgr2=0x7f977419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f9779cf3640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97741093e0 0x7f977419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f9779cf3640 1 -- 172.21.15.112:0/152616924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9774109d30 msgr2=0x7f977419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f9779cf3640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9774109d30 0x7f977419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f9779cf3640 1 -- 172.21.15.112:0/152616924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97741a0360 con 0x7f9774103ab0 2024-09-21T03:44:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f9779cf3640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9774103ab0 0x7f977419a430 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9770005940 tx=0x7f9770004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f976affd640 1 -- 172.21.15.112:0/152616924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977003f030 con 0x7f9774103ab0 2024-09-21T03:44:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f977bf57640 1 -- 172.21.15.112:0/152616924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97741a05f0 con 0x7f9774103ab0 2024-09-21T03:44:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f976affd640 1 -- 172.21.15.112:0/152616924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9770003aa0 con 0x7f9774103ab0 2024-09-21T03:44:23.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f976affd640 1 -- 172.21.15.112:0/152616924 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770035070 con 0x7f9774103ab0 2024-09-21T03:44:23.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.061+0000 7f977bf57640 1 -- 172.21.15.112:0/152616924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97741a6de0 con 0x7f9774103ab0 2024-09-21T03:44:23.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.062+0000 7f977bf57640 1 -- 172.21.15.112:0/152616924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9774103ec0 con 0x7f9774103ab0 2024-09-21T03:44:23.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.064+0000 7f976affd640 1 -- 172.21.15.112:0/152616924 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9770042030 con 0x7f9774103ab0 2024-09-21T03:44:23.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.065+0000 7f976affd640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97600b6680 0x7f97600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.065+0000 7f97794f2640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97600b6680 0x7f97600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.065+0000 7f976affd640 1 -- 172.21.15.112:0/152616924 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f9770108040 con 0x7f9774103ab0 2024-09-21T03:44:23.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.065+0000 7f97794f2640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97600b6680 0x7f97600b8b90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9774108f80 tx=0x7f9764005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:23.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.068+0000 7f976affd640 1 -- 172.21.15.112:0/152616924 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f97700d29a0 con 0x7f9774103ab0 2024-09-21T03:44:23.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.245+0000 7f977bf57640 1 -- 172.21.15.112:0/152616924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f977410bb30 con 0x7f9774103ab0 2024-09-21T03:44:23.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.245+0000 7f976affd640 1 -- 172.21.15.112:0/152616924 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f97700d78e0 con 0x7f9774103ab0 2024-09-21T03:44:23.247 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-09-21T03:44:23.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 -- 172.21.15.112:0/152616924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97600b6680 msgr2=0x7f97600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97600b6680 0x7f97600b8b90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9774108f80 tx=0x7f9764005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:23.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 -- 172.21.15.112:0/152616924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9774103ab0 msgr2=0x7f977419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9774103ab0 0x7f977419a430 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9770005940 tx=0x7f9770004270 comp rx=0 tx=0).stop 2024-09-21T03:44:23.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 -- 172.21.15.112:0/152616924 shutdown_connections 2024-09-21T03:44:23.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f97600b6680 0x7f97600b8b90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9774109d30 0x7f977419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97741093e0 0x7f977419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 --2- 172.21.15.112:0/152616924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9774103ab0 0x7f977419a430 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 -- 172.21.15.112:0/152616924 >> 172.21.15.112:0/152616924 conn(0x7f97740ff4d0 msgr2=0x7f97741018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:23.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 -- 172.21.15.112:0/152616924 shutdown_connections 2024-09-21T03:44:23.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.247+0000 7f977bf57640 1 -- 172.21.15.112:0/152616924 wait complete. 2024-09-21T03:44:23.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:44:23.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.373+0000 7f7648ade640 1 -- 172.21.15.112:0/2907592315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441093e0 msgr2=0x7f76441097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.373+0000 7f7648ade640 1 --2- 172.21.15.112:0/2907592315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441093e0 0x7f76441097f0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f762c009900 tx=0x7f762c02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.373+0000 7f7648ade640 1 -- 172.21.15.112:0/2907592315 shutdown_connections 2024-09-21T03:44:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.373+0000 7f7648ade640 1 --2- 172.21.15.112:0/2907592315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7644109d30 0x7f7644110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.373+0000 7f7648ade640 1 --2- 172.21.15.112:0/2907592315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441093e0 0x7f76441097f0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.373+0000 7f7648ade640 1 --2- 172.21.15.112:0/2907592315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644103ab0 0x7f7644103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.373+0000 7f7648ade640 1 -- 172.21.15.112:0/2907592315 >> 172.21.15.112:0/2907592315 conn(0x7f76440ff4b0 msgr2=0x7f7644101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.373+0000 7f7648ade640 1 -- 172.21.15.112:0/2907592315 shutdown_connections 2024-09-21T03:44:23.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.373+0000 7f7648ade640 1 -- 172.21.15.112:0/2907592315 wait complete. 2024-09-21T03:44:23.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.374+0000 7f7648ade640 1 Processor -- start 2024-09-21T03:44:23.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.374+0000 7f7648ade640 1 -- start start 2024-09-21T03:44:23.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.374+0000 7f7648ade640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644103ab0 0x7f764419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.374+0000 7f7648ade640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441093e0 0x7f764419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.374+0000 7f7648ade640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7644109d30 0x7f764419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.374+0000 7f7648ade640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76441139f0 con 0x7f76441093e0 2024-09-21T03:44:23.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.374+0000 7f7648ade640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7644113870 con 0x7f7644109d30 2024-09-21T03:44:23.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.374+0000 7f7648ade640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7644113b70 con 0x7f7644103ab0 2024-09-21T03:44:23.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f763bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441093e0 0x7f764419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f763bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441093e0 0x7f764419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45712/0 (socket says 172.21.15.112:45712) 2024-09-21T03:44:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f763bfff640 1 -- 172.21.15.112:0/3703684286 learned_addr learned my addr 172.21.15.112:0/3703684286 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f7642d9d640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7644109d30 0x7f764419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f764259c640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644103ab0 0x7f764419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f763bfff640 1 -- 172.21.15.112:0/3703684286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644103ab0 msgr2=0x7f764419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f763bfff640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644103ab0 0x7f764419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f763bfff640 1 -- 172.21.15.112:0/3703684286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7644109d30 msgr2=0x7f764419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f763bfff640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7644109d30 0x7f764419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f763bfff640 1 -- 172.21.15.112:0/3703684286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76441a0340 con 0x7f76441093e0 2024-09-21T03:44:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f763bfff640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441093e0 0x7f764419a9b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f762c02a930 tx=0x7f762c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.375+0000 7f763b7fe640 1 -- 172.21.15.112:0/3703684286 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c03f030 con 0x7f76441093e0 2024-09-21T03:44:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.376+0000 7f7648ade640 1 -- 172.21.15.112:0/3703684286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76441a05d0 con 0x7f76441093e0 2024-09-21T03:44:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.376+0000 7f763b7fe640 1 -- 172.21.15.112:0/3703684286 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f762c004490 con 0x7f76441093e0 2024-09-21T03:44:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.376+0000 7f763b7fe640 1 -- 172.21.15.112:0/3703684286 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c035070 con 0x7f76441093e0 2024-09-21T03:44:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.376+0000 7f7648ade640 1 -- 172.21.15.112:0/3703684286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76440755f0 con 0x7f76441093e0 2024-09-21T03:44:23.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.377+0000 7f7648ade640 1 -- 172.21.15.112:0/3703684286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7644104d20 con 0x7f76441093e0 2024-09-21T03:44:23.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.379+0000 7f763b7fe640 1 -- 172.21.15.112:0/3703684286 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f762c042030 con 0x7f76441093e0 2024-09-21T03:44:23.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.380+0000 7f763b7fe640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f76300b6680 0x7f76300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.380+0000 7f764259c640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f76300b6680 0x7f76300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.380+0000 7f763b7fe640 1 -- 172.21.15.112:0/3703684286 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f762c107db0 con 0x7f76441093e0 2024-09-21T03:44:23.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.380+0000 7f764259c640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f76300b6680 0x7f76300b8b90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7634010d80 tx=0x7f7634012040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:23.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.384+0000 7f763b7fe640 1 -- 172.21.15.112:0/3703684286 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f762c0d2710 con 0x7f76441093e0 2024-09-21T03:44:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.589+0000 7f7648ade640 1 -- 172.21.15.112:0/3703684286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7644103ec0 con 0x7f76441093e0 2024-09-21T03:44:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.589+0000 7f763b7fe640 1 -- 172.21.15.112:0/3703684286 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f762c0d7650 con 0x7f76441093e0 2024-09-21T03:44:23.592 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-09-21T03:44:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.591+0000 7f7648ade640 1 -- 172.21.15.112:0/3703684286 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f76300b6680 msgr2=0x7f76300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.591+0000 7f7648ade640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f76300b6680 0x7f76300b8b90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7634010d80 tx=0x7f7634012040 comp rx=0 tx=0).stop 2024-09-21T03:44:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.591+0000 7f7648ade640 1 -- 172.21.15.112:0/3703684286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441093e0 msgr2=0x7f764419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.591+0000 7f7648ade640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441093e0 0x7f764419a9b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f762c02a930 tx=0x7f762c031a20 comp rx=0 tx=0).stop 2024-09-21T03:44:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.591+0000 7f7648ade640 1 -- 172.21.15.112:0/3703684286 shutdown_connections 2024-09-21T03:44:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.591+0000 7f7648ade640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f76300b6680 0x7f76300b8b90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.591+0000 7f7648ade640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7644109d30 0x7f764419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.591+0000 7f7648ade640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441093e0 0x7f764419a9b0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.591+0000 7f7648ade640 1 --2- 172.21.15.112:0/3703684286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644103ab0 0x7f764419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.591+0000 7f7648ade640 1 -- 172.21.15.112:0/3703684286 >> 172.21.15.112:0/3703684286 conn(0x7f76440ff4b0 msgr2=0x7f7644106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:23.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.592+0000 7f7648ade640 1 -- 172.21.15.112:0/3703684286 shutdown_connections 2024-09-21T03:44:23.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.592+0000 7f7648ade640 1 -- 172.21.15.112:0/3703684286 wait complete. 2024-09-21T03:44:23.605 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T03:44:23.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.711+0000 7fc3e2928640 1 -- 172.21.15.112:0/2082707015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3dc102980 msgr2=0x7fc3dc102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.711+0000 7fc3e2928640 1 --2- 172.21.15.112:0/2082707015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3dc102980 0x7fc3dc102d90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009900 tx=0x7fc3cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:23.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.711+0000 7fc3e2928640 1 -- 172.21.15.112:0/2082707015 shutdown_connections 2024-09-21T03:44:23.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.711+0000 7fc3e2928640 1 --2- 172.21.15.112:0/2082707015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3dc108c00 0x7fc3dc10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.711+0000 7fc3e2928640 1 --2- 172.21.15.112:0/2082707015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3dc1082b0 0x7fc3dc1086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.711+0000 7fc3e2928640 1 --2- 172.21.15.112:0/2082707015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3dc102980 0x7fc3dc102d90 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.711+0000 7fc3e2928640 1 -- 172.21.15.112:0/2082707015 >> 172.21.15.112:0/2082707015 conn(0x7fc3dc0fe380 msgr2=0x7fc3dc1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:23.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.711+0000 7fc3e2928640 1 -- 172.21.15.112:0/2082707015 shutdown_connections 2024-09-21T03:44:23.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.711+0000 7fc3e2928640 1 -- 172.21.15.112:0/2082707015 wait complete. 2024-09-21T03:44:23.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3e2928640 1 Processor -- start 2024-09-21T03:44:23.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3e2928640 1 -- start start 2024-09-21T03:44:23.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3e2928640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3dc102980 0x7fc3dc199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3e2928640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3dc1082b0 0x7fc3dc199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3e2928640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3dc108c00 0x7fc3dc19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3e2928640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1128c0 con 0x7fc3dc102980 2024-09-21T03:44:23.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3e2928640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc112740 con 0x7fc3dc1082b0 2024-09-21T03:44:23.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3e2928640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc112a40 con 0x7fc3dc108c00 2024-09-21T03:44:23.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3dc102980 0x7fc3dc199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3dc102980 0x7fc3dc199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45728/0 (socket says 172.21.15.112:45728) 2024-09-21T03:44:23.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3dbfff640 1 -- 172.21.15.112:0/3426783166 learned_addr learned my addr 172.21.15.112:0/3426783166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:23.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3db7fe640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3dc1082b0 0x7fc3dc199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.712+0000 7fc3e0ec5640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3dc108c00 0x7fc3dc19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.713+0000 7fc3dbfff640 1 -- 172.21.15.112:0/3426783166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3dc108c00 msgr2=0x7fc3dc19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.713+0000 7fc3dbfff640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3dc108c00 0x7fc3dc19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.713+0000 7fc3dbfff640 1 -- 172.21.15.112:0/3426783166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3dc1082b0 msgr2=0x7fc3dc199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:23.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.713+0000 7fc3dbfff640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3dc1082b0 0x7fc3dc199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:23.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.713+0000 7fc3dbfff640 1 -- 172.21.15.112:0/3426783166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc19f2b0 con 0x7fc3dc102980 2024-09-21T03:44:23.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.713+0000 7fc3dbfff640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3dc102980 0x7fc3dc199380 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc005940 tx=0x7fc3cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:23.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.713+0000 7fc3d97fa640 1 -- 172.21.15.112:0/3426783166 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc03f030 con 0x7fc3dc102980 2024-09-21T03:44:23.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.713+0000 7fc3e2928640 1 -- 172.21.15.112:0/3426783166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3dc19f540 con 0x7fc3dc102980 2024-09-21T03:44:23.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.713+0000 7fc3d97fa640 1 -- 172.21.15.112:0/3426783166 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3cc003aa0 con 0x7fc3dc102980 2024-09-21T03:44:23.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.713+0000 7fc3d97fa640 1 -- 172.21.15.112:0/3426783166 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc035070 con 0x7fc3dc102980 2024-09-21T03:44:23.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.713+0000 7fc3e2928640 1 -- 172.21.15.112:0/3426783166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3dc1a5d30 con 0x7fc3dc102980 2024-09-21T03:44:23.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.714+0000 7fc3e2928640 1 -- 172.21.15.112:0/3426783166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3dc103bf0 con 0x7fc3dc102980 2024-09-21T03:44:23.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.716+0000 7fc3d97fa640 1 -- 172.21.15.112:0/3426783166 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc3cc042030 con 0x7fc3dc102980 2024-09-21T03:44:23.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.717+0000 7fc3d97fa640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3c80b6680 0x7fc3c80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.717+0000 7fc3db7fe640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3c80b6680 0x7fc3c80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.717+0000 7fc3d97fa640 1 -- 172.21.15.112:0/3426783166 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7fc3cc108060 con 0x7fc3dc102980 2024-09-21T03:44:23.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.717+0000 7fc3db7fe640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3c80b6680 0x7fc3c80b8b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc101a80 tx=0x7fc3c400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:23.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.720+0000 7fc3d97fa640 1 -- 172.21.15.112:0/3426783166 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3cc0d29c0 con 0x7fc3dc102980 2024-09-21T03:44:23.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:23 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:44:23.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:23 smithi203 ceph-mon[34723]: pgmap v799: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.6 MiB/s wr, 230 op/s 2024-09-21T03:44:23.911 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:23 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/2654724236' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:23.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:23 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3096033680' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:44:23.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:23 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/152616924' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:44:23.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:23 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:44:23.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:23 smithi187 ceph-mon[34603]: pgmap v799: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.6 MiB/s wr, 230 op/s 2024-09-21T03:44:23.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:23 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/2654724236' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:23.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:23 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3096033680' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:44:23.930 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:23 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/152616924' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:44:23.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.943+0000 7fc3e2928640 1 -- 172.21.15.112:0/3426783166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fc3dc10a440 con 0x7fc3dc102980 2024-09-21T03:44:23.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.951+0000 7fc3d97fa640 1 -- 172.21.15.112:0/3426783166 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+834061 (secure 0 0 0) 0x7fc3cc0d7900 con 0x7fc3dc102980 2024-09-21T03:44:23.964 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:44:23.964 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:44:23.964 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 24, 2024-09-21T03:44:23.964 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:23.964 INFO:teuthology.orchestra.run.smithi112.stdout: "active_gid": 14226, 2024-09-21T03:44:23.965 INFO:teuthology.orchestra.run.smithi112.stdout: "active_name": "x", 2024-09-21T03:44:23.965 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addrs": { 2024-09-21T03:44:23.965 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-09-21T03:44:23.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:23.965 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-09-21T03:44:23.965 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6800", 2024-09-21T03:44:23.965 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 4024464582 2024-09-21T03:44:23.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.966 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:23.966 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-09-21T03:44:23.966 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6801", 2024-09-21T03:44:23.966 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 4024464582 2024-09-21T03:44:23.966 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:23.966 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:44:23.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.966 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addr": "172.21.15.112:6801/4024464582", 2024-09-21T03:44:23.966 INFO:teuthology.orchestra.run.smithi112.stdout: "active_change": "2024-09-21T03:20:32.227422+0000", 2024-09-21T03:44:23.967 INFO:teuthology.orchestra.run.smithi112.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T03:44:23.967 INFO:teuthology.orchestra.run.smithi112.stdout: "available": true, 2024-09-21T03:44:23.967 INFO:teuthology.orchestra.run.smithi112.stdout: "standbys": [ 2024-09-21T03:44:23.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:23.967 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 24145, 2024-09-21T03:44:23.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "z", 2024-09-21T03:44:23.967 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:44:23.967 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-09-21T03:44:23.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:23.968 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-09-21T03:44:23.968 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:23.968 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:23.968 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:23.968 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:23.968 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:23.968 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:23.968 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.969 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.969 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:23.969 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.969 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.969 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.969 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:44:23.969 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.969 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.969 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.970 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:23.970 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:23.970 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:23.970 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.970 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.970 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:23.971 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.971 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.971 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:23.971 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:23.971 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:23.971 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:23.971 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:23.971 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:23.971 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:23.972 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:23.972 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:23.972 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.972 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.972 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.972 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:23.972 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:23.973 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:23.973 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.973 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.973 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:23.973 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.973 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.973 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.973 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:23.973 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.974 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.974 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.974 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:23.974 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:23.974 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:23.974 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.974 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.974 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:23.975 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.975 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.975 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:23.975 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:23.975 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:23.975 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:23.975 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:23.975 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:23.975 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:23.976 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:23.976 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:23.976 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.976 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.976 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.976 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:23.976 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:23.976 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:23.977 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.977 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.977 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:23.977 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.977 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.977 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.977 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:23.977 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.977 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.978 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.978 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-09-21T03:44:23.978 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-09-21T03:44:23.978 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:23.978 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.978 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.978 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:23.978 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.979 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.979 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.979 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:44:23.979 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.979 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.979 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.979 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-09-21T03:44:23.980 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-09-21T03:44:23.980 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:23.980 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.980 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.980 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-09-21T03:44:23.980 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.980 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.980 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.980 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-09-21T03:44:23.981 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.981 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.981 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.981 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-09-21T03:44:23.981 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-09-21T03:44:23.981 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:23.981 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.981 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.982 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:23.982 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.982 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.982 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.982 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-09-21T03:44:23.982 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.982 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.982 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.983 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-09-21T03:44:23.983 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-09-21T03:44:23.983 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:23.983 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.983 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:23.983 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.983 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.984 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.984 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-09-21T03:44:23.984 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.984 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.984 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.984 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-09-21T03:44:23.984 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-09-21T03:44:23.984 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:23.985 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.985 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.985 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-09-21T03:44:23.985 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.985 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.985 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.985 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-09-21T03:44:23.985 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.986 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.986 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-09-21T03:44:23.986 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-09-21T03:44:23.986 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:23.986 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.986 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.986 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:23.987 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.987 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.987 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.987 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:44:23.987 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.987 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.987 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.988 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-09-21T03:44:23.988 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-09-21T03:44:23.988 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:23.988 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.988 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.988 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:23.988 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.988 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.988 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.989 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:44:23.989 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.989 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.989 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.989 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-09-21T03:44:23.989 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-09-21T03:44:23.989 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:23.990 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.990 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.990 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:23.990 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.990 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.990 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.990 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-09-21T03:44:23.990 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.991 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.991 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.991 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:23.991 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:23.991 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:23.991 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:23.991 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.992 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:23.992 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.992 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.992 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.992 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:23.992 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.992 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.992 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.993 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:23.993 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:23.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:23.993 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-09-21T03:44:23.993 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:23.993 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:23.993 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:23.993 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-09-21T03:44:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-09-21T03:44:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:44:23.995 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:23.995 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.995 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.995 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-09-21T03:44:23.995 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-09-21T03:44:23.995 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:23.995 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.996 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.996 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-09-21T03:44:23.996 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:23.996 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:23.996 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.996 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:44:23.996 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:44:23.996 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.996 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.997 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-09-21T03:44:23.997 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-09-21T03:44:23.997 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:23.997 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.997 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.997 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:23.997 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-09-21T03:44:23.998 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-09-21T03:44:23.998 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.998 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:44:23.998 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:44:23.998 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:23.998 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:23.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:23.998 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-09-21T03:44:23.998 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:44:23.999 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:23.999 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:23.999 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:23.999 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-09-21T03:44:23.999 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:23.999 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-09-21T03:44:23.999 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:23.999 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:44:24.000 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.000 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.000 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.000 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-09-21T03:44:24.000 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-09-21T03:44:24.000 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.000 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.000 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:44:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:44:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:44:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.002 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-09-21T03:44:24.002 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-09-21T03:44:24.002 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:24.002 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.002 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.002 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-09-21T03:44:24.002 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-09-21T03:44:24.003 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-09-21T03:44:24.003 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.003 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:44:24.003 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:44:24.003 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.003 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.003 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-09-21T03:44:24.003 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-09-21T03:44:24.004 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.004 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.004 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-09-21T03:44:24.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.004 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.004 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.004 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:44:24.004 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:44:24.005 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.005 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.005 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-09-21T03:44:24.005 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-09-21T03:44:24.005 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:24.005 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.005 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.006 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.006 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-09-21T03:44:24.006 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-09-21T03:44:24.006 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.006 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:44:24.006 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:44:24.006 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.006 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.007 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.007 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.007 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.007 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.007 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.007 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.007 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.008 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.008 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.008 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.008 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.008 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.008 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.008 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.008 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.009 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.009 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.009 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.009 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.009 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.009 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.009 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.010 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.010 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.010 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.010 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.010 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.010 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.010 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.010 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.011 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.011 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.011 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.011 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.011 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.011 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.011 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.012 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.012 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.012 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.012 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.012 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.012 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.012 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.012 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.012 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.013 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.013 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.013 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.013 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.013 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.013 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.013 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.014 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.014 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.014 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.014 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.014 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.014 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.014 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.014 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.014 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.015 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.015 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-09-21T03:44:24.015 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-09-21T03:44:24.015 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:24.015 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.015 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.015 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.015 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.016 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.016 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.016 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:44:24.016 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.016 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.016 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.016 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-09-21T03:44:24.017 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-09-21T03:44:24.017 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.017 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.017 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.017 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-09-21T03:44:24.017 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.017 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.017 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.017 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-09-21T03:44:24.018 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-09-21T03:44:24.018 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-09-21T03:44:24.018 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-09-21T03:44:24.018 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-09-21T03:44:24.018 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.018 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-09-21T03:44:24.018 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.018 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.019 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.019 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-09-21T03:44:24.019 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-09-21T03:44:24.019 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.019 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.019 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.019 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.020 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.020 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.020 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.020 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:44:24.020 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.020 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.020 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.021 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:24.021 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:24.021 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.021 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.021 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.021 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:24.021 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.021 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.021 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.022 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:44:24.022 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.022 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.022 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.022 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.022 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.022 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.022 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.023 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.023 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.023 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.023 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.023 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.023 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.023 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.023 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.024 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.024 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-09-21T03:44:24.024 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-09-21T03:44:24.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.024 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.024 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.024 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:24.024 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:24.025 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.025 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.025 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:44:24.025 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:44:24.025 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.025 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.025 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-09-21T03:44:24.026 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:44:24.026 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:24.026 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.026 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.026 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.026 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.026 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.026 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.026 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:44:24.027 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.027 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.027 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.027 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.027 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.027 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.027 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-09-21T03:44:24.028 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.028 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.028 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.028 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-09-21T03:44:24.028 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-09-21T03:44:24.028 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:24.028 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.028 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.028 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-09-21T03:44:24.029 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.029 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.029 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.029 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:44:24.029 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.029 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.029 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.030 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-09-21T03:44:24.030 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-09-21T03:44:24.030 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.030 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.031 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.031 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-09-21T03:44:24.031 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.031 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.031 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.031 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:44:24.031 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.031 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.031 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.032 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-09-21T03:44:24.032 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-09-21T03:44:24.032 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.032 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.032 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.032 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-09-21T03:44:24.032 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.032 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.033 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.033 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:44:24.033 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.033 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.033 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.033 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-09-21T03:44:24.033 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-09-21T03:44:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:44:24.034 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:44:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-09-21T03:44:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-09-21T03:44:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.035 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:44:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:44:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:44:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-09-21T03:44:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.037 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:44:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-09-21T03:44:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-09-21T03:44:24.038 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.039 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-09-21T03:44:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-09-21T03:44:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-09-21T03:44:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:44:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.040 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-09-21T03:44:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-09-21T03:44:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.041 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:44:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-09-21T03:44:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-09-21T03:44:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.042 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:44:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-09-21T03:44:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-09-21T03:44:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.044 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:44:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-09-21T03:44:24.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-09-21T03:44:24.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.046 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.046 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.046 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:44:24.046 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.046 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.046 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.046 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:24.047 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.047 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.047 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.047 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-09-21T03:44:24.047 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-09-21T03:44:24.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.047 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.047 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.048 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:44:24.048 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.048 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.048 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.048 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:44:24.048 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.048 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.048 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.049 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-09-21T03:44:24.049 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:44:24.049 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.049 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.049 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:44:24.049 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.049 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.050 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.050 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-09-21T03:44:24.050 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.050 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.050 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.050 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-09-21T03:44:24.050 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-09-21T03:44:24.051 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.051 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.051 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.051 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:44:24.051 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.051 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.051 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.051 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:24.052 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.052 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.052 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.052 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-09-21T03:44:24.052 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-09-21T03:44:24.052 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.052 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.052 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.053 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:44:24.053 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.053 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.053 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.053 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-09-21T03:44:24.053 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.053 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.053 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.054 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-09-21T03:44:24.054 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:44:24.054 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.054 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.054 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.054 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:44:24.054 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:44:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-09-21T03:44:24.055 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:44:24.056 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.056 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.056 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.056 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:44:24.056 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.056 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.056 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.056 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:44:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-09-21T03:44:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:44:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.057 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.058 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:44:24.058 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.058 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.058 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.058 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-09-21T03:44:24.058 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.058 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.058 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.059 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-09-21T03:44:24.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-09-21T03:44:24.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.059 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.059 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.059 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:44:24.059 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.059 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.060 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.060 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-09-21T03:44:24.060 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.060 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.060 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-09-21T03:44:24.060 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-09-21T03:44:24.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.061 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.061 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.061 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:44:24.061 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.061 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.061 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.061 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-09-21T03:44:24.061 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.062 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.062 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.062 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nginx": { 2024-09-21T03:44:24.062 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nginx", 2024-09-21T03:44:24.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.062 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.062 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.063 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:44:24.063 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.063 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.063 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.063 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nginx container image", 2024-09-21T03:44:24.063 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.063 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.063 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.064 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-09-21T03:44:24.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-09-21T03:44:24.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.064 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.064 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.064 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:44:24.064 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.064 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.065 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.065 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:24.065 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.065 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.065 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.065 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-09-21T03:44:24.065 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-09-21T03:44:24.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.066 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.066 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:44:24.066 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.066 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.066 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.066 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-09-21T03:44:24.066 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.067 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.067 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.067 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:44:24.067 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:44:24.067 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.067 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.067 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:44:24.068 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.068 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.068 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.068 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:44:24.068 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.068 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.068 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.068 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-09-21T03:44:24.069 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-09-21T03:44:24.069 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.069 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.069 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.069 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:44:24.069 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.069 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.069 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.070 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:24.070 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.070 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.070 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.070 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-09-21T03:44:24.070 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-09-21T03:44:24.070 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.070 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.071 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.071 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:44:24.071 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.071 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.071 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.071 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-09-21T03:44:24.071 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.071 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.071 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.072 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-09-21T03:44:24.072 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-09-21T03:44:24.072 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.072 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.072 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.072 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:44:24.072 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.073 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.073 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.073 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:44:24.073 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.073 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.073 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.073 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba_metrics": { 2024-09-21T03:44:24.073 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:44:24.074 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.074 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.074 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.074 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:44:24.074 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.074 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.074 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.074 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:44:24.074 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.075 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.075 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.075 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-09-21T03:44:24.075 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:44:24.075 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.075 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.075 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.076 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:44:24.076 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.076 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.076 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.076 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:44:24.076 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.076 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.076 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.077 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-09-21T03:44:24.077 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-09-21T03:44:24.077 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.077 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.077 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.077 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.077 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.078 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:44:24.078 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.078 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.078 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.078 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-09-21T03:44:24.078 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:44:24.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.079 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.079 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.079 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:24.079 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.079 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.079 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.079 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:44:24.079 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.079 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.080 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.080 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:44:24.080 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:44:24.080 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.080 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.080 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.080 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-09-21T03:44:24.080 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.081 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.081 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.081 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:44:24.081 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.081 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.081 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.081 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-09-21T03:44:24.082 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-09-21T03:44:24.082 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.082 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.082 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.082 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-09-21T03:44:24.082 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.082 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.082 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.082 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:44:24.083 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.083 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.083 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.083 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-09-21T03:44:24.083 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-09-21T03:44:24.083 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.083 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.083 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.084 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-09-21T03:44:24.084 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.084 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.084 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.084 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:44:24.084 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.084 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.084 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.085 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-09-21T03:44:24.085 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-09-21T03:44:24.085 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.085 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.085 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.085 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.085 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.085 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.086 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.086 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:44:24.086 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.086 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.086 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.086 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-09-21T03:44:24.086 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-09-21T03:44:24.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.087 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.087 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.087 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:24.087 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.087 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.087 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.087 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:44:24.087 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.088 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.088 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.088 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-09-21T03:44:24.088 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:44:24.088 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.088 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.088 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.088 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:44:24.089 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.089 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.089 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.089 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:44:24.089 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.089 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.089 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.090 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-09-21T03:44:24.090 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-09-21T03:44:24.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.090 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.090 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:24.090 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.090 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.090 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.091 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:44:24.091 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.091 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.091 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.091 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-09-21T03:44:24.091 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-09-21T03:44:24.091 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.091 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.092 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.092 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.092 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.092 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.092 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:44:24.092 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.092 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.093 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.093 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-09-21T03:44:24.093 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-09-21T03:44:24.093 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.093 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.093 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.093 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.093 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.094 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.094 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.094 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:44:24.094 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.094 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.094 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.094 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.094 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.095 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.095 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.095 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.095 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.095 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.095 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.095 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.095 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.096 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.096 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.096 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.096 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.096 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.096 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.096 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.096 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.096 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.097 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.097 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-09-21T03:44:24.097 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-09-21T03:44:24.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.097 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.097 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.097 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.098 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.098 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.098 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:44:24.098 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.098 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.098 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.098 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.099 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.099 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.099 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.099 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.099 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.099 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.099 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.099 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.099 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:44:24.100 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.100 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.100 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.100 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.100 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.100 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.100 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.101 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.101 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.101 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.101 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.101 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.101 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.101 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.101 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.101 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.102 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.102 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.102 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.102 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.102 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.102 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.102 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.103 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.103 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.103 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.103 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.103 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.103 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.103 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.103 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.103 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.104 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.104 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.104 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.104 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.104 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:44:24.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:44:24.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.105 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.105 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.105 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.105 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.105 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.105 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.105 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:44:24.105 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.105 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.106 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.106 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:44:24.106 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:44:24.106 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.106 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.106 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.106 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-09-21T03:44:24.107 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.107 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.107 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.107 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:44:24.107 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.107 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.107 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.107 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-09-21T03:44:24.108 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-09-21T03:44:24.108 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.108 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.108 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.108 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.108 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.108 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.108 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.109 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:44:24.109 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.109 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.109 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.109 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-09-21T03:44:24.109 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-09-21T03:44:24.109 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.109 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.110 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.110 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.110 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.110 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.110 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:44:24.110 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.110 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.111 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.111 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-09-21T03:44:24.111 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-09-21T03:44:24.111 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.111 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.111 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.111 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.111 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.112 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.112 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.112 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-09-21T03:44:24.112 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.112 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.112 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.112 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-09-21T03:44:24.112 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-09-21T03:44:24.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.113 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.113 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.113 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-09-21T03:44:24.113 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.113 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.113 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.113 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-09-21T03:44:24.114 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-09-21T03:44:24.114 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.114 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:44:24.114 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.114 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.114 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.114 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-09-21T03:44:24.114 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-09-21T03:44:24.115 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.115 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.115 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.115 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-09-21T03:44:24.115 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.115 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.115 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.115 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:44:24.115 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.116 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.116 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.116 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-09-21T03:44:24.116 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:44:24.116 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.116 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.117 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.117 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:44:24.117 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.117 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.117 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.117 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:44:24.117 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.117 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.117 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.118 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-09-21T03:44:24.118 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-09-21T03:44:24.118 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.118 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.118 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.118 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.118 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.119 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.119 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.119 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:44:24.119 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.119 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.119 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.119 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-09-21T03:44:24.120 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-09-21T03:44:24.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.120 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.120 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.120 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.120 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.120 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.120 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.120 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:44:24.121 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.121 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.121 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.121 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-09-21T03:44:24.121 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-09-21T03:44:24.121 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.121 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.122 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.122 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.122 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.122 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.122 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.122 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:44:24.122 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.122 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.122 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.123 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-09-21T03:44:24.123 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-09-21T03:44:24.123 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.123 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.123 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.123 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.123 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.123 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.124 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.124 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:44:24.124 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.124 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.124 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.124 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-09-21T03:44:24.124 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:44:24.124 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:44:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-09-21T03:44:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-09-21T03:44:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-09-21T03:44:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:44:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.129 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.129 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.129 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.129 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-09-21T03:44:24.129 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-09-21T03:44:24.129 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.129 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.129 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.130 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.130 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.130 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.130 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.130 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:44:24.130 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.130 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.130 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.131 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:44:24.131 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:44:24.131 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.131 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.131 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-09-21T03:44:24.131 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.132 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.132 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.132 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:44:24.132 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.132 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.132 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.132 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_keepalive_interval": { 2024-09-21T03:44:24.132 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:44:24.133 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.133 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.133 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-09-21T03:44:24.133 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.133 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.133 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.133 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:44:24.133 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.134 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.134 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.134 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-09-21T03:44:24.134 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-09-21T03:44:24.134 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.134 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.134 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.135 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.135 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.135 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.135 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.135 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:44:24.135 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.135 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.135 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.136 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-09-21T03:44:24.136 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-09-21T03:44:24.136 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.136 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.136 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.136 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.136 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.136 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.137 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.137 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:44:24.137 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.137 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.137 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.137 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-09-21T03:44:24.137 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:44:24.138 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.138 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.138 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.138 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.138 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.138 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.138 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.138 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:44:24.138 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.139 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.139 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.139 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-09-21T03:44:24.139 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:44:24.139 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.139 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.139 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.139 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.140 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.140 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.140 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.140 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:44:24.140 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.140 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.140 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.141 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-09-21T03:44:24.141 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:44:24.141 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.141 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.141 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.141 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.141 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.142 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.142 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.142 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:44:24.142 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.142 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.142 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.142 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.142 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.143 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-09-21T03:44:24.143 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.143 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.143 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.143 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.143 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.143 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.144 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.144 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.144 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.144 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.144 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.144 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.144 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.145 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.145 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.145 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.145 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.145 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.145 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.145 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.145 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.146 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.146 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.146 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.146 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.146 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.146 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.146 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.146 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.147 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.147 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.147 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.147 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.147 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.147 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.148 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.148 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.148 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.148 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.148 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.148 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.148 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.148 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.149 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.149 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.149 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.149 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.149 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.149 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.149 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.149 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.150 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.150 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.150 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.150 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.150 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.150 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.150 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.151 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.151 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.151 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.151 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.151 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.151 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-09-21T03:44:24.151 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-09-21T03:44:24.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.152 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.152 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.152 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-09-21T03:44:24.152 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.152 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.152 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.152 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:44:24.152 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.153 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.153 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.153 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.153 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.153 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.153 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.154 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.154 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.154 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.154 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.154 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.154 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.155 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-09-21T03:44:24.155 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-09-21T03:44:24.155 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.155 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.155 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.155 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-09-21T03:44:24.155 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.155 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.155 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.156 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:44:24.156 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.156 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.156 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.156 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.156 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.156 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-09-21T03:44:24.157 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.157 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.157 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.157 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:44:24.157 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:44:24.157 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.157 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.157 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.158 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.158 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.158 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.158 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.158 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.158 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.158 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.158 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.159 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:44:24.159 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:44:24.159 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.159 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.159 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.159 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.159 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.159 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.159 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.160 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.160 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.160 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.160 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.160 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:44:24.160 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:44:24.160 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.160 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.161 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.161 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.161 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.161 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.161 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.161 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.161 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.161 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.162 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.162 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:44:24.162 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:44:24.162 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.162 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.162 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.162 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.163 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.163 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.163 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.163 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.163 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.163 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.163 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:44:24.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:44:24.164 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.164 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.164 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.164 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.164 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.164 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.164 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.164 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.165 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.165 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.165 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.165 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:44:24.165 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:44:24.165 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.165 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.165 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.166 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.166 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.166 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.166 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.166 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.166 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.166 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.166 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.167 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:44:24.167 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:44:24.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.167 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.167 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.167 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.167 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.168 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.168 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.168 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.168 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.168 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.168 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:44:24.168 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:44:24.169 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.169 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.169 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.169 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.169 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.169 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.169 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.169 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.169 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.170 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.170 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.170 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:44:24.170 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:44:24.170 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.170 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.170 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.171 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.171 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.171 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.171 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.171 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.171 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.171 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.172 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.172 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:44:24.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:44:24.172 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.172 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.172 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.172 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.173 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.173 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.173 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.173 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.173 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.173 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.173 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.173 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:44:24.174 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:44:24.174 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.174 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.174 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.174 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.174 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.174 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.174 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.175 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.175 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.175 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.175 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.175 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:44:24.175 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:44:24.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.175 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.176 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:44:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:44:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.177 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.178 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.178 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.178 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.178 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.178 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.178 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.178 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:44:24.179 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:44:24.179 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.179 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.179 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.179 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.179 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.179 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.179 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.180 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.180 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.180 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.180 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:44:24.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:44:24.180 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.180 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.180 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.181 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-09-21T03:44:24.181 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.181 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.181 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.181 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.181 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.181 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.181 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.182 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:44:24.182 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:44:24.182 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.182 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.182 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.182 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.182 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.182 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.183 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.183 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.183 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.183 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.183 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.183 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-09-21T03:44:24.183 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:44:24.183 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.184 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.184 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.184 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.184 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.184 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.184 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.184 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.184 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.185 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.185 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.185 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:44:24.185 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:44:24.185 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.185 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.185 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-09-21T03:44:24.186 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.186 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.186 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.186 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.186 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.186 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.186 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.186 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:44:24.187 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:44:24.187 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.187 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.187 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.187 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.187 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.187 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.187 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.188 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.188 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.188 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.188 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.188 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:44:24.188 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:44:24.188 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.188 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.189 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.189 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.189 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.189 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.189 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.189 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.189 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.189 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.189 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.190 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:44:24.190 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:44:24.190 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.190 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.190 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.190 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.190 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.190 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.191 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.191 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.191 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.191 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.191 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.191 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:44:24.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:44:24.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.192 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.192 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.192 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.192 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.192 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.192 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.192 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.193 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.193 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:44:24.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:44:24.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.193 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.194 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-09-21T03:44:24.194 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.194 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.194 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.194 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.194 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.194 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.194 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.195 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:44:24.195 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:44:24.195 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.195 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.195 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.195 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-09-21T03:44:24.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.196 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.196 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.196 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.196 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.196 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.196 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.196 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:44:24.196 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:44:24.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.197 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.197 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.197 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.197 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.197 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.197 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.198 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.198 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.198 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.198 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:44:24.198 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:44:24.198 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.198 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.198 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.199 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.199 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.199 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.199 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.199 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.199 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.199 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.199 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.200 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:44:24.200 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:44:24.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.200 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.200 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.200 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.200 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.200 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.200 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.201 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.201 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.201 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.201 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.201 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:44:24.201 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:44:24.201 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.202 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.202 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.202 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.202 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.202 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.202 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.202 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.202 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.203 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.203 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.203 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:44:24.203 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:44:24.203 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.203 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.203 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.203 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.204 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.204 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.204 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.204 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.204 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.204 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.204 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:44:24.205 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:44:24.205 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.205 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.205 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.205 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.205 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.205 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.205 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.206 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.206 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.206 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.206 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.206 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:44:24.206 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:44:24.206 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.207 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.207 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.207 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.207 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.207 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.207 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.207 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.207 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.208 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.208 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:44:24.208 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:44:24.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.208 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.208 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.208 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.209 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.209 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.209 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.209 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.209 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.209 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.209 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.209 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:44:24.210 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:44:24.210 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.210 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.210 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.210 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.210 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.210 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.210 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.211 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.211 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.211 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.211 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.211 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:44:24.211 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:44:24.211 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.211 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.212 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.212 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.212 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.212 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.212 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.212 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.212 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.213 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.213 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:44:24.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:44:24.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.213 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.213 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.213 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:44:24.213 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.214 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.214 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.214 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.214 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.214 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.214 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.215 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:44:24.215 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:44:24.215 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.215 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.215 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.215 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.215 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.215 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.216 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.216 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.216 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.216 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.216 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.216 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:44:24.216 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:44:24.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.217 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.217 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.217 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-09-21T03:44:24.217 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.217 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.217 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.217 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.217 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.217 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.218 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.218 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:44:24.218 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:44:24.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.218 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.219 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-09-21T03:44:24.219 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.219 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.219 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.219 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.219 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.219 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.220 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:44:24.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:44:24.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.220 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.220 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.220 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.221 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.221 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.221 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.221 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.221 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.221 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.221 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:44:24.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:44:24.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.222 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.222 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.222 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-09-21T03:44:24.222 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.222 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.222 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.223 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.223 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.223 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.223 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:44:24.223 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:44:24.223 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.223 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.224 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.224 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.224 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.224 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.224 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.224 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.224 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.224 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.225 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:44:24.225 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:44:24.225 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.225 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.225 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.225 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.225 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.225 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.226 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.226 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.226 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.226 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.226 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.226 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:44:24.226 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:44:24.226 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.227 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.227 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.227 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.227 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.227 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.227 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.227 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.227 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.228 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.228 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.228 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:44:24.228 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:44:24.228 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.228 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.228 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.228 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.229 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.229 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.229 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.229 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.229 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.229 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.229 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.230 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:44:24.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:44:24.230 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.230 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.230 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.230 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.231 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.231 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.231 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.231 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.233 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.233 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.233 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.234 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:44:24.234 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:44:24.234 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.234 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.234 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.234 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:24.234 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.235 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.235 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.235 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.235 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.235 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.235 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.235 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-09-21T03:44:24.236 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-09-21T03:44:24.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.236 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.236 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.236 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.236 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.236 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.236 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.236 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.237 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.237 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.237 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.237 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-09-21T03:44:24.237 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-09-21T03:44:24.237 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.237 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.237 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.238 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.238 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.238 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.238 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.238 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.238 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.238 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.238 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.239 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-09-21T03:44:24.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-09-21T03:44:24.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.239 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.239 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.239 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.240 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.240 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:44:24.240 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.240 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.240 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.240 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-09-21T03:44:24.241 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-09-21T03:44:24.241 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.241 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.241 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.241 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-09-21T03:44:24.241 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.241 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.241 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.242 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.242 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.242 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.242 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.242 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-09-21T03:44:24.242 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-09-21T03:44:24.242 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.242 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.243 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.243 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.243 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.243 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.243 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.243 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.243 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.243 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.244 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.244 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.244 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.244 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.244 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.244 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.245 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.245 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.245 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.245 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.245 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.245 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.245 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.245 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.245 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.246 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.246 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.246 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.246 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.246 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.246 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.246 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.246 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.247 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.247 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.247 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.247 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.247 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.247 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.247 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.248 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.248 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.248 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.248 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.248 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.248 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.248 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.248 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.249 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.249 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.249 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.249 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.249 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.249 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.249 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.249 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.250 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.250 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.250 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.250 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.250 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.250 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.250 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.250 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.251 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.251 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.251 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.251 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.251 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.251 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.251 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.251 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.252 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.252 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.252 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-09-21T03:44:24.252 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-09-21T03:44:24.252 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.252 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.252 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.252 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.253 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.253 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.253 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.253 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-09-21T03:44:24.253 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.253 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.253 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.253 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:44:24.254 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:44:24.254 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.254 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.254 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.254 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.254 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.254 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.254 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.255 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.255 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.255 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.255 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.255 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-09-21T03:44:24.255 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-09-21T03:44:24.255 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.255 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.256 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-09-21T03:44:24.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.256 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.256 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.256 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.256 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.256 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.256 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.257 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-09-21T03:44:24.257 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-09-21T03:44:24.257 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.257 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.257 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.257 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-09-21T03:44:24.257 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.257 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.258 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.258 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.258 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.258 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.258 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.258 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.259 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.259 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.259 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.259 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.259 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.259 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.260 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.260 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.260 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-09-21T03:44:24.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-09-21T03:44:24.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.260 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.260 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.261 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.261 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.261 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.261 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.261 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.261 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.261 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.262 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-09-21T03:44:24.262 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-09-21T03:44:24.262 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.262 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.262 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.262 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-09-21T03:44:24.262 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.262 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.263 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.263 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.263 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.263 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.263 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.263 INFO:teuthology.orchestra.run.smithi112.stdout: "sso_oauth2": { 2024-09-21T03:44:24.263 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sso_oauth2", 2024-09-21T03:44:24.263 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.264 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.264 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.264 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.264 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.264 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.264 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.264 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.264 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.265 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.265 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.265 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-09-21T03:44:24.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-09-21T03:44:24.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.265 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.265 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-09-21T03:44:24.266 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.266 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.266 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.266 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.266 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-09-21T03:44:24.266 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.266 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.266 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.267 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.267 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.267 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-09-21T03:44:24.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-09-21T03:44:24.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.267 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.267 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.267 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-09-21T03:44:24.268 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-09-21T03:44:24.268 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-09-21T03:44:24.268 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.268 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.268 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.268 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.268 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.268 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-09-21T03:44:24.269 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-09-21T03:44:24.269 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.269 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.269 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.269 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.269 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.269 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.269 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.270 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.270 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.270 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.270 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.270 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.270 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.270 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-09-21T03:44:24.271 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.271 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.271 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.271 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-09-21T03:44:24.271 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-09-21T03:44:24.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.271 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.271 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.271 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.272 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.272 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.272 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.272 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-09-21T03:44:24.272 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.272 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.272 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.273 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.273 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.273 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.273 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.273 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.273 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.273 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.274 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.274 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.274 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.274 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.274 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.274 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.274 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.274 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.275 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.275 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.275 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.275 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.275 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.275 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.275 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.275 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.276 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.276 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.276 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.276 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.276 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.276 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.276 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.276 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.276 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.277 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.277 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.277 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.277 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.277 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.277 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.278 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.278 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.278 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.278 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.278 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.278 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.278 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.278 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.278 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.279 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.279 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.279 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.279 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.279 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.279 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.280 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.280 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.280 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.280 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.280 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.280 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.280 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.280 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.281 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.281 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.281 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-09-21T03:44:24.281 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-09-21T03:44:24.281 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.281 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.281 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.281 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-09-21T03:44:24.282 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.282 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.282 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.282 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:44:24.282 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.282 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.282 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.283 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-09-21T03:44:24.283 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-09-21T03:44:24.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.283 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.283 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-09-21T03:44:24.283 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.283 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.283 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.284 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:44:24.284 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.284 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.284 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.284 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-09-21T03:44:24.284 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-09-21T03:44:24.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.284 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.285 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.285 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-09-21T03:44:24.285 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.285 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.285 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.285 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:44:24.285 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.285 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.286 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.286 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-09-21T03:44:24.286 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-09-21T03:44:24.286 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.286 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.286 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.286 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-09-21T03:44:24.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.287 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.287 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.287 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:44:24.287 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.287 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.287 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.287 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-09-21T03:44:24.288 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-09-21T03:44:24.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.288 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.288 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.288 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.288 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.288 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.288 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.288 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:44:24.289 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.289 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.289 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.289 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:24.289 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:24.289 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.289 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.289 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.290 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:24.290 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.290 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.290 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.290 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:44:24.290 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.290 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.290 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.291 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.291 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.291 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.291 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.291 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.291 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.292 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.292 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.292 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.292 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.292 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-09-21T03:44:24.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-09-21T03:44:24.292 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.293 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.293 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.293 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-09-21T03:44:24.293 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.293 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.293 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.293 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:44:24.293 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.294 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.294 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.294 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.294 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.294 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-09-21T03:44:24.294 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.294 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.295 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.295 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.295 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.295 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.295 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.295 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.295 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.295 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.296 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.296 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.296 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.296 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.296 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.296 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.296 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.296 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.296 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.297 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.297 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.297 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.297 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.297 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.297 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.297 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.297 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.298 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.298 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.298 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.298 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.298 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.298 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.298 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.298 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.299 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.299 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.299 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.299 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.299 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.299 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.299 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.299 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.300 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.300 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.300 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.300 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.300 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.300 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.300 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.300 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.300 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.301 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.301 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.301 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.301 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.301 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.301 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.301 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.301 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.302 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.302 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.302 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.302 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.302 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.302 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.302 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.302 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.303 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-09-21T03:44:24.303 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-09-21T03:44:24.303 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.303 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.303 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.303 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-09-21T03:44:24.303 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.304 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.304 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.304 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.304 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.304 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.304 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-09-21T03:44:24.304 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-09-21T03:44:24.305 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.305 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.305 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.305 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-09-21T03:44:24.305 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.305 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.305 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.305 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.306 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.306 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.306 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.306 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:24.306 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:24.306 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.306 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.307 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.307 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:24.307 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.307 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.307 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.307 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.307 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.307 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.307 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.308 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.308 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.308 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.308 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.308 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.309 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.309 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.309 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.309 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.309 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.309 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.309 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.309 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.310 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.310 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-09-21T03:44:24.310 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-09-21T03:44:24.310 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:44:24.310 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.310 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-09-21T03:44:24.310 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-09-21T03:44:24.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.311 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.311 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.311 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-09-21T03:44:24.311 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.311 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.311 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.311 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:44:24.311 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.312 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.312 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.312 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-09-21T03:44:24.312 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-09-21T03:44:24.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.312 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.312 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.313 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-09-21T03:44:24.313 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.313 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.313 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.313 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:44:24.313 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.313 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.313 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.314 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-09-21T03:44:24.314 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-09-21T03:44:24.314 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.314 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.314 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.314 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.314 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.314 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.315 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.315 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:44:24.315 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.315 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.315 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.315 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:24.315 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:24.315 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.316 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.316 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.316 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-09-21T03:44:24.316 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-09-21T03:44:24.316 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.316 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.316 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:44:24.316 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.316 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.317 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.317 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.317 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.317 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.317 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.317 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.317 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.318 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.318 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.318 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.318 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.318 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.318 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.318 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.318 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.319 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.319 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.319 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.319 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.319 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.319 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.319 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.319 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.320 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.320 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.320 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.320 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.320 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.320 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.321 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.321 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.321 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.321 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.321 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.322 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.322 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.322 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.322 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.322 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.322 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.322 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.323 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.323 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.323 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.323 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.323 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.324 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.324 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.324 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.324 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.324 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.324 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.324 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.324 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.325 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.325 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.325 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.325 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-09-21T03:44:24.325 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-09-21T03:44:24.325 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.325 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.325 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.326 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.326 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.326 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.326 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.326 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:44:24.326 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.326 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.326 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.327 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-09-21T03:44:24.327 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-09-21T03:44:24.327 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.327 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.327 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.327 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-09-21T03:44:24.327 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.328 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.328 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.328 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-09-21T03:44:24.328 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.328 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.328 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.328 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.328 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.329 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.329 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.329 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.329 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.330 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.330 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.330 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.330 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-09-21T03:44:24.330 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-09-21T03:44:24.330 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.330 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.330 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.331 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-09-21T03:44:24.331 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.331 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.331 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:44:24.331 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.331 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.331 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.332 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-09-21T03:44:24.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-09-21T03:44:24.332 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.332 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.332 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.332 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:24.332 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:24.332 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-09-21T03:44:24.333 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.333 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:44:24.333 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.333 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.333 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.333 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-09-21T03:44:24.333 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-09-21T03:44:24.333 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.334 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.334 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.334 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.334 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.334 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.334 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.334 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:44:24.334 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.335 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.335 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.335 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-09-21T03:44:24.335 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-09-21T03:44:24.335 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.335 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.335 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.335 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-09-21T03:44:24.336 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.336 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.336 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.336 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:44:24.336 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.336 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.336 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.336 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.336 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.337 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.337 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-09-21T03:44:24.337 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.337 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.337 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.337 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.337 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.338 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.338 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.338 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.338 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.338 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.338 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.338 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.338 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.338 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.339 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.339 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.339 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.339 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.339 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.339 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.339 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.339 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.340 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.340 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.340 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.340 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.340 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.340 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.340 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.340 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.341 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.342 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.343 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.344 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.346 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-09-21T03:44:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.349 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.351 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.352 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.353 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.354 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.355 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.356 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.356 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.356 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.356 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.356 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.356 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.356 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.357 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.357 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.357 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.357 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.357 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.357 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.357 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.357 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.358 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.358 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.358 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-09-21T03:44:24.358 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.358 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.358 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.359 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-09-21T03:44:24.359 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:44:24.359 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.359 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.359 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.359 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-09-21T03:44:24.359 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.359 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.359 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.360 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:44:24.360 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.360 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.360 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-09-21T03:44:24.360 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-09-21T03:44:24.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.361 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.361 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.361 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.361 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-09-21T03:44:24.361 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.361 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.361 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:44:24.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.362 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.362 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.362 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.362 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.362 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.363 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.363 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.363 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.363 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.363 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.363 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.363 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.363 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.363 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.364 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.364 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.364 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.364 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.364 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.364 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.364 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.365 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.365 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.365 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.365 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.365 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.365 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.365 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.366 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.366 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.366 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.366 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.366 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.366 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.366 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.367 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.367 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.367 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.367 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.367 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.367 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.367 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.367 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.368 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.368 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.368 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.368 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.368 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.368 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.369 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.369 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.369 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.369 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.369 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.370 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.370 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.370 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.370 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.370 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.370 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.371 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.371 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.371 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.371 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.371 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.371 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.371 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.371 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.372 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.372 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-09-21T03:44:24.372 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.372 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.372 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.373 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-09-21T03:44:24.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-09-21T03:44:24.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.373 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.373 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-09-21T03:44:24.373 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.373 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.374 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.374 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:44:24.374 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:44:24.374 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.374 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.374 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.374 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.375 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.375 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.375 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.375 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.375 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.375 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.375 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.375 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.376 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.376 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.376 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.376 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.376 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.376 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.376 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.376 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.377 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.377 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.377 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.377 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.377 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.378 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.378 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.378 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.378 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.378 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.378 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.379 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.379 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.379 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.379 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.379 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.379 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.379 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.379 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.379 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.380 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.380 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.380 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.380 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.380 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.380 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.380 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.380 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.380 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.381 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.381 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.381 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.381 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.381 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.381 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.381 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.382 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.382 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.382 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.382 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.382 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.382 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.382 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.382 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-09-21T03:44:24.383 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-09-21T03:44:24.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.383 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.383 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.383 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.383 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.383 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.383 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.383 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:44:24.384 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:44:24.384 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.384 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.384 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-09-21T03:44:24.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-09-21T03:44:24.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.384 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.385 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.385 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-09-21T03:44:24.385 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.385 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.385 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.385 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:44:24.385 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.385 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.385 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.386 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-09-21T03:44:24.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-09-21T03:44:24.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.386 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.386 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-09-21T03:44:24.386 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.387 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:44:24.387 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.387 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.387 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.387 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-09-21T03:44:24.387 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-09-21T03:44:24.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.388 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.388 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.388 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.388 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.388 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:44:24.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.389 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.389 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.389 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.389 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.389 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.389 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.389 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.390 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.390 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.390 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.390 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.390 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.390 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-09-21T03:44:24.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-09-21T03:44:24.391 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.391 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.391 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.391 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-09-21T03:44:24.391 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.391 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.391 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.391 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:44:24.392 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:44:24.392 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.392 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.392 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.392 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.392 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-09-21T03:44:24.393 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.393 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.393 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.393 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.393 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.393 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.393 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.393 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.393 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.394 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.395 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.395 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.395 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.395 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.395 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.395 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.395 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.396 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.396 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.396 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.396 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.396 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.396 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.397 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.397 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.397 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.397 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.397 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.397 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.397 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.397 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.398 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.398 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.398 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.398 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.398 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.398 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.398 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.398 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.399 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.399 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.399 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.399 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.399 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.400 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.400 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.400 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.400 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.400 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.401 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.401 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.401 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.401 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.401 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.401 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.402 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.402 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.402 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.402 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.402 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.402 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.402 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.402 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.403 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-09-21T03:44:24.403 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.403 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.403 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.403 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.404 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.404 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.404 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.405 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.405 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.405 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.405 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.405 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.405 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.406 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.406 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.406 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.406 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.406 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.406 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.407 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.407 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.407 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.407 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.407 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.407 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.407 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.408 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.408 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.408 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.408 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.408 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.408 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.408 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.408 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.409 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.409 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.409 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.409 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.409 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.409 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.409 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.409 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.409 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.410 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.410 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.410 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.410 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.410 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.410 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.410 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.410 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.411 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.411 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.412 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.412 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.412 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.412 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.413 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.413 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.413 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.413 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-09-21T03:44:24.413 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.413 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.414 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.414 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.414 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.414 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.414 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.414 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.414 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.414 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.415 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.415 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.415 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.415 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.415 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.415 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.415 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.415 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.416 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.416 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.416 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.416 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.416 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.416 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.417 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.417 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.417 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.417 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.417 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.418 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.418 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.418 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.418 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.419 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.419 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.419 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.419 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.419 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.419 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.419 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.420 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.420 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.420 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.421 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.421 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.421 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.421 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.421 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.422 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.422 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.422 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.422 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.422 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.423 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.423 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.423 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.423 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.424 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-09-21T03:44:24.424 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.424 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.424 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.424 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-09-21T03:44:24.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-09-21T03:44:24.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:44:24.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.426 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.426 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.427 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.427 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.427 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.427 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.428 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.428 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.428 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.428 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.428 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.429 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.429 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.430 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.430 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.430 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.430 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.430 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.431 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.431 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.431 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.431 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.431 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.431 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.432 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.432 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.432 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.432 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.432 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.432 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.432 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.432 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.433 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.433 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.433 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.433 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.433 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.433 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.433 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.434 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.434 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.434 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.435 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-09-21T03:44:24.435 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-09-21T03:44:24.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.435 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.437 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.437 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.438 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.438 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.438 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-09-21T03:44:24.438 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-09-21T03:44:24.438 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-09-21T03:44:24.438 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.438 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-09-21T03:44:24.438 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.439 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.439 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.439 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.439 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.439 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.440 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.440 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.440 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.440 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.440 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.440 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.440 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.440 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.440 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-09-21T03:44:24.441 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.441 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.441 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.441 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.442 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.442 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.442 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.442 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.442 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.442 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.442 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.443 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.443 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.443 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.443 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.443 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.443 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.443 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.443 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.444 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.444 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.444 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.445 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.445 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.445 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.445 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.445 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.445 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.445 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.446 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.446 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.446 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.446 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.446 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.446 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.446 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.447 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.447 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.447 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.447 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.447 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.447 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.447 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.447 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.448 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.448 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.448 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.448 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.448 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.449 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.449 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.449 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.449 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.449 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.449 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.449 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.449 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.450 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.450 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.450 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.450 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.451 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.451 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.451 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-09-21T03:44:24.452 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.452 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.452 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.452 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.453 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.453 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.453 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.453 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.454 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.454 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.454 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.454 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.454 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.454 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.454 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.454 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.454 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.455 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.455 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.455 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.455 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.455 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.455 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.455 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.456 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.456 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.456 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.456 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.456 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.456 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.456 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.457 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.457 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.457 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.457 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.457 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.457 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.457 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.458 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.458 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.458 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.458 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.458 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.458 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.458 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.458 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.459 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.459 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.459 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.459 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.459 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.459 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.459 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.459 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.460 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.460 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.460 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.460 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.460 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.460 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.461 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.461 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.461 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.461 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.461 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.461 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.461 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.461 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.462 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.462 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.462 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.462 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-09-21T03:44:24.462 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.462 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.463 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.463 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.463 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.463 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.463 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.463 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.463 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.463 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.464 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.464 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.464 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.464 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.464 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.464 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.464 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.464 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.464 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.465 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.465 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.465 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.465 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.465 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.466 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.466 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.467 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.467 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.467 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.467 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.467 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.467 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.468 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.468 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.468 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.468 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.468 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.468 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.468 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.468 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.469 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.469 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.469 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.469 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.469 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.469 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.469 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.470 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.470 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.470 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.470 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.470 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.470 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.470 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.471 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:24.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:24.471 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.471 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.471 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.471 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:24.471 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.472 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.472 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.472 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.472 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.472 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.472 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.472 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.473 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.473 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.473 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.473 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.473 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.474 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.474 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.474 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.474 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-09-21T03:44:24.474 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-09-21T03:44:24.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:24.474 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-09-21T03:44:24.475 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-09-21T03:44:24.475 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.475 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.475 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-09-21T03:44:24.475 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:44:24.475 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.475 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-09-21T03:44:24.476 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.476 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.476 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-09-21T03:44:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:44:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:44:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-09-21T03:44:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-09-21T03:44:24.478 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.479 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.481 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.483 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.484 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.485 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.486 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.487 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-09-21T03:44:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-09-21T03:44:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-09-21T03:44:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:44:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.489 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:24.490 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:44:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.491 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.492 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-09-21T03:44:24.493 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-09-21T03:44:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-09-21T03:44:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-09-21T03:44:24.495 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-09-21T03:44:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:44:24.496 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:44:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.497 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.498 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.499 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.500 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.501 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.502 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.503 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-09-21T03:44:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-09-21T03:44:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.505 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.506 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:44:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:44:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-09-21T03:44:24.507 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-09-21T03:44:24.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-09-21T03:44:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-09-21T03:44:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.509 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-09-21T03:44:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-09-21T03:44:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.510 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-09-21T03:44:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:44:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.511 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-09-21T03:44:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-09-21T03:44:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-09-21T03:44:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.512 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.513 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.513 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:44:24.513 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.513 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.513 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.513 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.514 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.514 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.514 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.514 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.514 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.514 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.514 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.515 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.515 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.515 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.515 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-09-21T03:44:24.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-09-21T03:44:24.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.516 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-09-21T03:44:24.516 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.516 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.516 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.517 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-09-21T03:44:24.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-09-21T03:44:24.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.517 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.517 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.517 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-09-21T03:44:24.517 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.517 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.517 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.518 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-09-21T03:44:24.518 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-09-21T03:44:24.518 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.518 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.518 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.518 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.518 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.519 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-09-21T03:44:24.519 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-09-21T03:44:24.519 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.519 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.519 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.519 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-09-21T03:44:24.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-09-21T03:44:24.519 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-09-21T03:44:24.519 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.520 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.520 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.520 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.520 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.520 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.520 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-09-21T03:44:24.521 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.521 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.521 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.521 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.521 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.521 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.521 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.522 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.522 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.522 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.522 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.522 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.522 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.523 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.523 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.523 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.523 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.523 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.523 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.524 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.524 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.524 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.524 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.524 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.524 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.524 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.524 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.525 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.525 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.525 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.525 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.525 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.525 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.526 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.526 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.526 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.526 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.526 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.526 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.526 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.526 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.526 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.527 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.527 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.527 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.527 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.527 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.527 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.527 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.527 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.528 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.529 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.529 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.529 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.529 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-09-21T03:44:24.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:44:24.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.530 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.530 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.530 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.530 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.530 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.530 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.530 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.530 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.531 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.531 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:44:24.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:44:24.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.531 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.532 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.532 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.532 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.533 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.533 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.533 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.533 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.533 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.534 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.534 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.534 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-09-21T03:44:24.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-09-21T03:44:24.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.535 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.535 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.535 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.535 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.535 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.535 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.535 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.535 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.536 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.536 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.536 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-09-21T03:44:24.536 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.536 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.536 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-09-21T03:44:24.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-09-21T03:44:24.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.537 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.537 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.538 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.538 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.538 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.538 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-09-21T03:44:24.538 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-09-21T03:44:24.538 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.538 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.539 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.539 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.539 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.539 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.539 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.540 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.540 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.541 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.541 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.541 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.541 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.541 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.541 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.541 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.541 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.542 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.542 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.542 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.542 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.543 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.543 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.543 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.543 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.543 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.543 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.544 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.544 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.544 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.544 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.544 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.545 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.545 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.545 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.545 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.545 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.545 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.546 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.546 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.546 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.546 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.546 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.546 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.547 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.547 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.547 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.547 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.547 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.547 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.547 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.547 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.548 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.548 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.548 INFO:teuthology.orchestra.run.smithi112.stdout: "max_requests": { 2024-09-21T03:44:24.548 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_requests", 2024-09-21T03:44:24.548 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.548 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.548 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-09-21T03:44:24.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.549 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:44:24.549 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.549 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.549 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.550 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-09-21T03:44:24.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-09-21T03:44:24.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.550 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.550 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.550 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.550 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.551 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.551 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.551 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.551 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-09-21T03:44:24.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-09-21T03:44:24.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.552 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.552 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.552 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.552 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.552 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.552 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.552 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.552 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.553 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.553 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.553 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.553 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.554 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.554 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.554 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.554 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.554 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.555 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-09-21T03:44:24.555 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.555 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.555 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.555 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.556 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.556 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.556 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.556 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.557 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.557 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.557 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.557 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.557 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.557 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.558 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.558 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.558 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.558 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.559 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.559 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.559 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.560 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.560 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.560 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.560 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.561 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.561 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.561 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.561 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.561 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.561 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.561 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.561 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.561 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.562 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.562 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.562 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.562 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.562 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.562 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.562 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.562 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.563 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.563 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.563 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.563 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.563 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.563 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.563 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.564 INFO:teuthology.orchestra.run.smithi112.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:44:24.564 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:44:24.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.564 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.564 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:24.564 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.564 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.564 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.565 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:44:24.565 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.565 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.565 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.565 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.565 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.565 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.566 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.566 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.566 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.566 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.566 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.566 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.566 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.566 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.567 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.567 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.567 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-09-21T03:44:24.567 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.567 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.567 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.568 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.568 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.568 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.568 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.568 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.569 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.569 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.569 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.569 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.569 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.569 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.569 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.569 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.569 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.570 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.570 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.570 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.570 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.570 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.570 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.570 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.571 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.571 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.571 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.571 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.571 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.572 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.572 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.572 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.572 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.572 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.572 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.573 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.573 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.573 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.573 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.573 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.573 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.573 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.573 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.574 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.574 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.574 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.574 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.574 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.574 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.574 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.575 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.575 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.575 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.575 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.575 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.575 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.575 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.576 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.576 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.576 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.576 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.576 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.576 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.576 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.577 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.577 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.577 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.577 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.577 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-09-21T03:44:24.577 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-09-21T03:44:24.578 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.578 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.578 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.578 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-09-21T03:44:24.578 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.578 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.578 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.578 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:44:24.579 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.579 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.579 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.579 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.579 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-09-21T03:44:24.579 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.580 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.580 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.580 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.580 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.580 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.580 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.580 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.580 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.581 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.581 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.581 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.581 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.581 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.581 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.581 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.581 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.582 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.582 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.582 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.582 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.582 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.582 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.582 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.583 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.583 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.583 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.583 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.584 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.584 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.584 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.584 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.584 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.584 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.584 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.585 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.585 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.585 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.585 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.585 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.585 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.585 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.585 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.586 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.586 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.586 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.586 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.586 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.586 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.586 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.587 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.587 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.587 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.587 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.587 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.587 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.588 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.588 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.588 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-09-21T03:44:24.588 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-09-21T03:44:24.588 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.588 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.589 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.589 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.589 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.589 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.589 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.589 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.589 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.589 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.590 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-09-21T03:44:24.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-09-21T03:44:24.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.590 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.590 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.590 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-09-21T03:44:24.590 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.591 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.591 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.591 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.591 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.591 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.591 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.591 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-09-21T03:44:24.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-09-21T03:44:24.592 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.592 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.592 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.592 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.592 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.592 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.592 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.592 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.593 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.593 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.593 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.593 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-09-21T03:44:24.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-09-21T03:44:24.593 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.593 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.594 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.594 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.594 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.594 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.594 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.594 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.594 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.594 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.595 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.595 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-09-21T03:44:24.595 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-09-21T03:44:24.595 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:24.595 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.595 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.595 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.595 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.596 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.596 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.596 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.596 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.596 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.596 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.596 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-09-21T03:44:24.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-09-21T03:44:24.597 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.597 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.597 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.597 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.597 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.597 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.597 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.597 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.598 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.598 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.598 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-09-21T03:44:24.598 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-09-21T03:44:24.598 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.598 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.598 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.599 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.599 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.599 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.599 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.599 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.599 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.599 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.600 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-09-21T03:44:24.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-09-21T03:44:24.600 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.600 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.600 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.600 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.600 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.601 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.601 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.601 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.601 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.601 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-09-21T03:44:24.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-09-21T03:44:24.602 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.602 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.602 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.602 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.602 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:24.602 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-09-21T03:44:24.602 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.602 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.603 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.603 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.603 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.603 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.604 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.604 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.604 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.604 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.605 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-09-21T03:44:24.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-09-21T03:44:24.605 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.605 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.605 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.605 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.606 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.606 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.606 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.606 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-09-21T03:44:24.606 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-09-21T03:44:24.606 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.607 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.607 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.607 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.607 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.607 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.607 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.607 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.608 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.608 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.608 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-09-21T03:44:24.608 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-09-21T03:44:24.608 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.608 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.608 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.608 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.609 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.609 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.609 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.609 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.609 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.610 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.610 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.610 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smb", 2024-09-21T03:44:24.610 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.610 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.610 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.610 INFO:teuthology.orchestra.run.smithi112.stdout: "internal_store_backend": { 2024-09-21T03:44:24.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "internal_store_backend", 2024-09-21T03:44:24.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.611 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.611 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.611 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.611 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:44:24.612 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.612 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.612 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.612 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.612 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.612 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.612 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.613 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.613 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.613 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.613 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.613 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.613 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.613 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.613 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.614 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.614 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.614 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.614 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.614 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.614 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.614 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.614 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.615 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.615 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.615 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.615 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.615 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.616 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.616 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.616 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.616 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.616 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.616 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.616 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.617 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.618 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.619 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.619 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.619 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.619 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.619 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.620 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.620 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.620 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.620 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.620 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.620 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.620 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.620 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.621 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.621 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.621 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.621 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.621 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.621 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.621 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.621 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.622 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.622 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.622 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.622 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.622 INFO:teuthology.orchestra.run.smithi112.stdout: "update_orchestration": { 2024-09-21T03:44:24.622 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "update_orchestration", 2024-09-21T03:44:24.623 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.623 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.623 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.623 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.623 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.623 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.623 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.623 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:44:24.624 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.624 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.624 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.624 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-09-21T03:44:24.624 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.625 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.625 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.625 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-09-21T03:44:24.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-09-21T03:44:24.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.625 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.625 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.626 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.626 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.626 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:44:24.626 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.626 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.626 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.626 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-09-21T03:44:24.627 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-09-21T03:44:24.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.627 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.627 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.627 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.627 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.627 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.627 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.627 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:44:24.628 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.628 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.628 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.628 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.628 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.628 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.629 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.629 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.629 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.629 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.629 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.629 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.629 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.630 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.630 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.630 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.630 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.630 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.630 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.630 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.631 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.631 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.631 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.631 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.631 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.631 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.633 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.633 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.633 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.633 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.633 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.633 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.633 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.633 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.634 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.634 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.634 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.634 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.634 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.634 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.634 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.634 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.635 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.635 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.635 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.635 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.635 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.635 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.635 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.636 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.636 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.636 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.636 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.636 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.636 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.637 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.637 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.637 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.637 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.637 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.637 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.637 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.637 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.638 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.638 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.638 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.638 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.638 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.638 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.638 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.638 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-09-21T03:44:24.639 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.639 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.639 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.639 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.639 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.639 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.640 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.640 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.640 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.640 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.641 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.641 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.641 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.642 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.642 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.642 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.642 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.642 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.642 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.642 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.643 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.643 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.643 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.643 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.643 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.643 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.643 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.643 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.644 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.644 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.644 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.644 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.644 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.645 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.645 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.645 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.645 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.645 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.645 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.645 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.645 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.645 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.646 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.646 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.646 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.646 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.646 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.646 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.646 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.646 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.647 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.647 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.647 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.647 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.647 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.647 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.648 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.648 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.648 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.648 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.648 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.648 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.648 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.648 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.649 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-09-21T03:44:24.651 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.651 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.651 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.651 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.651 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.652 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.652 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.652 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.652 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.652 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.652 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.652 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.652 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.652 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.653 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.653 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.653 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.653 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.653 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.653 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.653 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.654 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.654 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.654 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.654 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.654 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.654 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.654 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.654 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.655 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.655 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.655 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.655 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.655 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.655 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.655 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.656 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.656 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.656 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.656 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.656 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.656 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.656 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.657 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.657 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.657 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.657 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.657 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.657 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.657 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.658 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.658 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.658 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.658 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.658 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.658 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.658 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.659 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.659 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.659 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.659 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.659 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.659 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.659 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.659 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.660 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.660 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.660 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.660 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.660 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.660 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.661 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.661 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.661 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-09-21T03:44:24.661 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.661 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.662 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.662 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-09-21T03:44:24.662 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-09-21T03:44:24.662 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.662 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.662 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.662 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:44:24.662 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.663 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.663 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.663 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.663 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.663 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.663 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.663 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:24.663 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:24.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.664 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.664 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.664 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-09-21T03:44:24.664 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.664 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.664 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.664 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.665 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.665 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.665 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.665 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.665 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.665 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.665 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.666 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.666 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.666 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.666 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.666 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.666 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.666 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.666 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.666 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.667 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.667 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.667 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.667 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.667 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.667 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.667 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.668 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.668 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.668 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.668 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.668 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.668 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.668 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.668 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.668 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.669 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.669 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.669 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.669 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.669 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.669 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.670 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.670 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.670 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.670 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.670 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.670 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.670 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.670 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.671 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.671 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.671 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.671 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.671 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.671 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.671 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.672 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.672 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.672 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.672 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.672 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.672 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.672 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.672 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.673 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.673 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.673 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.673 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.673 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.673 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.674 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.674 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.674 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.674 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.674 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.674 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.674 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.674 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.674 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.675 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.675 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.675 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.675 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-09-21T03:44:24.675 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.675 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.676 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.676 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-09-21T03:44:24.676 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-09-21T03:44:24.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.676 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.676 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.676 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.676 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.676 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.677 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.677 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:44:24.677 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.677 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.677 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-09-21T03:44:24.677 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-09-21T03:44:24.678 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.678 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.678 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.678 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.678 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.678 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.678 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.678 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:44:24.678 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.679 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.679 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.679 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-09-21T03:44:24.679 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-09-21T03:44:24.679 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.679 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.679 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.680 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.680 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.680 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.680 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.680 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:44:24.680 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.680 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.680 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.681 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-09-21T03:44:24.681 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-09-21T03:44:24.681 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.681 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.681 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.681 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.681 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.681 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.682 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:44:24.682 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.682 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.682 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-09-21T03:44:24.682 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-09-21T03:44:24.682 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.683 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.683 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.683 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.683 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.683 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.683 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.683 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:44:24.683 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.684 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.684 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.684 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-09-21T03:44:24.684 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-09-21T03:44:24.684 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.684 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.684 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.684 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.685 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.685 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.685 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.685 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.685 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.685 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.685 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.685 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-09-21T03:44:24.686 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-09-21T03:44:24.686 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.686 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.686 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.686 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.686 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.686 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.686 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.687 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.687 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.687 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.687 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.687 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-09-21T03:44:24.687 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-09-21T03:44:24.687 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.687 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.688 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.688 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:44:24.688 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.688 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.688 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.688 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.688 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.688 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.689 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.689 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-09-21T03:44:24.689 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-09-21T03:44:24.689 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.689 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.689 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.689 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.690 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.690 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.690 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.690 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.690 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.690 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.690 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.690 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:24.691 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:24.691 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.691 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.691 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.691 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-09-21T03:44:24.691 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-09-21T03:44:24.691 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.691 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.691 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-09-21T03:44:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-09-21T03:44:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.692 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.693 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.693 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-09-21T03:44:24.693 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.693 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.693 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.693 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.693 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.693 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.693 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.694 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-09-21T03:44:24.694 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-09-21T03:44:24.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.694 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.694 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.694 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.694 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.695 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.695 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.695 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.695 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.695 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.695 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.695 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-09-21T03:44:24.696 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-09-21T03:44:24.696 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.696 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.696 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.696 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.696 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.696 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.696 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.696 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.697 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.697 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.697 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.697 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.697 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.697 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.697 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.697 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.698 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.698 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.698 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.698 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.698 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.698 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.698 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.698 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.699 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.699 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.699 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.699 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.699 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.699 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.699 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.699 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.700 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.700 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.700 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.700 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.700 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.700 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.700 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.700 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.700 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.701 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.701 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.701 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.701 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.701 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.701 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.701 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.701 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.702 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.702 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.702 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.702 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.702 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.702 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.702 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.702 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.703 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.703 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.703 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.703 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.703 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.703 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.703 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.703 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.704 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.704 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.704 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.704 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.704 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.704 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.704 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.704 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.705 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.705 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.705 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.705 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.705 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-09-21T03:44:24.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-09-21T03:44:24.705 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.705 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.706 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.706 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.706 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.706 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.706 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.706 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.706 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.706 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.707 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.707 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-09-21T03:44:24.707 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-09-21T03:44:24.707 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.707 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.707 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.707 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.707 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.708 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.708 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.708 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.708 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.708 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.708 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.708 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.709 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.709 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.709 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.709 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.709 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.709 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.709 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.710 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.710 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.710 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.710 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.710 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-09-21T03:44:24.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-09-21T03:44:24.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.710 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:44:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-09-21T03:44:24.712 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.712 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.712 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.712 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.713 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.713 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.713 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.713 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.713 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.713 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.713 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.713 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.713 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.714 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.714 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.714 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.714 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.714 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.714 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.714 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.714 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.715 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.715 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.715 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.715 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.715 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.715 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.715 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.716 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.716 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.716 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.716 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.716 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.716 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.716 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.716 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.717 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.717 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.717 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.717 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.717 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.717 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.717 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.718 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.718 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.718 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.718 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.718 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.718 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.718 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.718 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.718 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.719 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.719 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.719 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.719 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.719 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.719 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.720 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.720 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.720 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.720 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.720 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.720 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.720 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.720 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.721 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.721 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.721 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.721 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.721 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.721 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.721 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.722 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.722 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.722 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.722 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.722 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.722 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.722 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-09-21T03:44:24.723 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.723 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.723 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.723 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.723 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.723 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.723 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.724 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.724 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.724 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.724 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.724 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.724 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.724 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.724 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.724 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.725 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.725 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.725 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.725 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.725 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.725 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.725 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.725 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.726 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.726 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.726 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.726 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.726 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.726 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.726 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.727 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.727 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.727 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.727 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.727 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.727 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.727 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.727 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.729 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.729 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.729 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.729 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.729 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.729 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.729 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.730 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.730 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.730 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.730 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.730 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.730 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.730 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.730 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.731 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.731 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.731 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.731 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.731 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-09-21T03:44:24.731 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-09-21T03:44:24.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.731 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.732 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.732 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-09-21T03:44:24.732 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.732 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.732 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.732 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:44:24.732 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.732 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.733 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.733 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-09-21T03:44:24.733 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-09-21T03:44:24.733 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.733 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.733 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.733 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.733 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.734 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.734 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.734 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-09-21T03:44:24.734 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.734 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.734 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.734 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-09-21T03:44:24.734 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:44:24.735 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.735 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.735 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.735 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.735 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.735 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.735 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.735 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:44:24.735 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.736 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.736 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.736 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:44:24.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:44:24.736 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.736 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:44:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.738 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.739 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-09-21T03:44:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-09-21T03:44:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-09-21T03:44:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.740 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-09-21T03:44:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.742 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-09-21T03:44:24.742 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-09-21T03:44:24.742 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.742 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.742 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.742 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.742 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.742 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.743 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.743 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.743 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.743 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.743 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.743 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:24.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:24.744 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.744 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.744 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.744 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:24.744 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.744 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.744 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.744 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.744 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.745 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.745 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.745 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.745 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.745 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.745 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.746 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.746 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.746 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.746 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.746 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.746 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.746 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.746 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.746 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.747 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.747 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.747 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.747 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.747 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.747 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.747 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.747 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.748 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.748 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.748 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.748 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.748 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.748 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.748 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.748 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.749 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.749 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.749 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.749 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.749 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.749 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.749 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.749 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.750 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.750 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.750 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.750 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.750 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.750 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.750 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.750 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.751 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.751 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.751 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.751 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.751 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.751 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.751 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.752 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.752 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.752 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.752 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.752 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.752 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.752 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.752 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.752 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.753 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.753 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.753 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.753 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.753 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.753 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.753 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.754 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-09-21T03:44:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-09-21T03:44:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.755 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.756 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.756 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.756 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.756 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.756 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.756 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.756 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-09-21T03:44:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-09-21T03:44:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-09-21T03:44:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.757 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-09-21T03:44:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-09-21T03:44:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.758 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:44:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.759 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:44:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 24155, 2024-09-21T03:44:24.760 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "y", 2024-09-21T03:44:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:44:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-09-21T03:44:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-09-21T03:44:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.762 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:44:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.763 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.764 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.765 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.766 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.767 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.768 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.769 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.769 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.769 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.769 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.769 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.769 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.769 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.769 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.770 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.770 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.770 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.770 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.770 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.770 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.770 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.771 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.771 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.771 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.771 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.771 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.771 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.771 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.771 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-09-21T03:44:24.772 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-09-21T03:44:24.772 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.772 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.772 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.772 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.772 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.772 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.772 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.773 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:44:24.773 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.773 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.773 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.773 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-09-21T03:44:24.773 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-09-21T03:44:24.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.773 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.774 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.774 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-09-21T03:44:24.774 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.774 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.774 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.774 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-09-21T03:44:24.774 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.774 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.775 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.775 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-09-21T03:44:24.775 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-09-21T03:44:24.775 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.775 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.775 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.775 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.776 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.776 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.776 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-09-21T03:44:24.776 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.776 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.776 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.776 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-09-21T03:44:24.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-09-21T03:44:24.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.777 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.777 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.777 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.777 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.777 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.777 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.778 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-09-21T03:44:24.778 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.778 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.778 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.778 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-09-21T03:44:24.778 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-09-21T03:44:24.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.778 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.779 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.779 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-09-21T03:44:24.779 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.779 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.779 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.779 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-09-21T03:44:24.779 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.779 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.780 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.780 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-09-21T03:44:24.780 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-09-21T03:44:24.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.780 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.780 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.780 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.780 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.781 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.781 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.781 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:44:24.781 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.781 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.781 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.781 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-09-21T03:44:24.782 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-09-21T03:44:24.782 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.782 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.782 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.782 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.782 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.782 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.782 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.783 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:44:24.783 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.783 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.783 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.783 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-09-21T03:44:24.783 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-09-21T03:44:24.783 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.784 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.784 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.784 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.784 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.784 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.784 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.784 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-09-21T03:44:24.784 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.784 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.785 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.785 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.785 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.785 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.785 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.785 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.785 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.786 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.786 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.786 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.786 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.786 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.786 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.786 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.786 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.786 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.787 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-09-21T03:44:24.787 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.787 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.787 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.787 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-09-21T03:44:24.787 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-09-21T03:44:24.788 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.788 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.788 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.788 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.788 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.788 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.788 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.788 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:44:24.788 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.789 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.789 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.789 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-09-21T03:44:24.789 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-09-21T03:44:24.789 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.789 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.789 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.789 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-09-21T03:44:24.790 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.790 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.790 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.790 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:44:24.790 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:44:24.790 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.790 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.791 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-09-21T03:44:24.791 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-09-21T03:44:24.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:24.791 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.791 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.791 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.791 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-09-21T03:44:24.791 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-09-21T03:44:24.791 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.792 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:44:24.792 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:44:24.792 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.792 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.792 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-09-21T03:44:24.792 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:44:24.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:24.792 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.793 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.793 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-09-21T03:44:24.793 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:24.793 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-09-21T03:44:24.793 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.793 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:44:24.793 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.793 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.794 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.794 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-09-21T03:44:24.794 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-09-21T03:44:24.794 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.794 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.794 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.794 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:44:24.794 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.795 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.795 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.795 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:44:24.795 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:44:24.795 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.795 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.795 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-09-21T03:44:24.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-09-21T03:44:24.796 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:24.796 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.796 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.796 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-09-21T03:44:24.796 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-09-21T03:44:24.796 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-09-21T03:44:24.796 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.796 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:44:24.797 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:44:24.797 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.797 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.797 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-09-21T03:44:24.797 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-09-21T03:44:24.797 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.797 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.798 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.798 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-09-21T03:44:24.798 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.798 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.798 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.798 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:44:24.798 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:44:24.798 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.798 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.799 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-09-21T03:44:24.799 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-09-21T03:44:24.799 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:24.799 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.799 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.799 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.799 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-09-21T03:44:24.800 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-09-21T03:44:24.800 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.800 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:44:24.800 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:44:24.800 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.800 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.800 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.800 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.801 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.801 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.801 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.801 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.801 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.801 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.802 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.802 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.802 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.802 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.802 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.802 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.802 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.802 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.803 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.803 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.803 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.803 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.803 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.803 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.804 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.804 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.804 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.804 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.804 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.804 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.804 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.805 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.805 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.805 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.805 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.805 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.805 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.805 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.805 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.805 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.805 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.806 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.806 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.806 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.806 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.806 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.806 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.806 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.806 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.807 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.807 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.807 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.807 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.807 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.807 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.807 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.808 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.808 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.808 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.808 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.808 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.808 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.808 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.808 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.809 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-09-21T03:44:24.809 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-09-21T03:44:24.809 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:24.809 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.809 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.809 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.809 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.809 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.810 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.810 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:44:24.810 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.810 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.810 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.810 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-09-21T03:44:24.810 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-09-21T03:44:24.810 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.811 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.811 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.811 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-09-21T03:44:24.811 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.811 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.811 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.811 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-09-21T03:44:24.811 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-09-21T03:44:24.812 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-09-21T03:44:24.812 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-09-21T03:44:24.812 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-09-21T03:44:24.812 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.812 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-09-21T03:44:24.812 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.812 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.812 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.813 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-09-21T03:44:24.813 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-09-21T03:44:24.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.813 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.813 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.813 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.813 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.813 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:44:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.815 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.815 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.815 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:24.815 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.815 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.815 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.815 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:44:24.815 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.815 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.816 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.816 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.816 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.816 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.816 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.816 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.816 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.816 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.817 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.817 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.817 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.817 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.817 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.817 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.817 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-09-21T03:44:24.817 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-09-21T03:44:24.818 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.818 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.818 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.818 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:24.818 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:24.818 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.818 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.818 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:44:24.819 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:44:24.819 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.819 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.819 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-09-21T03:44:24.819 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:44:24.819 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:24.819 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.819 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.820 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.820 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.820 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.820 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.820 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:44:24.820 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.820 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.820 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.820 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.820 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.821 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-09-21T03:44:24.821 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.821 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.821 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.821 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-09-21T03:44:24.821 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-09-21T03:44:24.821 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:24.822 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.822 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.822 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-09-21T03:44:24.822 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.822 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.822 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.822 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:44:24.822 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.823 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.823 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.823 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-09-21T03:44:24.823 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-09-21T03:44:24.823 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.823 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.823 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.824 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-09-21T03:44:24.824 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.824 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.824 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.824 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:44:24.824 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.824 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.824 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.825 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-09-21T03:44:24.825 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-09-21T03:44:24.825 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.825 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.825 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.825 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-09-21T03:44:24.825 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.825 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.826 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.826 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:44:24.826 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.826 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.826 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.826 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-09-21T03:44:24.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-09-21T03:44:24.827 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.827 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.827 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.827 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.827 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.827 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.827 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.827 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:44:24.828 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:44:24.828 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.828 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.828 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-09-21T03:44:24.828 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-09-21T03:44:24.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.828 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.828 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.829 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:24.829 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.829 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.829 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.829 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:44:24.829 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.829 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.830 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.830 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:44:24.830 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:44:24.830 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:24.830 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.830 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.831 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-09-21T03:44:24.831 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.831 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.831 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.831 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:44:24.831 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.831 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.831 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.832 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-09-21T03:44:24.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-09-21T03:44:24.832 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.832 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.832 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.832 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.832 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.832 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.833 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.833 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-09-21T03:44:24.833 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-09-21T03:44:24.833 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-09-21T03:44:24.833 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.833 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:44:24.833 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.833 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.833 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.834 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-09-21T03:44:24.834 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-09-21T03:44:24.834 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.834 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.834 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.834 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.834 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.835 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.835 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.835 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:44:24.835 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.835 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.835 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.835 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-09-21T03:44:24.835 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-09-21T03:44:24.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.836 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.836 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.836 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.836 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.836 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.836 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.836 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:44:24.836 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.837 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.837 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.837 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-09-21T03:44:24.837 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-09-21T03:44:24.837 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.837 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.837 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.838 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.838 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.838 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.838 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.838 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:44:24.838 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.838 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.838 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.839 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-09-21T03:44:24.839 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-09-21T03:44:24.839 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.839 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.839 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.839 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:44:24.839 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.839 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.839 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.839 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:24.840 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.840 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.840 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.840 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-09-21T03:44:24.840 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-09-21T03:44:24.840 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.840 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.840 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.841 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:44:24.841 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.841 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.841 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.841 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:44:24.841 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.841 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.841 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.842 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-09-21T03:44:24.842 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:44:24.842 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.842 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.842 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.842 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:44:24.842 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.842 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.843 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.843 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-09-21T03:44:24.843 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.843 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.843 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.843 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-09-21T03:44:24.843 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-09-21T03:44:24.843 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.844 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.844 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.845 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:44:24.845 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.846 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.846 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.846 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:24.846 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.846 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.846 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.846 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-09-21T03:44:24.846 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-09-21T03:44:24.847 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.847 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.847 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.847 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:44:24.847 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.847 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.847 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.847 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-09-21T03:44:24.847 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.848 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.848 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.848 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-09-21T03:44:24.848 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:44:24.848 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.848 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.848 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.848 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:44:24.849 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.849 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.849 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.849 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:44:24.849 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.849 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.849 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.849 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-09-21T03:44:24.850 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:44:24.850 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.850 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.850 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.850 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:44:24.850 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.850 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.850 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.850 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:44:24.851 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.851 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.851 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.851 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-09-21T03:44:24.851 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:44:24.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.851 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.852 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:44:24.852 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.852 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.852 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.852 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-09-21T03:44:24.852 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.852 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.853 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-09-21T03:44:24.853 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-09-21T03:44:24.853 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.853 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.853 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.853 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:44:24.853 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.853 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.853 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.854 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-09-21T03:44:24.854 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.854 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.854 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.854 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-09-21T03:44:24.854 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-09-21T03:44:24.854 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.855 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.855 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:44:24.855 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.855 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.855 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.855 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-09-21T03:44:24.855 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.855 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.856 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.856 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nginx": { 2024-09-21T03:44:24.856 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nginx", 2024-09-21T03:44:24.856 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.856 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.856 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.856 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:44:24.856 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.856 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.857 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.857 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nginx container image", 2024-09-21T03:44:24.857 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.857 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.857 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.857 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-09-21T03:44:24.857 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-09-21T03:44:24.858 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.858 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.858 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.858 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:44:24.858 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.858 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.858 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.858 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:24.858 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.859 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.859 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.859 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-09-21T03:44:24.859 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-09-21T03:44:24.859 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.859 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.859 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.859 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:44:24.860 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.860 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.860 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.860 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-09-21T03:44:24.860 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.860 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.860 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.861 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:44:24.861 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:44:24.861 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.861 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.861 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.861 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:44:24.861 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.861 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.861 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.862 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:44:24.862 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.862 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.862 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.862 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-09-21T03:44:24.862 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-09-21T03:44:24.862 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.863 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.863 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.863 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:44:24.863 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.863 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.863 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.863 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:24.863 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.863 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.864 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.864 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-09-21T03:44:24.864 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-09-21T03:44:24.864 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.864 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.864 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.864 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:44:24.865 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.865 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.865 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.865 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-09-21T03:44:24.865 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.865 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.865 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.865 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-09-21T03:44:24.866 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-09-21T03:44:24.866 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.866 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.866 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.866 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:44:24.866 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.866 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.866 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.866 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:44:24.867 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.867 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.867 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.867 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba_metrics": { 2024-09-21T03:44:24.867 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:44:24.867 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.867 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.868 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.868 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:44:24.868 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.868 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.868 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.868 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:44:24.868 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.868 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.868 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.869 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-09-21T03:44:24.869 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:44:24.869 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.869 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.869 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.869 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:44:24.869 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.870 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.870 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.870 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:44:24.870 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.870 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.870 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.870 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-09-21T03:44:24.870 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-09-21T03:44:24.871 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.871 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.871 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.871 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.871 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.871 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.871 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.871 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:44:24.872 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.872 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.872 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.872 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-09-21T03:44:24.872 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:44:24.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.872 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.873 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.873 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:24.873 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.873 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.873 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.873 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:44:24.873 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.873 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.873 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.874 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:44:24.874 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:44:24.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.874 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.874 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.874 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-09-21T03:44:24.874 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.875 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.875 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.875 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:44:24.875 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.875 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.875 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.875 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-09-21T03:44:24.876 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-09-21T03:44:24.876 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.876 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.876 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.876 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-09-21T03:44:24.876 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.876 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.876 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.877 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:44:24.877 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.877 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.877 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.877 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-09-21T03:44:24.877 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-09-21T03:44:24.877 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.878 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.878 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.878 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-09-21T03:44:24.878 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.878 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.878 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.878 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:44:24.878 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.879 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.879 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.879 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-09-21T03:44:24.879 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-09-21T03:44:24.879 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.879 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.879 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.879 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.880 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.880 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.880 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.880 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:44:24.880 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.880 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.880 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.880 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-09-21T03:44:24.881 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-09-21T03:44:24.881 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.881 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.881 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.881 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:24.881 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.881 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.881 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.881 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:44:24.882 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.882 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.882 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.882 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-09-21T03:44:24.882 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:44:24.882 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.882 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.883 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:44:24.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.883 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.883 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.883 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:44:24.883 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.883 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.883 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.884 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-09-21T03:44:24.884 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-09-21T03:44:24.884 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.884 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.884 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.884 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:24.884 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.884 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.885 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:44:24.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.885 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.885 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.885 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-09-21T03:44:24.885 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-09-21T03:44:24.885 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.886 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.886 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.886 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.886 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.886 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.886 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.886 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:44:24.886 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.887 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.887 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.887 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-09-21T03:44:24.887 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-09-21T03:44:24.887 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.887 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.887 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.887 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.888 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.888 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.888 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.888 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:44:24.888 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.888 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.888 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.888 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.889 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.889 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.889 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.889 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.889 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.889 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.889 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.889 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.890 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.890 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.890 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.890 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.890 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.890 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.890 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.890 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.890 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.891 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.891 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.891 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-09-21T03:44:24.891 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-09-21T03:44:24.891 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.891 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.891 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.892 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.892 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.892 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.892 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.892 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:44:24.892 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.892 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.892 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.893 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.893 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.893 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.893 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.893 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.893 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.894 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:44:24.894 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.894 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.894 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.894 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.894 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.894 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.895 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.895 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.895 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.895 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.895 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.895 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.895 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.895 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.895 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.896 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.896 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.896 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.896 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.896 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.896 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.896 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.896 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.897 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.897 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.897 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.897 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.897 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.897 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.897 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.898 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.898 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.898 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.898 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.898 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.898 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:44:24.898 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:44:24.899 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.899 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.899 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.899 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.899 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.899 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.899 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.899 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:44:24.899 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:44:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:44:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.900 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-09-21T03:44:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:44:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.901 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-09-21T03:44:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-09-21T03:44:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.902 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:44:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-09-21T03:44:24.903 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-09-21T03:44:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.904 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:44:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-09-21T03:44:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-09-21T03:44:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.905 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-09-21T03:44:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-09-21T03:44:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-09-21T03:44:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-09-21T03:44:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.907 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-09-21T03:44:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-09-21T03:44:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:44:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-09-21T03:44:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-09-21T03:44:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-09-21T03:44:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.909 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:44:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-09-21T03:44:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:44:24.910 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.911 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.911 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.911 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:44:24.911 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.911 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.911 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.911 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:44:24.911 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.911 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.912 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.912 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-09-21T03:44:24.912 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-09-21T03:44:24.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.912 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.912 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.912 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.912 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.913 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.913 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.913 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:44:24.913 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.913 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.913 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.913 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-09-21T03:44:24.914 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-09-21T03:44:24.914 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.914 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.914 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.914 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.914 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.914 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.914 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.914 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:44:24.915 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.915 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.915 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.915 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-09-21T03:44:24.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-09-21T03:44:24.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.915 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.915 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:44:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-09-21T03:44:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-09-21T03:44:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:44:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-09-21T03:44:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:44:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:44:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.920 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-09-21T03:44:24.920 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-09-21T03:44:24.920 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.920 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.920 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.920 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-09-21T03:44:24.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.921 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.921 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.921 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:44:24.921 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.921 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.921 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.921 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.922 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.922 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.922 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.922 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.922 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.923 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.923 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.923 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.923 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.923 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.923 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.923 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-09-21T03:44:24.924 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-09-21T03:44:24.924 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.924 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.924 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.924 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.924 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.924 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.925 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.925 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:44:24.925 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.925 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.925 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.925 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:44:24.926 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:44:24.926 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.926 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.926 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.926 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-09-21T03:44:24.926 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.926 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.926 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.927 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:44:24.927 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.927 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.927 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.927 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_keepalive_interval": { 2024-09-21T03:44:24.927 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:44:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-09-21T03:44:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:44:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-09-21T03:44:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-09-21T03:44:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:44:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-09-21T03:44:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-09-21T03:44:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:44:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-09-21T03:44:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:44:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:44:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-09-21T03:44:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:44:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:44:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-09-21T03:44:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:44:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:44:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-09-21T03:44:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-09-21T03:44:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-09-21T03:44:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-09-21T03:44:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:44:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-09-21T03:44:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-09-21T03:44:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-09-21T03:44:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:44:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-09-21T03:44:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:44:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:44:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:44:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:44:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:44:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:44:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:44:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:44:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:44:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:44:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:44:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:44:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:44:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:44:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:44:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:44:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:44:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:44:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:44:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:44:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:44:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:44:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:44:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:44:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:44:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:44:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:44:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:44:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:44:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:44:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-09-21T03:44:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:44:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:44:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-09-21T03:44:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:44:24.983 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.983 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.983 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.983 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.983 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.983 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.983 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:44:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:44:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-09-21T03:44:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:44:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:44:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:44:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:44:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:44:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:44:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:44:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:44:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:44:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:44:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-09-21T03:44:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:44:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:44:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-09-21T03:44:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:44:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:44:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:44:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:44:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:44:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:44:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:44:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:44:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:44:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:44:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.006 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.006 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.006 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:44:25.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:44:25.006 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.008 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.008 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.008 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:44:25.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:44:25.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.008 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.008 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.008 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.009 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.009 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.009 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.009 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.009 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:44:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:44:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:44:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:44:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:44:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:44:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:44:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:44:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:44:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:44:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:44:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:44:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:44:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-09-21T03:44:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:44:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:44:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-09-21T03:44:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:44:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:44:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:44:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:44:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-09-21T03:44:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:44:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:44:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:44:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:44:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:44:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:44:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:44:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:44:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:44:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:44:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:44:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:44:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-09-21T03:44:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-09-21T03:44:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-09-21T03:44:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-09-21T03:44:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-09-21T03:44:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-09-21T03:44:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:44:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-09-21T03:44:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-09-21T03:44:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-09-21T03:44:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-09-21T03:44:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-09-21T03:44:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-09-21T03:44:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-09-21T03:44:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-09-21T03:44:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:44:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:44:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-09-21T03:44:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-09-21T03:44:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-09-21T03:44:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-09-21T03:44:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-09-21T03:44:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-09-21T03:44:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-09-21T03:44:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-09-21T03:44:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-09-21T03:44:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-09-21T03:44:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-09-21T03:44:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: "sso_oauth2": { 2024-09-21T03:44:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sso_oauth2", 2024-09-21T03:44:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-09-21T03:44:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-09-21T03:44:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.071 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.071 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-09-21T03:44:25.071 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.071 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.073 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.073 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.073 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-09-21T03:44:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-09-21T03:44:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-09-21T03:44:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-09-21T03:44:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-09-21T03:44:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-09-21T03:44:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-09-21T03:44:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-09-21T03:44:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-09-21T03:44:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-09-21T03:44:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-09-21T03:44:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-09-21T03:44:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-09-21T03:44:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-09-21T03:44:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-09-21T03:44:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:44:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-09-21T03:44:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-09-21T03:44:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-09-21T03:44:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:44:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-09-21T03:44:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-09-21T03:44:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-09-21T03:44:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:44:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-09-21T03:44:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-09-21T03:44:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-09-21T03:44:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:44:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-09-21T03:44:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-09-21T03:44:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:44:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:44:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-09-21T03:44:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-09-21T03:44:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-09-21T03:44:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:44:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-09-21T03:44:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-09-21T03:44:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-09-21T03:44:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-09-21T03:44:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-09-21T03:44:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-09-21T03:44:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-09-21T03:44:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.119 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.119 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:25.119 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.119 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.119 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.119 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-09-21T03:44:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-09-21T03:44:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:44:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-09-21T03:44:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-09-21T03:44:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-09-21T03:44:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:44:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-09-21T03:44:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-09-21T03:44:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-09-21T03:44:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:44:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-09-21T03:44:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-09-21T03:44:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:44:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-09-21T03:44:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-09-21T03:44:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:44:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-09-21T03:44:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-09-21T03:44:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:44:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-09-21T03:44:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-09-21T03:44:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-09-21T03:44:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-09-21T03:44:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-09-21T03:44:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-09-21T03:44:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-09-21T03:44:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.146 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.146 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.146 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:44:25.146 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.146 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.146 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-09-21T03:44:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-09-21T03:44:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-09-21T03:44:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:44:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-09-21T03:44:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-09-21T03:44:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:44:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-09-21T03:44:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-09-21T03:44:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-09-21T03:44:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:44:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-09-21T03:44:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.159 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.160 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.160 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.160 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.160 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.160 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.160 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.160 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.161 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.161 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.161 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.161 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.161 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.161 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.161 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.161 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.162 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.162 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.162 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.162 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.162 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.162 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.163 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.163 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.163 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.163 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.163 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.163 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.163 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.163 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.164 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.164 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.164 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-09-21T03:44:25.164 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.164 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.164 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.165 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.165 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.165 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.165 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.165 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.165 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.165 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.165 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.166 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.166 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.166 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.166 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.166 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.166 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.166 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.167 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.167 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.167 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.167 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.167 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.167 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.167 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.168 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.168 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.168 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.168 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.168 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.168 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.168 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.168 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.169 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.169 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.169 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.169 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.169 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.169 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.170 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.170 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.170 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.170 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.170 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.170 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.170 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.170 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.171 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.171 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.171 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.171 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.171 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.171 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.171 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.171 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.172 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.172 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.172 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.172 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.172 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.172 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.173 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.173 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.173 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.173 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.173 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.173 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.173 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.174 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.174 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.174 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.174 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.174 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.174 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.174 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.175 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.175 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.175 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.175 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.175 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.175 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.175 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.176 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.176 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-09-21T03:44:25.176 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.176 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.176 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.177 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-09-21T03:44:25.177 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:44:25.177 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.177 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.177 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.177 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-09-21T03:44:25.177 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.177 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.178 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.178 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:44:25.178 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.178 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.178 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.178 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-09-21T03:44:25.179 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-09-21T03:44:25.179 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.179 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.179 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.179 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:25.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-09-21T03:44:25.179 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.180 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.180 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:44:25.180 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.180 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.180 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.180 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.181 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.181 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.181 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.181 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.181 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.182 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.182 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.182 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.182 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.182 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.182 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.182 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.182 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.183 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.183 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.183 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.183 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.183 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.183 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.183 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.184 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.184 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.184 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.184 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.184 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.184 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.184 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.185 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.185 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.185 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.185 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.185 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.185 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.186 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.186 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.186 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.186 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.186 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.186 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.186 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.186 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.187 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.187 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.187 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.187 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.187 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.187 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.187 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.188 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.188 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.188 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.188 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.188 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.188 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.188 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.188 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.189 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.189 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.189 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.189 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.189 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.189 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.190 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.190 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.190 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.190 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.190 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.190 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.190 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.190 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.191 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.191 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.191 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.191 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.191 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.191 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.192 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-09-21T03:44:25.192 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.192 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.192 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.192 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-09-21T03:44:25.192 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-09-21T03:44:25.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.192 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.193 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-09-21T03:44:25.193 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.193 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.193 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.193 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:44:25.193 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:44:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.195 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.195 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.195 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.195 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.195 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.195 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.196 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.196 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.196 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.196 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.196 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.196 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.196 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.196 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.197 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.197 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.197 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.197 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.198 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.198 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.198 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.198 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.198 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.198 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.199 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.199 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.199 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.199 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.199 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.199 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.200 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.200 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.200 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.200 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.200 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.200 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.200 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.200 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.201 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.201 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.201 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.201 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.201 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.201 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.202 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.202 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.202 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.202 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.202 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.202 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.202 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.202 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.203 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.203 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.203 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.203 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.203 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.203 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-09-21T03:44:25.203 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-09-21T03:44:25.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.204 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.204 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.204 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.204 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.204 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:44:25.205 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:44:25.205 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.205 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.205 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-09-21T03:44:25.205 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-09-21T03:44:25.205 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.205 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.206 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.206 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-09-21T03:44:25.206 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.206 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.206 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.206 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:44:25.206 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.207 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.207 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.207 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-09-21T03:44:25.207 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-09-21T03:44:25.207 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.207 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.207 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-09-21T03:44:25.208 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.208 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.208 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.208 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:44:25.208 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.208 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.208 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.209 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-09-21T03:44:25.209 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-09-21T03:44:25.209 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.209 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.209 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.209 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.209 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.209 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.210 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.210 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:44:25.210 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.210 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.210 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.210 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.211 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.211 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.211 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.211 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.211 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.211 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.211 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.211 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.211 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.212 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.212 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.212 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-09-21T03:44:25.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-09-21T03:44:25.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.213 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.213 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.213 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-09-21T03:44:25.213 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.213 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.213 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.213 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:44:25.213 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:44:25.214 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.214 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.214 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.214 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.214 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-09-21T03:44:25.214 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.215 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.215 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.215 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.215 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.215 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.215 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.215 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.216 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.216 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.216 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.216 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.216 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.216 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.216 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.217 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.217 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.217 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.217 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.217 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.217 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.217 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.217 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.218 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.218 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.218 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.218 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.219 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.219 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.219 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.219 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.219 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.219 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.220 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.220 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.220 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.220 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.220 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.221 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.221 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.221 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.221 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.221 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.221 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.221 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.222 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.222 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.222 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.222 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.222 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.222 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.223 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.223 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.223 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.223 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.223 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.223 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.223 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.225 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.225 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.225 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.225 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.225 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.225 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.225 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.225 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.226 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.226 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.226 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.226 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.226 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-09-21T03:44:25.226 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.227 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.227 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.227 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.227 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.227 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.227 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.227 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.228 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.228 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.228 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.228 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.228 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.228 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.228 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.228 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.229 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.229 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.229 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.229 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.229 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.229 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.230 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.230 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.230 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.230 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.230 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.230 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.230 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.231 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.231 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.231 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.231 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.231 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.231 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.232 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.232 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.232 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.232 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.232 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.232 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.232 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.232 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.233 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.233 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.233 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.233 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.233 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.233 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.233 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.233 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.234 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.234 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.234 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.234 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.234 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.234 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.235 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.235 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.235 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.235 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.235 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.235 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.235 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.236 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.236 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.236 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.236 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.236 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.237 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.237 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.237 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.237 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.237 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.237 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.237 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.237 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.238 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.238 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.238 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.238 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.238 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-09-21T03:44:25.239 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.239 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.239 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.239 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.240 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.240 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.240 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.240 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.240 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.240 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.240 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.240 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.241 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.241 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.241 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.241 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.241 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.241 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.241 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.241 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.242 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.242 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.242 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.242 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.242 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.242 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.242 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.242 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.243 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.243 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.243 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.243 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.243 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.243 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.243 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.244 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.244 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.244 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.244 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.244 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.244 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.245 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.245 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.245 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.245 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.245 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.245 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.245 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.245 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.246 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.246 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.246 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.246 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.246 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.246 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.246 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.247 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.247 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.247 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.247 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.247 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.247 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.247 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.247 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.248 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.248 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.248 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.248 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.248 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.248 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.248 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.249 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.249 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.249 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.249 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.249 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.249 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.249 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.250 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.250 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-09-21T03:44:25.250 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.250 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.250 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.251 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-09-21T03:44:25.251 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-09-21T03:44:25.251 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.251 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.251 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.251 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.251 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.252 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.252 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:44:25.252 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.252 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.252 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.252 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.253 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.253 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.253 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.253 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.253 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.253 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.253 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.253 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.254 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.254 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.254 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.254 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.254 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.254 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.254 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.255 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.255 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.255 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.255 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.255 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.255 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.256 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.256 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.256 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.256 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.256 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.256 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.257 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.257 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.257 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.257 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.257 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.257 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.258 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.258 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.258 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.258 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.258 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.258 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.258 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.258 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.259 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.259 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.259 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.259 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.259 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.259 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.259 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.260 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.260 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.260 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.261 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.261 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.261 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.261 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.261 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.261 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.261 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.262 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-09-21T03:44:25.262 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-09-21T03:44:25.262 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.262 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.262 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.262 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.262 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.263 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.263 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.263 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-09-21T03:44:25.263 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-09-21T03:44:25.263 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-09-21T03:44:25.263 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.263 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-09-21T03:44:25.263 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.264 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.264 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.264 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.264 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.264 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.264 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.264 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.265 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.265 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.265 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.265 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.265 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.265 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.265 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.266 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.266 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.266 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.266 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-09-21T03:44:25.266 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.266 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.267 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.267 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.267 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.267 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.267 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.268 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.268 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.268 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.268 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.268 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.268 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.268 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.268 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.269 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.269 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.269 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.269 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.269 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.269 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.270 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.270 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.270 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.270 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.270 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.271 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.271 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.271 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.271 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.271 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.271 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.272 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.272 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.272 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.272 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.272 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.272 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.272 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.272 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.273 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.273 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.273 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.273 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.273 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.273 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.273 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.273 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.273 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.274 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.274 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.274 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.274 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.274 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.274 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.275 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.275 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.275 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.275 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.275 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.275 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.275 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.276 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.276 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.276 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.276 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.276 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.276 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.277 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.277 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.277 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.277 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.277 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.277 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.277 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.278 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.278 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.278 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.278 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.278 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-09-21T03:44:25.278 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.279 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.279 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.279 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.279 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.279 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.279 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.280 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.280 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.280 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.280 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.280 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.280 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.280 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.281 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.281 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.281 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.281 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.281 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.281 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.281 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.282 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.283 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.284 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.285 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.286 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.287 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.289 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.290 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-09-21T03:44:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.293 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.294 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.294 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.294 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.294 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.295 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.295 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.295 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.295 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.295 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.295 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.295 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.296 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.296 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.296 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.296 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.296 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.296 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.296 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.297 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.297 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.297 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.297 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.297 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.297 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.297 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.297 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.298 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.298 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.298 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.298 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.298 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.298 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.299 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.299 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.299 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.299 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.299 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.299 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.299 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.300 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.300 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.300 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.300 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.300 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.300 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:25.300 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:25.301 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.301 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.301 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.301 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:25.301 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.301 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.301 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.301 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.302 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.302 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.302 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.302 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.302 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.302 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.303 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.303 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.303 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.303 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.303 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.303 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.303 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.304 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.304 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.304 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-09-21T03:44:25.304 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-09-21T03:44:25.304 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:25.304 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.305 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.305 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-09-21T03:44:25.305 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-09-21T03:44:25.305 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.305 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.305 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-09-21T03:44:25.305 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:44:25.305 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.306 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.306 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.306 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.306 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-09-21T03:44:25.306 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.307 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.307 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.307 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-09-21T03:44:25.307 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:44:25.307 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.307 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.307 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.307 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.308 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.308 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:44:25.308 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.308 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.308 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.309 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-09-21T03:44:25.309 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-09-21T03:44:25.309 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.309 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.309 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.309 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.309 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.310 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.310 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.310 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.310 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.310 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.310 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.310 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.311 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.311 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.311 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.311 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.311 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.312 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.312 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.312 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.312 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.312 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.312 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.312 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.312 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.313 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.313 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.313 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.313 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.313 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.313 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.314 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.314 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.314 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.314 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.314 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.314 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.314 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.315 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.315 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.315 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.315 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.315 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.315 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.315 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.316 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.316 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.316 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.316 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.316 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.316 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.316 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.317 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.317 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.317 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.317 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.317 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.317 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.317 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.317 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.318 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.318 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.318 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.318 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.318 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.318 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.318 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.319 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.319 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.319 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.319 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.319 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.319 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.319 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.319 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-09-21T03:44:25.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-09-21T03:44:25.320 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.320 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.320 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.320 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-09-21T03:44:25.321 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.321 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.321 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.321 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:44:25.321 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.321 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.321 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.322 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:25.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:25.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.322 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.322 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:25.322 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.322 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.323 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:44:25.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.323 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.323 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.323 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.324 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.324 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.324 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.324 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.324 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.324 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.325 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.325 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.325 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.325 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.325 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.325 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.326 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.326 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-09-21T03:44:25.326 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.326 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.326 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.326 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-09-21T03:44:25.326 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-09-21T03:44:25.326 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.327 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.327 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.327 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.327 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.327 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.327 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.327 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.327 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.328 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.328 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.328 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-09-21T03:44:25.328 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-09-21T03:44:25.328 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.328 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.329 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.329 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.329 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:44:25.329 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:44:25.329 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.330 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.330 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.330 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.330 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.330 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.330 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.330 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.331 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.331 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.331 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.331 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.331 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.331 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.332 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.332 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.332 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.332 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.332 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.332 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.332 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.333 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.333 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.333 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.333 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.333 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.333 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.333 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.333 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.334 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.334 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.334 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.334 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.334 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.334 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.334 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.335 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.335 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.335 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.335 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.335 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.335 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.335 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.335 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.336 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.336 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.336 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.336 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.336 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.336 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.337 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.337 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.337 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.337 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.337 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.337 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.338 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.338 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.338 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.338 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.338 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.338 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.338 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.339 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.339 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-09-21T03:44:25.339 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-09-21T03:44:25.339 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.339 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.339 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.339 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.339 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.340 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.340 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.340 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.340 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.340 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.340 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.340 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:44:25.341 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:44:25.341 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.341 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.341 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-09-21T03:44:25.341 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.341 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.341 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.341 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.342 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.342 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.342 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.342 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-09-21T03:44:25.342 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-09-21T03:44:25.342 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:25.342 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.343 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-09-21T03:44:25.343 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.343 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.343 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.343 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.343 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.343 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.343 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.344 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-09-21T03:44:25.344 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-09-21T03:44:25.344 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.344 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.344 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.345 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-09-21T03:44:25.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.345 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.345 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.345 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:44:25.345 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.345 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.345 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.346 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-09-21T03:44:25.346 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-09-21T03:44:25.346 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.346 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.346 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.346 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-09-21T03:44:25.346 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.347 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.347 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.347 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:44:25.347 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.347 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.347 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.348 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.348 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.348 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.348 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.348 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.348 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.348 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.349 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.349 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.349 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.349 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.349 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.349 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-09-21T03:44:25.349 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-09-21T03:44:25.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.350 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.350 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.350 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-09-21T03:44:25.350 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.350 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.350 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.350 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.351 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.351 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.351 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.351 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-09-21T03:44:25.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-09-21T03:44:25.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.352 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.352 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.352 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-09-21T03:44:25.352 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.352 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.352 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.352 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-09-21T03:44:25.352 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-09-21T03:44:25.353 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.353 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.353 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.353 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.353 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.353 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-09-21T03:44:25.353 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-09-21T03:44:25.354 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.354 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.354 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.354 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-09-21T03:44:25.354 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-09-21T03:44:25.354 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-09-21T03:44:25.354 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.355 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.355 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.355 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.355 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.355 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.355 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.356 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-09-21T03:44:25.356 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.356 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.356 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.356 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.356 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.356 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.357 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.357 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.357 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.357 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.357 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.357 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.357 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.358 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.358 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.358 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.358 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.358 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.358 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.358 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.358 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.359 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.359 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.359 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.359 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.359 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.359 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.359 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.360 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.360 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.360 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.360 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.360 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.360 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.361 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.361 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.361 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.361 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.361 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.361 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.361 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.362 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.362 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.362 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.362 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.362 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.362 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.362 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.362 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.363 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.363 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.363 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.363 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.363 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.363 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.364 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.364 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.364 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.364 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.364 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.364 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.364 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.365 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.365 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.365 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.365 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.365 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-09-21T03:44:25.365 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:44:25.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.366 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.366 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.366 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:25.366 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.366 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.366 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.366 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.367 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.367 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.367 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.367 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:44:25.367 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:44:25.367 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.367 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.368 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.368 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.368 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.368 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.368 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.368 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.368 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.369 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.369 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.369 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.370 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.370 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.370 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.370 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.370 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.370 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.371 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-09-21T03:44:25.371 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-09-21T03:44:25.371 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.371 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.371 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.371 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.371 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.371 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.372 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.372 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.372 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.372 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.372 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-09-21T03:44:25.373 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.373 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.373 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.373 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-09-21T03:44:25.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-09-21T03:44:25.374 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.374 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.374 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.374 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.374 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.374 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.374 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.375 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.375 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.375 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.375 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.375 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-09-21T03:44:25.375 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-09-21T03:44:25.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.376 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.376 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.376 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.376 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.376 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.376 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.377 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.377 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.377 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.377 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.377 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.378 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.378 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.378 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.378 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.378 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.378 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.379 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.379 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.379 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.379 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.379 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.379 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.379 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.379 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.380 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.380 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.380 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.380 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.380 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.380 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.381 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.381 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.381 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.381 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.381 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.381 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.381 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.382 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.382 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.382 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.382 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.382 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.382 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.382 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.383 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.383 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.383 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.383 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.383 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.383 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.383 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.384 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.384 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.384 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.384 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.384 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.384 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.385 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.385 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.385 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.385 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.385 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.385 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.385 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.385 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.386 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.386 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.386 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.386 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.386 INFO:teuthology.orchestra.run.smithi112.stdout: "max_requests": { 2024-09-21T03:44:25.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_requests", 2024-09-21T03:44:25.387 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.387 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-09-21T03:44:25.387 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.387 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.388 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:44:25.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.388 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.388 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.388 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-09-21T03:44:25.388 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-09-21T03:44:25.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.389 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-09-21T03:44:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-09-21T03:44:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.391 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.392 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.393 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-09-21T03:44:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.394 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.395 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.396 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.398 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.400 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.402 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.402 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.402 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.402 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.402 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.403 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.403 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.403 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.403 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.403 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.403 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.403 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.404 INFO:teuthology.orchestra.run.smithi112.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:44:25.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:44:25.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:25.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:44:25.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.405 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.405 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.406 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.406 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.406 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.406 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.406 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.406 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.407 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.407 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.407 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.407 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.407 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.407 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.407 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-09-21T03:44:25.408 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.408 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.408 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.408 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.408 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.409 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.409 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.409 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.409 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.410 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.410 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.410 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.410 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.410 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.410 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.411 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.411 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.411 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.412 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.412 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.412 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.413 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.413 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.413 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.413 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.413 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.414 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.414 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.414 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.414 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.414 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.414 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.414 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.415 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.415 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.415 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.415 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.415 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.415 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.416 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.416 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.416 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.416 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.416 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.416 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.416 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.416 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.417 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.417 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.417 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.417 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.417 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.418 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.418 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.418 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.418 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.418 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.419 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.419 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.419 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.419 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.419 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-09-21T03:44:25.419 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-09-21T03:44:25.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.420 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-09-21T03:44:25.420 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.420 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.420 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:44:25.421 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.421 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.421 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.421 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.421 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.421 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-09-21T03:44:25.422 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.422 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.422 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.422 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.422 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.423 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.423 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.424 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.424 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.424 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.424 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.424 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.424 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.424 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.425 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.425 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.425 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.426 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.426 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.426 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.426 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.427 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.427 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.427 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.428 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.428 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.428 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.428 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.428 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.428 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.428 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.428 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.429 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.429 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.429 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.429 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.429 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.430 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.430 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.430 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.430 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.430 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.430 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.431 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-09-21T03:44:25.431 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-09-21T03:44:25.431 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.431 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.431 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.431 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.432 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.432 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.432 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.432 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.432 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.432 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.432 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-09-21T03:44:25.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-09-21T03:44:25.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.433 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.433 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.433 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-09-21T03:44:25.433 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.433 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.433 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.433 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.433 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.434 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-09-21T03:44:25.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-09-21T03:44:25.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.434 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.434 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.435 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.435 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.435 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.435 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.436 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.436 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-09-21T03:44:25.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-09-21T03:44:25.436 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.436 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.436 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.436 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.436 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.437 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.437 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.437 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.437 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.437 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.437 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.438 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-09-21T03:44:25.438 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-09-21T03:44:25.438 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:25.438 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.438 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.438 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.438 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.438 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.439 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.439 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.439 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.439 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.439 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-09-21T03:44:25.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-09-21T03:44:25.440 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.440 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.440 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.440 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.440 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.440 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.440 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.440 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.441 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.441 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.441 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.441 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-09-21T03:44:25.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-09-21T03:44:25.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.442 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.442 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.442 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.442 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.442 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.442 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.442 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.443 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.443 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.443 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-09-21T03:44:25.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-09-21T03:44:25.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.443 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.443 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.444 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.444 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.444 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.445 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-09-21T03:44:25.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-09-21T03:44:25.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.445 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.445 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.445 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:25.446 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-09-21T03:44:25.446 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.446 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.446 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.446 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.446 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.447 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.447 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.447 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.447 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.447 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.448 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.448 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.448 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.448 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.448 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.448 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-09-21T03:44:25.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-09-21T03:44:25.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.449 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.449 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.449 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.449 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.449 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.449 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.450 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.450 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.450 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.450 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-09-21T03:44:25.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-09-21T03:44:25.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.451 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.451 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.451 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.451 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.451 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.452 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.452 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-09-21T03:44:25.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-09-21T03:44:25.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.452 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.452 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.452 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.453 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.453 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.453 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.454 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smb", 2024-09-21T03:44:25.454 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.454 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.454 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.455 INFO:teuthology.orchestra.run.smithi112.stdout: "internal_store_backend": { 2024-09-21T03:44:25.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "internal_store_backend", 2024-09-21T03:44:25.455 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.455 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.455 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.455 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.455 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.456 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.456 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.456 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:44:25.456 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.456 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.456 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.456 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.458 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.458 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.458 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.458 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.458 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.458 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.458 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.458 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.459 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.461 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.461 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.462 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.463 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.463 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.463 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.463 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.463 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.463 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.464 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.464 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.464 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.464 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.464 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.464 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.464 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.464 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.465 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.465 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.465 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.466 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.467 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.467 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.467 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.467 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.467 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.468 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.468 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.468 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.468 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.468 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.468 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.468 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.468 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.469 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.469 INFO:teuthology.orchestra.run.smithi112.stdout: "update_orchestration": { 2024-09-21T03:44:25.469 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "update_orchestration", 2024-09-21T03:44:25.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.469 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.469 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.470 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.470 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.470 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.470 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:44:25.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.470 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-09-21T03:44:25.471 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.471 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.471 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.472 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-09-21T03:44:25.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-09-21T03:44:25.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.472 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.472 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.472 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.472 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.472 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:44:25.473 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.473 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.473 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-09-21T03:44:25.474 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-09-21T03:44:25.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.474 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.474 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.474 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.475 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.475 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:44:25.475 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.475 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.475 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.475 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.476 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.476 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.476 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.476 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.476 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.476 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.477 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.477 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.477 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.477 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.477 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.477 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.477 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.477 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.478 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.478 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.478 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.478 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.479 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.479 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.479 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.480 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.481 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.481 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.481 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.481 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.481 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.482 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.482 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.482 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.482 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.482 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.483 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.484 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.484 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.484 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.484 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.484 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.485 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.485 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.485 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.485 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.486 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.486 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.486 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.486 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-09-21T03:44:25.487 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.487 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.487 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.487 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.488 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.488 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.488 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.488 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.488 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.489 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.489 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.489 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.489 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.489 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.489 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.489 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.489 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.490 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.490 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.490 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.490 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.491 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.491 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.491 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.491 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.491 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.491 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.491 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.492 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.492 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.492 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.492 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.492 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.493 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.493 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.493 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.493 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.493 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.493 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.493 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.493 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.494 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.494 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.494 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.494 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.494 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.494 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.494 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.494 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.495 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.495 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.495 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.495 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.495 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.496 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.496 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.496 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.496 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.496 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.496 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.496 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.497 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.497 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.497 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.497 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.497 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.498 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.498 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.498 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.498 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.498 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.498 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.499 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.499 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-09-21T03:44:25.499 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.499 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.499 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.499 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.500 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.500 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.500 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.500 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.501 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.501 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.501 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.501 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.501 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.501 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.501 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.501 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.502 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.502 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.502 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.503 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.503 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.504 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.505 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.505 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.505 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.505 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.505 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.505 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.506 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.506 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.506 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.506 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.506 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.506 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.506 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.507 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.507 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.507 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.508 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.508 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.508 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.508 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.508 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.508 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.509 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.509 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.509 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.509 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.509 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.510 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.510 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.510 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.510 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.511 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.511 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.511 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-09-21T03:44:25.511 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.511 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.512 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.512 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-09-21T03:44:25.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-09-21T03:44:25.512 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.512 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.512 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.512 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:44:25.512 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.512 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.513 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.513 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.513 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.513 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.513 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.513 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:25.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:25.514 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.514 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.514 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.514 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-09-21T03:44:25.514 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.514 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.515 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.515 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.515 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.515 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.515 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.516 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.516 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.516 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.516 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.516 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.517 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.517 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.517 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.517 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.517 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.517 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.517 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.517 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.518 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.518 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.518 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.518 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.518 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.518 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.519 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.519 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.519 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.519 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.519 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.519 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.520 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.520 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.520 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.520 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.520 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.520 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.520 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.521 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.521 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.521 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.521 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.521 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.521 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.521 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.521 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.522 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.522 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.522 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.522 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.522 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.522 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.522 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.523 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.523 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.523 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.523 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.523 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.523 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.524 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.524 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.524 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.524 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.524 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.524 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.525 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.525 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.525 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.525 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.525 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.526 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.526 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.526 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.526 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.526 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.526 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.526 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.527 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-09-21T03:44:25.527 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.527 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.527 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.527 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-09-21T03:44:25.527 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-09-21T03:44:25.528 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.528 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.528 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.528 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.528 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.528 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:44:25.529 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.529 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.529 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.529 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-09-21T03:44:25.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-09-21T03:44:25.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.530 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.530 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.530 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.530 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.530 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.530 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:44:25.530 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.531 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.531 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.531 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-09-21T03:44:25.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-09-21T03:44:25.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.532 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.532 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:44:25.532 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.533 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-09-21T03:44:25.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-09-21T03:44:25.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.533 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.533 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.533 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.534 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.534 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:44:25.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.534 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.534 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.535 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-09-21T03:44:25.535 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-09-21T03:44:25.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.535 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.535 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.535 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.535 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.535 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.536 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:44:25.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.536 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.536 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-09-21T03:44:25.536 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-09-21T03:44:25.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.537 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.537 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.537 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.538 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.538 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.538 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-09-21T03:44:25.538 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-09-21T03:44:25.538 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.538 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.539 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.539 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.539 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.539 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.540 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.540 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-09-21T03:44:25.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-09-21T03:44:25.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.540 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:44:25.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.541 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.541 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.541 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.541 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.541 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.541 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.541 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-09-21T03:44:25.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-09-21T03:44:25.542 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.542 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.542 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.542 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.542 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.542 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.542 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.543 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.543 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.543 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.543 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:25.543 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:25.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.543 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.544 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-09-21T03:44:25.544 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-09-21T03:44:25.544 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.544 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.544 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.544 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.545 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.545 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-09-21T03:44:25.545 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-09-21T03:44:25.545 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.545 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.546 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-09-21T03:44:25.546 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.546 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.546 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.546 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.546 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.546 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.547 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-09-21T03:44:25.547 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-09-21T03:44:25.547 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.547 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.547 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.547 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.548 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.548 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.548 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.548 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.548 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.548 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.548 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-09-21T03:44:25.548 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-09-21T03:44:25.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.549 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.549 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.550 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.550 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.550 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.550 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.550 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.551 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.551 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.551 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.551 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.551 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.551 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.551 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.552 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.552 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.552 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.552 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.552 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.552 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.552 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.552 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.553 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.553 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.553 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.554 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.554 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.554 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.554 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.555 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.555 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.555 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.555 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.556 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.556 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.556 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.556 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.557 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.557 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.557 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.557 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.558 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.558 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.558 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.558 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.559 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.559 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-09-21T03:44:25.560 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-09-21T03:44:25.560 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.560 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.560 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.561 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.561 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.561 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.561 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.561 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-09-21T03:44:25.561 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-09-21T03:44:25.561 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.562 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.562 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.562 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.562 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.562 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.562 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.562 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.563 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.563 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.563 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.563 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.563 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.563 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.563 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.564 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.564 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.564 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.564 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.564 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.564 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.564 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.565 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.565 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-09-21T03:44:25.565 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-09-21T03:44:25.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.565 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.565 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:44:25.566 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.566 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.566 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.566 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.566 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.566 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.566 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.567 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.567 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-09-21T03:44:25.567 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.567 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.568 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.568 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.568 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.568 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.568 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.569 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.569 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.569 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.569 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.569 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.569 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.569 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.569 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.569 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.570 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.570 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.570 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.570 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.570 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.571 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.571 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.571 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.571 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.572 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.572 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.572 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.572 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.572 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.572 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.573 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.573 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.573 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.573 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.573 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.573 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.574 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.574 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.574 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.574 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.574 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.574 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.574 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.574 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.575 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.575 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.575 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.575 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.575 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.576 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.576 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.576 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.576 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.576 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.576 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.576 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.577 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.577 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.577 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.577 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.577 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.577 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.577 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.578 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.578 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.578 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.578 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.578 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.578 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-09-21T03:44:25.579 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.579 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.579 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.579 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.579 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.580 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.580 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.580 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.580 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.580 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.580 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.580 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.580 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.581 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.581 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.581 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.581 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.581 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.581 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.581 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.581 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.582 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.582 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.582 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.587 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.587 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.587 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.587 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.587 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-09-21T03:44:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-09-21T03:44:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-09-21T03:44:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:44:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-09-21T03:44:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-09-21T03:44:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-09-21T03:44:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-09-21T03:44:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:44:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:44:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:44:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:44:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:44:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-09-21T03:44:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-09-21T03:44:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-09-21T03:44:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-09-21T03:44:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-09-21T03:44:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-09-21T03:44:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-09-21T03:44:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-09-21T03:44:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-09-21T03:44:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-09-21T03:44:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-09-21T03:44:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-09-21T03:44:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-09-21T03:44:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:44:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:44:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: "modules": [ 2024-09-21T03:44:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-09-21T03:44:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard", 2024-09-21T03:44:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: "iostat", 2024-09-21T03:44:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: "nfs", 2024-09-21T03:44:25.622 INFO:teuthology.orchestra.run.smithi112.stdout: "restful", 2024-09-21T03:44:25.622 INFO:teuthology.orchestra.run.smithi112.stdout: "snap_schedule" 2024-09-21T03:44:25.622 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.622 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-09-21T03:44:25.622 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.622 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-09-21T03:44:25.622 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:44:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.630 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.630 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.630 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.630 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.630 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.630 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.630 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-09-21T03:44:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-09-21T03:44:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:44:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-09-21T03:44:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-09-21T03:44:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-09-21T03:44:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-09-21T03:44:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-09-21T03:44:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-09-21T03:44:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-09-21T03:44:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-09-21T03:44:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-09-21T03:44:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-09-21T03:44:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-09-21T03:44:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-09-21T03:44:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-09-21T03:44:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-09-21T03:44:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-09-21T03:44:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-09-21T03:44:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:44:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-09-21T03:44:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-09-21T03:44:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:44:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-09-21T03:44:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-09-21T03:44:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-09-21T03:44:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-09-21T03:44:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-09-21T03:44:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-09-21T03:44:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:44:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-09-21T03:44:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-09-21T03:44:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-09-21T03:44:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:44:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:44:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-09-21T03:44:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-09-21T03:44:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-09-21T03:44:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-09-21T03:44:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:44:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:44:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-09-21T03:44:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:44:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-09-21T03:44:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-09-21T03:44:25.657 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.657 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:44:25.657 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.657 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.657 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.657 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-09-21T03:44:25.657 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-09-21T03:44:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:44:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:44:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:44:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-09-21T03:44:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-09-21T03:44:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-09-21T03:44:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-09-21T03:44:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-09-21T03:44:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:44:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:44:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-09-21T03:44:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-09-21T03:44:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-09-21T03:44:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:44:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:44:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-09-21T03:44:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-09-21T03:44:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-09-21T03:44:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-09-21T03:44:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:44:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:44:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-09-21T03:44:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-09-21T03:44:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:44:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-09-21T03:44:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-09-21T03:44:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-09-21T03:44:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-09-21T03:44:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-09-21T03:44:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-09-21T03:44:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-09-21T03:44:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-09-21T03:44:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-09-21T03:44:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-09-21T03:44:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-09-21T03:44:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:44:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-09-21T03:44:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-09-21T03:44:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:25.685 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.685 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.685 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:44:25.685 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:44:25.685 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.685 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.685 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-09-21T03:44:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:44:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:44:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-09-21T03:44:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-09-21T03:44:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-09-21T03:44:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-09-21T03:44:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:44:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-09-21T03:44:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-09-21T03:44:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-09-21T03:44:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:44:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-09-21T03:44:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-09-21T03:44:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-09-21T03:44:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:44:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-09-21T03:44:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-09-21T03:44:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:44:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:44:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-09-21T03:44:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-09-21T03:44:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:44:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:44:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:44:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-09-21T03:44:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:44:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-09-21T03:44:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-09-21T03:44:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-09-21T03:44:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-09-21T03:44:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-09-21T03:44:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:44:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-09-21T03:44:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-09-21T03:44:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:44:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-09-21T03:44:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-09-21T03:44:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:44:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-09-21T03:44:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-09-21T03:44:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:44:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-09-21T03:44:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-09-21T03:44:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:44:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-09-21T03:44:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-09-21T03:44:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:44:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:44:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-09-21T03:44:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:44:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:44:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-09-21T03:44:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-09-21T03:44:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-09-21T03:44:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:44:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-09-21T03:44:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-09-21T03:44:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:44:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.715 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.715 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.715 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-09-21T03:44:25.715 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.715 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.715 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-09-21T03:44:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:44:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:44:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:44:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-09-21T03:44:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:44:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:44:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:44:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-09-21T03:44:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:44:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:44:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-09-21T03:44:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-09-21T03:44:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-09-21T03:44:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:44:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-09-21T03:44:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-09-21T03:44:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-09-21T03:44:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:44:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-09-21T03:44:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nginx": { 2024-09-21T03:44:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nginx", 2024-09-21T03:44:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:44:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nginx container image", 2024-09-21T03:44:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-09-21T03:44:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-09-21T03:44:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:44:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-09-21T03:44:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-09-21T03:44:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:44:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-09-21T03:44:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:44:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:44:25.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.731 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.731 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.731 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:44:25.731 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:44:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-09-21T03:44:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-09-21T03:44:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:44:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-09-21T03:44:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-09-21T03:44:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-09-21T03:44:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:44:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-09-21T03:44:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-09-21T03:44:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-09-21T03:44:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:44:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:44:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba_metrics": { 2024-09-21T03:44:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:44:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:44:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:44:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-09-21T03:44:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:44:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:44:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:44:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-09-21T03:44:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-09-21T03:44:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:44:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-09-21T03:44:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:44:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:44:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:44:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:44:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.746 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.746 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-09-21T03:44:25.746 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.746 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.746 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.746 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:44:25.746 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-09-21T03:44:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-09-21T03:44:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-09-21T03:44:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:44:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-09-21T03:44:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-09-21T03:44:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-09-21T03:44:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:44:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-09-21T03:44:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-09-21T03:44:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:44:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-09-21T03:44:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-09-21T03:44:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:44:25.755 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.755 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.755 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.755 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-09-21T03:44:25.755 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:44:25.755 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.756 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.756 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.756 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:44:25.756 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.756 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.756 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.756 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:44:25.756 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.757 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.757 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.757 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-09-21T03:44:25.757 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-09-21T03:44:25.757 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.757 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.757 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:44:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-09-21T03:44:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-09-21T03:44:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:44:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-09-21T03:44:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-09-21T03:44:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:44:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-09-21T03:44:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-09-21T03:44:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:44:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:44:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:44:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:44:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:44:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:44:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:44:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-09-21T03:44:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:44:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-09-21T03:44:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-09-21T03:44:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:44:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-09-21T03:44:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-09-21T03:44:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:44:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-09-21T03:44:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-09-21T03:44:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-09-21T03:44:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-09-21T03:44:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-09-21T03:44:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-09-21T03:44:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-09-21T03:44:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-09-21T03:44:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:44:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-09-21T03:44:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-09-21T03:44:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-09-21T03:44:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:44:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-09-21T03:44:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:44:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:44:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:44:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-09-21T03:44:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-09-21T03:44:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-09-21T03:44:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-09-21T03:44:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:44:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-09-21T03:44:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-09-21T03:44:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:44:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-09-21T03:44:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:44:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:44:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-09-21T03:44:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-09-21T03:44:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-09-21T03:44:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:44:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-09-21T03:44:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-09-21T03:44:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:44:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:44:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:44:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-09-21T03:44:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:44:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_keepalive_interval": { 2024-09-21T03:44:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:44:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-09-21T03:44:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:44:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-09-21T03:44:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-09-21T03:44:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:44:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-09-21T03:44:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-09-21T03:44:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:44:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-09-21T03:44:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:44:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:44:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-09-21T03:44:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:44:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:44:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-09-21T03:44:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:44:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.815 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:44:25.815 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.815 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.815 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.815 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.815 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-09-21T03:44:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.823 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.823 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.823 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.823 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.823 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.823 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.826 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.826 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-09-21T03:44:25.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-09-21T03:44:25.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.826 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.826 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-09-21T03:44:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:44:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.829 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.829 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.829 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.829 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.829 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.829 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.829 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-09-21T03:44:25.830 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-09-21T03:44:25.830 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.830 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.830 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.830 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-09-21T03:44:25.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.830 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.831 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.831 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:44:25.831 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.831 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.831 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.831 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.831 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-09-21T03:44:25.832 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.832 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.832 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.832 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:44:25.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:44:25.833 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.833 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.833 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.833 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:25.833 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.833 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.833 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.833 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.834 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.834 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.834 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.834 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:44:25.834 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:44:25.834 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.834 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.835 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.835 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.835 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.835 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.835 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.835 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.835 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.836 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.836 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.836 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:44:25.836 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:44:25.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.836 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.836 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.837 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.838 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.838 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:44:25.838 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:44:25.838 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.838 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.838 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.838 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.839 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.839 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.839 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.839 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.839 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.839 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.839 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.840 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:44:25.840 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:44:25.840 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.840 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.840 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.840 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.840 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.840 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.841 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.841 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.841 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.841 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.841 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.841 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:44:25.841 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:44:25.842 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.842 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.842 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.842 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.842 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.842 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.842 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.843 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.843 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.843 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.843 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.843 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:44:25.843 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:44:25.843 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.844 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.844 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.844 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.844 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.844 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.844 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.844 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.844 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.845 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.845 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.845 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:44:25.845 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:44:25.845 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.845 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.846 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.846 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.846 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.846 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.846 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.846 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.846 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.846 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.847 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.847 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:44:25.847 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:44:25.847 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.847 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.847 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.848 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.848 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.848 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.848 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.848 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.848 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.848 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.849 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.849 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:44:25.849 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:44:25.849 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.849 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.849 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.849 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.850 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.850 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.850 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.850 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.850 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.850 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.850 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.851 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:44:25.851 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:44:25.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.851 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.851 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.851 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.851 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.852 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.852 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.852 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.852 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.852 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:44:25.853 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:44:25.853 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.853 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.853 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.853 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.853 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.853 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.853 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.854 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.854 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.854 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.854 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.854 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:44:25.854 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:44:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.855 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:44:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:44:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.856 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.857 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.857 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.857 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.857 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.857 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.857 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.857 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.858 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.858 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.858 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:44:25.858 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:44:25.858 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.858 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.858 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.859 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-09-21T03:44:25.859 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.859 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.859 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.859 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.859 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.859 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.859 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.860 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:44:25.860 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:44:25.860 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.860 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.860 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.860 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.861 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.861 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.861 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.861 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.861 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.861 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.861 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.862 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-09-21T03:44:25.862 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:44:25.862 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.862 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.862 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.862 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.862 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.862 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.863 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.863 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.863 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.863 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.863 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.863 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:44:25.863 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:44:25.864 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.864 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.864 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.864 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-09-21T03:44:25.864 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.864 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.864 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.865 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.865 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.865 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.865 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.865 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:44:25.865 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:44:25.865 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.866 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.866 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.866 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.866 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.866 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.866 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.867 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.867 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.867 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.867 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.867 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:44:25.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:44:25.868 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.868 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.868 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.868 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.868 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.869 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.870 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.870 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.870 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.870 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.870 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.870 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:44:25.871 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:44:25.871 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.871 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.871 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.871 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.871 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.871 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.871 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.871 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.872 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.872 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.872 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.872 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:44:25.872 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:44:25.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.872 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.872 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.873 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.873 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.873 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.873 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.873 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.873 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.873 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.873 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.874 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:44:25.874 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:44:25.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.874 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.874 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.874 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-09-21T03:44:25.874 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.875 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.875 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.875 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.875 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.875 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.875 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.875 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:44:25.875 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:44:25.876 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.876 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.876 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.876 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-09-21T03:44:25.876 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.876 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.876 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.876 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.877 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.877 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.877 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.877 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:44:25.877 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:44:25.877 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.877 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.877 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.878 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.878 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.878 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.878 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.878 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.878 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.878 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.878 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.879 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:44:25.879 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:44:25.879 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.879 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.879 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.879 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.879 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.879 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.880 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.880 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.880 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.880 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.880 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.880 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:44:25.880 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:44:25.880 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.881 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.881 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.881 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.881 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.881 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.881 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.881 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.881 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.882 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.882 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.882 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:44:25.882 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:44:25.882 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.882 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.882 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.882 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.883 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.883 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.883 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.883 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.883 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.883 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.883 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:44:25.884 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:44:25.884 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.884 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.884 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.884 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.884 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.884 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.884 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.885 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.885 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.885 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:44:25.885 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:44:25.885 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.885 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.886 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.886 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.886 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.886 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.886 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.886 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.886 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.886 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.886 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.887 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:44:25.887 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:44:25.887 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.887 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.887 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.887 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.887 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.887 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.888 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.888 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.888 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.888 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.888 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.888 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:44:25.888 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:44:25.889 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.889 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.889 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.889 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.889 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.889 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.889 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.889 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.889 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.890 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.890 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.890 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:44:25.890 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:44:25.890 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.890 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.890 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.890 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.891 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.891 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.891 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.891 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.891 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.891 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.891 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:44:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:44:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.892 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.893 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.893 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.893 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.893 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.893 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:44:25.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:44:25.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.894 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.894 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.894 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:44:25.894 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.894 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.894 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.894 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.894 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.895 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.895 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.895 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:44:25.895 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:44:25.895 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.895 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.895 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.895 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:25.896 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.896 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.896 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.896 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.896 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.896 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.896 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.896 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:44:25.897 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:44:25.897 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.897 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.897 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.897 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-09-21T03:44:25.897 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.897 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.898 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.898 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.898 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.898 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.898 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:44:25.898 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:44:25.898 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.898 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.899 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.899 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-09-21T03:44:25.899 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.899 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.899 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.899 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.899 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.899 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.900 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.900 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:44:25.900 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:44:25.900 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.900 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.900 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.900 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.901 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.901 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.901 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.901 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.901 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.901 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.901 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.902 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:44:25.902 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:44:25.902 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.902 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.902 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.902 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-09-21T03:44:25.902 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.902 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.903 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.903 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.903 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.903 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.903 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.903 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:44:25.903 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:44:25.903 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.904 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.904 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.904 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.904 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.904 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.904 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.904 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.905 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.905 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.905 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:44:25.905 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:44:25.905 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.905 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.905 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.905 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.906 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.906 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.906 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.906 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.906 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.906 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.906 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.907 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:44:25.907 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:44:25.907 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.907 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.907 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.907 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.907 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.907 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.907 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.908 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.908 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.908 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.908 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.908 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:44:25.908 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:44:25.908 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.908 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.909 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.909 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.909 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.909 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.909 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.909 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.909 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.909 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.909 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.910 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:44:25.910 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:44:25.910 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.910 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.910 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.910 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:25.910 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.911 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.911 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.911 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.911 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.911 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.911 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.911 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:44:25.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:44:25.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.912 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.912 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.912 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:25.912 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.912 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.912 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.912 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.912 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.913 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.913 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.913 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-09-21T03:44:25.913 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-09-21T03:44:25.913 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.913 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.913 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.914 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.914 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.914 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.914 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.914 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.914 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.914 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.914 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.915 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-09-21T03:44:25.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-09-21T03:44:25.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.915 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.915 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.915 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.915 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.915 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.916 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.916 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.916 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.916 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.916 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.916 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-09-21T03:44:25.916 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-09-21T03:44:25.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.917 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.917 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.917 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.917 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.917 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.917 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.917 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:44:25.917 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.917 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.918 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.918 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-09-21T03:44:25.918 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-09-21T03:44:25.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.918 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.919 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.919 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-09-21T03:44:25.919 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.919 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.919 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.919 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.919 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.919 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.919 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.920 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-09-21T03:44:25.920 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-09-21T03:44:25.920 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.920 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.920 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.920 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.920 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.921 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.921 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.921 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.921 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.921 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.921 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.921 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.921 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.922 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.922 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.922 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.922 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.922 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.922 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.922 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.923 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.923 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.923 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.923 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.923 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.923 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.923 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.923 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.923 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.924 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.924 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.924 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.924 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.924 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.924 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.924 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.925 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.925 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.925 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.925 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.925 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.925 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.925 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.925 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.926 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.926 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.926 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.926 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.926 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.926 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.926 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.926 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.927 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.927 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.927 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.927 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.927 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.927 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.927 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.927 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.928 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.928 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.928 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.928 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.928 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.928 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.928 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.929 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.929 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.929 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.929 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.929 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.929 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.929 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.929 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.930 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-09-21T03:44:25.930 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-09-21T03:44:25.930 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.930 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.930 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.930 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.930 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.931 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.931 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.931 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-09-21T03:44:25.931 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.931 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.931 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.931 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:44:25.932 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:44:25.932 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.932 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.932 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.932 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.932 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.932 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.932 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.932 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.933 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.933 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.933 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.933 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-09-21T03:44:25.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-09-21T03:44:25.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.933 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.934 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.934 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-09-21T03:44:25.934 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.934 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.934 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.934 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.934 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.934 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.935 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.935 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-09-21T03:44:25.935 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-09-21T03:44:25.935 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.935 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.935 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.935 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-09-21T03:44:25.935 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.936 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.936 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.936 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.936 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.936 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.936 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.936 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.936 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.937 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.937 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.937 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.937 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.937 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.937 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.937 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.938 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.938 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.938 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.938 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-09-21T03:44:25.938 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-09-21T03:44:25.938 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.938 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.939 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.939 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.939 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.939 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.939 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.939 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.939 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.939 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.940 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.940 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-09-21T03:44:25.940 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-09-21T03:44:25.940 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.940 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.940 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.940 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-09-21T03:44:25.940 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.941 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.941 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.941 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.941 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.941 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.941 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.941 INFO:teuthology.orchestra.run.smithi112.stdout: "sso_oauth2": { 2024-09-21T03:44:25.941 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sso_oauth2", 2024-09-21T03:44:25.942 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.942 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.942 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.942 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.942 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.942 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.942 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.942 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.943 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.943 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.943 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.943 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-09-21T03:44:25.943 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-09-21T03:44:25.943 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.943 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.943 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.944 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-09-21T03:44:25.944 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.944 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.944 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.944 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.944 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-09-21T03:44:25.944 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.944 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.944 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.945 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.945 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.945 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-09-21T03:44:25.945 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-09-21T03:44:25.945 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.945 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.945 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.946 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-09-21T03:44:25.946 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-09-21T03:44:25.946 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-09-21T03:44:25.946 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.946 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.946 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.946 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.946 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.947 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-09-21T03:44:25.947 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-09-21T03:44:25.947 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.947 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.947 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.947 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.947 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.947 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.948 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.948 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.948 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.948 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.948 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.948 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.948 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.948 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-09-21T03:44:25.949 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.949 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.949 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.949 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-09-21T03:44:25.949 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-09-21T03:44:25.949 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.949 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.949 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.949 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.950 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.950 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.950 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.950 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-09-21T03:44:25.950 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.950 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.950 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.950 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.951 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.951 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.951 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.951 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.951 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.951 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.951 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.951 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.951 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.952 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.952 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.952 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.952 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.952 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.952 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.952 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.952 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.952 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.953 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.953 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.953 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.953 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.953 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.953 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.953 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.953 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.954 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.954 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.954 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.954 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.954 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.954 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.955 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.955 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.955 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.955 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.955 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.955 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.955 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.955 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.957 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.957 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.957 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.957 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.957 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.957 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.957 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.958 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.958 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.958 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.958 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.958 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.958 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.958 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.958 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.959 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-09-21T03:44:25.959 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-09-21T03:44:25.959 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.959 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.959 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.959 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-09-21T03:44:25.959 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.960 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.960 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.960 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:44:25.960 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.960 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.960 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.960 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-09-21T03:44:25.960 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-09-21T03:44:25.961 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.961 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.961 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.961 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-09-21T03:44:25.961 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.961 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.961 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.961 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:44:25.961 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.962 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.962 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.962 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-09-21T03:44:25.962 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-09-21T03:44:25.962 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.962 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.962 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.963 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-09-21T03:44:25.963 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.963 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.963 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.963 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:44:25.963 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.963 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.963 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.964 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-09-21T03:44:25.964 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-09-21T03:44:25.964 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.964 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.964 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.964 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-09-21T03:44:25.964 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.964 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.965 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.965 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:44:25.965 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.965 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.965 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.965 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-09-21T03:44:25.965 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-09-21T03:44:25.965 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.966 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.966 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.966 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:25.966 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.966 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.966 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.966 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:44:25.966 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.967 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.967 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.967 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:25.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:25.967 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.967 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.968 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:25.968 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.968 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.968 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.968 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:44:25.968 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.968 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.968 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.969 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.969 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.969 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.969 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.969 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.969 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.969 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.969 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.970 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.970 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.970 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.970 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.970 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.970 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-09-21T03:44:25.970 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-09-21T03:44:25.971 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.971 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.971 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.971 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-09-21T03:44:25.971 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.971 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.971 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.971 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:44:25.971 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.972 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.972 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.972 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.972 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.972 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-09-21T03:44:25.972 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:25.972 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:25.973 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.973 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.973 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.973 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.973 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.973 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.973 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.973 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.974 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.974 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.974 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.974 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.974 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.974 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.974 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.974 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.974 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.975 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.975 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.975 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.975 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.975 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.975 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.975 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.976 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.976 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.976 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.976 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.976 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.976 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.976 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.976 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.976 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.977 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.977 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.977 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.977 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.977 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.977 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.977 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:25.977 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.978 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.978 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.978 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.978 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.978 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.978 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.978 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.978 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.979 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.979 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.979 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.979 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.979 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.979 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:25.979 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:25.979 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.980 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.980 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.980 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.980 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.980 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.980 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.980 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.980 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.980 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.981 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.981 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-09-21T03:44:25.981 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-09-21T03:44:25.981 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.981 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.981 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.981 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-09-21T03:44:25.982 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.982 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.982 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.982 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.982 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.982 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.982 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.982 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-09-21T03:44:25.983 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-09-21T03:44:25.983 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.983 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.983 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-09-21T03:44:25.983 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.983 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.983 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.984 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.984 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.984 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.984 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.984 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:25.984 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:25.984 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.984 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.985 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.985 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-09-21T03:44:25.985 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.985 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.985 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.985 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.985 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.985 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.986 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:25.986 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:25.986 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.986 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:25.986 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.986 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:25.986 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.986 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.987 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.987 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.987 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.987 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.987 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.987 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.987 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:25.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:25.988 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-09-21T03:44:25.988 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-09-21T03:44:25.988 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:44:25.988 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:25.988 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-09-21T03:44:25.988 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-09-21T03:44:25.988 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:25.989 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.989 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.989 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-09-21T03:44:25.989 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.989 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.989 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.989 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:44:25.989 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.989 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.990 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.990 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-09-21T03:44:25.990 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-09-21T03:44:25.990 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.990 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.990 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.990 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-09-21T03:44:25.991 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.991 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.991 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.991 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:44:25.991 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.991 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.991 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.991 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-09-21T03:44:25.992 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-09-21T03:44:25.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.992 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.992 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.992 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.992 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.992 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.992 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.993 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:44:25.993 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.993 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.993 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.993 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:25.993 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:25.993 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:25.993 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:25.994 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-09-21T03:44:25.994 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-09-21T03:44:25.994 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.994 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.994 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:44:25.994 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.994 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.995 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.995 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:25.995 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:25.995 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.995 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.995 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.995 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:25.996 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.996 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.996 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:25.996 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:25.996 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:25.996 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:25.996 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:25.996 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:25.996 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:25.997 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:25.997 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.997 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.997 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.997 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.997 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:25.997 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:25.998 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:25.998 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.998 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:25.998 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:25.998 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:25.998 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:25.998 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:25.998 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:25.999 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:25.999 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:25.999 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:25.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:25.999 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:25.999 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:25.999 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:25.999 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:25.999 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.000 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.000 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.000 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.000 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.000 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.000 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.000 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.000 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.001 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.001 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.001 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.001 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.001 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.001 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.001 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.001 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.002 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.002 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.002 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.002 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.002 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.002 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.002 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.002 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.003 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.003 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.003 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.003 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.003 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-09-21T03:44:26.003 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-09-21T03:44:26.003 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.003 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.004 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.004 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.004 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.004 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:44:26.004 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.004 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.004 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.005 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-09-21T03:44:26.005 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-09-21T03:44:26.005 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.005 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.005 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.005 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-09-21T03:44:26.005 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.006 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.006 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.006 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-09-21T03:44:26.006 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.006 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.006 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.006 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.007 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.007 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.007 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.007 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.007 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.007 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.007 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.008 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.008 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.008 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-09-21T03:44:26.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-09-21T03:44:26.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.008 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.008 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.009 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-09-21T03:44:26.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.009 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.009 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:44:26.009 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.009 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.009 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.010 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-09-21T03:44:26.010 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-09-21T03:44:26.010 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.010 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.010 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.010 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:26.010 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:26.010 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-09-21T03:44:26.010 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.011 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:44:26.011 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.011 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.011 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.011 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-09-21T03:44:26.011 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-09-21T03:44:26.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.012 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.012 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.012 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.012 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.012 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.012 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.012 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:44:26.012 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.012 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.013 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.013 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-09-21T03:44:26.013 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-09-21T03:44:26.013 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.013 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.013 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.013 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-09-21T03:44:26.014 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.014 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.014 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.014 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:44:26.014 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.014 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.014 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.014 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.014 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.015 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.015 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-09-21T03:44:26.015 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.015 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.015 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.015 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.015 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.015 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.016 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.016 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.016 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.016 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.016 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.016 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.016 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.017 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.017 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.017 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.017 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.017 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.017 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.017 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.017 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.017 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.018 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.018 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.018 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.018 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.018 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.018 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.018 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.018 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.019 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.019 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.019 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.019 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.019 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.019 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.020 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.020 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.020 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.020 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.020 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.020 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.020 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.020 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.020 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.021 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.021 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.021 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.021 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.021 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.021 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.021 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.021 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.021 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.022 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.022 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.022 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.022 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.022 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.022 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.022 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.023 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.023 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.023 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.023 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.023 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.023 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.023 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.023 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.024 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.024 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.024 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.024 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.024 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.024 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.024 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.025 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.025 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.025 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.025 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.025 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-09-21T03:44:26.026 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.026 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.026 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.026 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.026 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.026 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.026 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.026 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.027 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.027 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.027 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.027 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.027 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.027 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.027 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.027 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.027 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.028 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.028 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.028 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.028 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.028 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.028 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.028 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.029 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.029 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.029 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.029 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.029 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.029 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.029 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.030 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.030 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.030 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.030 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.030 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.030 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.031 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.031 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.031 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.031 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.031 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.031 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.031 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.031 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.032 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.032 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.032 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.032 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.032 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.032 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.032 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.032 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.033 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.033 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.033 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.033 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.033 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.033 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.033 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.034 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.034 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.034 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.034 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.034 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.034 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.034 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.034 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.035 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.035 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.035 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.035 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.035 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.035 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.036 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.036 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.036 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.036 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-09-21T03:44:26.037 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.037 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.037 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.037 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-09-21T03:44:26.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:44:26.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.037 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.038 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.038 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-09-21T03:44:26.038 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.038 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.038 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.038 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:44:26.038 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.038 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.039 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.039 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-09-21T03:44:26.039 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-09-21T03:44:26.039 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.039 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.039 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.040 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:26.040 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-09-21T03:44:26.040 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.040 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.040 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:44:26.040 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.040 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.040 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.041 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.041 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.041 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.041 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.041 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.041 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.041 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.042 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.042 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.042 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.042 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.042 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.042 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.042 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.042 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.043 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.043 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.043 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.043 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.043 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.043 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.044 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.044 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.044 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.044 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.044 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.044 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.044 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.044 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.045 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.045 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.045 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.045 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.045 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.045 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.045 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.046 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.046 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.046 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.046 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.046 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.046 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.046 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.046 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.046 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.047 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.047 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.047 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.047 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.047 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.047 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.048 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.048 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.048 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.048 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.048 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.048 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.048 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.048 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.048 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.049 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.049 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.049 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.049 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.049 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.049 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.049 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.050 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.050 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.050 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.050 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.050 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.050 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.050 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.051 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-09-21T03:44:26.051 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.051 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.051 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.051 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-09-21T03:44:26.051 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-09-21T03:44:26.051 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.051 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.051 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.052 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-09-21T03:44:26.052 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.052 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.052 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.052 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:44:26.052 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:44:26.052 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.052 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.053 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.053 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.053 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.053 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.053 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.053 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.053 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.053 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.053 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.054 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.054 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.054 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.054 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.054 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.054 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.054 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.054 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.055 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.055 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.055 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.055 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.055 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.055 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.055 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.056 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.056 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.056 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.056 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.056 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.056 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.056 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.056 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.057 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.057 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.057 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.057 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.057 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.057 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.057 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.057 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.058 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.058 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.058 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.058 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.058 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.058 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.058 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.058 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.059 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.059 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.059 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.059 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.059 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.060 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.060 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.060 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.060 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.060 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.060 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.060 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.060 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.061 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.061 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-09-21T03:44:26.061 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-09-21T03:44:26.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.061 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.061 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.061 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.062 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.062 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.062 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.062 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:44:26.062 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:44:26.062 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.062 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.062 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-09-21T03:44:26.063 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-09-21T03:44:26.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.063 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.063 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.063 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-09-21T03:44:26.063 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.063 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.063 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.064 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:44:26.064 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.064 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.064 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.064 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-09-21T03:44:26.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-09-21T03:44:26.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.065 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.065 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.065 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-09-21T03:44:26.065 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.065 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.065 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.065 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:44:26.065 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.065 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.066 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.066 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-09-21T03:44:26.066 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-09-21T03:44:26.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.066 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.066 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.066 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.067 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.067 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.067 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:44:26.067 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.067 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.067 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.068 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.068 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.068 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.068 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.068 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.068 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.068 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.068 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.068 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.069 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.069 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.069 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.069 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.069 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-09-21T03:44:26.069 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-09-21T03:44:26.069 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.070 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.070 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.070 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-09-21T03:44:26.070 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.070 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.070 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.070 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:44:26.070 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:44:26.070 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.071 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.071 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.071 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.071 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-09-21T03:44:26.071 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.071 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.072 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.072 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.072 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.072 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.072 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.072 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.072 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.072 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.072 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.073 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.073 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.073 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.073 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.073 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.073 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.073 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.073 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.074 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.074 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.074 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.074 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.074 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.074 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.074 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.074 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.075 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.075 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.075 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.075 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.075 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.075 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.075 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.075 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.075 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.076 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.076 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.076 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.076 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.076 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.076 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.077 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.077 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.077 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.077 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.077 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.077 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.077 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.078 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.078 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.078 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.078 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.078 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.078 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.079 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.079 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.079 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.079 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.079 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.079 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.079 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.079 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.080 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.080 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.080 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.080 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.080 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.080 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.080 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.080 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.081 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.081 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.081 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.081 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.081 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.081 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.081 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.081 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.081 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.082 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-09-21T03:44:26.082 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.082 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.082 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.082 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.082 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.082 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.083 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.083 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.083 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.083 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.083 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.083 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.083 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.083 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.083 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.084 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.084 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.084 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.084 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.084 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.084 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.084 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.084 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.085 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.085 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.085 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.085 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.085 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.085 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.085 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.085 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.086 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.086 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.086 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.086 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.086 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.086 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.086 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.087 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.087 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.087 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.087 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.087 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.087 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.087 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.087 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.088 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.088 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.088 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.088 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.088 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.088 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.088 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.088 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.088 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.089 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.089 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.089 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.089 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.089 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.089 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.089 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.089 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.090 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.090 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.090 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.090 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.090 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.090 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.090 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.091 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.091 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.091 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.091 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.091 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.091 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.091 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.091 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.092 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.092 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.092 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.092 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.092 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.092 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-09-21T03:44:26.093 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.093 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.093 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.093 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.093 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.093 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.093 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.093 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.094 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.094 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.094 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.094 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.094 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.094 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.094 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.094 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.094 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.095 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.095 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.095 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.095 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.095 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.095 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.095 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.095 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.096 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.096 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.096 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.096 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.096 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.096 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.096 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.096 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.097 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.097 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.097 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.097 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.097 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.097 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.098 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.098 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.098 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.098 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.098 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.098 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.098 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.098 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.098 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.099 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.099 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.099 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.099 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.099 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.099 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.099 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.099 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.100 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.100 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.100 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.100 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.100 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.100 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.100 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.100 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.101 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.101 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.101 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.101 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.101 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.101 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.101 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.101 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.102 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.102 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.102 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.102 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.102 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.102 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.102 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.102 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.102 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.103 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.103 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-09-21T03:44:26.103 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.103 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.103 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.103 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-09-21T03:44:26.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-09-21T03:44:26.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.104 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.104 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.104 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.104 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.104 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.104 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:44:26.105 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.105 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.105 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.105 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.105 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.105 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.105 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.106 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.106 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.106 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.106 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.106 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.106 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.106 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.106 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.106 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.107 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.107 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.107 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.107 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.107 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.107 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.107 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.107 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.108 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.108 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.108 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.108 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.108 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.108 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.108 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.108 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.109 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.109 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.109 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.109 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.109 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.109 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.109 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.109 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.110 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.110 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.110 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.110 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.110 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.110 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.110 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.110 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.111 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.111 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.111 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.111 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.111 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.111 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.111 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.111 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.112 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.112 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.112 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.112 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.112 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.112 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.112 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.112 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.112 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.113 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.113 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.113 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.113 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-09-21T03:44:26.113 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-09-21T03:44:26.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.113 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.113 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.114 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.114 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.114 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.114 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.114 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-09-21T03:44:26.114 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-09-21T03:44:26.114 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-09-21T03:44:26.114 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.115 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-09-21T03:44:26.115 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.115 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.115 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.115 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.115 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.115 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.115 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.116 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.116 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.116 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.116 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.116 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.116 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.116 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.116 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.116 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.117 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.117 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.117 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-09-21T03:44:26.117 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.117 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.117 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.117 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.118 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.118 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.118 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.118 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.118 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.118 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.118 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.118 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.119 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.119 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.119 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.119 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.119 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.119 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.119 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.119 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.119 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.120 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.120 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.120 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.120 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.120 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.120 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.121 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.121 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.121 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.121 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.121 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.121 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.121 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.121 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.122 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.122 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.122 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.122 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.122 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.122 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.122 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.122 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.122 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.123 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.123 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.123 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.123 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.123 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.123 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.123 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.123 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.124 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.124 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.124 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.124 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.124 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.124 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.124 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.124 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.125 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.125 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.125 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.125 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.125 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.125 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.125 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.125 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.125 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.126 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.126 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.126 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.126 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.126 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.126 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.126 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.127 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.127 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.127 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.127 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.127 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.127 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.128 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-09-21T03:44:26.128 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.128 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.128 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.128 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.128 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.128 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.128 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.128 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.129 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.129 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.129 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.129 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.129 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.129 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.129 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.129 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.130 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.130 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.130 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.130 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.130 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.130 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.130 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.130 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.131 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.131 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.131 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.131 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.131 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.131 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.131 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.131 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.132 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.132 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.132 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.132 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.132 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.132 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.132 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-09-21T03:44:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.145 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.145 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.145 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.145 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.145 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.145 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.145 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.146 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.146 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.146 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.146 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.146 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.146 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.146 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.146 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.146 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.147 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.147 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.147 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:26.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:26.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:26.147 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.147 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.148 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:26.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.148 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.148 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.148 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.148 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.148 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.148 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.149 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.149 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.149 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.149 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.149 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.149 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.149 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.149 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.150 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.150 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.150 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.150 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-09-21T03:44:26.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-09-21T03:44:26.150 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:26.150 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.151 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.151 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-09-21T03:44:26.151 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-09-21T03:44:26.151 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.151 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.151 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-09-21T03:44:26.151 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:44:26.151 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.151 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.152 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.152 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.152 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-09-21T03:44:26.152 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.152 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.152 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.152 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-09-21T03:44:26.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:44:26.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.153 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.153 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.153 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.153 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.153 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.153 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.153 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:44:26.154 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.154 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.154 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.154 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-09-21T03:44:26.154 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-09-21T03:44:26.154 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.154 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.154 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.155 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:26.155 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.155 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.155 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.155 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.155 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.155 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.155 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.156 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.156 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.156 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.156 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.156 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.156 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.156 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.157 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.157 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.157 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.157 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.157 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.157 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.157 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.157 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.157 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.158 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.158 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.158 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.158 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.158 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.158 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.158 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.159 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.159 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.159 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.159 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.159 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.159 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.159 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.159 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.160 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.160 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.160 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.160 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.160 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.160 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.160 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.160 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.160 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.161 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.161 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.161 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.161 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.161 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.161 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.161 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.161 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.161 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.162 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.162 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.162 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.162 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.162 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.162 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.162 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.163 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.163 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.163 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.163 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.163 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.163 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.163 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.163 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-09-21T03:44:26.164 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-09-21T03:44:26.164 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.164 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.164 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.164 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-09-21T03:44:26.164 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.164 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.164 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.165 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:44:26.165 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.165 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.165 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.165 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-09-21T03:44:26.165 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-09-21T03:44:26.165 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:26.165 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.166 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.166 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:26.166 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.166 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.166 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.166 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:44:26.166 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.166 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.167 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.167 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.167 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.167 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.167 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.167 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.168 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.168 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.168 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.168 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.168 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.168 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.168 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.168 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.169 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-09-21T03:44:26.169 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.169 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.169 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.169 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-09-21T03:44:26.169 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-09-21T03:44:26.169 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.170 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.170 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.170 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:26.170 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.170 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.170 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.170 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.170 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.170 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.171 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.171 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-09-21T03:44:26.171 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-09-21T03:44:26.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.171 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.171 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.171 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:26.172 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.172 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.172 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.172 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:44:26.172 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:44:26.172 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.172 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.173 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.173 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.173 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.173 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.173 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.173 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.173 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.173 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.174 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.174 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.174 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.174 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.174 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.174 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.174 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.174 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.175 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.175 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.175 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.175 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.175 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.175 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.175 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.176 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.176 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.176 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.176 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.176 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.176 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.176 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.176 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.176 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.177 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.177 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.177 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.177 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.177 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.177 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.177 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.178 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.178 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.178 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.178 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.178 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.178 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.178 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.178 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.178 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.179 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.179 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.179 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.179 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.179 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.179 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.179 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.180 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.180 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.180 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.180 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.180 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.180 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.180 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.180 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.180 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.181 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.181 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-09-21T03:44:26.181 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-09-21T03:44:26.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.181 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.181 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.181 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.182 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.182 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.182 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.182 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.182 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.182 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.183 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:44:26.183 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:44:26.183 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.183 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.183 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.183 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-09-21T03:44:26.183 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.183 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.184 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.184 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.184 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.184 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.184 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.184 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-09-21T03:44:26.184 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-09-21T03:44:26.184 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:26.185 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.185 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-09-21T03:44:26.185 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.185 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.185 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.185 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.185 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.185 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.186 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.186 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-09-21T03:44:26.186 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-09-21T03:44:26.186 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.186 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.186 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.186 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-09-21T03:44:26.186 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.187 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.187 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.187 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:44:26.187 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.187 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.187 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.187 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-09-21T03:44:26.188 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-09-21T03:44:26.188 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.188 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.188 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.188 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-09-21T03:44:26.188 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.188 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.188 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.188 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:44:26.189 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.189 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.189 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.189 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.189 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.189 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.189 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.190 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.190 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.190 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.190 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.190 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.190 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.190 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.190 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.191 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.191 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-09-21T03:44:26.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-09-21T03:44:26.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.191 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.191 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.191 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-09-21T03:44:26.191 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.192 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.192 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.192 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.192 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.192 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.192 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-09-21T03:44:26.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-09-21T03:44:26.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.193 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.193 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-09-21T03:44:26.193 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.193 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.193 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.193 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-09-21T03:44:26.193 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-09-21T03:44:26.194 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.194 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.194 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.194 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.194 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.194 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-09-21T03:44:26.194 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-09-21T03:44:26.195 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.195 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.195 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.195 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-09-21T03:44:26.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-09-21T03:44:26.195 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-09-21T03:44:26.195 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.195 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.195 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.196 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.196 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.196 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.196 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.196 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-09-21T03:44:26.196 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.196 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.196 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.197 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.197 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.197 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.197 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.197 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.197 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.197 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.198 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.198 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.198 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.198 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.198 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.198 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.198 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.198 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.198 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.199 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.199 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.199 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.199 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.199 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.199 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.200 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.200 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.200 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.200 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.200 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.200 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.200 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.200 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.201 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.201 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.201 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.201 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.201 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.201 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.201 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.201 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.201 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.202 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.202 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.202 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.202 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.202 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.202 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.202 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.202 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.202 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.203 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.203 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.203 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.203 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.203 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.203 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.203 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.203 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.204 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.204 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.204 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.204 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.204 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.204 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.205 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-09-21T03:44:26.205 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:44:26.205 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.205 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.205 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.205 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-09-21T03:44:26.205 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.205 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.205 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.206 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.206 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.206 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.206 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.206 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:44:26.206 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:44:26.206 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.207 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.207 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.207 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.207 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.207 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.207 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.207 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.208 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.208 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.208 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.208 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.208 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.208 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.209 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.209 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.209 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.209 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.209 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.209 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.209 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.209 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.210 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-09-21T03:44:26.210 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-09-21T03:44:26.210 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.210 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.210 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.210 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.210 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.210 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.210 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.211 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.211 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.211 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.211 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.211 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.211 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-09-21T03:44:26.212 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.212 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.212 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.212 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-09-21T03:44:26.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-09-21T03:44:26.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.212 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.212 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.213 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:26.213 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.213 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.213 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.213 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.213 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.213 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.213 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.214 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-09-21T03:44:26.214 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-09-21T03:44:26.214 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.214 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.214 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.214 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.214 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.214 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.215 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.215 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.215 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.215 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.215 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.215 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.215 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.215 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.216 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.216 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.216 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.216 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.216 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.216 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.216 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.216 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.216 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.217 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.217 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.217 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.217 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.217 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.217 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.217 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.217 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.218 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.218 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.218 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.218 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.218 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.218 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.219 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.220 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.221 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.222 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: "max_requests": { 2024-09-21T03:44:26.223 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_requests", 2024-09-21T03:44:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-09-21T03:44:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:44:26.224 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-09-21T03:44:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-09-21T03:44:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.225 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.226 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-09-21T03:44:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-09-21T03:44:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.227 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.228 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.229 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-09-21T03:44:26.230 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.231 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.232 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.233 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.234 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.235 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.236 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.237 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.238 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:44:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:44:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.240 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-09-21T03:44:26.240 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.240 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.240 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.240 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:44:26.241 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.241 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.241 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.241 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.241 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.241 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.241 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.242 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.242 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.242 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.242 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.242 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.242 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.242 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.242 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.243 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.243 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.243 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.243 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-09-21T03:44:26.243 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.243 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.243 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.244 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.244 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.244 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.244 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.244 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.244 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.244 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.245 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.245 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.245 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.245 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.245 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.245 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.245 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.245 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.245 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.246 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.246 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.246 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.246 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.246 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.246 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.246 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.247 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.247 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.247 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.247 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.247 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.247 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.247 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.247 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.248 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.248 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.248 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.248 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.248 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.248 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.248 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.248 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.249 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.249 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.249 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.249 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.249 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.249 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.249 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.249 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.249 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.250 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.250 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.250 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.250 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.250 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.250 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.250 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.251 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.251 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.251 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.251 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.251 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.251 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.251 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.251 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.252 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.252 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.252 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.252 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.252 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.252 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.252 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.252 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.252 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.253 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.253 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.253 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.253 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.253 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-09-21T03:44:26.253 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-09-21T03:44:26.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.254 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.254 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.254 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-09-21T03:44:26.254 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.254 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.254 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.254 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:44:26.254 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.255 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.255 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.255 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.255 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.255 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-09-21T03:44:26.255 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.256 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.256 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.256 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.256 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.256 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.256 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.256 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.257 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.257 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.257 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.257 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.257 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.257 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.257 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.257 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.257 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.257 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.258 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.258 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.258 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.258 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.258 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.258 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.259 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.259 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.259 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.259 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.259 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.260 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.260 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.260 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.260 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.261 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.261 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.261 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.261 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.261 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.261 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.261 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.261 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.261 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.262 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.262 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.262 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.262 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.262 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.262 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.262 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.262 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.263 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.263 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.263 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.263 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.263 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.263 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.263 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.263 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.263 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.264 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.264 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-09-21T03:44:26.264 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-09-21T03:44:26.264 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.264 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.264 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.264 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.264 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.264 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.265 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.265 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.265 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.265 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.265 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.265 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-09-21T03:44:26.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-09-21T03:44:26.266 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.266 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.266 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.266 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-09-21T03:44:26.266 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.266 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.266 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.267 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.267 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.267 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.267 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.267 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-09-21T03:44:26.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-09-21T03:44:26.268 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.268 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.268 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.268 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.268 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.268 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.268 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.268 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.269 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.269 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.269 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.269 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-09-21T03:44:26.269 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-09-21T03:44:26.269 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.269 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.270 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.270 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.270 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.270 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.270 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.270 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.270 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.271 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.271 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-09-21T03:44:26.271 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-09-21T03:44:26.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-09-21T03:44:26.271 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.271 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.272 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.272 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.272 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.272 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.272 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.272 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.272 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.272 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.273 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-09-21T03:44:26.273 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-09-21T03:44:26.273 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.273 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.273 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.273 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.273 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.273 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.273 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.274 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.274 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.274 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.274 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.274 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-09-21T03:44:26.274 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-09-21T03:44:26.274 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.275 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.275 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.275 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.275 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.275 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.275 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.275 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.275 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.276 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.276 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.276 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-09-21T03:44:26.276 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-09-21T03:44:26.276 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.276 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.276 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.276 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:26.277 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.277 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.277 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.277 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.277 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.277 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.277 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.278 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-09-21T03:44:26.278 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-09-21T03:44:26.278 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.278 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.278 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.278 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.278 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-09-21T03:44:26.278 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-09-21T03:44:26.278 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.279 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.280 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-09-21T03:44:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-09-21T03:44:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.281 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.282 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-09-21T03:44:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-09-21T03:44:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.283 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-09-21T03:44:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-09-21T03:44:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.284 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.285 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smb", 2024-09-21T03:44:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.286 INFO:teuthology.orchestra.run.smithi112.stdout: "internal_store_backend": { 2024-09-21T03:44:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "internal_store_backend", 2024-09-21T03:44:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.287 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:44:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.288 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.289 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.290 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.291 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.291 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.291 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.291 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.291 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.291 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.291 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.292 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.292 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.292 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.292 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.292 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.292 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.293 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.293 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.293 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.293 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.293 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.293 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.293 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.293 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.294 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.294 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.294 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.294 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.294 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.294 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.294 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.294 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.295 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.295 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.295 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.295 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.295 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.295 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.295 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.296 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.296 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.296 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.296 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.296 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.296 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.296 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.296 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.297 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.297 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.297 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.297 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.297 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.297 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.297 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.297 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.297 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.298 INFO:teuthology.orchestra.run.smithi112.stdout: "update_orchestration": { 2024-09-21T03:44:26.298 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "update_orchestration", 2024-09-21T03:44:26.298 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.298 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.298 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.298 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:26.298 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.299 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.299 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.299 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:44:26.299 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.299 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.299 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.299 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.299 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.300 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-09-21T03:44:26.300 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.300 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.300 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.300 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-09-21T03:44:26.300 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-09-21T03:44:26.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.300 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.301 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.301 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.301 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.301 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.301 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.301 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:44:26.301 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.301 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.302 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.302 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-09-21T03:44:26.302 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-09-21T03:44:26.302 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.302 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.302 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.302 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.302 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.303 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.303 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.303 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:44:26.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.303 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.303 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.303 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.303 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.304 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.304 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.304 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.304 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.304 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.304 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.304 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.305 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.305 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.305 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.305 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.305 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.305 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.305 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.305 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.305 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.306 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.307 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.308 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.309 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.310 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.312 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-09-21T03:44:26.313 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.314 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.315 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.315 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.315 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.315 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.315 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.315 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.315 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.315 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.316 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.316 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.316 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.316 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.316 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.316 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.316 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.317 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.317 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.317 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.317 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.317 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.317 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.317 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.317 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.318 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.318 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.318 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.318 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.318 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.318 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.318 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.319 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.319 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.319 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.319 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.319 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.319 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.319 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.319 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.319 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.320 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.320 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.320 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.320 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.320 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.320 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.321 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.321 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.321 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.321 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.321 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.321 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.321 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.321 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.322 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.322 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.322 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.322 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.323 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.323 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.323 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.323 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.323 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.324 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-09-21T03:44:26.324 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.324 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.324 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.324 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.325 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.325 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.325 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.325 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.325 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.325 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.325 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.325 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.325 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.326 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.326 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.326 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.326 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.326 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.326 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.326 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.326 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.326 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.327 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.327 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.327 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.327 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.327 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.327 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.327 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.328 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.328 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.328 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.328 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.328 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.328 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.328 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.328 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.329 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.329 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.329 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.329 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.329 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.330 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.330 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.330 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.330 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.330 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.330 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.330 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.330 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.330 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.331 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.331 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.331 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.331 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.331 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.331 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.331 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.332 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.332 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.332 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.332 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.332 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.332 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.332 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.333 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.333 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.333 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.333 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.333 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.333 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.333 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.333 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.334 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.334 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.334 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.334 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.334 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-09-21T03:44:26.334 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.334 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.335 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.335 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-09-21T03:44:26.335 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-09-21T03:44:26.335 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.335 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.335 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.335 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:44:26.335 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.335 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.336 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.336 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.336 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.336 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.336 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.336 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:26.336 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:26.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:26.337 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.337 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-09-21T03:44:26.337 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.337 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.337 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.337 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.337 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.338 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.338 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.338 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.338 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.338 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.338 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.338 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.339 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.339 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.339 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.339 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.339 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.339 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.339 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.339 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.339 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.340 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.340 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.340 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.340 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.340 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.340 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.340 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.341 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.341 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.341 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.341 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.341 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.341 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.341 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.342 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.342 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.342 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.342 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.342 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.342 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.342 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.343 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.343 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.343 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.343 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.343 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.343 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.343 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.343 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.344 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.344 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.344 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.344 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.344 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.344 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.344 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.344 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.345 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.345 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.345 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.345 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.345 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.346 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.346 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.346 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.346 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.346 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.346 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.346 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.346 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.347 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.347 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.347 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.347 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.347 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.347 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.347 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.347 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.348 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.348 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.348 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.348 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-09-21T03:44:26.348 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.348 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.349 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.349 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-09-21T03:44:26.349 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-09-21T03:44:26.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.349 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.349 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.349 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:26.349 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.350 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.350 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.350 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:44:26.350 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.350 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.350 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.350 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-09-21T03:44:26.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-09-21T03:44:26.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.351 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.351 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:26.351 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.351 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.351 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.352 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:44:26.352 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.352 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.352 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.352 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-09-21T03:44:26.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-09-21T03:44:26.352 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.352 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.353 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.353 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:26.353 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.353 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.353 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.353 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:44:26.353 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.353 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.354 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.354 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-09-21T03:44:26.354 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-09-21T03:44:26.354 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.354 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.354 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.354 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.354 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.355 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.355 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.355 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:44:26.355 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.355 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.355 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.355 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-09-21T03:44:26.356 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-09-21T03:44:26.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.356 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.356 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.356 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.356 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.356 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.356 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.356 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:44:26.356 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.357 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.357 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.357 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-09-21T03:44:26.357 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-09-21T03:44:26.357 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.357 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.357 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.357 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.358 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.358 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.358 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.358 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.358 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.358 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.358 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.358 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-09-21T03:44:26.359 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-09-21T03:44:26.359 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.359 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.359 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.359 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.359 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.359 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.359 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.360 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.360 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.360 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.360 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-09-21T03:44:26.360 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-09-21T03:44:26.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.360 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.361 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.361 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:44:26.361 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.361 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.361 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.361 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.362 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.362 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-09-21T03:44:26.362 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-09-21T03:44:26.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.362 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.362 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.362 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.363 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.363 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.363 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.363 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.363 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.363 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.363 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-09-21T03:44:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-09-21T03:44:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.364 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.365 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.365 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.365 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.365 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-09-21T03:44:26.365 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-09-21T03:44:26.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.365 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.366 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-09-21T03:44:26.366 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.366 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.366 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.366 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.366 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.366 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.366 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.367 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-09-21T03:44:26.367 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-09-21T03:44:26.367 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.367 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.367 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.367 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.368 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.368 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.368 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.368 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.368 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.368 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-09-21T03:44:26.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-09-21T03:44:26.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.369 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.369 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.369 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.369 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.369 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.370 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.370 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.370 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.370 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.370 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.370 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.371 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.371 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.371 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.371 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.371 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.371 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.371 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.371 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.371 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.372 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.372 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.372 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.372 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.372 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.372 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.372 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.373 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.373 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.373 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.373 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.373 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.373 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.374 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.374 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.374 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.374 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.374 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.374 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.374 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.375 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.375 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.375 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.375 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.375 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.375 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.375 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.375 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.376 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.376 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.376 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.376 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.376 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.376 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.376 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.376 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.376 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.377 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.377 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.377 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.377 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.377 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.378 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.378 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.378 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.378 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.378 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-09-21T03:44:26.378 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-09-21T03:44:26.378 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.379 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.379 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.379 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.379 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.379 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.379 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.379 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.379 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.379 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.380 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.380 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-09-21T03:44:26.380 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-09-21T03:44:26.380 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.380 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.380 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.380 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.381 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.381 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.381 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.381 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.381 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.381 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.381 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.381 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.382 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.382 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.382 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.382 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.382 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.382 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.382 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.383 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.383 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.383 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.383 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.383 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-09-21T03:44:26.383 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-09-21T03:44:26.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.383 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.384 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:44:26.384 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.384 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.384 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.384 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.384 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.384 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.385 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.385 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.385 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.385 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-09-21T03:44:26.385 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.385 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.385 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.386 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.386 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.386 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.386 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.386 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.387 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.387 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.387 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.387 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.387 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.387 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.387 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.388 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.388 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.388 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.388 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.388 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.389 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.389 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.389 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.389 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.389 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.389 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.389 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.389 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.390 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.390 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.390 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.390 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.391 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.391 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.391 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.391 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.391 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.391 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.391 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.391 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.392 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.392 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.392 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.392 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.392 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.392 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.392 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.392 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.393 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.393 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.393 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.393 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.393 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.393 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.393 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.393 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.394 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.394 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.394 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.394 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.394 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.394 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.394 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.394 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.395 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.395 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.395 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.395 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.395 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.395 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.395 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.395 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.396 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-09-21T03:44:26.396 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.396 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.396 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.396 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.396 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.397 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.397 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.397 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.397 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.397 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.397 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.397 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.397 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.397 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.398 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.398 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.398 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.398 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.398 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.398 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.398 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.398 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.399 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.399 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.399 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.399 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.399 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.400 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.400 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.400 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.400 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.400 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.401 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.401 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.401 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.401 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.401 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.401 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.401 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.401 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.402 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.402 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.402 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.402 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.402 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.402 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.402 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.402 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.403 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.403 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.403 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.403 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.403 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.403 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.403 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.404 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.404 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.404 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.404 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.404 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-09-21T03:44:26.405 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-09-21T03:44:26.405 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.405 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-09-21T03:44:26.405 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.405 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:44:26.406 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.406 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.406 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.406 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-09-21T03:44:26.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-09-21T03:44:26.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.406 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.406 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.407 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.407 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.407 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.407 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.407 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-09-21T03:44:26.407 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.407 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.407 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.408 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-09-21T03:44:26.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:44:26.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.408 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.408 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.408 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.408 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.408 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.409 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:44:26.409 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.409 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.409 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.409 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:44:26.409 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:44:26.410 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.410 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.410 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.410 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-09-21T03:44:26.410 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.410 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.410 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:44:26.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.411 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.411 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.411 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.412 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.412 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.412 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-09-21T03:44:26.413 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-09-21T03:44:26.413 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-09-21T03:44:26.413 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-09-21T03:44:26.413 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-09-21T03:44:26.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-09-21T03:44:26.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-09-21T03:44:26.414 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.414 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-09-21T03:44:26.414 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.414 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.414 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.414 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.415 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.415 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.415 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-09-21T03:44:26.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-09-21T03:44:26.415 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.415 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.415 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.416 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.416 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.416 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.416 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.416 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.417 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-09-21T03:44:26.417 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-09-21T03:44:26.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-09-21T03:44:26.417 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.417 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-09-21T03:44:26.417 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.417 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.418 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.418 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.418 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.418 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-09-21T03:44:26.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-09-21T03:44:26.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.419 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.419 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.419 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.419 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.419 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.419 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.420 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.420 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.420 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.420 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.420 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.420 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.421 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-09-21T03:44:26.421 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-09-21T03:44:26.421 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.421 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.421 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.422 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.422 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.422 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.422 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.422 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.422 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-09-21T03:44:26.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-09-21T03:44:26.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.423 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-09-21T03:44:26.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-09-21T03:44:26.423 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-09-21T03:44:26.423 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-09-21T03:44:26.424 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-09-21T03:44:26.424 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-09-21T03:44:26.424 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-09-21T03:44:26.424 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-09-21T03:44:26.424 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.424 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.424 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.425 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-09-21T03:44:26.425 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-09-21T03:44:26.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-09-21T03:44:26.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-09-21T03:44:26.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.426 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.426 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.426 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.426 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-09-21T03:44:26.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:44:26.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.427 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-09-21T03:44:26.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-09-21T03:44:26.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-09-21T03:44:26.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.427 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.428 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-09-21T03:44:26.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-09-21T03:44:26.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.428 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.428 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.428 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-09-21T03:44:26.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.429 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.429 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.429 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.430 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-09-21T03:44:26.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-09-21T03:44:26.430 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-09-21T03:44:26.430 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.430 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-09-21T03:44:26.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.431 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.431 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.431 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.431 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-09-21T03:44:26.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-09-21T03:44:26.432 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-09-21T03:44:26.432 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-09-21T03:44:26.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-09-21T03:44:26.432 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:44:26.432 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-09-21T03:44:26.432 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-09-21T03:44:26.432 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-09-21T03:44:26.432 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-09-21T03:44:26.433 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-09-21T03:44:26.433 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-09-21T03:44:26.433 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-09-21T03:44:26.433 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.433 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.433 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.433 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.433 INFO:teuthology.orchestra.run.smithi112.stdout: "services": { 2024-09-21T03:44:26.433 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard": "https://172.21.15.112:8443/" 2024-09-21T03:44:26.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.434 INFO:teuthology.orchestra.run.smithi112.stdout: "always_on_modules": { 2024-09-21T03:44:26.434 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus": [ 2024-09-21T03:44:26.434 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-09-21T03:44:26.434 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-21T03:44:26.434 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-09-21T03:44:26.434 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-09-21T03:44:26.434 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-09-21T03:44:26.435 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-09-21T03:44:26.435 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-09-21T03:44:26.435 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-09-21T03:44:26.435 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-09-21T03:44:26.435 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-09-21T03:44:26.435 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.435 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific": [ 2024-09-21T03:44:26.435 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-09-21T03:44:26.435 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-21T03:44:26.436 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-09-21T03:44:26.436 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-09-21T03:44:26.436 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-09-21T03:44:26.436 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-09-21T03:44:26.436 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-09-21T03:44:26.436 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-09-21T03:44:26.436 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-09-21T03:44:26.436 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-09-21T03:44:26.436 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.437 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy": [ 2024-09-21T03:44:26.437 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-09-21T03:44:26.437 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-21T03:44:26.437 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-09-21T03:44:26.437 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-09-21T03:44:26.437 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-09-21T03:44:26.437 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-09-21T03:44:26.437 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-09-21T03:44:26.438 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-09-21T03:44:26.438 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-09-21T03:44:26.438 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-09-21T03:44:26.438 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.438 INFO:teuthology.orchestra.run.smithi112.stdout: "reef": [ 2024-09-21T03:44:26.438 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-09-21T03:44:26.438 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-21T03:44:26.438 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-09-21T03:44:26.439 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-09-21T03:44:26.439 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-09-21T03:44:26.439 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-09-21T03:44:26.439 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-09-21T03:44:26.439 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-09-21T03:44:26.439 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-09-21T03:44:26.439 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-09-21T03:44:26.439 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-09-21T03:44:26.439 INFO:teuthology.orchestra.run.smithi112.stdout: "squid": [ 2024-09-21T03:44:26.440 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-09-21T03:44:26.440 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-09-21T03:44:26.440 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-09-21T03:44:26.440 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-09-21T03:44:26.440 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-09-21T03:44:26.440 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-09-21T03:44:26.440 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-09-21T03:44:26.440 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-09-21T03:44:26.441 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-09-21T03:44:26.441 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-09-21T03:44:26.441 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:44:26.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.441 INFO:teuthology.orchestra.run.smithi112.stdout: "last_failure_osd_epoch": 5, 2024-09-21T03:44:26.441 INFO:teuthology.orchestra.run.smithi112.stdout: "active_clients": [ 2024-09-21T03:44:26.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-09-21T03:44:26.441 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-09-21T03:44:26.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.442 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-09-21T03:44:26.442 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-09-21T03:44:26.442 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3312220801 2024-09-21T03:44:26.442 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.442 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:44:26.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "libcephsqlite", 2024-09-21T03:44:26.443 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-09-21T03:44:26.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-09-21T03:44:26.443 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-09-21T03:44:26.443 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3017432490 2024-09-21T03:44:26.443 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.444 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:44:26.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.444 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-09-21T03:44:26.444 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-09-21T03:44:26.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-09-21T03:44:26.444 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-09-21T03:44:26.444 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 79700488 2024-09-21T03:44:26.445 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.445 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:44:26.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-09-21T03:44:26.445 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-09-21T03:44:26.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-09-21T03:44:26.445 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-09-21T03:44:26.446 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1420469135 2024-09-21T03:44:26.446 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.446 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:44:26.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.446 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-09-21T03:44:26.446 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-09-21T03:44:26.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-09-21T03:44:26.447 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-09-21T03:44:26.447 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 920936562 2024-09-21T03:44:26.447 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.447 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:44:26.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:26.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-09-21T03:44:26.447 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-09-21T03:44:26.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-09-21T03:44:26.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-09-21T03:44:26.448 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-09-21T03:44:26.448 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2215771593 2024-09-21T03:44:26.448 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.448 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:44:26.448 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:26.448 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-09-21T03:44:26.449 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:44:26.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 -- 172.21.15.112:0/3426783166 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3c80b6680 msgr2=0x7fc3c80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:26.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3c80b6680 0x7fc3c80b8b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc101a80 tx=0x7fc3c400b390 comp rx=0 tx=0).stop 2024-09-21T03:44:26.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 -- 172.21.15.112:0/3426783166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3dc102980 msgr2=0x7fc3dc199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:26.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3dc102980 0x7fc3dc199380 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc005940 tx=0x7fc3cc004270 comp rx=0 tx=0).stop 2024-09-21T03:44:26.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 -- 172.21.15.112:0/3426783166 shutdown_connections 2024-09-21T03:44:26.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7fc3c80b6680 0x7fc3c80b8b90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:26.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3dc108c00 0x7fc3dc19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:26.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3dc1082b0 0x7fc3dc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:26.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 --2- 172.21.15.112:0/3426783166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3dc102980 0x7fc3dc199380 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:26.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 -- 172.21.15.112:0/3426783166 >> 172.21.15.112:0/3426783166 conn(0x7fc3dc0fe380 msgr2=0x7fc3dc107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:26.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 -- 172.21.15.112:0/3426783166 shutdown_connections 2024-09-21T03:44:26.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:23.957+0000 7fc3e2928640 1 -- 172.21.15.112:0/3426783166 wait complete. 2024-09-21T03:44:26.451 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:23 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:44:26.451 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:23 smithi112 ceph-mon[31442]: pgmap v799: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.6 MiB/s wr, 230 op/s 2024-09-21T03:44:26.451 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:23 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/2654724236' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:26.452 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:23 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3096033680' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:44:26.452 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:23 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/152616924' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:44:26.459 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T03:44:26.459 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T03:44:26.460 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:24 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3703684286' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:44:26.460 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:24 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3426783166' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:44:26.460 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:25 smithi112 ceph-mon[31442]: pgmap v800: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.8 MiB/s wr, 230 op/s 2024-09-21T03:44:26.461 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:24 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3703684286' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:44:26.461 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:24 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3426783166' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:44:26.462 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:25 smithi187 ceph-mon[34603]: pgmap v800: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.8 MiB/s wr, 230 op/s 2024-09-21T03:44:26.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:24 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3703684286' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:44:26.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:24 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3426783166' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:44:26.462 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:25 smithi203 ceph-mon[34723]: pgmap v800: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.8 MiB/s wr, 230 op/s 2024-09-21T03:44:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:27.328 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:44:27.328 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:27.328 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-09-21T03:44:27.328 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-09-21T03:44:27.329 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:44:27.329 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-09-21T03:44:27.329 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:27.329 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-09-21T03:44:27.329 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001196685, 2024-09-21T03:44:27.329 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:44:27.329 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000236553, 2024-09-21T03:44:27.329 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:27.330 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:27.330 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:27.330 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:27.330 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:27.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:27.330 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:27.330 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:27.330 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:27.330 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:27.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:27.331 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:44:27.331 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:44:27.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:27.331 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:27.331 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 287, 2024-09-21T03:44:27.331 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 282, 2024-09-21T03:44:27.331 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 856480, 2024-09-21T03:44:27.331 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 174144, 2024-09-21T03:44:27.332 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-09-21T03:44:27.332 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-09-21T03:44:27.332 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.041258720, 2024-09-21T03:44:27.332 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.020278487, 2024-09-21T03:44:27.332 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.028339157, 2024-09-21T03:44:27.332 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:27.332 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:27.332 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 281, 2024-09-21T03:44:27.332 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.012415339, 2024-09-21T03:44:27.332 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000044182 2024-09-21T03:44:27.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:27.333 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:27.333 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:27.333 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:27.333 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:27.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:27.333 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 856480, 2024-09-21T03:44:27.333 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 174144 2024-09-21T03:44:27.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 6, 2024-09-21T03:44:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 218, 2024-09-21T03:44:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3264, 2024-09-21T03:44:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 199200, 2024-09-21T03:44:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-09-21T03:44:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.019532416, 2024-09-21T03:44:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.013696744, 2024-09-21T03:44:27.335 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.004142652, 2024-09-21T03:44:27.335 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:27.335 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:27.335 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 217, 2024-09-21T03:44:27.335 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.012074616, 2024-09-21T03:44:27.335 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000055643 2024-09-21T03:44:27.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:27.335 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:27.335 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:27.336 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:27.336 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:27.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:27.336 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T03:44:27.336 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 199200 2024-09-21T03:44:27.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:27.336 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-21T03:44:27.336 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-09-21T03:44:27.336 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-21T03:44:27.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:27.337 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-21T03:44:27.337 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:27.337 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:27.337 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:27.337 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:27.337 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:27.337 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:27.337 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:27.337 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:27.338 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:27.338 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:27.338 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:27.338 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:27.338 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:27.338 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:27.338 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:27.338 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:27.338 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:27.339 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:27.339 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:27.339 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:27.339 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:27.339 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:27.339 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:27.339 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:27.339 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:27.340 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:27.340 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:27.340 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:27.340 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:27.340 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:27.340 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:27.340 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-21T03:44:27.340 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:27.340 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:27.341 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:27.341 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:27.341 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 147960, 2024-09-21T03:44:27.341 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 28, 2024-09-21T03:44:27.341 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:44:27.341 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-09-21T03:44:27.341 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-21T03:44:27.341 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-21T03:44:27.341 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:27.342 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:27.342 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:27.342 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:27.342 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-09-21T03:44:27.342 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-09-21T03:44:27.342 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:27.342 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:27.342 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:27.342 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-21T03:44:27.343 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 968, 2024-09-21T03:44:27.343 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-09-21T03:44:27.343 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:27.343 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-21T03:44:27.343 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:27.343 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-21T03:44:27.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:27.343 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:27.343 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-21T03:44:27.344 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-21T03:44:27.344 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-21T03:44:27.344 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 289, 2024-09-21T03:44:27.344 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 815918, 2024-09-21T03:44:27.344 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:27.344 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 289, 2024-09-21T03:44:27.344 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-21T03:44:27.344 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-21T03:44:27.345 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 289, 2024-09-21T03:44:27.345 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 815918, 2024-09-21T03:44:27.345 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-21T03:44:27.345 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:27.345 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:27.345 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:27.345 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:27.345 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:27.345 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:44:27.923 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:27 smithi112 ceph-mon[31442]: pgmap v801: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.5 MiB/s wr, 291 op/s 2024-09-21T03:44:27.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:27 smithi187 ceph-mon[34603]: pgmap v801: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.5 MiB/s wr, 291 op/s 2024-09-21T03:44:27.951 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-21T03:44:27.951 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-21T03:44:27.951 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-21T03:44:27.951 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-21T03:44:27.951 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-21T03:44:27.951 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T03:44:27.951 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T03:44:28.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:27 smithi203 ceph-mon[34723]: pgmap v801: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.5 MiB/s wr, 291 op/s 2024-09-21T03:44:28.199 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:28.713 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:44:28.713 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:28.713 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4, 2024-09-21T03:44:28.713 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4, 2024-09-21T03:44:28.713 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 2944, 2024-09-21T03:44:28.714 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 448, 2024-09-21T03:44:28.714 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:28.714 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-09-21T03:44:28.714 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.003126641, 2024-09-21T03:44:28.714 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000039853, 2024-09-21T03:44:28.714 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.002280933, 2024-09-21T03:44:28.714 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:28.714 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:28.714 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-09-21T03:44:28.715 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003939864, 2024-09-21T03:44:28.715 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001313288 2024-09-21T03:44:28.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:28.715 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:28.715 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:28.715 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:28.715 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:28.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:28.715 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:44:28.716 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:44:28.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:28.716 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:28.716 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 293, 2024-09-21T03:44:28.716 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 288, 2024-09-21T03:44:28.716 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 858176, 2024-09-21T03:44:28.716 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 177504, 2024-09-21T03:44:28.716 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-09-21T03:44:28.716 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-09-21T03:44:28.717 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.043114386, 2024-09-21T03:44:28.717 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.021553798, 2024-09-21T03:44:28.717 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.028708927, 2024-09-21T03:44:28.717 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:28.717 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:28.717 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 287, 2024-09-21T03:44:28.717 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.011714494, 2024-09-21T03:44:28.717 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000040817 2024-09-21T03:44:28.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:28.718 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:28.718 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:28.718 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:28.718 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:28.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:28.718 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 858176, 2024-09-21T03:44:28.718 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 177504 2024-09-21T03:44:28.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:28.719 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:28.719 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2, 2024-09-21T03:44:28.719 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 220, 2024-09-21T03:44:28.719 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:44:28.719 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 199456, 2024-09-21T03:44:28.719 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:28.719 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-09-21T03:44:28.719 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.017057602, 2024-09-21T03:44:28.719 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.013624747, 2024-09-21T03:44:28.720 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001695687, 2024-09-21T03:44:28.720 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:28.720 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:28.720 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 218, 2024-09-21T03:44:28.720 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.009186435, 2024-09-21T03:44:28.720 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000042139 2024-09-21T03:44:28.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:28.720 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:28.720 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:28.721 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:28.721 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:28.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:28.721 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:44:28.721 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 199456 2024-09-21T03:44:28.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:28.721 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-21T03:44:28.721 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-09-21T03:44:28.721 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-21T03:44:28.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:28.722 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-21T03:44:28.722 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:28.722 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:28.722 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:28.722 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:28.722 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:28.722 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:28.722 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:28.722 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:28.723 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:28.724 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-21T03:44:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 147937, 2024-09-21T03:44:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 28, 2024-09-21T03:44:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:44:28.726 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-09-21T03:44:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-21T03:44:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-21T03:44:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-09-21T03:44:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-09-21T03:44:28.727 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-21T03:44:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 968, 2024-09-21T03:44:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-09-21T03:44:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-21T03:44:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:28.728 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-21T03:44:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-21T03:44:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-21T03:44:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-21T03:44:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 295, 2024-09-21T03:44:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 816762, 2024-09-21T03:44:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 295, 2024-09-21T03:44:28.729 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-21T03:44:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-21T03:44:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 295, 2024-09-21T03:44:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 816762, 2024-09-21T03:44:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-21T03:44:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:28.730 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:28.731 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:44:29.650 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T03:44:29.651 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T03:44:29.651 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T03:44:29.651 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T03:44:29.651 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T03:44:29.651 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T03:44:29.651 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T03:44:29.898 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:29.929 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:29 smithi187 ceph-mon[34603]: pgmap v802: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 6.2 MiB/s wr, 209 op/s 2024-09-21T03:44:30.016 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:29 smithi112 ceph-mon[31442]: pgmap v802: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 6.2 MiB/s wr, 209 op/s 2024-09-21T03:44:30.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:29 smithi203 ceph-mon[34723]: pgmap v802: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 6.2 MiB/s wr, 209 op/s 2024-09-21T03:44:30.363 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:44:30.364 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:30.364 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-09-21T03:44:30.364 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-09-21T03:44:30.364 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:44:30.364 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-09-21T03:44:30.364 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:30.364 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-09-21T03:44:30.364 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001182109, 2024-09-21T03:44:30.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:44:30.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000283208, 2024-09-21T03:44:30.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:30.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:30.365 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:30.365 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:30.365 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:30.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:30.365 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:30.365 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:30.366 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:30.366 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:30.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:30.366 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:44:30.366 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:44:30.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:30.366 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:30.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 6, 2024-09-21T03:44:30.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 219, 2024-09-21T03:44:30.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3264, 2024-09-21T03:44:30.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 199376, 2024-09-21T03:44:30.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:30.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-09-21T03:44:30.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.019658745, 2024-09-21T03:44:30.367 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.014354991, 2024-09-21T03:44:30.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.003631715, 2024-09-21T03:44:30.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:30.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:30.368 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 218, 2024-09-21T03:44:30.368 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.012980974, 2024-09-21T03:44:30.368 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000059545 2024-09-21T03:44:30.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:30.368 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:30.369 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:30.369 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:30.369 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:30.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:30.369 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T03:44:30.369 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 199376 2024-09-21T03:44:30.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:30.369 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:30.369 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 288, 2024-09-21T03:44:30.370 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 283, 2024-09-21T03:44:30.370 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 856752, 2024-09-21T03:44:30.370 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 174784, 2024-09-21T03:44:30.370 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-09-21T03:44:30.370 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-09-21T03:44:30.370 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.042432239, 2024-09-21T03:44:30.370 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.021120983, 2024-09-21T03:44:30.370 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.029033590, 2024-09-21T03:44:30.370 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:30.371 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:30.371 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 282, 2024-09-21T03:44:30.371 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.011638715, 2024-09-21T03:44:30.371 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000041272 2024-09-21T03:44:30.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:30.372 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:30.372 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:30.373 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:30.373 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:30.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:30.373 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 856752, 2024-09-21T03:44:30.374 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 174784 2024-09-21T03:44:30.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:30.374 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-21T03:44:30.374 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-09-21T03:44:30.374 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-21T03:44:30.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:30.374 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-21T03:44:30.374 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:30.375 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:30.375 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:30.375 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:30.375 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:30.375 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:30.375 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:30.375 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:30.375 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:30.375 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:30.376 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:30.376 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:30.376 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:30.376 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:30.376 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:30.376 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:30.376 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:30.376 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:30.376 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:30.377 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:30.377 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:30.377 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:30.377 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:30.377 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:30.377 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:30.377 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:30.377 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:30.377 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:30.378 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:30.378 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:30.378 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:30.378 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-21T03:44:30.378 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:30.378 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:30.378 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:30.378 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:30.378 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 147960, 2024-09-21T03:44:30.379 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 28, 2024-09-21T03:44:30.379 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:44:30.379 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-09-21T03:44:30.379 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-21T03:44:30.379 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-21T03:44:30.379 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:30.379 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:30.379 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:30.379 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:30.380 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-09-21T03:44:30.380 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-09-21T03:44:30.380 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:30.380 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:30.380 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:30.380 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-21T03:44:30.380 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 968, 2024-09-21T03:44:30.380 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-09-21T03:44:30.380 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:30.381 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-21T03:44:30.381 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:30.381 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-21T03:44:30.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:30.381 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:30.381 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-21T03:44:30.381 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-21T03:44:30.381 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-21T03:44:30.382 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 290, 2024-09-21T03:44:30.382 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 816048, 2024-09-21T03:44:30.382 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:30.382 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 290, 2024-09-21T03:44:30.382 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-21T03:44:30.382 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-21T03:44:30.383 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 290, 2024-09-21T03:44:30.383 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 816048, 2024-09-21T03:44:30.383 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-21T03:44:30.383 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:30.383 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:30.383 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:30.383 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:30.383 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:30.383 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:44:31.210 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-21T03:44:31.210 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-21T03:44:31.210 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-21T03:44:31.211 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-21T03:44:31.211 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-21T03:44:31.211 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T03:44:31.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T03:44:31.455 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:32.011 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:31 smithi112 ceph-mon[31442]: pgmap v803: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.4 MiB/s wr, 246 op/s 2024-09-21T03:44:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-09-21T03:44:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 295, 2024-09-21T03:44:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 290, 2024-09-21T03:44:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 859344, 2024-09-21T03:44:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 178480, 2024-09-21T03:44:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-09-21T03:44:32.132 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.044613907, 2024-09-21T03:44:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.021072920, 2024-09-21T03:44:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.031123183, 2024-09-21T03:44:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 289, 2024-09-21T03:44:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.011805112, 2024-09-21T03:44:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000040848 2024-09-21T03:44:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:32.133 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 859344, 2024-09-21T03:44:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 178480 2024-09-21T03:44:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:32.134 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2, 2024-09-21T03:44:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 221, 2024-09-21T03:44:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:44:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 199632, 2024-09-21T03:44:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-09-21T03:44:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.018468360, 2024-09-21T03:44:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.014843175, 2024-09-21T03:44:32.135 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001697347, 2024-09-21T03:44:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 219, 2024-09-21T03:44:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.009273939, 2024-09-21T03:44:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000042346 2024-09-21T03:44:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:32.136 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:44:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 199632 2024-09-21T03:44:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4, 2024-09-21T03:44:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4, 2024-09-21T03:44:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 2944, 2024-09-21T03:44:32.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 448, 2024-09-21T03:44:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-09-21T03:44:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-09-21T03:44:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.002139716, 2024-09-21T03:44:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000041575, 2024-09-21T03:44:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.002077669, 2024-09-21T03:44:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-09-21T03:44:32.138 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003773460, 2024-09-21T03:44:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001257820 2024-09-21T03:44:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:44:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:44:32.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-09-21T03:44:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-09-21T03:44:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-09-21T03:44:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-09-21T03:44:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:32.140 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:32.141 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:32.142 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:32.143 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-09-21T03:44:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:32.144 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 147977, 2024-09-21T03:44:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 29, 2024-09-21T03:44:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:44:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-09-21T03:44:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-09-21T03:44:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-09-21T03:44:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:32.145 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-09-21T03:44:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-09-21T03:44:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-09-21T03:44:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 968, 2024-09-21T03:44:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-09-21T03:44:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:32.146 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-09-21T03:44:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-09-21T03:44:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-09-21T03:44:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-09-21T03:44:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-09-21T03:44:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-09-21T03:44:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 297, 2024-09-21T03:44:32.147 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 817659, 2024-09-21T03:44:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 297, 2024-09-21T03:44:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-09-21T03:44:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-09-21T03:44:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 297, 2024-09-21T03:44:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 817659, 2024-09-21T03:44:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-09-21T03:44:32.148 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-09-21T03:44:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-09-21T03:44:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-09-21T03:44:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:32.149 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-09-21T03:44:32.149 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-09-21T03:44:32.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:31 smithi203 ceph-mon[34723]: pgmap v803: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.4 MiB/s wr, 246 op/s 2024-09-21T03:44:32.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:31 smithi187 ceph-mon[34603]: pgmap v803: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.4 MiB/s wr, 246 op/s 2024-09-21T03:44:32.793 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T03:44:32.793 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T03:44:32.793 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T03:44:32.794 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T03:44:32.794 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T03:44:32.794 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T03:44:32.794 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T03:44:33.043 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:44:33.598 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-09-21T03:44:33.598 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:33.599 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 185830, 2024-09-21T03:44:33.599 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 134178, 2024-09-21T03:44:33.599 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 60364290, 2024-09-21T03:44:33.599 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 205410426, 2024-09-21T03:44:33.599 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 13, 2024-09-21T03:44:33.599 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 6, 2024-09-21T03:44:33.599 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 11.142382268, 2024-09-21T03:44:33.599 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 5.276334196, 2024-09-21T03:44:33.600 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 6.850982974, 2024-09-21T03:44:33.600 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.537506174, 2024-09-21T03:44:33.600 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:33.600 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 134174, 2024-09-21T03:44:33.600 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 2.762458464, 2024-09-21T03:44:33.600 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000020588 2024-09-21T03:44:33.600 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.600 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:33.601 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 281916, 2024-09-21T03:44:33.601 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.042172645, 2024-09-21T03:44:33.601 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000149 2024-09-21T03:44:33.601 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.601 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 2208, 2024-09-21T03:44:33.601 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 9152 2024-09-21T03:44:33.601 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.601 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:33.602 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 213102, 2024-09-21T03:44:33.602 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 293754, 2024-09-21T03:44:33.602 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 75086084, 2024-09-21T03:44:33.602 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 375544174, 2024-09-21T03:44:33.602 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 56, 2024-09-21T03:44:33.602 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 9, 2024-09-21T03:44:33.602 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 17.910047831, 2024-09-21T03:44:33.602 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 10.103442584, 2024-09-21T03:44:33.602 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 9.419389264, 2024-09-21T03:44:33.603 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.504376668, 2024-09-21T03:44:33.603 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:33.603 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 293752, 2024-09-21T03:44:33.603 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 9.336779143, 2024-09-21T03:44:33.603 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000031784 2024-09-21T03:44:33.603 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.603 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:33.603 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 456198, 2024-09-21T03:44:33.604 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.073791779, 2024-09-21T03:44:33.604 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000161 2024-09-21T03:44:33.604 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.604 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 72095792, 2024-09-21T03:44:33.604 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 301500896 2024-09-21T03:44:33.604 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.604 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:33.605 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 24703, 2024-09-21T03:44:33.605 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 27013, 2024-09-21T03:44:33.605 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 8511183, 2024-09-21T03:44:33.605 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 120776593, 2024-09-21T03:44:33.605 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 7, 2024-09-21T03:44:33.605 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 5, 2024-09-21T03:44:33.605 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 2.337680332, 2024-09-21T03:44:33.605 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 1.094403152, 2024-09-21T03:44:33.606 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.744236850, 2024-09-21T03:44:33.606 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.499194739, 2024-09-21T03:44:33.606 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:33.606 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 27015, 2024-09-21T03:44:33.606 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.123015576, 2024-09-21T03:44:33.606 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000041570 2024-09-21T03:44:33.606 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.606 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:33.607 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 21206, 2024-09-21T03:44:33.607 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.003896961, 2024-09-21T03:44:33.607 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000183 2024-09-21T03:44:33.607 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.607 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 964528, 2024-09-21T03:44:33.607 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 202368 2024-09-21T03:44:33.607 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.607 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-09-21T03:44:33.608 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2024-09-21T03:44:33.608 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-09-21T03:44:33.608 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.608 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2024-09-21T03:44:33.608 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-09-21T03:44:33.608 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-09-21T03:44:33.608 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 45664, 2024-09-21T03:44:33.608 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 54.024251892, 2024-09-21T03:44:33.608 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001183081 2024-09-21T03:44:33.609 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:33.609 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.609 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2024-09-21T03:44:33.609 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-09-21T03:44:33.609 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-09-21T03:44:33.609 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 28292, 2024-09-21T03:44:33.609 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 3.777978014, 2024-09-21T03:44:33.609 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000133535 2024-09-21T03:44:33.610 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:33.610 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.610 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2024-09-21T03:44:33.610 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 147009, 2024-09-21T03:44:33.610 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 140931, 2024-09-21T03:44:33.610 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2024-09-21T03:44:33.610 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 140931, 2024-09-21T03:44:33.610 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 324.872237554, 2024-09-21T03:44:33.611 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002305186 2024-09-21T03:44:33.611 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.611 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2024-09-21T03:44:33.611 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 6078, 2024-09-21T03:44:33.611 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2024-09-21T03:44:33.611 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2024-09-21T03:44:33.611 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 86, 2024-09-21T03:44:33.611 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 9, 2024-09-21T03:44:33.612 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 2, 2024-09-21T03:44:33.612 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 2452, 2024-09-21T03:44:33.612 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 286, 2024-09-21T03:44:33.612 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 1555, 2024-09-21T03:44:33.612 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 611, 2024-09-21T03:44:33.612 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 2044, 2024-09-21T03:44:33.612 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 381, 2024-09-21T03:44:33.613 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 227, 2024-09-21T03:44:33.613 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 227, 2024-09-21T03:44:33.613 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 11, 2024-09-21T03:44:33.613 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 220367, 2024-09-21T03:44:33.613 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 127676, 2024-09-21T03:44:33.613 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 6070, 2024-09-21T03:44:33.613 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 61, 2024-09-21T03:44:33.613 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:44:33.613 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2024-09-21T03:44:33.614 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 1145, 2024-09-21T03:44:33.614 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 82478, 2024-09-21T03:44:33.614 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 2, 2024-09-21T03:44:33.614 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 9, 2024-09-21T03:44:33.614 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 313, 2024-09-21T03:44:33.614 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 4, 2024-09-21T03:44:33.614 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 497, 2024-09-21T03:44:33.614 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2024-09-21T03:44:33.614 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 871, 2024-09-21T03:44:33.615 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2024-09-21T03:44:33.615 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 723, 2024-09-21T03:44:33.615 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 212053256, 2024-09-21T03:44:33.615 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 6, 2024-09-21T03:44:33.615 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 871, 2024-09-21T03:44:33.615 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2024-09-21T03:44:33.615 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:44:33.615 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:44:33.616 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 60, 2024-09-21T03:44:33.616 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:44:33.616 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 60, 2024-09-21T03:44:33.616 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 811, 2024-09-21T03:44:33.616 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 7, 2024-09-21T03:44:33.616 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 5426, 2024-09-21T03:44:33.616 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 71043, 2024-09-21T03:44:33.616 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:44:33.616 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:44:33.617 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:44:33.617 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 36350, 2024-09-21T03:44:33.617 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 16877, 2024-09-21T03:44:33.617 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 9740, 2024-09-21T03:44:33.617 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 116414 2024-09-21T03:44:33.617 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.617 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2024-09-21T03:44:33.617 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 51, 2024-09-21T03:44:33.617 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 293, 2024-09-21T03:44:33.618 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 162, 2024-09-21T03:44:33.618 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 341, 2024-09-21T03:44:33.618 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 100, 2024-09-21T03:44:33.618 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 81, 2024-09-21T03:44:33.618 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2024-09-21T03:44:33.618 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:44:33.618 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 16342, 2024-09-21T03:44:33.618 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 16261, 2024-09-21T03:44:33.619 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2024-09-21T03:44:33.619 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2024-09-21T03:44:33.619 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2024-09-21T03:44:33.619 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:44:33.619 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:44:33.619 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2024-09-21T03:44:33.619 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2024-09-21T03:44:33.619 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:44:33.619 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:44:33.620 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:44:33.620 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 18, 2024-09-21T03:44:33.620 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2024-09-21T03:44:33.620 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 11, 2024-09-21T03:44:33.620 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2024-09-21T03:44:33.620 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0, 2024-09-21T03:44:33.620 INFO:teuthology.orchestra.run.smithi187.stdout: "uninline_started": 0, 2024-09-21T03:44:33.620 INFO:teuthology.orchestra.run.smithi187.stdout: "uninline_succeeded": 0, 2024-09-21T03:44:33.620 INFO:teuthology.orchestra.run.smithi187.stdout: "uninline_write_failed": 0 2024-09-21T03:44:33.621 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.621 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2024-09-21T03:44:33.621 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2024-09-21T03:44:33.621 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 140949, 2024-09-21T03:44:33.621 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 37895, 2024-09-21T03:44:33.621 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 36871, 2024-09-21T03:44:33.621 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 104078, 2024-09-21T03:44:33.621 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2024-09-21T03:44:33.621 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 1024, 2024-09-21T03:44:33.622 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 173, 2024-09-21T03:44:33.622 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 45, 2024-09-21T03:44:33.622 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 44, 2024-09-21T03:44:33.622 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 129, 2024-09-21T03:44:33.622 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 9, 2024-09-21T03:44:33.622 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2024-09-21T03:44:33.622 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 1, 2024-09-21T03:44:33.622 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 54653610, 2024-09-21T03:44:33.622 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 359182151, 2024-09-21T03:44:33.623 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 0, 2024-09-21T03:44:33.623 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2024-09-21T03:44:33.623 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 25084, 2024-09-21T03:44:33.623 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 159.499032033, 2024-09-21T03:44:33.623 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.006358596 2024-09-21T03:44:33.623 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.623 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 0 2024-09-21T03:44:33.623 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.623 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2024-09-21T03:44:33.624 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 837, 2024-09-21T03:44:33.624 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 17221, 2024-09-21T03:44:33.624 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 16384, 2024-09-21T03:44:33.624 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 143, 2024-09-21T03:44:33.624 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 333, 2024-09-21T03:44:33.624 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 190, 2024-09-21T03:44:33.624 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 2346, 2024-09-21T03:44:33.624 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 33952, 2024-09-21T03:44:33.624 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 31606, 2024-09-21T03:44:33.625 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 227, 2024-09-21T03:44:33.625 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 18077, 2024-09-21T03:44:33.625 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 17850, 2024-09-21T03:44:33.625 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 149344, 2024-09-21T03:44:33.625 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 357449728 2024-09-21T03:44:33.625 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.625 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2024-09-21T03:44:33.625 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 197390, 2024-09-21T03:44:33.625 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 578, 2024-09-21T03:44:33.625 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 147009, 2024-09-21T03:44:33.626 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 2426, 2024-09-21T03:44:33.626 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 10980, 2024-09-21T03:44:33.626 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2024-09-21T03:44:33.626 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 16334, 2024-09-21T03:44:33.626 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 48.210569081, 2024-09-21T03:44:33.626 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002951547 2024-09-21T03:44:33.626 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.626 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2024-09-21T03:44:33.626 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1004, 2024-09-21T03:44:33.627 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 20.384169183, 2024-09-21T03:44:33.627 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.020302957 2024-09-21T03:44:33.627 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.627 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2024-09-21T03:44:33.627 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.627 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.627 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.627 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.627 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2024-09-21T03:44:33.628 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.628 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.628 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.628 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.628 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2024-09-21T03:44:33.628 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 30395, 2024-09-21T03:44:33.628 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 48.493227723, 2024-09-21T03:44:33.628 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001595434 2024-09-21T03:44:33.629 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.629 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2024-09-21T03:44:33.629 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.629 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.629 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.629 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.629 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2024-09-21T03:44:33.629 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.630 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.630 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.630 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.630 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2024-09-21T03:44:33.630 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.630 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.630 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.630 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.630 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2024-09-21T03:44:33.631 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.631 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.632 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.632 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.632 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2024-09-21T03:44:33.632 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.632 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.632 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.632 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.632 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2024-09-21T03:44:33.633 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.633 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.633 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.633 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.633 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2024-09-21T03:44:33.633 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.633 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.633 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.633 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.634 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2024-09-21T03:44:33.634 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.634 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.634 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.634 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.634 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2024-09-21T03:44:33.634 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.634 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.634 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.635 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.635 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2024-09-21T03:44:33.635 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.635 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.635 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.635 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.635 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2024-09-21T03:44:33.635 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 74030, 2024-09-21T03:44:33.635 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 152.641279938, 2024-09-21T03:44:33.636 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002061884 2024-09-21T03:44:33.636 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.636 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2024-09-21T03:44:33.636 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2500, 2024-09-21T03:44:33.636 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 8.222410514, 2024-09-21T03:44:33.636 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.003288964 2024-09-21T03:44:33.636 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.636 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2024-09-21T03:44:33.636 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.636 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.637 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.637 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.637 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2024-09-21T03:44:33.637 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.637 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.637 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.637 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.637 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2024-09-21T03:44:33.637 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.638 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.638 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.638 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.638 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2024-09-21T03:44:33.638 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.638 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.638 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.638 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.638 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2024-09-21T03:44:33.639 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.639 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.639 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.639 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.639 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2024-09-21T03:44:33.639 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.639 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.639 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.640 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.640 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2024-09-21T03:44:33.640 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.640 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.640 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.640 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.640 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2024-09-21T03:44:33.640 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 326, 2024-09-21T03:44:33.640 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.568222978, 2024-09-21T03:44:33.641 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001743015 2024-09-21T03:44:33.641 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.641 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2024-09-21T03:44:33.641 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.641 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.641 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.641 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.641 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2024-09-21T03:44:33.641 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.642 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.642 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.642 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.642 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2024-09-21T03:44:33.642 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.642 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.642 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.642 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.643 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2024-09-21T03:44:33.643 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 16342, 2024-09-21T03:44:33.643 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 46.352358137, 2024-09-21T03:44:33.643 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002836394 2024-09-21T03:44:33.643 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.643 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:44:33.643 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:44:33.643 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2024-09-21T03:44:33.644 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.644 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.644 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.644 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:33.644 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.644 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2024-09-21T03:44:33.644 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 2, 2024-09-21T03:44:33.644 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 2, 2024-09-21T03:44:33.644 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 0, 2024-09-21T03:44:33.645 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 2, 2024-09-21T03:44:33.645 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2024-09-21T03:44:33.645 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 34611, 2024-09-21T03:44:33.645 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 17305, 2024-09-21T03:44:33.645 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 939, 2024-09-21T03:44:33.645 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2024-09-21T03:44:33.645 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.645 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-09-21T03:44:33.646 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:33.646 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:33.646 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:33.646 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:33.646 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:33.646 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:33.646 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:33.646 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:33.647 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:33.647 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:33.647 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:33.647 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:33.647 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:33.647 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:33.647 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:33.647 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:33.647 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:33.648 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:33.648 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:33.648 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:33.648 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:33.648 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:33.648 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:33.648 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:33.648 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:33.648 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:33.649 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:33.649 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:33.649 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:33.649 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:33.649 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:33.649 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-09-21T03:44:33.649 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:33.649 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:33.650 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:33.650 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:33.650 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 4600502, 2024-09-21T03:44:33.650 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 1734, 2024-09-21T03:44:33.650 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 528, 2024-09-21T03:44:33.650 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 6, 2024-09-21T03:44:33.650 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-09-21T03:44:33.650 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-09-21T03:44:33.650 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:33.651 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:33.651 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:33.651 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:33.651 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 8064, 2024-09-21T03:44:33.651 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 178, 2024-09-21T03:44:33.651 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:33.651 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:33.651 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:33.651 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-09-21T03:44:33.652 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 4970668, 2024-09-21T03:44:33.652 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 106240, 2024-09-21T03:44:33.652 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:33.652 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-09-21T03:44:33.652 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:33.652 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-09-21T03:44:33.652 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.652 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2024-09-21T03:44:33.653 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2024-09-21T03:44:33.653 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2024-09-21T03:44:33.653 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 44674, 2024-09-21T03:44:33.653 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 359330824, 2024-09-21T03:44:33.653 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 33, 2024-09-21T03:44:33.653 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 44674, 2024-09-21T03:44:33.653 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2024-09-21T03:44:33.653 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 44674, 2024-09-21T03:44:33.654 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 246.501074313, 2024-09-21T03:44:33.654 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.005517774 2024-09-21T03:44:33.654 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.654 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2024-09-21T03:44:33.654 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2024-09-21T03:44:33.654 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 44674, 2024-09-21T03:44:33.654 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 45835 2024-09-21T03:44:33.654 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.654 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 44674, 2024-09-21T03:44:33.655 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 1427, 2024-09-21T03:44:33.655 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 43247, 2024-09-21T03:44:33.655 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2024-09-21T03:44:33.655 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2024-09-21T03:44:33.655 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 0, 2024-09-21T03:44:33.655 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 21, 2024-09-21T03:44:33.655 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 556, 2024-09-21T03:44:33.655 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 25640, 2024-09-21T03:44:33.656 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 325, 2024-09-21T03:44:33.656 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2024-09-21T03:44:33.656 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2024-09-21T03:44:33.656 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2024-09-21T03:44:33.656 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 9, 2024-09-21T03:44:33.656 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 16385, 2024-09-21T03:44:33.656 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2024-09-21T03:44:33.656 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2024-09-21T03:44:33.656 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2024-09-21T03:44:33.657 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 871, 2024-09-21T03:44:33.657 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 42, 2024-09-21T03:44:33.657 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:44:33.657 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2024-09-21T03:44:33.657 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:44:33.657 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2024-09-21T03:44:33.657 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2024-09-21T03:44:33.657 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2024-09-21T03:44:33.658 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:44:33.658 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2024-09-21T03:44:33.658 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:44:33.658 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2024-09-21T03:44:33.658 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2024-09-21T03:44:33.658 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2024-09-21T03:44:33.658 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2024-09-21T03:44:33.658 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2024-09-21T03:44:33.659 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2024-09-21T03:44:33.659 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2024-09-21T03:44:33.659 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2024-09-21T03:44:33.659 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2024-09-21T03:44:33.659 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2024-09-21T03:44:33.659 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2024-09-21T03:44:33.659 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2024-09-21T03:44:33.659 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2024-09-21T03:44:33.659 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2024-09-21T03:44:33.660 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2024-09-21T03:44:33.660 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2024-09-21T03:44:33.660 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2024-09-21T03:44:33.660 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 96, 2024-09-21T03:44:33.660 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2024-09-21T03:44:33.660 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 22, 2024-09-21T03:44:33.660 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 12, 2024-09-21T03:44:33.660 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 12, 2024-09-21T03:44:33.661 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2024-09-21T03:44:33.661 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2024-09-21T03:44:33.661 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 1693, 2024-09-21T03:44:33.661 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 0, 2024-09-21T03:44:33.661 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 293 2024-09-21T03:44:33.661 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.661 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2024-09-21T03:44:33.661 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 1, 2024-09-21T03:44:33.661 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:44:33.661 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2024-09-21T03:44:33.662 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2024-09-21T03:44:33.662 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.662 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2024-09-21T03:44:33.662 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2024-09-21T03:44:33.662 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 10, 2024-09-21T03:44:33.662 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2024-09-21T03:44:33.662 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 10, 2024-09-21T03:44:33.662 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed_ops": 16261, 2024-09-21T03:44:33.663 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 16261, 2024-09-21T03:44:33.663 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2024-09-21T03:44:33.663 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.663 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:33.663 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:33.663 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-09-21T03:44:33.663 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:33.663 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 423631, 2024-09-21T03:44:33.664 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 96842954, 2024-09-21T03:44:33.664 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:33.664 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 423631, 2024-09-21T03:44:33.664 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-09-21T03:44:33.664 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-09-21T03:44:33.664 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 423631, 2024-09-21T03:44:33.664 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 96842954, 2024-09-21T03:44:33.664 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:33.665 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.665 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.665 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.665 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:33.665 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.665 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2024-09-21T03:44:33.665 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:33.665 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-09-21T03:44:33.665 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:33.666 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-09-21T03:44:33.666 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-09-21T03:44:33.666 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:33.666 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:33.666 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 44674, 2024-09-21T03:44:33.666 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 360976369, 2024-09-21T03:44:33.666 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 44674, 2024-09-21T03:44:33.666 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 360976369, 2024-09-21T03:44:33.667 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:33.667 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.667 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.667 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.667 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:33.667 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.667 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2024-09-21T03:44:33.667 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:33.668 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2024-09-21T03:44:33.668 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:33.668 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-09-21T03:44:33.668 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-09-21T03:44:33.668 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:33.668 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:33.668 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 44674, 2024-09-21T03:44:33.668 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 44674, 2024-09-21T03:44:33.668 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 44674, 2024-09-21T03:44:33.669 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 44674, 2024-09-21T03:44:33.669 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:33.669 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.669 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.669 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.669 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:33.669 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.669 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:44:33.669 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:33.670 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-09-21T03:44:33.670 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:33.670 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 16261, 2024-09-21T03:44:33.670 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 1645545, 2024-09-21T03:44:33.670 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:33.670 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 16261, 2024-09-21T03:44:33.670 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-09-21T03:44:33.670 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-09-21T03:44:33.670 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 556, 2024-09-21T03:44:33.671 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 1645545, 2024-09-21T03:44:33.671 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:33.671 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.671 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.671 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.671 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:33.671 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.671 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x563b43be1380": { 2024-09-21T03:44:33.671 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 98236, 2024-09-21T03:44:33.672 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-09-21T03:44:33.672 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:33.672 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 140949, 2024-09-21T03:44:33.672 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 354987847, 2024-09-21T03:44:33.672 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:33.672 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 140949, 2024-09-21T03:44:33.672 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-09-21T03:44:33.672 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-09-21T03:44:33.672 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 25084, 2024-09-21T03:44:33.673 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 354889611, 2024-09-21T03:44:33.673 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:33.673 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:33.673 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:33.673 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:33.673 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:33.673 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:33.673 INFO:teuthology.orchestra.run.smithi187.stdout: "trackedop": { 2024-09-21T03:44:33.674 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2024-09-21T03:44:33.674 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:33.674 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-09-21T03:44:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:33 smithi112 ceph-mon[31442]: pgmap v804: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 239 op/s 2024-09-21T03:44:34.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:33 smithi203 ceph-mon[34723]: pgmap v804: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 239 op/s 2024-09-21T03:44:34.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:33 smithi187 ceph-mon[34603]: pgmap v804: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 239 op/s 2024-09-21T03:44:34.743 INFO:tasks.check_counter:Daemon mds.b mds.exported=9 2024-09-21T03:44:34.743 INFO:tasks.check_counter:Daemon mds.b mds.imported=4 2024-09-21T03:44:34.743 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-21T03:44:34.743 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:44:34.743 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:44:34.744 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T03:44:34.744 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T03:44:34.991 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:44:35.522 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-09-21T03:44:35.523 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:35.523 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1031, 2024-09-21T03:44:35.523 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1031, 2024-09-21T03:44:35.523 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 150088522, 2024-09-21T03:44:35.523 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 327098, 2024-09-21T03:44:35.523 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 8, 2024-09-21T03:44:35.523 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2024-09-21T03:44:35.523 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.164827899, 2024-09-21T03:44:35.524 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.040018696, 2024-09-21T03:44:35.524 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.095007171, 2024-09-21T03:44:35.524 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.022416662, 2024-09-21T03:44:35.524 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:35.524 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1030, 2024-09-21T03:44:35.524 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.036829694, 2024-09-21T03:44:35.524 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000035756 2024-09-21T03:44:35.524 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.524 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:35.525 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.525 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.525 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.525 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.525 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:44:35.525 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:44:35.525 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.525 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:35.525 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1143, 2024-09-21T03:44:35.526 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1362, 2024-09-21T03:44:35.526 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 155486167, 2024-09-21T03:44:35.526 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 926203, 2024-09-21T03:44:35.526 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 8, 2024-09-21T03:44:35.526 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 6, 2024-09-21T03:44:35.526 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.193999791, 2024-09-21T03:44:35.526 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.047358798, 2024-09-21T03:44:35.526 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.111745949, 2024-09-21T03:44:35.526 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.025768119, 2024-09-21T03:44:35.527 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:35.527 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1360, 2024-09-21T03:44:35.527 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.040250222, 2024-09-21T03:44:35.527 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000029595 2024-09-21T03:44:35.527 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.527 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:35.527 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.527 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.527 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.528 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.528 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:44:35.528 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 563360 2024-09-21T03:44:35.528 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.528 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:35.528 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 2333, 2024-09-21T03:44:35.528 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 2314, 2024-09-21T03:44:35.528 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 144144113, 2024-09-21T03:44:35.528 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 818230, 2024-09-21T03:44:35.528 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 5, 2024-09-21T03:44:35.529 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2024-09-21T03:44:35.529 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.310981883, 2024-09-21T03:44:35.529 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.097385988, 2024-09-21T03:44:35.529 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.158548236, 2024-09-21T03:44:35.529 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.051189550, 2024-09-21T03:44:35.529 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:35.529 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2313, 2024-09-21T03:44:35.529 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.076541777, 2024-09-21T03:44:35.529 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000033091 2024-09-21T03:44:35.530 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.530 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:35.530 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.530 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.530 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.530 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.530 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 943536, 2024-09-21T03:44:35.530 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 183184 2024-09-21T03:44:35.531 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.531 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-09-21T03:44:35.531 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2024-09-21T03:44:35.531 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-09-21T03:44:35.531 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.531 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2024-09-21T03:44:35.531 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-09-21T03:44:35.531 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-09-21T03:44:35.531 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 6950, 2024-09-21T03:44:35.531 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.175176889, 2024-09-21T03:44:35.532 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000025205 2024-09-21T03:44:35.532 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:35.532 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.532 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2024-09-21T03:44:35.532 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-09-21T03:44:35.532 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-09-21T03:44:35.532 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.532 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.532 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.533 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:35.533 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.533 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2024-09-21T03:44:35.533 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 0, 2024-09-21T03:44:35.533 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 0, 2024-09-21T03:44:35.533 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2024-09-21T03:44:35.533 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.533 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.533 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.534 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.534 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2024-09-21T03:44:35.534 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 0, 2024-09-21T03:44:35.534 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2024-09-21T03:44:35.534 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2024-09-21T03:44:35.534 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 0, 2024-09-21T03:44:35.534 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2024-09-21T03:44:35.534 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2024-09-21T03:44:35.534 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 32963, 2024-09-21T03:44:35.535 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 8353, 2024-09-21T03:44:35.535 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 24610, 2024-09-21T03:44:35.535 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 0, 2024-09-21T03:44:35.535 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 21030, 2024-09-21T03:44:35.535 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 0, 2024-09-21T03:44:35.535 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 0, 2024-09-21T03:44:35.535 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 0, 2024-09-21T03:44:35.535 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 7, 2024-09-21T03:44:35.536 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 0, 2024-09-21T03:44:35.536 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 0, 2024-09-21T03:44:35.536 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2024-09-21T03:44:35.536 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 0, 2024-09-21T03:44:35.536 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:44:35.536 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2024-09-21T03:44:35.536 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2024-09-21T03:44:35.536 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 0, 2024-09-21T03:44:35.536 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2024-09-21T03:44:35.537 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 0, 2024-09-21T03:44:35.537 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 0, 2024-09-21T03:44:35.537 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 0, 2024-09-21T03:44:35.537 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 0, 2024-09-21T03:44:35.537 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2024-09-21T03:44:35.537 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:44:35.537 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2024-09-21T03:44:35.537 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 0, 2024-09-21T03:44:35.538 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 0, 2024-09-21T03:44:35.538 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2024-09-21T03:44:35.538 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:44:35.538 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2024-09-21T03:44:35.538 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:44:35.538 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:44:35.538 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:44:35.538 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:44:35.538 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:44:35.539 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 0, 2024-09-21T03:44:35.539 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:44:35.539 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:44:35.539 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:44:35.539 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:44:35.539 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:44:35.539 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:44:35.539 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 0, 2024-09-21T03:44:35.540 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:44:35.540 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 0, 2024-09-21T03:44:35.540 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 0 2024-09-21T03:44:35.540 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.540 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2024-09-21T03:44:35.540 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2024-09-21T03:44:35.540 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2024-09-21T03:44:35.540 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2024-09-21T03:44:35.540 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 0, 2024-09-21T03:44:35.540 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2024-09-21T03:44:35.541 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2024-09-21T03:44:35.541 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2024-09-21T03:44:35.541 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:44:35.541 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 0, 2024-09-21T03:44:35.541 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 0, 2024-09-21T03:44:35.541 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2024-09-21T03:44:35.541 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2024-09-21T03:44:35.541 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2024-09-21T03:44:35.541 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:44:35.542 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:44:35.542 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2024-09-21T03:44:35.542 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2024-09-21T03:44:35.542 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:44:35.542 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:44:35.542 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:44:35.542 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 0, 2024-09-21T03:44:35.542 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2024-09-21T03:44:35.542 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:44:35.542 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2024-09-21T03:44:35.543 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0, 2024-09-21T03:44:35.543 INFO:teuthology.orchestra.run.smithi187.stdout: "uninline_started": 0, 2024-09-21T03:44:35.543 INFO:teuthology.orchestra.run.smithi187.stdout: "uninline_succeeded": 0, 2024-09-21T03:44:35.543 INFO:teuthology.orchestra.run.smithi187.stdout: "uninline_write_failed": 0 2024-09-21T03:44:35.543 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.543 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2024-09-21T03:44:35.543 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2024-09-21T03:44:35.543 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 0, 2024-09-21T03:44:35.543 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2024-09-21T03:44:35.544 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2024-09-21T03:44:35.544 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 160069, 2024-09-21T03:44:35.544 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2024-09-21T03:44:35.544 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2024-09-21T03:44:35.544 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 0, 2024-09-21T03:44:35.544 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2024-09-21T03:44:35.544 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2024-09-21T03:44:35.544 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 129, 2024-09-21T03:44:35.544 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 15, 2024-09-21T03:44:35.545 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2024-09-21T03:44:35.545 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2024-09-21T03:44:35.545 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 106498153, 2024-09-21T03:44:35.545 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 451809154, 2024-09-21T03:44:35.545 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 451799025, 2024-09-21T03:44:35.545 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2024-09-21T03:44:35.545 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.545 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.545 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.545 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.546 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 160069 2024-09-21T03:44:35.546 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.546 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2024-09-21T03:44:35.546 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 988, 2024-09-21T03:44:35.546 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 32886, 2024-09-21T03:44:35.546 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 31898, 2024-09-21T03:44:35.546 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 116, 2024-09-21T03:44:35.546 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 143, 2024-09-21T03:44:35.546 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 27, 2024-09-21T03:44:35.547 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 32963, 2024-09-21T03:44:35.547 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 33214, 2024-09-21T03:44:35.547 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 251, 2024-09-21T03:44:35.547 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 0, 2024-09-21T03:44:35.547 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 0, 2024-09-21T03:44:35.547 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 0, 2024-09-21T03:44:35.547 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 158100, 2024-09-21T03:44:35.547 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 366448640 2024-09-21T03:44:35.547 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.547 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2024-09-21T03:44:35.548 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 0, 2024-09-21T03:44:35.548 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 0, 2024-09-21T03:44:35.548 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 0, 2024-09-21T03:44:35.548 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 0, 2024-09-21T03:44:35.548 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 0, 2024-09-21T03:44:35.548 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2024-09-21T03:44:35.548 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.548 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.548 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.549 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.549 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2024-09-21T03:44:35.549 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.549 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.549 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.549 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.549 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2024-09-21T03:44:35.549 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.549 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.549 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.550 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.550 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2024-09-21T03:44:35.550 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.550 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.550 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.550 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.550 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2024-09-21T03:44:35.550 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.551 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.551 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.551 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.551 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2024-09-21T03:44:35.551 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.551 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.551 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.551 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.551 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2024-09-21T03:44:35.552 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.552 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.552 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.552 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.552 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2024-09-21T03:44:35.552 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.552 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.552 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.552 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.553 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2024-09-21T03:44:35.553 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.553 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.553 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.553 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.553 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2024-09-21T03:44:35.553 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.553 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.553 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.554 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.554 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2024-09-21T03:44:35.554 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.554 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.555 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.555 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.555 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2024-09-21T03:44:35.555 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.555 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.555 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.555 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.555 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2024-09-21T03:44:35.556 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.556 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.556 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.556 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.556 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2024-09-21T03:44:35.556 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.556 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.556 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.556 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.556 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2024-09-21T03:44:35.557 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.557 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.557 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.557 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.557 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2024-09-21T03:44:35.557 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.557 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.557 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.557 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.557 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2024-09-21T03:44:35.558 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.558 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.558 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.558 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.558 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2024-09-21T03:44:35.558 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.558 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.558 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.559 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.559 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2024-09-21T03:44:35.559 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.559 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.559 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.559 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.559 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2024-09-21T03:44:35.559 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.559 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.559 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.560 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.560 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2024-09-21T03:44:35.560 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.560 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.560 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.560 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.560 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2024-09-21T03:44:35.560 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.560 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.560 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.561 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.561 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2024-09-21T03:44:35.561 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.561 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.561 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.561 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.561 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2024-09-21T03:44:35.561 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.561 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.562 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.562 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.562 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2024-09-21T03:44:35.562 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.562 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.562 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.562 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.562 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2024-09-21T03:44:35.562 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.563 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.563 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.563 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.563 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2024-09-21T03:44:35.563 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.563 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.563 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.563 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.563 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2024-09-21T03:44:35.564 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.564 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.564 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.564 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.564 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2024-09-21T03:44:35.564 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.564 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.564 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.564 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.565 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:44:35.565 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:44:35.565 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2024-09-21T03:44:35.565 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.565 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.565 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.565 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:35.565 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.565 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2024-09-21T03:44:35.566 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 2, 2024-09-21T03:44:35.566 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 3, 2024-09-21T03:44:35.566 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 1, 2024-09-21T03:44:35.566 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 2, 2024-09-21T03:44:35.566 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2024-09-21T03:44:35.566 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 0, 2024-09-21T03:44:35.566 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 0, 2024-09-21T03:44:35.566 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 2235, 2024-09-21T03:44:35.567 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2024-09-21T03:44:35.567 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.567 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-09-21T03:44:35.567 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:35.567 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:35.567 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:35.567 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:35.567 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:35.567 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:35.568 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:35.568 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:35.568 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:35.568 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:35.568 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:35.568 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:35.568 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:35.568 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:35.568 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:35.569 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:35.569 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:35.569 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:35.569 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:35.569 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:35.569 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:35.569 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:35.569 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:35.569 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:35.569 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:35.570 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:35.570 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:35.570 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:35.570 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:35.570 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:35.570 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:35.570 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-09-21T03:44:35.570 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:35.571 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:35.571 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:35.571 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:35.571 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 489201, 2024-09-21T03:44:35.571 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 1159, 2024-09-21T03:44:35.571 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 88, 2024-09-21T03:44:35.571 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 1, 2024-09-21T03:44:35.571 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-09-21T03:44:35.571 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-09-21T03:44:35.571 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:35.572 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:35.572 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:35.572 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:35.572 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 8064, 2024-09-21T03:44:35.572 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 178, 2024-09-21T03:44:35.572 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:35.572 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:35.572 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:35.573 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-09-21T03:44:35.573 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 25164040, 2024-09-21T03:44:35.573 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 180813, 2024-09-21T03:44:35.573 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:35.573 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-09-21T03:44:35.573 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:35.573 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-09-21T03:44:35.573 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.573 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2024-09-21T03:44:35.574 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2024-09-21T03:44:35.574 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2024-09-21T03:44:35.574 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 4165, 2024-09-21T03:44:35.574 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 21, 2024-09-21T03:44:35.574 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 3, 2024-09-21T03:44:35.574 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 4165, 2024-09-21T03:44:35.574 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2024-09-21T03:44:35.574 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4165, 2024-09-21T03:44:35.574 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 4.168932640, 2024-09-21T03:44:35.574 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001000944 2024-09-21T03:44:35.575 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.575 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2024-09-21T03:44:35.575 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2024-09-21T03:44:35.575 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4165, 2024-09-21T03:44:35.575 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 4166 2024-09-21T03:44:35.575 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.575 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 4165, 2024-09-21T03:44:35.575 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 4165, 2024-09-21T03:44:35.575 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 0, 2024-09-21T03:44:35.576 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2024-09-21T03:44:35.576 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2024-09-21T03:44:35.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 2227, 2024-09-21T03:44:35.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 0, 2024-09-21T03:44:35.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 1936, 2024-09-21T03:44:35.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 0, 2024-09-21T03:44:35.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 0, 2024-09-21T03:44:35.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2024-09-21T03:44:35.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2024-09-21T03:44:35.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2024-09-21T03:44:35.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2024-09-21T03:44:35.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 0, 2024-09-21T03:44:35.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2024-09-21T03:44:35.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2024-09-21T03:44:35.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2024-09-21T03:44:35.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 1, 2024-09-21T03:44:35.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 0, 2024-09-21T03:44:35.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:44:35.578 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2024-09-21T03:44:35.578 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:44:35.578 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2024-09-21T03:44:35.578 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2024-09-21T03:44:35.578 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2024-09-21T03:44:35.578 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:44:35.578 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2024-09-21T03:44:35.578 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:44:35.578 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2024-09-21T03:44:35.579 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2024-09-21T03:44:35.579 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2024-09-21T03:44:35.579 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2024-09-21T03:44:35.579 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2024-09-21T03:44:35.579 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2024-09-21T03:44:35.579 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2024-09-21T03:44:35.579 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2024-09-21T03:44:35.579 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2024-09-21T03:44:35.579 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2024-09-21T03:44:35.580 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2024-09-21T03:44:35.580 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2024-09-21T03:44:35.580 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2024-09-21T03:44:35.580 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2024-09-21T03:44:35.580 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2024-09-21T03:44:35.580 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2024-09-21T03:44:35.580 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2024-09-21T03:44:35.580 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 96, 2024-09-21T03:44:35.580 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2024-09-21T03:44:35.581 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 22, 2024-09-21T03:44:35.581 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 11, 2024-09-21T03:44:35.581 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 11, 2024-09-21T03:44:35.581 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2024-09-21T03:44:35.581 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2024-09-21T03:44:35.581 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 0, 2024-09-21T03:44:35.581 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 2, 2024-09-21T03:44:35.581 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 0 2024-09-21T03:44:35.581 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.582 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2024-09-21T03:44:35.582 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 0, 2024-09-21T03:44:35.582 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:44:35.582 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2024-09-21T03:44:35.582 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2024-09-21T03:44:35.582 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.582 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2024-09-21T03:44:35.582 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2024-09-21T03:44:35.582 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:44:35.583 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2024-09-21T03:44:35.583 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 0, 2024-09-21T03:44:35.583 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed_ops": 0, 2024-09-21T03:44:35.583 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 0, 2024-09-21T03:44:35.583 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2024-09-21T03:44:35.583 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.583 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:35.583 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:35.583 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-09-21T03:44:35.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:35.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 4503, 2024-09-21T03:44:35.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 449293411, 2024-09-21T03:44:35.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:35.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 4503, 2024-09-21T03:44:35.584 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-09-21T03:44:35.584 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-09-21T03:44:35.584 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 4503, 2024-09-21T03:44:35.584 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 449293411, 2024-09-21T03:44:35.585 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:35.585 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.585 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.585 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.585 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:35.585 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.585 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2024-09-21T03:44:35.585 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:35.586 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-09-21T03:44:35.586 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:35.586 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-09-21T03:44:35.586 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-09-21T03:44:35.586 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:35.586 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:35.586 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 4165, 2024-09-21T03:44:35.586 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 447624844, 2024-09-21T03:44:35.586 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 4165, 2024-09-21T03:44:35.587 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 447624844, 2024-09-21T03:44:35.587 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:35.587 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.587 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.587 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.587 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:35.587 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.587 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2024-09-21T03:44:35.587 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:35.588 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2024-09-21T03:44:35.588 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:35.588 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-09-21T03:44:35.588 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-09-21T03:44:35.588 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:35.588 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:35.588 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 4165, 2024-09-21T03:44:35.588 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 4165, 2024-09-21T03:44:35.588 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 4165, 2024-09-21T03:44:35.589 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 4165, 2024-09-21T03:44:35.589 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:35.589 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.589 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.589 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.589 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:35.589 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.589 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:44:35.590 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:35.590 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-09-21T03:44:35.590 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:35.590 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-09-21T03:44:35.590 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-09-21T03:44:35.590 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:35.590 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:35.590 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-09-21T03:44:35.590 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-09-21T03:44:35.591 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-09-21T03:44:35.591 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-09-21T03:44:35.591 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:35.591 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.591 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.591 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.591 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:35.591 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.591 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x56067d2f7450": { 2024-09-21T03:44:35.591 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:35.592 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-09-21T03:44:35.592 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:35.592 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-09-21T03:44:35.592 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-09-21T03:44:35.592 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:35.592 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:35.592 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-09-21T03:44:35.592 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-09-21T03:44:35.592 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-09-21T03:44:35.593 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-09-21T03:44:35.593 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:35.593 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:35.593 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:35.593 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:35.593 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:35.593 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:35.593 INFO:teuthology.orchestra.run.smithi187.stdout: "trackedop": { 2024-09-21T03:44:35.593 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2024-09-21T03:44:35.594 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:35.594 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-09-21T03:44:36.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:35 smithi112 ceph-mon[31442]: pgmap v805: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.8 MiB/s wr, 202 op/s 2024-09-21T03:44:36.138 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:35 smithi187 ceph-mon[34603]: pgmap v805: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.8 MiB/s wr, 202 op/s 2024-09-21T03:44:36.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:35 smithi203 ceph-mon[34723]: pgmap v805: 97 pgs: 97 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.8 MiB/s wr, 202 op/s 2024-09-21T03:44:36.165 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T03:44:36.165 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T03:44:36.165 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-21T03:44:36.165 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:44:36.165 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:44:36.165 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T03:44:36.165 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T03:44:36.410 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:44:36.946 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-09-21T03:44:36.946 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:36.946 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 2593, 2024-09-21T03:44:36.947 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 2594, 2024-09-21T03:44:36.947 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 175287221, 2024-09-21T03:44:36.947 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 824263, 2024-09-21T03:44:36.947 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 8, 2024-09-21T03:44:36.947 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2024-09-21T03:44:36.947 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.321892303, 2024-09-21T03:44:36.947 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.098163409, 2024-09-21T03:44:36.947 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.148459267, 2024-09-21T03:44:36.948 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.058690093, 2024-09-21T03:44:36.948 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:36.948 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2592, 2024-09-21T03:44:36.948 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.081349286, 2024-09-21T03:44:36.948 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000031384 2024-09-21T03:44:36.948 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.948 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:36.948 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.948 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.948 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.949 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.949 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:44:36.949 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:44:36.949 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.949 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:36.949 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1223, 2024-09-21T03:44:36.949 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1205, 2024-09-21T03:44:36.949 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 101831538, 2024-09-21T03:44:36.949 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 464968, 2024-09-21T03:44:36.950 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 6, 2024-09-21T03:44:36.950 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 5, 2024-09-21T03:44:36.950 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.164554996, 2024-09-21T03:44:36.950 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.048912010, 2024-09-21T03:44:36.950 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.101379258, 2024-09-21T03:44:36.950 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.018597534, 2024-09-21T03:44:36.950 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:36.950 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1204, 2024-09-21T03:44:36.950 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.034919090, 2024-09-21T03:44:36.951 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000029002 2024-09-21T03:44:36.951 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.951 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:36.951 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.951 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.951 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.951 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.951 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 942096, 2024-09-21T03:44:36.951 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 181568 2024-09-21T03:44:36.952 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.952 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:36.952 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 652, 2024-09-21T03:44:36.952 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 867, 2024-09-21T03:44:36.952 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 79878906, 2024-09-21T03:44:36.952 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 768868, 2024-09-21T03:44:36.952 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 7, 2024-09-21T03:44:36.952 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 5, 2024-09-21T03:44:36.952 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.110631482, 2024-09-21T03:44:36.953 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.031108727, 2024-09-21T03:44:36.953 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.059870078, 2024-09-21T03:44:36.953 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.014200144, 2024-09-21T03:44:36.953 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:36.953 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 866, 2024-09-21T03:44:36.953 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.028740506, 2024-09-21T03:44:36.953 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000033187 2024-09-21T03:44:36.953 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.953 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:36.954 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.954 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.954 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.954 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.954 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T03:44:36.954 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 563440 2024-09-21T03:44:36.954 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.954 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-09-21T03:44:36.955 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2024-09-21T03:44:36.955 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-09-21T03:44:36.955 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.955 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2024-09-21T03:44:36.955 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-09-21T03:44:36.955 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-09-21T03:44:36.955 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 6938, 2024-09-21T03:44:36.955 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.143678878, 2024-09-21T03:44:36.955 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000020708 2024-09-21T03:44:36.956 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:36.956 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.956 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2024-09-21T03:44:36.956 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-09-21T03:44:36.956 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-09-21T03:44:36.956 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.956 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.956 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.956 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:36.957 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.957 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2024-09-21T03:44:36.957 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 0, 2024-09-21T03:44:36.957 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 0, 2024-09-21T03:44:36.957 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2024-09-21T03:44:36.957 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.957 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.957 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.958 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.958 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2024-09-21T03:44:36.958 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 0, 2024-09-21T03:44:36.958 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2024-09-21T03:44:36.958 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2024-09-21T03:44:36.958 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 0, 2024-09-21T03:44:36.958 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2024-09-21T03:44:36.958 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2024-09-21T03:44:36.959 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 17081, 2024-09-21T03:44:36.959 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 2062, 2024-09-21T03:44:36.959 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 15019, 2024-09-21T03:44:36.959 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 0, 2024-09-21T03:44:36.959 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 14135, 2024-09-21T03:44:36.959 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 0, 2024-09-21T03:44:36.959 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 0, 2024-09-21T03:44:36.959 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 0, 2024-09-21T03:44:36.959 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 6, 2024-09-21T03:44:36.960 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 0, 2024-09-21T03:44:36.960 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 0, 2024-09-21T03:44:36.960 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2024-09-21T03:44:36.960 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 0, 2024-09-21T03:44:36.960 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:44:36.960 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2024-09-21T03:44:36.960 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2024-09-21T03:44:36.960 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 0, 2024-09-21T03:44:36.960 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2024-09-21T03:44:36.961 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 0, 2024-09-21T03:44:36.961 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 0, 2024-09-21T03:44:36.961 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 0, 2024-09-21T03:44:36.961 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 0, 2024-09-21T03:44:36.961 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2024-09-21T03:44:36.961 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:44:36.961 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2024-09-21T03:44:36.961 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 0, 2024-09-21T03:44:36.962 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 0, 2024-09-21T03:44:36.962 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2024-09-21T03:44:36.962 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:44:36.962 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2024-09-21T03:44:36.962 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:44:36.962 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:44:36.962 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:44:36.962 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:44:36.962 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:44:36.963 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 0, 2024-09-21T03:44:36.963 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:44:36.963 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:44:36.963 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:44:36.963 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:44:36.963 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:44:36.963 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:44:36.963 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 0, 2024-09-21T03:44:36.963 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:44:36.964 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 0, 2024-09-21T03:44:36.964 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 0 2024-09-21T03:44:36.964 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.964 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2024-09-21T03:44:36.964 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2024-09-21T03:44:36.964 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2024-09-21T03:44:36.964 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2024-09-21T03:44:36.964 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 0, 2024-09-21T03:44:36.964 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2024-09-21T03:44:36.965 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2024-09-21T03:44:36.965 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2024-09-21T03:44:36.965 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:44:36.965 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 0, 2024-09-21T03:44:36.965 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 0, 2024-09-21T03:44:36.965 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2024-09-21T03:44:36.965 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2024-09-21T03:44:36.965 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2024-09-21T03:44:36.965 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:44:36.966 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:44:36.966 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2024-09-21T03:44:36.966 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2024-09-21T03:44:36.966 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:44:36.966 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:44:36.966 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:44:36.966 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 0, 2024-09-21T03:44:36.966 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2024-09-21T03:44:36.966 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:44:36.967 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2024-09-21T03:44:36.967 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0, 2024-09-21T03:44:36.967 INFO:teuthology.orchestra.run.smithi187.stdout: "uninline_started": 0, 2024-09-21T03:44:36.967 INFO:teuthology.orchestra.run.smithi187.stdout: "uninline_succeeded": 0, 2024-09-21T03:44:36.967 INFO:teuthology.orchestra.run.smithi187.stdout: "uninline_write_failed": 0 2024-09-21T03:44:36.967 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.967 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2024-09-21T03:44:36.967 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2024-09-21T03:44:36.967 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 0, 2024-09-21T03:44:36.968 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2024-09-21T03:44:36.968 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2024-09-21T03:44:36.968 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 140785, 2024-09-21T03:44:36.968 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2024-09-21T03:44:36.968 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2024-09-21T03:44:36.968 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 0, 2024-09-21T03:44:36.968 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2024-09-21T03:44:36.968 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2024-09-21T03:44:36.968 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 129, 2024-09-21T03:44:36.969 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 13, 2024-09-21T03:44:36.969 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2024-09-21T03:44:36.969 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2024-09-21T03:44:36.969 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 54653610, 2024-09-21T03:44:36.969 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 359083915, 2024-09-21T03:44:36.969 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 359074259, 2024-09-21T03:44:36.969 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2024-09-21T03:44:36.969 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.970 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.970 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.970 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.970 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 140785 2024-09-21T03:44:36.970 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.970 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2024-09-21T03:44:36.970 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 792, 2024-09-21T03:44:36.970 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 17164, 2024-09-21T03:44:36.970 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 16372, 2024-09-21T03:44:36.971 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 94, 2024-09-21T03:44:36.971 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 129, 2024-09-21T03:44:36.971 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 35, 2024-09-21T03:44:36.971 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 17081, 2024-09-21T03:44:36.971 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 17396, 2024-09-21T03:44:36.971 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 315, 2024-09-21T03:44:36.971 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 0, 2024-09-21T03:44:36.971 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 0, 2024-09-21T03:44:36.971 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 0, 2024-09-21T03:44:36.972 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 138324, 2024-09-21T03:44:36.972 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 355209216 2024-09-21T03:44:36.972 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.972 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2024-09-21T03:44:36.972 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 0, 2024-09-21T03:44:36.972 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 0, 2024-09-21T03:44:36.972 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 0, 2024-09-21T03:44:36.972 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 0, 2024-09-21T03:44:36.972 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 0, 2024-09-21T03:44:36.973 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2024-09-21T03:44:36.973 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.973 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.973 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.973 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.973 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2024-09-21T03:44:36.973 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.973 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.974 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.974 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.974 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2024-09-21T03:44:36.974 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.974 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.974 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.974 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.974 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2024-09-21T03:44:36.974 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.975 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.975 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.975 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.975 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2024-09-21T03:44:36.975 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.975 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.975 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.975 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.975 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2024-09-21T03:44:36.976 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.976 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.976 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.976 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.976 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2024-09-21T03:44:36.976 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.976 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.976 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.976 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.977 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2024-09-21T03:44:36.977 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.977 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.977 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.977 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.977 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2024-09-21T03:44:36.977 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.977 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.977 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.978 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.978 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2024-09-21T03:44:36.978 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.978 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.978 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.978 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.978 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2024-09-21T03:44:36.979 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.979 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.979 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.979 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.980 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2024-09-21T03:44:36.980 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.980 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.980 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.980 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.980 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2024-09-21T03:44:36.980 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.980 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.981 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.981 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.981 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2024-09-21T03:44:36.981 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.981 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.981 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.981 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.981 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2024-09-21T03:44:36.981 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.982 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.982 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.982 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.982 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2024-09-21T03:44:36.982 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.982 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.982 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.982 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.982 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2024-09-21T03:44:36.983 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.983 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.983 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.983 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.983 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2024-09-21T03:44:36.983 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.983 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.983 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.983 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.984 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2024-09-21T03:44:36.984 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.984 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.984 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.984 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.984 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2024-09-21T03:44:36.984 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.984 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.984 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.985 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.985 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2024-09-21T03:44:36.985 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.985 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.985 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.985 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.985 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2024-09-21T03:44:36.985 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.985 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.986 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.986 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.986 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2024-09-21T03:44:36.986 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.986 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.986 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.986 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.987 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2024-09-21T03:44:36.987 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.987 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.987 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.987 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.987 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2024-09-21T03:44:36.987 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.987 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.987 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.987 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.988 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2024-09-21T03:44:36.988 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.988 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.988 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.988 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.988 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2024-09-21T03:44:36.988 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.988 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.988 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.989 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.989 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2024-09-21T03:44:36.989 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.989 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.989 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.989 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.989 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2024-09-21T03:44:36.989 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.990 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.990 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.990 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.990 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:44:36.990 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:44:36.990 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2024-09-21T03:44:36.990 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:36.990 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:36.990 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:36.991 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:36.991 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.991 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2024-09-21T03:44:36.991 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 2, 2024-09-21T03:44:36.991 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 2, 2024-09-21T03:44:36.991 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 0, 2024-09-21T03:44:36.991 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 2, 2024-09-21T03:44:36.991 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2024-09-21T03:44:36.991 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 0, 2024-09-21T03:44:36.992 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 0, 2024-09-21T03:44:36.992 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 2237, 2024-09-21T03:44:36.992 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2024-09-21T03:44:36.992 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.992 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-09-21T03:44:36.992 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:36.992 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:36.992 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:36.992 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:36.993 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:36.993 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:36.993 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:36.993 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:36.993 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:36.993 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:36.993 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:36.993 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:36.993 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:36.993 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:36.994 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:36.994 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:36.994 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:36.994 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:36.994 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:36.994 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:36.994 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:36.994 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:36.994 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:36.994 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:36.995 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:36.995 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:36.995 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:36.995 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:36.995 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:36.995 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:36.995 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:36.995 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-09-21T03:44:36.995 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:36.996 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:36.996 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:36.996 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:36.996 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 272865, 2024-09-21T03:44:36.996 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 935, 2024-09-21T03:44:36.996 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:44:36.996 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2024-09-21T03:44:36.996 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-09-21T03:44:36.996 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-09-21T03:44:36.997 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:36.997 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:36.997 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:36.997 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:36.997 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 8064, 2024-09-21T03:44:36.997 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 178, 2024-09-21T03:44:36.997 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:36.997 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:36.997 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:36.998 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-09-21T03:44:36.998 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 13841064, 2024-09-21T03:44:36.998 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 115197, 2024-09-21T03:44:36.998 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:36.998 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-09-21T03:44:36.998 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:36.998 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-09-21T03:44:36.998 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:36.998 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2024-09-21T03:44:36.999 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2024-09-21T03:44:36.999 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2024-09-21T03:44:36.999 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 4129, 2024-09-21T03:44:36.999 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 21, 2024-09-21T03:44:36.999 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 3, 2024-09-21T03:44:36.999 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 4129, 2024-09-21T03:44:36.999 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2024-09-21T03:44:36.999 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4129, 2024-09-21T03:44:36.999 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 5.415912376, 2024-09-21T03:44:37.000 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001311676 2024-09-21T03:44:37.000 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:37.000 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2024-09-21T03:44:37.000 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2024-09-21T03:44:37.000 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4129, 2024-09-21T03:44:37.000 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 4130 2024-09-21T03:44:37.000 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:37.000 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 4129, 2024-09-21T03:44:37.001 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 4129, 2024-09-21T03:44:37.001 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 0, 2024-09-21T03:44:37.001 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2024-09-21T03:44:37.001 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2024-09-21T03:44:37.001 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 2224, 2024-09-21T03:44:37.001 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 0, 2024-09-21T03:44:37.001 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 1903, 2024-09-21T03:44:37.001 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 0, 2024-09-21T03:44:37.001 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 0, 2024-09-21T03:44:37.002 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2024-09-21T03:44:37.002 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2024-09-21T03:44:37.002 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2024-09-21T03:44:37.002 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2024-09-21T03:44:37.002 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 0, 2024-09-21T03:44:37.002 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2024-09-21T03:44:37.002 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2024-09-21T03:44:37.002 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2024-09-21T03:44:37.003 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 1, 2024-09-21T03:44:37.003 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 0, 2024-09-21T03:44:37.003 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:44:37.003 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2024-09-21T03:44:37.003 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:44:37.003 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2024-09-21T03:44:37.003 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2024-09-21T03:44:37.003 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2024-09-21T03:44:37.004 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:44:37.004 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2024-09-21T03:44:37.004 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:44:37.004 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2024-09-21T03:44:37.004 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2024-09-21T03:44:37.004 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2024-09-21T03:44:37.004 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2024-09-21T03:44:37.004 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2024-09-21T03:44:37.005 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2024-09-21T03:44:37.005 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2024-09-21T03:44:37.005 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2024-09-21T03:44:37.005 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2024-09-21T03:44:37.005 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2024-09-21T03:44:37.005 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2024-09-21T03:44:37.005 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2024-09-21T03:44:37.005 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2024-09-21T03:44:37.005 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2024-09-21T03:44:37.005 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2024-09-21T03:44:37.006 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2024-09-21T03:44:37.006 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2024-09-21T03:44:37.006 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 96, 2024-09-21T03:44:37.006 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2024-09-21T03:44:37.006 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 22, 2024-09-21T03:44:37.006 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 11, 2024-09-21T03:44:37.006 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 11, 2024-09-21T03:44:37.006 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2024-09-21T03:44:37.006 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2024-09-21T03:44:37.006 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 0, 2024-09-21T03:44:37.007 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 2, 2024-09-21T03:44:37.007 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 0 2024-09-21T03:44:37.007 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:37.007 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2024-09-21T03:44:37.007 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 0, 2024-09-21T03:44:37.007 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:44:37.007 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2024-09-21T03:44:37.007 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2024-09-21T03:44:37.007 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:37.008 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2024-09-21T03:44:37.008 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2024-09-21T03:44:37.008 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:44:37.008 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2024-09-21T03:44:37.008 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 0, 2024-09-21T03:44:37.008 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed_ops": 0, 2024-09-21T03:44:37.008 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 0, 2024-09-21T03:44:37.008 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2024-09-21T03:44:37.008 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:37.009 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:37.009 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:37.009 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-09-21T03:44:37.009 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:37.009 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 4464, 2024-09-21T03:44:37.009 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 356575927, 2024-09-21T03:44:37.009 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:37.009 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 4464, 2024-09-21T03:44:37.010 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-09-21T03:44:37.010 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-09-21T03:44:37.010 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 4464, 2024-09-21T03:44:37.010 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 356575927, 2024-09-21T03:44:37.010 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:37.010 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:37.010 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:37.010 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:37.011 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:37.011 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:37.011 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2024-09-21T03:44:37.011 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:37.011 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-09-21T03:44:37.011 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:37.011 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-09-21T03:44:37.011 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-09-21T03:44:37.011 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:37.011 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:37.012 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 4129, 2024-09-21T03:44:37.012 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 354916939, 2024-09-21T03:44:37.012 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 4129, 2024-09-21T03:44:37.012 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 354916939, 2024-09-21T03:44:37.012 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:37.012 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:37.012 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:37.012 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:37.012 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:37.013 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:37.013 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2024-09-21T03:44:37.013 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:37.013 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2024-09-21T03:44:37.013 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:37.013 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-09-21T03:44:37.013 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-09-21T03:44:37.013 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:37.013 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:37.014 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 4129, 2024-09-21T03:44:37.014 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 4129, 2024-09-21T03:44:37.014 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 4129, 2024-09-21T03:44:37.014 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 4129, 2024-09-21T03:44:37.014 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:37.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:37.014 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:37.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:37.015 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:37.015 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:37.015 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:44:37.015 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:37.015 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-09-21T03:44:37.015 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:37.015 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-09-21T03:44:37.015 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-09-21T03:44:37.016 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:37.016 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:37.016 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-09-21T03:44:37.016 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-09-21T03:44:37.016 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-09-21T03:44:37.016 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-09-21T03:44:37.016 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:37.016 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:37.016 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:37.017 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:37.017 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:37.017 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:37.017 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x5604130896c0": { 2024-09-21T03:44:37.017 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:37.017 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-09-21T03:44:37.017 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:37.017 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-09-21T03:44:37.018 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-09-21T03:44:37.018 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:37.018 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:37.018 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-09-21T03:44:37.018 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-09-21T03:44:37.018 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-09-21T03:44:37.018 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-09-21T03:44:37.018 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:37.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:37.019 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:37.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:37.019 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:37.019 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:37.019 INFO:teuthology.orchestra.run.smithi187.stdout: "trackedop": { 2024-09-21T03:44:37.019 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2024-09-21T03:44:37.019 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:37.019 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-09-21T03:44:37.783 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-21T03:44:37.784 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-21T03:44:37.784 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-09-21T03:44:37.784 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:44:37.784 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:44:37.784 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T03:44:37.784 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T03:44:37.807 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:37 smithi187 ceph-mon[34603]: pgmap v806: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.7 MiB/s wr, 254 op/s 2024-09-21T03:44:38.035 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:44:38.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:37 smithi112 ceph-mon[31442]: pgmap v806: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.7 MiB/s wr, 254 op/s 2024-09-21T03:44:38.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:37 smithi203 ceph-mon[34723]: pgmap v806: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.7 MiB/s wr, 254 op/s 2024-09-21T03:44:38.546 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-09-21T03:44:38.547 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:38.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 4, 2024-09-21T03:44:38.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 4, 2024-09-21T03:44:38.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 2944, 2024-09-21T03:44:38.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 448, 2024-09-21T03:44:38.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:38.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 0, 2024-09-21T03:44:38.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.002958243, 2024-09-21T03:44:38.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.000045108, 2024-09-21T03:44:38.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.002181327, 2024-09-21T03:44:38.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:38.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:38.548 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3, 2024-09-21T03:44:38.548 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.003900699, 2024-09-21T03:44:38.548 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001300233 2024-09-21T03:44:38.548 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:38.549 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:38.549 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:38.549 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:38.549 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:38.549 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:38.549 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:44:38.549 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:44:38.549 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:38.550 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:38.550 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 2, 2024-09-21T03:44:38.550 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 223, 2024-09-21T03:44:38.550 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:44:38.550 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 199984, 2024-09-21T03:44:38.550 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:38.550 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 2, 2024-09-21T03:44:38.550 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.014683064, 2024-09-21T03:44:38.551 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.011413101, 2024-09-21T03:44:38.551 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.001413200, 2024-09-21T03:44:38.551 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:38.551 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:38.551 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 221, 2024-09-21T03:44:38.551 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.011420199, 2024-09-21T03:44:38.551 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000051675 2024-09-21T03:44:38.551 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:38.551 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:38.552 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:38.552 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:38.552 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:38.552 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:38.552 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:44:38.552 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 199984 2024-09-21T03:44:38.552 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:38.552 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:38.552 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 298, 2024-09-21T03:44:38.553 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 293, 2024-09-21T03:44:38.553 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 860160, 2024-09-21T03:44:38.553 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 180400, 2024-09-21T03:44:38.553 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 2, 2024-09-21T03:44:38.553 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 1, 2024-09-21T03:44:38.553 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.042835738, 2024-09-21T03:44:38.553 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.019617999, 2024-09-21T03:44:38.553 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.032103905, 2024-09-21T03:44:38.554 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:38.554 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:38.554 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 292, 2024-09-21T03:44:38.554 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.012131594, 2024-09-21T03:44:38.554 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000041546 2024-09-21T03:44:38.554 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:38.554 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:38.554 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:38.554 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:38.555 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:38.555 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:38.555 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 860160, 2024-09-21T03:44:38.555 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 180400 2024-09-21T03:44:38.555 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:38.555 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-09-21T03:44:38.555 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 0, 2024-09-21T03:44:38.555 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-09-21T03:44:38.556 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:38.556 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-09-21T03:44:38.556 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:38.556 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:38.556 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:38.556 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:38.556 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:38.556 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:38.556 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:38.557 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:38.557 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:38.557 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:38.557 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:38.557 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:38.557 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:38.557 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:38.557 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:38.557 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:38.558 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:38.558 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:38.558 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:38.558 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:38.558 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:38.558 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:38.558 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:38.558 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:38.559 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:38.559 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:38.559 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:38.559 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:38.559 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:38.559 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:38.559 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:38.559 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-09-21T03:44:38.560 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:38.560 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:38.560 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:38.560 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:38.560 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 148721, 2024-09-21T03:44:38.560 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 30, 2024-09-21T03:44:38.560 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:44:38.560 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2024-09-21T03:44:38.560 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-09-21T03:44:38.561 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-09-21T03:44:38.561 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:38.561 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:38.561 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:38.561 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:38.561 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 0, 2024-09-21T03:44:38.561 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 0, 2024-09-21T03:44:38.561 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:38.562 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:38.562 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:38.562 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-09-21T03:44:38.562 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 968, 2024-09-21T03:44:38.562 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 2, 2024-09-21T03:44:38.562 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:38.562 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-09-21T03:44:38.562 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:38.562 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-09-21T03:44:38.563 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-09-21T03:44:38.563 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:38.563 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-09-21T03:44:38.563 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-09-21T03:44:38.563 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-09-21T03:44:38.563 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 300, 2024-09-21T03:44:38.563 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 818049, 2024-09-21T03:44:38.563 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:38.564 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 300, 2024-09-21T03:44:38.564 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-09-21T03:44:38.564 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-09-21T03:44:38.564 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 300, 2024-09-21T03:44:38.564 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 818049, 2024-09-21T03:44:38.564 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-09-21T03:44:38.564 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-09-21T03:44:38.564 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-09-21T03:44:38.564 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-09-21T03:44:38.565 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:38.565 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-09-21T03:44:38.565 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-09-21T03:44:39.235 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-21T03:44:39.235 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-21T03:44:39.235 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-21T03:44:39.235 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-21T03:44:39.235 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-21T03:44:39.236 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T03:44:39.236 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T03:44:39.901 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:44:40.023 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:39 smithi203 ceph-mon[34723]: pgmap v807: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 4.1 MiB/s wr, 168 op/s 2024-09-21T03:44:40.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:39 smithi112 ceph-mon[31442]: pgmap v807: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 4.1 MiB/s wr, 168 op/s 2024-09-21T03:44:40.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:39 smithi187 ceph-mon[34603]: pgmap v807: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 4.1 MiB/s wr, 168 op/s 2024-09-21T03:44:40.873 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-21T03:44:40.873 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:40.873 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 248087, 2024-09-21T03:44:40.873 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 218915, 2024-09-21T03:44:40.873 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 77551781, 2024-09-21T03:44:40.874 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 274093226, 2024-09-21T03:44:40.874 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 16, 2024-09-21T03:44:40.874 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 4, 2024-09-21T03:44:40.874 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 17.108009108, 2024-09-21T03:44:40.874 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 8.696158874, 2024-09-21T03:44:40.874 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 9.210270856, 2024-09-21T03:44:40.874 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 1.058288125, 2024-09-21T03:44:40.874 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:40.874 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 218916, 2024-09-21T03:44:40.875 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 5.333284780, 2024-09-21T03:44:40.875 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000024362 2024-09-21T03:44:40.875 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.875 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:40.875 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 400800, 2024-09-21T03:44:40.875 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.053564872, 2024-09-21T03:44:40.875 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000133 2024-09-21T03:44:40.875 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.876 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:44:40.876 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:44:40.876 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.876 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:40.876 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 262245, 2024-09-21T03:44:40.876 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 427023, 2024-09-21T03:44:40.876 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 92861879, 2024-09-21T03:44:40.876 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 492958720, 2024-09-21T03:44:40.876 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 50, 2024-09-21T03:44:40.877 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 6, 2024-09-21T03:44:40.877 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 27.796100427, 2024-09-21T03:44:40.877 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 15.894151572, 2024-09-21T03:44:40.877 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 12.671515469, 2024-09-21T03:44:40.877 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 1.325577806, 2024-09-21T03:44:40.877 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:40.877 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 427022, 2024-09-21T03:44:40.877 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 19.242435115, 2024-09-21T03:44:40.877 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000045061 2024-09-21T03:44:40.878 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.878 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:40.878 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 573345, 2024-09-21T03:44:40.878 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.107865695, 2024-09-21T03:44:40.878 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000188 2024-09-21T03:44:40.878 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.878 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 85039120, 2024-09-21T03:44:40.878 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 317093920 2024-09-21T03:44:40.878 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.879 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:40.879 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 30289, 2024-09-21T03:44:40.879 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 28233, 2024-09-21T03:44:40.879 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 11452813, 2024-09-21T03:44:40.879 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 73325708, 2024-09-21T03:44:40.879 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 12, 2024-09-21T03:44:40.879 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 7, 2024-09-21T03:44:40.879 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 2.825155090, 2024-09-21T03:44:40.880 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 1.124847144, 2024-09-21T03:44:40.880 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.910679389, 2024-09-21T03:44:40.880 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.789780993, 2024-09-21T03:44:40.880 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:40.880 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 28229, 2024-09-21T03:44:40.880 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.196183980, 2024-09-21T03:44:40.880 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000077798 2024-09-21T03:44:40.880 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.880 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:40.881 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 21439, 2024-09-21T03:44:40.881 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.004264211, 2024-09-21T03:44:40.881 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000198 2024-09-21T03:44:40.881 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.881 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 5520, 2024-09-21T03:44:40.881 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 495408 2024-09-21T03:44:40.881 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.882 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-09-21T03:44:40.882 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-09-21T03:44:40.882 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-09-21T03:44:40.882 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.882 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-09-21T03:44:40.882 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-21T03:44:40.883 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-21T03:44:40.883 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 85089, 2024-09-21T03:44:40.883 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 83.489653896, 2024-09-21T03:44:40.883 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000981203 2024-09-21T03:44:40.883 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:40.883 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.883 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-09-21T03:44:40.883 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-21T03:44:40.884 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-21T03:44:40.884 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 52341, 2024-09-21T03:44:40.884 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 7.659659308, 2024-09-21T03:44:40.884 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000146341 2024-09-21T03:44:40.884 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:40.884 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.884 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-09-21T03:44:40.884 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 169250, 2024-09-21T03:44:40.884 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 163648, 2024-09-21T03:44:40.885 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-09-21T03:44:40.885 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 163648, 2024-09-21T03:44:40.885 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 460.524323627, 2024-09-21T03:44:40.885 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002814115 2024-09-21T03:44:40.885 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.885 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-09-21T03:44:40.885 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 5602, 2024-09-21T03:44:40.885 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 1, 2024-09-21T03:44:40.885 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2024-09-21T03:44:40.886 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 89, 2024-09-21T03:44:40.886 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 9, 2024-09-21T03:44:40.886 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 3, 2024-09-21T03:44:40.886 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 2468, 2024-09-21T03:44:40.886 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 120, 2024-09-21T03:44:40.886 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 1627, 2024-09-21T03:44:40.886 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 721, 2024-09-21T03:44:40.886 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 2297, 2024-09-21T03:44:40.886 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 465, 2024-09-21T03:44:40.887 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 234, 2024-09-21T03:44:40.887 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 234, 2024-09-21T03:44:40.887 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 12, 2024-09-21T03:44:40.887 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 327725, 2024-09-21T03:44:40.887 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 155829, 2024-09-21T03:44:40.887 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 5590, 2024-09-21T03:44:40.887 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 54, 2024-09-21T03:44:40.887 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 1, 2024-09-21T03:44:40.888 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2024-09-21T03:44:40.888 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 3668, 2024-09-21T03:44:40.888 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 27586, 2024-09-21T03:44:40.888 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-09-21T03:44:40.888 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 11, 2024-09-21T03:44:40.888 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 282, 2024-09-21T03:44:40.888 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 7, 2024-09-21T03:44:40.888 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 709, 2024-09-21T03:44:40.888 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-09-21T03:44:40.889 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 1024, 2024-09-21T03:44:40.889 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 1, 2024-09-21T03:44:40.889 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 723, 2024-09-21T03:44:40.889 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 212053256, 2024-09-21T03:44:40.889 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 6, 2024-09-21T03:44:40.889 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 1024, 2024-09-21T03:44:40.889 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-09-21T03:44:40.889 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:44:40.890 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:44:40.890 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 80, 2024-09-21T03:44:40.890 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:44:40.890 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 80, 2024-09-21T03:44:40.890 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 944, 2024-09-21T03:44:40.890 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 8, 2024-09-21T03:44:40.890 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 5956, 2024-09-21T03:44:40.890 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 116338, 2024-09-21T03:44:40.890 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:44:40.891 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:44:40.891 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:44:40.891 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 37540, 2024-09-21T03:44:40.891 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 21337, 2024-09-21T03:44:40.891 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 6983, 2024-09-21T03:44:40.891 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 144104 2024-09-21T03:44:40.891 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.891 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-09-21T03:44:40.892 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 128, 2024-09-21T03:44:40.892 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 159, 2024-09-21T03:44:40.892 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 89, 2024-09-21T03:44:40.892 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 189, 2024-09-21T03:44:40.892 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 187, 2024-09-21T03:44:40.892 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2024-09-21T03:44:40.892 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-09-21T03:44:40.892 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:44:40.893 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 31893, 2024-09-21T03:44:40.893 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 31893, 2024-09-21T03:44:40.893 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2024-09-21T03:44:40.893 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2024-09-21T03:44:40.893 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-09-21T03:44:40.893 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:44:40.893 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:44:40.893 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-09-21T03:44:40.893 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-09-21T03:44:40.894 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:44:40.894 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:44:40.894 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:44:40.894 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 14, 2024-09-21T03:44:40.894 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-09-21T03:44:40.894 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 12, 2024-09-21T03:44:40.894 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-09-21T03:44:40.894 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0, 2024-09-21T03:44:40.895 INFO:teuthology.orchestra.run.smithi203.stdout: "uninline_started": 0, 2024-09-21T03:44:40.895 INFO:teuthology.orchestra.run.smithi203.stdout: "uninline_succeeded": 0, 2024-09-21T03:44:40.895 INFO:teuthology.orchestra.run.smithi203.stdout: "uninline_write_failed": 0 2024-09-21T03:44:40.895 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.895 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-09-21T03:44:40.895 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2024-09-21T03:44:40.895 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 160452, 2024-09-21T03:44:40.895 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 62660, 2024-09-21T03:44:40.895 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 61451, 2024-09-21T03:44:40.896 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 99001, 2024-09-21T03:44:40.896 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-09-21T03:44:40.896 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 1209, 2024-09-21T03:44:40.896 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 206, 2024-09-21T03:44:40.896 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 78, 2024-09-21T03:44:40.896 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 76, 2024-09-21T03:44:40.896 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 130, 2024-09-21T03:44:40.896 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 9, 2024-09-21T03:44:40.896 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-09-21T03:44:40.897 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 2, 2024-09-21T03:44:40.897 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 106498153, 2024-09-21T03:44:40.897 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 452985579, 2024-09-21T03:44:40.897 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 0, 2024-09-21T03:44:40.897 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-09-21T03:44:40.897 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 45351, 2024-09-21T03:44:40.897 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 203.911764665, 2024-09-21T03:44:40.897 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004496301 2024-09-21T03:44:40.898 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.898 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 0 2024-09-21T03:44:40.898 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.898 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-09-21T03:44:40.898 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 829, 2024-09-21T03:44:40.898 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 33019, 2024-09-21T03:44:40.898 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 32190, 2024-09-21T03:44:40.898 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 180, 2024-09-21T03:44:40.898 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 286, 2024-09-21T03:44:40.899 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 106, 2024-09-21T03:44:40.899 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 2260, 2024-09-21T03:44:40.899 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 51208, 2024-09-21T03:44:40.899 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 48948, 2024-09-21T03:44:40.899 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 234, 2024-09-21T03:44:40.899 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 33307, 2024-09-21T03:44:40.899 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 33073, 2024-09-21T03:44:40.899 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 153280, 2024-09-21T03:44:40.900 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 361648128 2024-09-21T03:44:40.900 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.900 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-09-21T03:44:40.900 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 281202, 2024-09-21T03:44:40.900 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 322, 2024-09-21T03:44:40.900 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 169250, 2024-09-21T03:44:40.900 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 3647, 2024-09-21T03:44:40.900 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 27298, 2024-09-21T03:44:40.900 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-09-21T03:44:40.901 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 31942, 2024-09-21T03:44:40.901 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 91.227284595, 2024-09-21T03:44:40.901 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002856029 2024-09-21T03:44:40.901 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.901 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-09-21T03:44:40.901 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 575, 2024-09-21T03:44:40.901 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 8.381091540, 2024-09-21T03:44:40.901 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.014575811 2024-09-21T03:44:40.902 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.902 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-09-21T03:44:40.902 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.902 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.902 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.902 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.902 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-09-21T03:44:40.902 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.902 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.903 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.903 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.903 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-09-21T03:44:40.903 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 45864, 2024-09-21T03:44:40.903 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 87.285389582, 2024-09-21T03:44:40.903 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001903135 2024-09-21T03:44:40.903 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.903 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-09-21T03:44:40.904 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.904 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.904 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.904 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.904 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-09-21T03:44:40.904 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.904 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.904 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.904 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.905 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-09-21T03:44:40.905 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.905 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.905 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.905 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.905 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-09-21T03:44:40.905 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.906 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.906 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.906 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.906 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-09-21T03:44:40.907 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.907 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.907 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.907 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.907 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-09-21T03:44:40.907 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.907 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.907 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.908 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.908 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-09-21T03:44:40.908 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.908 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.908 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.908 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.908 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-09-21T03:44:40.908 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.908 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.909 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.909 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.909 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-09-21T03:44:40.909 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.909 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.909 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.909 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.909 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-09-21T03:44:40.909 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.910 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.910 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.910 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.910 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-09-21T03:44:40.910 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 48112, 2024-09-21T03:44:40.910 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 180.791921555, 2024-09-21T03:44:40.910 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.003757730 2024-09-21T03:44:40.910 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.911 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-09-21T03:44:40.911 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 4939, 2024-09-21T03:44:40.911 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 16.445509381, 2024-09-21T03:44:40.911 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.003329724 2024-09-21T03:44:40.911 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.911 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-09-21T03:44:40.911 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.911 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.911 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.912 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.912 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-09-21T03:44:40.912 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.912 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.912 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.912 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.912 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-09-21T03:44:40.912 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.912 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.913 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.913 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.913 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-09-21T03:44:40.913 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.913 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.913 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.913 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.913 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-09-21T03:44:40.913 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.914 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.914 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.914 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.914 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-09-21T03:44:40.914 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 11, 2024-09-21T03:44:40.914 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.152249972, 2024-09-21T03:44:40.914 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.013840906 2024-09-21T03:44:40.914 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.915 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-09-21T03:44:40.915 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.915 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.915 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.915 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.915 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-09-21T03:44:40.915 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 312, 2024-09-21T03:44:40.915 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.216043027, 2024-09-21T03:44:40.915 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.003897573 2024-09-21T03:44:40.916 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.916 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-09-21T03:44:40.916 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.916 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.916 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.916 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.916 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-09-21T03:44:40.916 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.916 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.917 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.917 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.917 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-09-21T03:44:40.917 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.917 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.917 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.917 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.917 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-09-21T03:44:40.918 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 31893, 2024-09-21T03:44:40.918 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 75.024833975, 2024-09-21T03:44:40.918 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002352391 2024-09-21T03:44:40.918 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.918 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:44:40.918 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:44:40.918 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-09-21T03:44:40.918 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.918 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.919 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.919 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:40.919 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.919 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-09-21T03:44:40.919 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 2, 2024-09-21T03:44:40.919 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 3, 2024-09-21T03:44:40.919 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 1, 2024-09-21T03:44:40.919 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 2, 2024-09-21T03:44:40.920 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-09-21T03:44:40.920 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 34351, 2024-09-21T03:44:40.920 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 17175, 2024-09-21T03:44:40.920 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 940, 2024-09-21T03:44:40.920 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-09-21T03:44:40.920 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.920 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-09-21T03:44:40.920 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:40.920 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:40.921 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:40.921 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:40.921 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:40.921 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:40.921 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:40.921 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:40.921 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:40.921 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:40.921 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:40.922 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:40.922 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:40.922 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:40.922 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:40.922 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:40.922 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:40.922 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:40.922 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:40.923 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:40.923 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:40.923 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:40.923 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:40.923 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:40.923 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:40.923 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:40.923 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:40.923 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:40.924 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:40.924 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:40.924 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:40.924 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-09-21T03:44:40.924 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:40.924 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:40.924 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:40.924 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:40.925 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 4035979, 2024-09-21T03:44:40.925 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 1651, 2024-09-21T03:44:40.925 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 264, 2024-09-21T03:44:40.925 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 3, 2024-09-21T03:44:40.925 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-09-21T03:44:40.925 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-09-21T03:44:40.925 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:40.925 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:40.926 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:40.926 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:40.926 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8064, 2024-09-21T03:44:40.926 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 178, 2024-09-21T03:44:40.926 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:40.926 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:40.926 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:40.926 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-09-21T03:44:40.926 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 5314611, 2024-09-21T03:44:40.927 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 84563, 2024-09-21T03:44:40.927 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:40.927 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-09-21T03:44:40.927 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:40.927 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-09-21T03:44:40.927 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.927 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-09-21T03:44:40.927 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-09-21T03:44:40.928 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-09-21T03:44:40.928 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 81795, 2024-09-21T03:44:40.928 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 456041187, 2024-09-21T03:44:40.928 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 33, 2024-09-21T03:44:40.928 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 81795, 2024-09-21T03:44:40.928 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-09-21T03:44:40.928 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 81795, 2024-09-21T03:44:40.928 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 355.497138527, 2024-09-21T03:44:40.928 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004346196 2024-09-21T03:44:40.929 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.929 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-09-21T03:44:40.929 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-09-21T03:44:40.929 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 81795, 2024-09-21T03:44:40.929 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 83235 2024-09-21T03:44:40.929 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.929 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 81795, 2024-09-21T03:44:40.929 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 1999, 2024-09-21T03:44:40.930 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 79796, 2024-09-21T03:44:40.930 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-09-21T03:44:40.930 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-09-21T03:44:40.930 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 0, 2024-09-21T03:44:40.930 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 21, 2024-09-21T03:44:40.930 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 968, 2024-09-21T03:44:40.930 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 46319, 2024-09-21T03:44:40.930 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 492, 2024-09-21T03:44:40.930 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-09-21T03:44:40.931 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-09-21T03:44:40.931 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2024-09-21T03:44:40.931 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 9, 2024-09-21T03:44:40.931 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 32042, 2024-09-21T03:44:40.931 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-09-21T03:44:40.931 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-09-21T03:44:40.931 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-09-21T03:44:40.931 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 1025, 2024-09-21T03:44:40.932 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 42, 2024-09-21T03:44:40.932 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:44:40.932 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-09-21T03:44:40.932 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:44:40.932 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-09-21T03:44:40.932 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-09-21T03:44:40.932 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-09-21T03:44:40.932 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:44:40.932 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-09-21T03:44:40.933 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:44:40.933 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2024-09-21T03:44:40.933 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-09-21T03:44:40.933 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-09-21T03:44:40.933 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-09-21T03:44:40.933 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-09-21T03:44:40.933 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-09-21T03:44:40.933 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-09-21T03:44:40.933 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-09-21T03:44:40.934 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-09-21T03:44:40.934 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-09-21T03:44:40.934 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-09-21T03:44:40.934 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-09-21T03:44:40.934 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-09-21T03:44:40.934 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-09-21T03:44:40.934 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-09-21T03:44:40.934 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-09-21T03:44:40.935 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-09-21T03:44:40.935 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 96, 2024-09-21T03:44:40.935 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-09-21T03:44:40.935 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 22, 2024-09-21T03:44:40.935 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 12, 2024-09-21T03:44:40.935 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 12, 2024-09-21T03:44:40.935 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-09-21T03:44:40.935 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-09-21T03:44:40.935 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 1792, 2024-09-21T03:44:40.936 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 8, 2024-09-21T03:44:40.936 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 517 2024-09-21T03:44:40.936 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.936 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-09-21T03:44:40.936 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 1, 2024-09-21T03:44:40.936 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:44:40.936 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-09-21T03:44:40.936 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-09-21T03:44:40.936 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.937 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-09-21T03:44:40.937 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-09-21T03:44:40.937 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 10, 2024-09-21T03:44:40.937 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-09-21T03:44:40.937 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 10, 2024-09-21T03:44:40.937 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 31893, 2024-09-21T03:44:40.937 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 31893, 2024-09-21T03:44:40.937 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-09-21T03:44:40.938 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.938 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:40.938 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:40.938 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-21T03:44:40.938 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:40.938 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 540617, 2024-09-21T03:44:40.938 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 122665200, 2024-09-21T03:44:40.938 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:40.938 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 540617, 2024-09-21T03:44:40.939 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-21T03:44:40.939 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-21T03:44:40.939 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 540617, 2024-09-21T03:44:40.939 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 122665200, 2024-09-21T03:44:40.939 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:40.939 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.939 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.939 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.939 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:40.940 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.940 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-09-21T03:44:40.940 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:40.940 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-21T03:44:40.940 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:40.940 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-21T03:44:40.940 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-21T03:44:40.941 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:40.941 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:40.941 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 81795, 2024-09-21T03:44:40.941 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 459265125, 2024-09-21T03:44:40.941 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 81795, 2024-09-21T03:44:40.941 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 459265125, 2024-09-21T03:44:40.941 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:40.941 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.941 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.942 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.942 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:40.942 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.942 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-09-21T03:44:40.942 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:40.942 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-09-21T03:44:40.942 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:40.942 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-21T03:44:40.943 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-21T03:44:40.943 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:40.943 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:40.943 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 81795, 2024-09-21T03:44:40.943 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 81795, 2024-09-21T03:44:40.943 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 81795, 2024-09-21T03:44:40.943 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 81795, 2024-09-21T03:44:40.943 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:40.943 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.944 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.944 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.944 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:40.944 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.944 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:44:40.944 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:40.944 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-21T03:44:40.944 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:40.944 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 31893, 2024-09-21T03:44:40.945 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 3224129, 2024-09-21T03:44:40.945 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:40.945 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 31893, 2024-09-21T03:44:40.945 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-21T03:44:40.945 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-21T03:44:40.945 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 968, 2024-09-21T03:44:40.945 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 3224129, 2024-09-21T03:44:40.945 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:40.945 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.946 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.946 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.946 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:40.946 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.946 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x55d1519f5520": { 2024-09-21T03:44:40.946 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 747, 2024-09-21T03:44:40.946 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-21T03:44:40.946 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:40.947 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 160452, 2024-09-21T03:44:40.947 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 448791275, 2024-09-21T03:44:40.947 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:40.947 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 160452, 2024-09-21T03:44:40.947 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-21T03:44:40.947 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-21T03:44:40.947 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 45351, 2024-09-21T03:44:40.947 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 448790528, 2024-09-21T03:44:40.947 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:40.948 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:40.948 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:40.948 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:40.948 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:40.948 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:40.948 INFO:teuthology.orchestra.run.smithi203.stdout: "trackedop": { 2024-09-21T03:44:40.948 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2024-09-21T03:44:40.948 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:40.948 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-21T03:44:41.928 INFO:tasks.check_counter:Daemon mds.c mds.exported=11 2024-09-21T03:44:41.928 INFO:tasks.check_counter:Daemon mds.c mds.imported=7 2024-09-21T03:44:41.929 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-21T03:44:41.929 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:44:41.929 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:44:41.929 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T03:44:41.929 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T03:44:41.955 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:41 smithi203 ceph-mon[34723]: pgmap v808: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 186 op/s 2024-09-21T03:44:42.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:41 smithi112 ceph-mon[31442]: pgmap v808: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 186 op/s 2024-09-21T03:44:42.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:41 smithi187 ceph-mon[34603]: pgmap v808: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 186 op/s 2024-09-21T03:44:42.185 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:44:42.739 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-21T03:44:42.739 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:42.739 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 2647, 2024-09-21T03:44:42.739 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 2647, 2024-09-21T03:44:42.739 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 424467319, 2024-09-21T03:44:42.739 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 840932, 2024-09-21T03:44:42.740 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 8, 2024-09-21T03:44:42.740 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 4, 2024-09-21T03:44:42.740 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.505963096, 2024-09-21T03:44:42.740 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.098003860, 2024-09-21T03:44:42.740 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.276006365, 2024-09-21T03:44:42.740 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.113891270, 2024-09-21T03:44:42.740 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:42.740 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2646, 2024-09-21T03:44:42.740 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.085300506, 2024-09-21T03:44:42.741 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000032237 2024-09-21T03:44:42.741 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.741 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:42.741 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.741 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.741 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.741 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.741 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:44:42.742 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:44:42.742 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.742 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:42.742 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 1905, 2024-09-21T03:44:42.742 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 1887, 2024-09-21T03:44:42.742 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 436859635, 2024-09-21T03:44:42.742 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 682674, 2024-09-21T03:44:42.742 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 6, 2024-09-21T03:44:42.743 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-09-21T03:44:42.743 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.374810867, 2024-09-21T03:44:42.743 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.072995535, 2024-09-21T03:44:42.743 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.269324325, 2024-09-21T03:44:42.743 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.033914907, 2024-09-21T03:44:42.743 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:42.743 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1886, 2024-09-21T03:44:42.743 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.053760791, 2024-09-21T03:44:42.743 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000028505 2024-09-21T03:44:42.744 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.744 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:42.744 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.744 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.744 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.744 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.744 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 943968, 2024-09-21T03:44:42.744 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 183984 2024-09-21T03:44:42.744 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.745 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:42.745 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 1118, 2024-09-21T03:44:42.745 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 1337, 2024-09-21T03:44:42.745 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 260332579, 2024-09-21T03:44:42.745 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 921387, 2024-09-21T03:44:42.745 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 7, 2024-09-21T03:44:42.745 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-09-21T03:44:42.745 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.231335861, 2024-09-21T03:44:42.746 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.053707798, 2024-09-21T03:44:42.746 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.141863707, 2024-09-21T03:44:42.746 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.026966501, 2024-09-21T03:44:42.746 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:42.746 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1335, 2024-09-21T03:44:42.746 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.039309385, 2024-09-21T03:44:42.746 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000029445 2024-09-21T03:44:42.746 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.747 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:42.747 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.747 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.747 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.747 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.747 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:44:42.747 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 566816 2024-09-21T03:44:42.747 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.747 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-09-21T03:44:42.748 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-09-21T03:44:42.748 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-09-21T03:44:42.748 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.748 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-09-21T03:44:42.748 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-21T03:44:42.748 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-21T03:44:42.748 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 8151, 2024-09-21T03:44:42.748 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.211132432, 2024-09-21T03:44:42.749 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000025902 2024-09-21T03:44:42.749 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:42.749 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.749 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-09-21T03:44:42.749 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-21T03:44:42.749 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-21T03:44:42.749 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.749 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.750 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.750 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:42.750 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.750 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-09-21T03:44:42.750 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 0, 2024-09-21T03:44:42.750 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 0, 2024-09-21T03:44:42.750 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-09-21T03:44:42.750 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.750 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.751 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.751 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.751 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-09-21T03:44:42.751 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 0, 2024-09-21T03:44:42.751 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2024-09-21T03:44:42.751 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2024-09-21T03:44:42.751 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 0, 2024-09-21T03:44:42.751 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2024-09-21T03:44:42.752 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2024-09-21T03:44:42.752 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 31140, 2024-09-21T03:44:42.752 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 10670, 2024-09-21T03:44:42.752 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 20470, 2024-09-21T03:44:42.752 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 0, 2024-09-21T03:44:42.752 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 15897, 2024-09-21T03:44:42.752 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 0, 2024-09-21T03:44:42.752 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2024-09-21T03:44:42.752 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2024-09-21T03:44:42.753 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 11, 2024-09-21T03:44:42.753 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 0, 2024-09-21T03:44:42.753 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 0, 2024-09-21T03:44:42.753 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 0, 2024-09-21T03:44:42.753 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 0, 2024-09-21T03:44:42.753 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:44:42.753 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2024-09-21T03:44:42.753 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2024-09-21T03:44:42.754 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2024-09-21T03:44:42.754 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-09-21T03:44:42.754 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2024-09-21T03:44:42.754 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2024-09-21T03:44:42.754 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 0, 2024-09-21T03:44:42.754 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 0, 2024-09-21T03:44:42.754 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-09-21T03:44:42.754 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:44:42.754 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2024-09-21T03:44:42.755 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 723, 2024-09-21T03:44:42.755 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 212057352, 2024-09-21T03:44:42.755 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 6, 2024-09-21T03:44:42.755 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:44:42.755 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-09-21T03:44:42.755 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:44:42.755 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:44:42.755 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:44:42.756 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:44:42.756 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:44:42.756 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 0, 2024-09-21T03:44:42.756 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:44:42.756 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:44:42.756 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:44:42.756 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:44:42.756 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:44:42.756 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:44:42.757 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 0, 2024-09-21T03:44:42.757 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:44:42.757 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 0, 2024-09-21T03:44:42.757 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 0 2024-09-21T03:44:42.757 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.757 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-09-21T03:44:42.757 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2024-09-21T03:44:42.757 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 0, 2024-09-21T03:44:42.757 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 0, 2024-09-21T03:44:42.758 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 0, 2024-09-21T03:44:42.758 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 0, 2024-09-21T03:44:42.758 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2024-09-21T03:44:42.758 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-09-21T03:44:42.758 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:44:42.758 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2024-09-21T03:44:42.758 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2024-09-21T03:44:42.758 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2024-09-21T03:44:42.758 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2024-09-21T03:44:42.759 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-09-21T03:44:42.759 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:44:42.759 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:44:42.759 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-09-21T03:44:42.759 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-09-21T03:44:42.759 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:44:42.759 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:44:42.759 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:44:42.760 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2024-09-21T03:44:42.760 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-09-21T03:44:42.760 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:44:42.760 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-09-21T03:44:42.760 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0, 2024-09-21T03:44:42.760 INFO:teuthology.orchestra.run.smithi203.stdout: "uninline_started": 0, 2024-09-21T03:44:42.760 INFO:teuthology.orchestra.run.smithi203.stdout: "uninline_succeeded": 0, 2024-09-21T03:44:42.760 INFO:teuthology.orchestra.run.smithi203.stdout: "uninline_write_failed": 0 2024-09-21T03:44:42.760 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.761 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-09-21T03:44:42.761 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2024-09-21T03:44:42.761 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 0, 2024-09-21T03:44:42.761 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2024-09-21T03:44:42.761 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2024-09-21T03:44:42.761 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 179979, 2024-09-21T03:44:42.761 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-09-21T03:44:42.761 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2024-09-21T03:44:42.761 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 0, 2024-09-21T03:44:42.762 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2024-09-21T03:44:42.762 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2024-09-21T03:44:42.762 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 146, 2024-09-21T03:44:42.762 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 16, 2024-09-21T03:44:42.762 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-09-21T03:44:42.762 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2024-09-21T03:44:42.762 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 548653743, 2024-09-21T03:44:42.762 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 1123364734, 2024-09-21T03:44:42.762 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 1123359148, 2024-09-21T03:44:42.763 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-09-21T03:44:42.763 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.763 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.763 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.763 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.763 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 179979 2024-09-21T03:44:42.763 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.763 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-09-21T03:44:42.763 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 1371, 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 31463, 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 30092, 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 140, 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 262, 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 122, 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 31140, 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 32253, 2024-09-21T03:44:42.764 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 1113, 2024-09-21T03:44:42.765 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2024-09-21T03:44:42.765 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 0, 2024-09-21T03:44:42.765 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 0, 2024-09-21T03:44:42.765 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 155444, 2024-09-21T03:44:42.765 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 380018688 2024-09-21T03:44:42.765 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.765 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-09-21T03:44:42.765 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 0, 2024-09-21T03:44:42.766 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2024-09-21T03:44:42.766 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 0, 2024-09-21T03:44:42.766 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 0, 2024-09-21T03:44:42.766 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 0, 2024-09-21T03:44:42.766 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-09-21T03:44:42.766 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.766 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.766 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.767 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.767 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-09-21T03:44:42.767 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.767 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.767 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.767 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.767 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-09-21T03:44:42.767 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.767 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.768 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.769 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.770 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.770 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-09-21T03:44:42.770 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.770 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.770 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.770 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.770 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-09-21T03:44:42.770 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.771 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-09-21T03:44:42.772 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.772 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.772 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.772 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.773 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-09-21T03:44:42.773 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.773 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.773 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.773 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.773 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-09-21T03:44:42.774 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.774 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.774 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.774 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.774 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-09-21T03:44:42.774 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.774 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.775 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.775 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.775 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-09-21T03:44:42.775 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.775 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.775 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.775 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.775 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-09-21T03:44:42.776 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.776 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.776 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.776 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.776 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-09-21T03:44:42.776 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.776 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.776 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.777 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.777 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-09-21T03:44:42.777 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.777 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.777 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.777 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.777 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-09-21T03:44:42.777 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.778 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.778 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.778 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.778 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-09-21T03:44:42.778 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.778 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.778 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.778 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.778 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-09-21T03:44:42.779 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.779 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.779 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.779 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.779 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-09-21T03:44:42.779 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.779 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.779 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.779 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.780 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-09-21T03:44:42.780 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.780 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.780 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.780 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.780 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-09-21T03:44:42.780 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.780 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.781 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.781 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.781 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-09-21T03:44:42.781 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.781 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.781 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.781 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.781 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-09-21T03:44:42.781 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.782 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.782 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.782 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.782 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-09-21T03:44:42.782 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.782 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.782 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.782 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.783 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-09-21T03:44:42.783 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.783 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.783 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.783 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.783 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-09-21T03:44:42.783 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.784 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.784 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.784 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.784 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-09-21T03:44:42.784 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.784 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.784 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.784 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.785 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:44:42.785 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:44:42.785 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-09-21T03:44:42.785 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.785 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.785 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.785 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:42.785 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.785 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-09-21T03:44:42.786 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 2, 2024-09-21T03:44:42.786 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 3, 2024-09-21T03:44:42.786 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 1, 2024-09-21T03:44:42.786 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 2, 2024-09-21T03:44:42.786 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-09-21T03:44:42.786 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 0, 2024-09-21T03:44:42.786 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 0, 2024-09-21T03:44:42.786 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 2230, 2024-09-21T03:44:42.787 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-09-21T03:44:42.787 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.787 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-09-21T03:44:42.787 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:42.787 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:42.787 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:42.787 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:42.787 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:42.788 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:42.788 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:42.788 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:42.788 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:42.788 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:42.788 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:42.788 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:42.788 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:42.789 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:42.789 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:42.789 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:42.789 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:42.789 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:42.789 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:42.789 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:42.789 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:42.789 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:42.790 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:42.790 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:42.790 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:42.790 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:42.790 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:42.790 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:42.790 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:42.790 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:42.791 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:42.791 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-09-21T03:44:42.791 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:42.791 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:42.791 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:42.791 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:42.791 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 300179, 2024-09-21T03:44:42.791 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 1700, 2024-09-21T03:44:42.791 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:44:42.792 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-09-21T03:44:42.792 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-09-21T03:44:42.792 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-09-21T03:44:42.792 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:42.792 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:42.792 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:42.792 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:42.792 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8064, 2024-09-21T03:44:42.793 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 178, 2024-09-21T03:44:42.793 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:42.793 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:42.793 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:42.793 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-09-21T03:44:42.793 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 25031422, 2024-09-21T03:44:42.793 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 161743, 2024-09-21T03:44:42.793 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:42.794 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-09-21T03:44:42.794 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:42.794 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-09-21T03:44:42.794 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.794 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-09-21T03:44:42.794 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-09-21T03:44:42.794 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-09-21T03:44:42.794 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 5326, 2024-09-21T03:44:42.795 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 26, 2024-09-21T03:44:42.795 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 4, 2024-09-21T03:44:42.795 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 5326, 2024-09-21T03:44:42.795 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-09-21T03:44:42.795 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 5326, 2024-09-21T03:44:42.795 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 7.603808608, 2024-09-21T03:44:42.795 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001427677 2024-09-21T03:44:42.795 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.796 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-09-21T03:44:42.796 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-09-21T03:44:42.796 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 5326, 2024-09-21T03:44:42.796 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 5327 2024-09-21T03:44:42.796 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.796 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 5326, 2024-09-21T03:44:42.796 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 5326, 2024-09-21T03:44:42.796 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 0, 2024-09-21T03:44:42.796 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-09-21T03:44:42.797 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-09-21T03:44:42.797 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 3190, 2024-09-21T03:44:42.797 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 0, 2024-09-21T03:44:42.797 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 2133, 2024-09-21T03:44:42.797 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 0, 2024-09-21T03:44:42.797 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 0, 2024-09-21T03:44:42.797 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-09-21T03:44:42.797 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-09-21T03:44:42.797 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2024-09-21T03:44:42.798 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2024-09-21T03:44:42.798 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 0, 2024-09-21T03:44:42.798 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-09-21T03:44:42.798 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-09-21T03:44:42.798 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-09-21T03:44:42.798 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 2, 2024-09-21T03:44:42.798 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 0, 2024-09-21T03:44:42.798 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:44:42.798 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-09-21T03:44:42.799 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:44:42.799 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-09-21T03:44:42.799 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-09-21T03:44:42.799 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-09-21T03:44:42.799 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:44:42.799 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-09-21T03:44:42.799 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:44:42.799 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2024-09-21T03:44:42.800 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-09-21T03:44:42.800 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-09-21T03:44:42.800 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-09-21T03:44:42.800 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-09-21T03:44:42.800 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-09-21T03:44:42.800 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-09-21T03:44:42.800 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-09-21T03:44:42.800 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-09-21T03:44:42.800 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-09-21T03:44:42.801 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-09-21T03:44:42.801 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-09-21T03:44:42.801 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-09-21T03:44:42.801 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-09-21T03:44:42.801 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-09-21T03:44:42.801 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-09-21T03:44:42.801 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-09-21T03:44:42.801 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 96, 2024-09-21T03:44:42.801 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-09-21T03:44:42.801 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 22, 2024-09-21T03:44:42.802 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 11, 2024-09-21T03:44:42.802 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 11, 2024-09-21T03:44:42.802 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-09-21T03:44:42.802 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-09-21T03:44:42.802 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 0, 2024-09-21T03:44:42.802 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2024-09-21T03:44:42.802 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 0 2024-09-21T03:44:42.802 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.803 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-09-21T03:44:42.803 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 0, 2024-09-21T03:44:42.803 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:44:42.803 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-09-21T03:44:42.803 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-09-21T03:44:42.803 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.803 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-09-21T03:44:42.803 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-09-21T03:44:42.803 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:44:42.804 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-09-21T03:44:42.804 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2024-09-21T03:44:42.804 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 0, 2024-09-21T03:44:42.804 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2024-09-21T03:44:42.804 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-09-21T03:44:42.804 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.804 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:42.805 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:42.805 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-21T03:44:42.805 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:42.805 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 5666, 2024-09-21T03:44:42.805 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 1121129374, 2024-09-21T03:44:42.805 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:42.805 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 5666, 2024-09-21T03:44:42.805 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-21T03:44:42.805 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-21T03:44:42.806 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 5666, 2024-09-21T03:44:42.806 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1121129374, 2024-09-21T03:44:42.806 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:42.806 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.806 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.806 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.806 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:42.806 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.806 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-09-21T03:44:42.807 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:42.807 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-21T03:44:42.807 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:42.807 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-21T03:44:42.807 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-21T03:44:42.807 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:42.807 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:42.807 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 5326, 2024-09-21T03:44:42.808 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 1119261743, 2024-09-21T03:44:42.808 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 5326, 2024-09-21T03:44:42.808 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1119261743, 2024-09-21T03:44:42.808 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:42.808 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.808 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.808 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.808 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:42.808 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.809 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-09-21T03:44:42.809 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:42.809 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-09-21T03:44:42.809 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:42.809 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-21T03:44:42.809 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-21T03:44:42.809 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:42.809 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:42.809 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 5326, 2024-09-21T03:44:42.810 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 5326, 2024-09-21T03:44:42.810 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 5326, 2024-09-21T03:44:42.810 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 5326, 2024-09-21T03:44:42.810 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:42.810 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.810 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.810 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.810 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:44:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-21T03:44:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-21T03:44:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-21T03:44:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:42.811 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-21T03:44:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-21T03:44:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-09-21T03:44:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-09-21T03:44:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.812 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:42.813 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.813 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x55f2256832b0": { 2024-09-21T03:44:42.813 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:42.813 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-21T03:44:42.813 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:42.813 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-21T03:44:42.813 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-21T03:44:42.813 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:42.814 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:42.814 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-21T03:44:42.814 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-21T03:44:42.814 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-09-21T03:44:42.814 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-09-21T03:44:42.814 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:42.814 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:42.814 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:42.814 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:42.815 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:42.815 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:42.815 INFO:teuthology.orchestra.run.smithi203.stdout: "trackedop": { 2024-09-21T03:44:42.815 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2024-09-21T03:44:42.815 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:42.815 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-21T03:44:43.460 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T03:44:43.460 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T03:44:43.460 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-21T03:44:43.460 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:44:43.460 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:44:43.461 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T03:44:43.461 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T03:44:43.749 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:44:44.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:43 smithi203 ceph-mon[34723]: pgmap v809: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 4.1 MiB/s wr, 160 op/s 2024-09-21T03:44:44.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:43 smithi112 ceph-mon[31442]: pgmap v809: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 4.1 MiB/s wr, 160 op/s 2024-09-21T03:44:44.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:43 smithi187 ceph-mon[34603]: pgmap v809: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 4.1 MiB/s wr, 160 op/s 2024-09-21T03:44:44.318 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-21T03:44:44.319 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:44.319 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 225415, 2024-09-21T03:44:44.319 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 253686, 2024-09-21T03:44:44.319 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 54083443, 2024-09-21T03:44:44.319 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 471382921, 2024-09-21T03:44:44.319 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 13, 2024-09-21T03:44:44.319 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-09-21T03:44:44.320 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 18.390451536, 2024-09-21T03:44:44.320 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 9.750415105, 2024-09-21T03:44:44.320 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 8.742587594, 2024-09-21T03:44:44.320 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 1.206600366, 2024-09-21T03:44:44.320 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:44.320 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 253685, 2024-09-21T03:44:44.320 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 5.953124094, 2024-09-21T03:44:44.320 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000023466 2024-09-21T03:44:44.321 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.321 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:44.321 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 402342, 2024-09-21T03:44:44.321 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.061169081, 2024-09-21T03:44:44.321 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000152 2024-09-21T03:44:44.321 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.321 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 1474096, 2024-09-21T03:44:44.321 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 375584 2024-09-21T03:44:44.321 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.322 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:44.322 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 389385, 2024-09-21T03:44:44.322 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 537693, 2024-09-21T03:44:44.322 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 137352561, 2024-09-21T03:44:44.322 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 746641263, 2024-09-21T03:44:44.322 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 188, 2024-09-21T03:44:44.322 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 9, 2024-09-21T03:44:44.322 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 35.281108411, 2024-09-21T03:44:44.322 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 19.767099888, 2024-09-21T03:44:44.323 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 18.834437337, 2024-09-21T03:44:44.323 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 1.582025820, 2024-09-21T03:44:44.323 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:44.323 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 537691, 2024-09-21T03:44:44.323 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 21.178734042, 2024-09-21T03:44:44.323 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000039388 2024-09-21T03:44:44.323 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.323 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:44.324 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 769312, 2024-09-21T03:44:44.324 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.138417932, 2024-09-21T03:44:44.324 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000179 2024-09-21T03:44:44.324 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.324 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 123996368, 2024-09-21T03:44:44.324 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 375825792 2024-09-21T03:44:44.324 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.324 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:44.325 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 154277, 2024-09-21T03:44:44.325 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 205988, 2024-09-21T03:44:44.325 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 40502968, 2024-09-21T03:44:44.325 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 443745911, 2024-09-21T03:44:44.325 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 10, 2024-09-21T03:44:44.325 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 6, 2024-09-21T03:44:44.325 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 13.240536966, 2024-09-21T03:44:44.325 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 6.755572957, 2024-09-21T03:44:44.325 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 5.954783460, 2024-09-21T03:44:44.326 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 1.112349433, 2024-09-21T03:44:44.326 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:44.326 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 205987, 2024-09-21T03:44:44.326 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 6.312486361, 2024-09-21T03:44:44.326 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000030645 2024-09-21T03:44:44.326 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.326 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:44.326 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 283024, 2024-09-21T03:44:44.326 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.050817013, 2024-09-21T03:44:44.327 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000179 2024-09-21T03:44:44.327 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.327 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:44:44.327 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:44:44.327 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.327 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-09-21T03:44:44.327 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-09-21T03:44:44.327 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-09-21T03:44:44.327 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.328 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-09-21T03:44:44.328 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-21T03:44:44.328 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-21T03:44:44.328 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 155973, 2024-09-21T03:44:44.328 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 132.819065760, 2024-09-21T03:44:44.328 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000851551 2024-09-21T03:44:44.328 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:44.328 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.328 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-09-21T03:44:44.329 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-21T03:44:44.329 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-21T03:44:44.329 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 49213, 2024-09-21T03:44:44.329 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 7.308886348, 2024-09-21T03:44:44.329 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000148515 2024-09-21T03:44:44.329 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:44.329 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.329 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-09-21T03:44:44.329 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 240737, 2024-09-21T03:44:44.330 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 222118, 2024-09-21T03:44:44.330 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-09-21T03:44:44.330 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 222118, 2024-09-21T03:44:44.330 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 822.795638907, 2024-09-21T03:44:44.330 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.003704317 2024-09-21T03:44:44.330 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.330 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-09-21T03:44:44.330 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 18619, 2024-09-21T03:44:44.330 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 13, 2024-09-21T03:44:44.331 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 2, 2024-09-21T03:44:44.331 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 147, 2024-09-21T03:44:44.331 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 13, 2024-09-21T03:44:44.331 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 7, 2024-09-21T03:44:44.331 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 3022, 2024-09-21T03:44:44.331 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 489, 2024-09-21T03:44:44.331 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 1709, 2024-09-21T03:44:44.331 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 824, 2024-09-21T03:44:44.332 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 2322, 2024-09-21T03:44:44.332 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 1858, 2024-09-21T03:44:44.332 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 238, 2024-09-21T03:44:44.332 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 239, 2024-09-21T03:44:44.332 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 13, 2024-09-21T03:44:44.332 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 367159, 2024-09-21T03:44:44.332 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 240681, 2024-09-21T03:44:44.332 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 18604, 2024-09-21T03:44:44.332 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 26, 2024-09-21T03:44:44.333 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 7, 2024-09-21T03:44:44.333 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2024-09-21T03:44:44.333 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 1563, 2024-09-21T03:44:44.333 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 17316, 2024-09-21T03:44:44.333 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-09-21T03:44:44.333 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 11, 2024-09-21T03:44:44.333 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 1206, 2024-09-21T03:44:44.333 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 20, 2024-09-21T03:44:44.334 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 595, 2024-09-21T03:44:44.334 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-09-21T03:44:44.334 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 2287, 2024-09-21T03:44:44.334 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2024-09-21T03:44:44.334 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 723, 2024-09-21T03:44:44.334 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 214756952, 2024-09-21T03:44:44.334 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 6, 2024-09-21T03:44:44.334 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 2287, 2024-09-21T03:44:44.334 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-09-21T03:44:44.335 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:44:44.335 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:44:44.335 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 598, 2024-09-21T03:44:44.335 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 23, 2024-09-21T03:44:44.335 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 575, 2024-09-21T03:44:44.335 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 1689, 2024-09-21T03:44:44.335 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 9, 2024-09-21T03:44:44.335 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 42496, 2024-09-21T03:44:44.335 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 116414, 2024-09-21T03:44:44.336 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:44:44.336 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:44:44.336 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:44:44.336 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 85886, 2024-09-21T03:44:44.336 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 25020, 2024-09-21T03:44:44.336 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 13273, 2024-09-21T03:44:44.336 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 122568 2024-09-21T03:44:44.336 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.337 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-09-21T03:44:44.337 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 31, 2024-09-21T03:44:44.337 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 27, 2024-09-21T03:44:44.337 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 18, 2024-09-21T03:44:44.337 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 35, 2024-09-21T03:44:44.337 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 278, 2024-09-21T03:44:44.337 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 1, 2024-09-21T03:44:44.337 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-09-21T03:44:44.337 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:44:44.338 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 29494, 2024-09-21T03:44:44.338 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 29493, 2024-09-21T03:44:44.338 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2024-09-21T03:44:44.338 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2024-09-21T03:44:44.338 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-09-21T03:44:44.338 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:44:44.338 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:44:44.338 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-09-21T03:44:44.339 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-09-21T03:44:44.339 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:44:44.339 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:44:44.339 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 23, 2024-09-21T03:44:44.339 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 11, 2024-09-21T03:44:44.339 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-09-21T03:44:44.339 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 20, 2024-09-21T03:44:44.339 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-09-21T03:44:44.339 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0, 2024-09-21T03:44:44.340 INFO:teuthology.orchestra.run.smithi203.stdout: "uninline_started": 0, 2024-09-21T03:44:44.340 INFO:teuthology.orchestra.run.smithi203.stdout: "uninline_succeeded": 0, 2024-09-21T03:44:44.340 INFO:teuthology.orchestra.run.smithi203.stdout: "uninline_write_failed": 0 2024-09-21T03:44:44.340 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.340 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-09-21T03:44:44.340 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2024-09-21T03:44:44.340 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 179987, 2024-09-21T03:44:44.340 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 106566, 2024-09-21T03:44:44.341 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 98321, 2024-09-21T03:44:44.341 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 81666, 2024-09-21T03:44:44.341 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-09-21T03:44:44.341 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 8245, 2024-09-21T03:44:44.341 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 303, 2024-09-21T03:44:44.341 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 175, 2024-09-21T03:44:44.341 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 157, 2024-09-21T03:44:44.341 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 146, 2024-09-21T03:44:44.341 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 7, 2024-09-21T03:44:44.342 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-09-21T03:44:44.342 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 18, 2024-09-21T03:44:44.342 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 548653743, 2024-09-21T03:44:44.342 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 1123537019, 2024-09-21T03:44:44.342 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 0, 2024-09-21T03:44:44.342 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-09-21T03:44:44.342 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 80830, 2024-09-21T03:44:44.342 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 346.832177504, 2024-09-21T03:44:44.342 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004290884 2024-09-21T03:44:44.343 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.343 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 0 2024-09-21T03:44:44.343 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.343 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-09-21T03:44:44.343 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 1236, 2024-09-21T03:44:44.343 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 31566, 2024-09-21T03:44:44.343 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 30330, 2024-09-21T03:44:44.344 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 190, 2024-09-21T03:44:44.344 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 320, 2024-09-21T03:44:44.344 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 130, 2024-09-21T03:44:44.344 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 3022, 2024-09-21T03:44:44.344 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 58950, 2024-09-21T03:44:44.344 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 55928, 2024-09-21T03:44:44.344 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 239, 2024-09-21T03:44:44.344 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 31422, 2024-09-21T03:44:44.344 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 31183, 2024-09-21T03:44:44.345 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 158752, 2024-09-21T03:44:44.345 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 375271424 2024-09-21T03:44:44.345 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.345 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-09-21T03:44:44.345 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 327851, 2024-09-21T03:44:44.345 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 36647, 2024-09-21T03:44:44.345 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 240737, 2024-09-21T03:44:44.345 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 5154, 2024-09-21T03:44:44.346 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 74021, 2024-09-21T03:44:44.346 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-09-21T03:44:44.346 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 30176, 2024-09-21T03:44:44.346 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 85.528346011, 2024-09-21T03:44:44.346 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002834316 2024-09-21T03:44:44.346 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.347 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-09-21T03:44:44.347 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 37971, 2024-09-21T03:44:44.347 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 263.535996909, 2024-09-21T03:44:44.347 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.006940454 2024-09-21T03:44:44.347 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.347 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-09-21T03:44:44.347 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.347 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.348 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.348 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.348 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-09-21T03:44:44.348 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.348 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.348 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.348 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.348 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-09-21T03:44:44.348 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 63004, 2024-09-21T03:44:44.349 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 102.373895386, 2024-09-21T03:44:44.349 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001624879 2024-09-21T03:44:44.349 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.349 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-09-21T03:44:44.349 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.349 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.349 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.349 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.350 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-09-21T03:44:44.350 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2, 2024-09-21T03:44:44.350 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000294738, 2024-09-21T03:44:44.350 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000147369 2024-09-21T03:44:44.350 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.350 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-09-21T03:44:44.350 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.350 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.351 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.351 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.351 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-09-21T03:44:44.352 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.352 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.352 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.352 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.352 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-09-21T03:44:44.352 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.352 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.353 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.353 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.353 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-09-21T03:44:44.353 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 16, 2024-09-21T03:44:44.353 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.005037940, 2024-09-21T03:44:44.353 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000314871 2024-09-21T03:44:44.353 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.353 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-09-21T03:44:44.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 119, 2024-09-21T03:44:44.354 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.502063900, 2024-09-21T03:44:44.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004219024 2024-09-21T03:44:44.354 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.354 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-09-21T03:44:44.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.354 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.355 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.355 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-09-21T03:44:44.355 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 16, 2024-09-21T03:44:44.355 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.366688735, 2024-09-21T03:44:44.355 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.147918045 2024-09-21T03:44:44.355 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.355 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-09-21T03:44:44.355 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 10, 2024-09-21T03:44:44.355 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.015387738, 2024-09-21T03:44:44.356 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001538773 2024-09-21T03:44:44.356 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.356 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-09-21T03:44:44.356 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 50510, 2024-09-21T03:44:44.356 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 252.537271728, 2024-09-21T03:44:44.356 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004999748 2024-09-21T03:44:44.356 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.356 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-09-21T03:44:44.356 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 8895, 2024-09-21T03:44:44.357 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 26.009385339, 2024-09-21T03:44:44.357 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002924045 2024-09-21T03:44:44.357 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.357 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-09-21T03:44:44.357 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.357 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.357 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.357 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.358 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-09-21T03:44:44.358 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.358 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.358 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.358 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.358 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-09-21T03:44:44.358 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1, 2024-09-21T03:44:44.358 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.001908009, 2024-09-21T03:44:44.358 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001908009 2024-09-21T03:44:44.359 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.359 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-09-21T03:44:44.359 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 10, 2024-09-21T03:44:44.359 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.762221739, 2024-09-21T03:44:44.359 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.176222173 2024-09-21T03:44:44.359 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.359 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-09-21T03:44:44.359 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.359 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.360 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.360 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.360 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-09-21T03:44:44.360 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 15, 2024-09-21T03:44:44.360 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.353262744, 2024-09-21T03:44:44.360 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.023550849 2024-09-21T03:44:44.360 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.360 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-09-21T03:44:44.361 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.361 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.361 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.361 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.361 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-09-21T03:44:44.361 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1870, 2024-09-21T03:44:44.361 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.027978418, 2024-09-21T03:44:44.361 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001084480 2024-09-21T03:44:44.362 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.362 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-09-21T03:44:44.362 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.362 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.362 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.362 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.362 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-09-21T03:44:44.362 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 9, 2024-09-21T03:44:44.363 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.021380029, 2024-09-21T03:44:44.363 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002375558 2024-09-21T03:44:44.363 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.363 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-09-21T03:44:44.363 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.363 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.363 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.363 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.363 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-09-21T03:44:44.364 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 29494, 2024-09-21T03:44:44.364 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 85.754519544, 2024-09-21T03:44:44.364 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002907524 2024-09-21T03:44:44.364 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.364 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:44:44.364 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:44:44.364 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-09-21T03:44:44.364 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.365 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.365 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.365 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:44.365 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.365 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-09-21T03:44:44.365 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 2, 2024-09-21T03:44:44.365 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 3, 2024-09-21T03:44:44.365 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 1, 2024-09-21T03:44:44.366 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 2, 2024-09-21T03:44:44.366 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-09-21T03:44:44.366 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 36342, 2024-09-21T03:44:44.366 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 18171, 2024-09-21T03:44:44.366 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 951, 2024-09-21T03:44:44.366 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-09-21T03:44:44.366 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.366 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-09-21T03:44:44.366 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:44.367 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:44.367 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:44.367 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:44.367 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:44.367 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:44.367 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:44.367 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:44.367 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:44.368 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:44.368 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:44.368 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:44.368 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:44.368 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:44.368 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:44.368 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:44.368 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:44.368 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:44.369 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:44.369 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:44.369 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:44.369 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:44.369 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:44.369 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:44.369 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:44.369 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:44.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:44.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:44.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:44.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:44.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:44.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-09-21T03:44:44.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:44.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:44.371 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:44.371 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:44.371 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 368197, 2024-09-21T03:44:44.371 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 1055, 2024-09-21T03:44:44.371 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 264, 2024-09-21T03:44:44.371 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 3, 2024-09-21T03:44:44.371 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-09-21T03:44:44.371 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-09-21T03:44:44.371 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:44.372 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:44.372 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:44.372 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:44.372 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8064, 2024-09-21T03:44:44.372 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 178, 2024-09-21T03:44:44.372 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:44.372 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:44.372 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:44.372 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-09-21T03:44:44.373 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 5925230, 2024-09-21T03:44:44.373 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 60926, 2024-09-21T03:44:44.373 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:44.373 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-09-21T03:44:44.373 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:44.373 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-09-21T03:44:44.373 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.373 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-09-21T03:44:44.374 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-09-21T03:44:44.374 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-09-21T03:44:44.374 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 116899, 2024-09-21T03:44:44.374 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 1127111828, 2024-09-21T03:44:44.374 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 37, 2024-09-21T03:44:44.374 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 116899, 2024-09-21T03:44:44.374 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-09-21T03:44:44.374 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 116899, 2024-09-21T03:44:44.374 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 515.940128675, 2024-09-21T03:44:44.375 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004413554 2024-09-21T03:44:44.375 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.375 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-09-21T03:44:44.375 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-09-21T03:44:44.375 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 116899, 2024-09-21T03:44:44.375 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 118509 2024-09-21T03:44:44.375 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.375 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 116899, 2024-09-21T03:44:44.375 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 3232, 2024-09-21T03:44:44.376 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 113667, 2024-09-21T03:44:44.376 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-09-21T03:44:44.376 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-09-21T03:44:44.376 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 8, 2024-09-21T03:44:44.376 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 47, 2024-09-21T03:44:44.376 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 885, 2024-09-21T03:44:44.376 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 81692, 2024-09-21T03:44:44.376 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 462, 2024-09-21T03:44:44.376 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-09-21T03:44:44.377 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-09-21T03:44:44.377 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2024-09-21T03:44:44.377 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 13, 2024-09-21T03:44:44.377 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 30437, 2024-09-21T03:44:44.377 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-09-21T03:44:44.377 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-09-21T03:44:44.377 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-09-21T03:44:44.377 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 2325, 2024-09-21T03:44:44.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 94, 2024-09-21T03:44:44.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:44:44.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-09-21T03:44:44.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:44:44.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-09-21T03:44:44.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-09-21T03:44:44.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-09-21T03:44:44.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:44:44.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-09-21T03:44:44.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:44:44.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2024-09-21T03:44:44.379 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-09-21T03:44:44.379 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-09-21T03:44:44.379 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-09-21T03:44:44.379 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-09-21T03:44:44.379 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-09-21T03:44:44.380 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-09-21T03:44:44.380 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-09-21T03:44:44.380 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-09-21T03:44:44.380 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-09-21T03:44:44.380 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-09-21T03:44:44.380 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-09-21T03:44:44.380 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-09-21T03:44:44.380 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-09-21T03:44:44.380 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-09-21T03:44:44.381 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-09-21T03:44:44.381 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-09-21T03:44:44.381 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 96, 2024-09-21T03:44:44.381 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-09-21T03:44:44.381 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 22, 2024-09-21T03:44:44.381 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 12, 2024-09-21T03:44:44.381 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 12, 2024-09-21T03:44:44.381 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-09-21T03:44:44.382 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-09-21T03:44:44.382 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 1994, 2024-09-21T03:44:44.382 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 52, 2024-09-21T03:44:44.382 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 500 2024-09-21T03:44:44.382 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.382 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-09-21T03:44:44.382 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 1, 2024-09-21T03:44:44.382 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:44:44.382 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-09-21T03:44:44.383 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-09-21T03:44:44.383 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.383 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-09-21T03:44:44.383 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-09-21T03:44:44.383 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 10, 2024-09-21T03:44:44.383 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-09-21T03:44:44.383 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 10, 2024-09-21T03:44:44.383 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 29493, 2024-09-21T03:44:44.384 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 29493, 2024-09-21T03:44:44.384 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-09-21T03:44:44.384 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.384 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:44.384 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:44.384 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-21T03:44:44.384 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:44.384 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 769073, 2024-09-21T03:44:44.384 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 146770896, 2024-09-21T03:44:44.385 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:44.385 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 769073, 2024-09-21T03:44:44.385 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-21T03:44:44.385 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-21T03:44:44.385 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 769073, 2024-09-21T03:44:44.385 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 146770896, 2024-09-21T03:44:44.385 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:44.385 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.386 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.386 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.386 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:44.386 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.386 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-09-21T03:44:44.386 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:44.386 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-21T03:44:44.386 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:44.387 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-21T03:44:44.387 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-21T03:44:44.387 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:44.387 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:44.387 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 116899, 2024-09-21T03:44:44.387 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 1130093537, 2024-09-21T03:44:44.387 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 116899, 2024-09-21T03:44:44.387 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1130093537, 2024-09-21T03:44:44.387 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:44.387 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.388 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.388 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.388 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:44.388 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.388 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-09-21T03:44:44.388 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:44.388 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-09-21T03:44:44.388 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:44.389 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-21T03:44:44.389 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-21T03:44:44.389 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:44.389 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-21T03:44:44.389 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 116899, 2024-09-21T03:44:44.389 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 116899, 2024-09-21T03:44:44.389 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 116899, 2024-09-21T03:44:44.389 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 116899, 2024-09-21T03:44:44.389 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:44.390 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.390 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.390 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.390 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:44.390 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.390 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:44:44.390 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:44.390 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-21T03:44:44.391 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:44.391 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 29493, 2024-09-21T03:44:44.391 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 2982569, 2024-09-21T03:44:44.391 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:44.391 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 29493, 2024-09-21T03:44:44.391 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-21T03:44:44.391 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-21T03:44:44.391 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 862, 2024-09-21T03:44:44.392 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 2982569, 2024-09-21T03:44:44.392 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:44.392 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.392 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.392 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.392 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:44.392 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.392 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x55df18395450": { 2024-09-21T03:44:44.392 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 574, 2024-09-21T03:44:44.392 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-21T03:44:44.393 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:44.393 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 179987, 2024-09-21T03:44:44.393 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 1119342715, 2024-09-21T03:44:44.393 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:44.393 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 179987, 2024-09-21T03:44:44.393 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-21T03:44:44.393 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-21T03:44:44.393 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 80830, 2024-09-21T03:44:44.394 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1119342141, 2024-09-21T03:44:44.394 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:44.394 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:44.394 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:44.394 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:44.394 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:44.394 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:44.394 INFO:teuthology.orchestra.run.smithi203.stdout: "trackedop": { 2024-09-21T03:44:44.394 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2024-09-21T03:44:44.395 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:44.395 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-21T03:44:45.111 INFO:tasks.check_counter:Daemon mds.i mds.exported=11 2024-09-21T03:44:45.111 INFO:tasks.check_counter:Daemon mds.i mds.imported=20 2024-09-21T03:44:45.111 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-21T03:44:45.111 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=16 2024-09-21T03:44:45.112 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=10 2024-09-21T03:44:45.112 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T03:44:45.112 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T03:44:45.359 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:44:45.744 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:45 smithi203 ceph-mon[34723]: pgmap v810: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 3.3 MiB/s wr, 129 op/s 2024-09-21T03:44:45.820 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-21T03:44:45.820 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:44:45.821 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 300, 2024-09-21T03:44:45.821 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 295, 2024-09-21T03:44:45.821 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 860704, 2024-09-21T03:44:45.821 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 181680, 2024-09-21T03:44:45.821 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:45.821 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 0, 2024-09-21T03:44:45.821 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.043578883, 2024-09-21T03:44:45.821 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.019152809, 2024-09-21T03:44:45.822 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.032299102, 2024-09-21T03:44:45.822 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:45.822 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:45.822 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 294, 2024-09-21T03:44:45.822 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.011034718, 2024-09-21T03:44:45.822 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000037533 2024-09-21T03:44:45.822 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:45.822 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:45.823 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:45.823 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:45.823 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:45.823 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:45.823 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 860704, 2024-09-21T03:44:45.823 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 181680 2024-09-21T03:44:45.823 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:45.823 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:44:45.823 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 2, 2024-09-21T03:44:45.824 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 225, 2024-09-21T03:44:45.824 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 320, 2024-09-21T03:44:45.824 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 200336, 2024-09-21T03:44:45.824 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2024-09-21T03:44:45.824 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 2, 2024-09-21T03:44:45.824 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.014405425, 2024-09-21T03:44:45.824 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.011254089, 2024-09-21T03:44:45.824 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.001242912, 2024-09-21T03:44:45.824 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:45.825 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:45.825 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 223, 2024-09-21T03:44:45.825 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.009372076, 2024-09-21T03:44:45.825 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000042027 2024-09-21T03:44:45.825 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:45.825 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:45.825 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:45.825 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:45.825 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:45.826 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:45.826 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T03:44:45.826 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 200336 2024-09-21T03:44:45.826 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:45.826 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:44:45.826 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 4, 2024-09-21T03:44:45.826 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 4, 2024-09-21T03:44:45.826 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 2944, 2024-09-21T03:44:45.827 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 448, 2024-09-21T03:44:45.827 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 2, 2024-09-21T03:44:45.827 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 1, 2024-09-21T03:44:45.827 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.002183072, 2024-09-21T03:44:45.827 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.000044760, 2024-09-21T03:44:45.827 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.002122969, 2024-09-21T03:44:45.827 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:44:45.827 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:44:45.827 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 3, 2024-09-21T03:44:45.828 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.003824536, 2024-09-21T03:44:45.828 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001274845 2024-09-21T03:44:45.828 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:45.828 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:44:45.828 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:45.828 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:45.828 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:45.828 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:45.828 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:44:45.829 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:44:45.829 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:45.829 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-09-21T03:44:45.829 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 0, 2024-09-21T03:44:45.829 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-09-21T03:44:45.829 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:45.829 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-09-21T03:44:45.829 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:44:45.829 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-09-21T03:44:45.830 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:44:45.830 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:44:45.830 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:44:45.830 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:44:45.830 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:44:45.830 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:44:45.830 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:44:45.830 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-09-21T03:44:45.831 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-09-21T03:44:45.832 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:44:45.833 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:44:45.833 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:44:45.833 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-09-21T03:44:45.833 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-09-21T03:44:45.833 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-09-21T03:44:45.833 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:44:45.833 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:44:45.833 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:44:45.833 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 148704, 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 29, 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:44:45.834 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 0, 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 0, 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-09-21T03:44:45.835 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-09-21T03:44:45.836 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 968, 2024-09-21T03:44:45.836 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 2, 2024-09-21T03:44:45.836 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-09-21T03:44:45.836 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-09-21T03:44:45.836 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-09-21T03:44:45.836 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-09-21T03:44:45.836 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-21T03:44:45.836 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:44:45.836 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-21T03:44:45.837 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-21T03:44:45.837 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-21T03:44:45.837 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 302, 2024-09-21T03:44:45.837 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 818309, 2024-09-21T03:44:45.837 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-21T03:44:45.837 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 302, 2024-09-21T03:44:45.837 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-21T03:44:45.837 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-21T03:44:45.838 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 302, 2024-09-21T03:44:45.838 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 818309, 2024-09-21T03:44:45.838 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-21T03:44:45.838 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-21T03:44:45.838 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-21T03:44:45.838 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-21T03:44:45.838 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:45.838 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-21T03:44:45.839 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-21T03:44:46.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:45 smithi112 ceph-mon[31442]: pgmap v810: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 3.3 MiB/s wr, 129 op/s 2024-09-21T03:44:46.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:45 smithi187 ceph-mon[34603]: pgmap v810: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 3.3 MiB/s wr, 129 op/s 2024-09-21T03:44:46.528 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-21T03:44:46.528 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-21T03:44:46.528 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-21T03:44:46.528 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-21T03:44:46.528 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-21T03:44:46.529 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T03:44:46.539 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T03:44:46.539 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:44:46.539 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:46.561 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T03:44:46.561 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:44:46.619 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:44:46.920 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.443+0000 7f29aa548640 1 -- 172.21.15.112:0/487471968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a4107be0 msgr2=0x7f29a4107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.443+0000 7f29aa548640 1 --2- 172.21.15.112:0/487471968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a4107be0 0x7f29a4107ff0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f298c009900 tx=0x7f298c02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.443+0000 7f29aa548640 1 -- 172.21.15.112:0/487471968 shutdown_connections 2024-09-21T03:44:47.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.443+0000 7f29aa548640 1 --2- 172.21.15.112:0/487471968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29a4108530 0x7f29a410ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.443+0000 7f29aa548640 1 --2- 172.21.15.112:0/487471968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a4107be0 0x7f29a4107ff0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.443+0000 7f29aa548640 1 --2- 172.21.15.112:0/487471968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29a41022a0 0x7f29a41026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.443+0000 7f29aa548640 1 -- 172.21.15.112:0/487471968 >> 172.21.15.112:0/487471968 conn(0x7f29a4079ec0 msgr2=0x7f29a407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.443+0000 7f29aa548640 1 -- 172.21.15.112:0/487471968 shutdown_connections 2024-09-21T03:44:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.443+0000 7f29aa548640 1 -- 172.21.15.112:0/487471968 wait complete. 2024-09-21T03:44:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.444+0000 7f29aa548640 1 Processor -- start 2024-09-21T03:44:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.444+0000 7f29aa548640 1 -- start start 2024-09-21T03:44:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29aa548640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29a41022a0 0x7f29a419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29aa548640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29a4107be0 0x7f29a419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29aa548640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a4108530 0x7f29a419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29aa548640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4113ba0 con 0x7f29a4108530 2024-09-21T03:44:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29aa548640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4113a20 con 0x7f29a4107be0 2024-09-21T03:44:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29aa548640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4113d20 con 0x7f29a41022a0 2024-09-21T03:44:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a8ae5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a4108530 0x7f29a419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a8ae5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a4108530 0x7f29a419fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49160/0 (socket says 172.21.15.112:49160) 2024-09-21T03:44:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a8ae5640 1 -- 172.21.15.112:0/4001218199 learned_addr learned my addr 172.21.15.112:0/4001218199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:47.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a3fff640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29a41022a0 0x7f29a419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a37fe640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29a4107be0 0x7f29a419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a8ae5640 1 -- 172.21.15.112:0/4001218199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29a41022a0 msgr2=0x7f29a419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a8ae5640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29a41022a0 0x7f29a419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a8ae5640 1 -- 172.21.15.112:0/4001218199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29a4107be0 msgr2=0x7f29a419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a8ae5640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29a4107be0 0x7f29a419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a8ae5640 1 -- 172.21.15.112:0/4001218199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a41a04c0 con 0x7f29a4108530 2024-09-21T03:44:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a37fe640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29a4107be0 0x7f29a419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:47.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a8ae5640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a4108530 0x7f29a419fd90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f299c00b970 tx=0x7f299c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a17fa640 1 -- 172.21.15.112:0/4001218199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c019070 con 0x7f29a4108530 2024-09-21T03:44:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a17fa640 1 -- 172.21.15.112:0/4001218199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f299c014440 con 0x7f29a4108530 2024-09-21T03:44:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29aa548640 1 -- 172.21.15.112:0/4001218199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a41a07b0 con 0x7f29a4108530 2024-09-21T03:44:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.445+0000 7f29a17fa640 1 -- 172.21.15.112:0/4001218199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c0075d0 con 0x7f29a4108530 2024-09-21T03:44:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.446+0000 7f29aa548640 1 -- 172.21.15.112:0/4001218199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a41a6f10 con 0x7f29a4108530 2024-09-21T03:44:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.447+0000 7f29aa548640 1 -- 172.21.15.112:0/4001218199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a41026b0 con 0x7f29a4108530 2024-09-21T03:44:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.448+0000 7f29a17fa640 1 -- 172.21.15.112:0/4001218199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f299c01c070 con 0x7f29a4108530 2024-09-21T03:44:47.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.449+0000 7f29a17fa640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f29900b6700 0x7f29900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.449+0000 7f29a3fff640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f29900b6700 0x7f29900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.449+0000 7f29a17fa640 1 -- 172.21.15.112:0/4001218199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f299c0e7070 con 0x7f29a4108530 2024-09-21T03:44:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.449+0000 7f29a3fff640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f29900b6700 0x7f29900b8c10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2994009bd0 tx=0x7f2994005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:47.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.452+0000 7f29a17fa640 1 -- 172.21.15.112:0/4001218199 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f299c0b1970 con 0x7f29a4108530 2024-09-21T03:44:47.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.643+0000 7f29aa548640 1 -- 172.21.15.112:0/4001218199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f29a419b680 con 0x7f29a4108530 2024-09-21T03:44:47.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.644+0000 7f29a17fa640 1 -- 172.21.15.112:0/4001218199 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v96) ==== 81+0+899 (secure 0 0 0) 0x7f29a419b680 con 0x7f29a4108530 2024-09-21T03:44:47.648 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/4206336392 2024-09-22T03:20:32.227264+0000 2024-09-21T03:44:47.648 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/839399771 2024-09-22T03:19:06.754886+0000 2024-09-21T03:44:47.648 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1495858515 2024-09-22T03:19:06.754886+0000 2024-09-21T03:44:47.649 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2996077964 2024-09-22T03:19:31.820776+0000 2024-09-21T03:44:47.649 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3545481828 2024-09-22T03:19:06.754886+0000 2024-09-21T03:44:47.649 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/3643082307 2024-09-22T03:20:32.227264+0000 2024-09-21T03:44:47.649 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/839399771 2024-09-22T03:19:06.754886+0000 2024-09-21T03:44:47.649 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1177659103 2024-09-22T03:19:06.754886+0000 2024-09-21T03:44:47.650 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3522404182 2024-09-22T03:19:31.820776+0000 2024-09-21T03:44:47.650 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/914655252 2024-09-22T03:19:31.820776+0000 2024-09-21T03:44:47.650 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/3643082307 2024-09-22T03:20:32.227264+0000 2024-09-21T03:44:47.650 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1049357498 2024-09-22T03:20:32.227264+0000 2024-09-21T03:44:47.651 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2996077964 2024-09-22T03:19:31.820776+0000 2024-09-21T03:44:47.651 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/597633710 2024-09-22T03:19:31.820776+0000 2024-09-21T03:44:47.651 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2169183307 2024-09-22T03:20:32.227264+0000 2024-09-21T03:44:47.652 INFO:teuthology.orchestra.run.smithi112.stderr:listed 15 entries 2024-09-21T03:44:47.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 -- 172.21.15.112:0/4001218199 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f29900b6700 msgr2=0x7f29900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f29900b6700 0x7f29900b8c10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2994009bd0 tx=0x7f2994005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:47.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 -- 172.21.15.112:0/4001218199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a4108530 msgr2=0x7f29a419fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:47.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a4108530 0x7f29a419fd90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f299c00b970 tx=0x7f299c00be30 comp rx=0 tx=0).stop 2024-09-21T03:44:47.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 -- 172.21.15.112:0/4001218199 shutdown_connections 2024-09-21T03:44:47.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f29900b6700 0x7f29900b8c10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a4108530 0x7f29a419fd90 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29a4107be0 0x7f29a419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 --2- 172.21.15.112:0/4001218199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29a41022a0 0x7f29a419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 -- 172.21.15.112:0/4001218199 >> 172.21.15.112:0/4001218199 conn(0x7f29a4079ec0 msgr2=0x7f29a4105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 -- 172.21.15.112:0/4001218199 shutdown_connections 2024-09-21T03:44:47.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:47.647+0000 7f29aa548640 1 -- 172.21.15.112:0/4001218199 wait complete. 2024-09-21T03:44:48.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:47 smithi112 ceph-mon[31442]: pgmap v811: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.0 MiB/s wr, 122 op/s 2024-09-21T03:44:48.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:47 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/4001218199' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:44:48.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:47 smithi203 ceph-mon[34723]: pgmap v811: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.0 MiB/s wr, 122 op/s 2024-09-21T03:44:48.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:47 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/4001218199' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:44:48.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:47 smithi187 ceph-mon[34603]: pgmap v811: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.0 MiB/s wr, 122 op/s 2024-09-21T03:44:48.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:47 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/4001218199' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:44:48.437 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:44:48.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:48.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:44:48.614 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T03:44:48.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:48.615 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:44:48.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:48.671 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T03:44:48.709 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:44:48.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:48.710 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:44:48.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:44:48.812 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-09-21T03:44:48.883 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T03:44:48.899 INFO:tasks.cephadm:Teardown begin 2024-09-21T03:44:48.900 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:44:48.900 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:44:49.150 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:49.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.769+0000 7f71e1020640 1 -- 172.21.15.112:0/2409596813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc103b60 msgr2=0x7f71dc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.769+0000 7f71e1020640 1 --2- 172.21.15.112:0/2409596813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc103b60 0x7f71dc103f70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009900 tx=0x7f71cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:44:49.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.769+0000 7f71e1020640 1 -- 172.21.15.112:0/2409596813 shutdown_connections 2024-09-21T03:44:49.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.769+0000 7f71e1020640 1 --2- 172.21.15.112:0/2409596813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc109df0 0x7f71dc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.769+0000 7f71e1020640 1 --2- 172.21.15.112:0/2409596813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc1094a0 0x7f71dc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.769+0000 7f71e1020640 1 --2- 172.21.15.112:0/2409596813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc103b60 0x7f71dc103f70 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.769+0000 7f71e1020640 1 -- 172.21.15.112:0/2409596813 >> 172.21.15.112:0/2409596813 conn(0x7f71dc07b850 msgr2=0x7f71dc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:49.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.769+0000 7f71e1020640 1 -- 172.21.15.112:0/2409596813 shutdown_connections 2024-09-21T03:44:49.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.769+0000 7f71e1020640 1 -- 172.21.15.112:0/2409596813 wait complete. 2024-09-21T03:44:49.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.770+0000 7f71e1020640 1 Processor -- start 2024-09-21T03:44:49.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.770+0000 7f71e1020640 1 -- start start 2024-09-21T03:44:49.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.770+0000 7f71e1020640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc103b60 0x7f71dc19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.770+0000 7f71e1020640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1094a0 0x7f71dc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.770+0000 7f71e1020640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc109df0 0x7f71dc1a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.770+0000 7f71e1020640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc113ab0 con 0x7f71dc109df0 2024-09-21T03:44:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.770+0000 7f71e1020640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc113930 con 0x7f71dc1094a0 2024-09-21T03:44:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.770+0000 7f71e1020640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc113c30 con 0x7f71dc103b60 2024-09-21T03:44:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71db59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc109df0 0x7f71dc1a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71da59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1094a0 0x7f71dc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71db59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc109df0 0x7f71dc1a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49188/0 (socket says 172.21.15.112:49188) 2024-09-21T03:44:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71db59e640 1 -- 172.21.15.112:0/1678780563 learned_addr learned my addr 172.21.15.112:0/1678780563 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71da59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1094a0 0x7f71dc19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:54122/0 (socket says 172.21.15.112:54122) 2024-09-21T03:44:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71dad9d640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc103b60 0x7f71dc19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71db59e640 1 -- 172.21.15.112:0/1678780563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc103b60 msgr2=0x7f71dc19c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71db59e640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc103b60 0x7f71dc19c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71db59e640 1 -- 172.21.15.112:0/1678780563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1094a0 msgr2=0x7f71dc19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71db59e640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1094a0 0x7f71dc19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71db59e640 1 -- 172.21.15.112:0/1678780563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc1a26b0 con 0x7f71dc109df0 2024-09-21T03:44:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71da59c640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1094a0 0x7f71dc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:44:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71db59e640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc109df0 0x7f71dc1a1f80 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f71d400eb10 tx=0x7f71d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71c3fff640 1 -- 172.21.15.112:0/1678780563 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4004b20 con 0x7f71dc109df0 2024-09-21T03:44:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71e1020640 1 -- 172.21.15.112:0/1678780563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc1a29a0 con 0x7f71dc109df0 2024-09-21T03:44:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71c3fff640 1 -- 172.21.15.112:0/1678780563 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d4004cc0 con 0x7f71dc109df0 2024-09-21T03:44:49.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71c3fff640 1 -- 172.21.15.112:0/1678780563 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4010670 con 0x7f71dc109df0 2024-09-21T03:44:49.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.771+0000 7f71e1020640 1 -- 172.21.15.112:0/1678780563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc1a9100 con 0x7f71dc109df0 2024-09-21T03:44:49.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.772+0000 7f71e1020640 1 -- 172.21.15.112:0/1678780563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71dc103f70 con 0x7f71dc109df0 2024-09-21T03:44:49.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.774+0000 7f71c3fff640 1 -- 172.21.15.112:0/1678780563 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f71d4005000 con 0x7f71dc109df0 2024-09-21T03:44:49.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:49 smithi112 ceph-mon[31442]: pgmap v812: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.7 MiB/s wr, 60 op/s 2024-09-21T03:44:49.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.775+0000 7f71c3fff640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71c40b6680 0x7f71c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:49.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.775+0000 7f71dad9d640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71c40b6680 0x7f71c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:49.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.775+0000 7f71c3fff640 1 -- 172.21.15.112:0/1678780563 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f71d40e7e00 con 0x7f71dc109df0 2024-09-21T03:44:49.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.775+0000 7f71dad9d640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71c40b6680 0x7f71c40b8b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f71cc031440 tx=0x7f71cc009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:49.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.778+0000 7f71c3fff640 1 -- 172.21.15.112:0/1678780563 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f71d4056710 con 0x7f71dc109df0 2024-09-21T03:44:49.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.994+0000 7f71e1020640 1 -- 172.21.15.112:0/1678780563 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71dc19d870 con 0x7f71dc109df0 2024-09-21T03:44:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.995+0000 7f71c3fff640 1 -- 172.21.15.112:0/1678780563 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11934 (secure 0 0 0) 0x7f71dc19d870 con 0x7f71dc109df0 2024-09-21T03:44:49.997 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:44:49.997 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":36,"btime":"2024-09-21T03:44:22:526380+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:44:22.336872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:44:49.998 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 36 2024-09-21T03:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 -- 172.21.15.112:0/1678780563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71c40b6680 msgr2=0x7f71c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71c40b6680 0x7f71c40b8b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f71cc031440 tx=0x7f71cc009560 comp rx=0 tx=0).stop 2024-09-21T03:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 -- 172.21.15.112:0/1678780563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc109df0 msgr2=0x7f71dc1a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:49.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc109df0 0x7f71dc1a1f80 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f71d400eb10 tx=0x7f71d400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 -- 172.21.15.112:0/1678780563 shutdown_connections 2024-09-21T03:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f71c40b6680 0x7f71c40b8b90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc109df0 0x7f71dc1a1f80 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1094a0 0x7f71dc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 --2- 172.21.15.112:0/1678780563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc103b60 0x7f71dc19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 -- 172.21.15.112:0/1678780563 >> 172.21.15.112:0/1678780563 conn(0x7f71dc07b850 msgr2=0x7f71dc107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 -- 172.21.15.112:0/1678780563 shutdown_connections 2024-09-21T03:44:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:49.997+0000 7f71e1020640 1 -- 172.21.15.112:0/1678780563 wait complete. 2024-09-21T03:44:50.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:49 smithi203 ceph-mon[34723]: pgmap v812: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.7 MiB/s wr, 60 op/s 2024-09-21T03:44:50.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:49 smithi187 ceph-mon[34603]: pgmap v812: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.7 MiB/s wr, 60 op/s 2024-09-21T03:44:50.679 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:44:50.927 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:51.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:50 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1678780563' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:51.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:50 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1678780563' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:51.179 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:50 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1678780563' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:51.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.392+0000 7f827bc44640 1 -- 172.21.15.112:0/1073821909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82741094a0 msgr2=0x7f82741098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.392+0000 7f827bc44640 1 --2- 172.21.15.112:0/1073821909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82741094a0 0x7f82741098b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8268009ca0 tx=0x7f826802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.392+0000 7f827bc44640 1 -- 172.21.15.112:0/1073821909 shutdown_connections 2024-09-21T03:44:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.392+0000 7f827bc44640 1 --2- 172.21.15.112:0/1073821909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8274109df0 0x7f8274110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.392+0000 7f827bc44640 1 --2- 172.21.15.112:0/1073821909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82741094a0 0x7f82741098b0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.392+0000 7f827bc44640 1 --2- 172.21.15.112:0/1073821909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8274103b60 0x7f8274103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.392+0000 7f827bc44640 1 -- 172.21.15.112:0/1073821909 >> 172.21.15.112:0/1073821909 conn(0x7f827407b850 msgr2=0x7f827407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.392+0000 7f827bc44640 1 -- 172.21.15.112:0/1073821909 shutdown_connections 2024-09-21T03:44:51.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.392+0000 7f827bc44640 1 -- 172.21.15.112:0/1073821909 wait complete. 2024-09-21T03:44:51.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.392+0000 7f827bc44640 1 Processor -- start 2024-09-21T03:44:51.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.392+0000 7f827bc44640 1 -- start start 2024-09-21T03:44:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827bc44640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8274103b60 0x7f827419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827bc44640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82741094a0 0x7f827419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827bc44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8274109df0 0x7f827419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827bc44640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8274113ab0 con 0x7f8274109df0 2024-09-21T03:44:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827bc44640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8274113930 con 0x7f82741094a0 2024-09-21T03:44:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827bc44640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8274113c30 con 0x7f8274103b60 2024-09-21T03:44:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827a1e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8274109df0 0x7f827419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827a1e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8274109df0 0x7f827419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49216/0 (socket says 172.21.15.112:49216) 2024-09-21T03:44:51.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827a1e1640 1 -- 172.21.15.112:0/3714250679 learned_addr learned my addr 172.21.15.112:0/3714250679 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:51.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f82791df640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82741094a0 0x7f827419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f82799e0640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8274103b60 0x7f827419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827a1e1640 1 -- 172.21.15.112:0/3714250679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8274103b60 msgr2=0x7f827419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827a1e1640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8274103b60 0x7f827419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827a1e1640 1 -- 172.21.15.112:0/3714250679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82741094a0 msgr2=0x7f827419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827a1e1640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82741094a0 0x7f827419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827a1e1640 1 -- 172.21.15.112:0/3714250679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82741a0370 con 0x7f8274109df0 2024-09-21T03:44:51.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f82799e0640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8274103b60 0x7f827419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:51.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.393+0000 7f827a1e1640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8274109df0 0x7f827419fd00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f826400c970 tx=0x7f826400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.394+0000 7f8272ffd640 1 -- 172.21.15.112:0/3714250679 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264007bb0 con 0x7f8274109df0 2024-09-21T03:44:51.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.394+0000 7f827bc44640 1 -- 172.21.15.112:0/3714250679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82741a0660 con 0x7f8274109df0 2024-09-21T03:44:51.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.394+0000 7f8272ffd640 1 -- 172.21.15.112:0/3714250679 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8264007d50 con 0x7f8274109df0 2024-09-21T03:44:51.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.394+0000 7f8272ffd640 1 -- 172.21.15.112:0/3714250679 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264018770 con 0x7f8274109df0 2024-09-21T03:44:51.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.394+0000 7f827bc44640 1 -- 172.21.15.112:0/3714250679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82741a6e80 con 0x7f8274109df0 2024-09-21T03:44:51.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.395+0000 7f827bc44640 1 -- 172.21.15.112:0/3714250679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8274103f70 con 0x7f8274109df0 2024-09-21T03:44:51.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.396+0000 7f8272ffd640 1 -- 172.21.15.112:0/3714250679 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f826400f990 con 0x7f8274109df0 2024-09-21T03:44:51.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.397+0000 7f8272ffd640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f825c0b66b0 0x7f825c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:51.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.397+0000 7f82799e0640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f825c0b66b0 0x7f825c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:51.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.397+0000 7f8272ffd640 1 -- 172.21.15.112:0/3714250679 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f82640e7770 con 0x7f8274109df0 2024-09-21T03:44:51.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.397+0000 7f82799e0640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f825c0b66b0 0x7f825c0b8bc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f826c005d00 tx=0x7f826c005c70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:51.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.400+0000 7f8272ffd640 1 -- 172.21.15.112:0/3714250679 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8264014030 con 0x7f8274109df0 2024-09-21T03:44:51.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.612+0000 7f827bc44640 1 -- 172.21.15.112:0/3714250679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f827410bbf0 con 0x7f8274109df0 2024-09-21T03:44:51.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.613+0000 7f8272ffd640 1 -- 172.21.15.112:0/3714250679 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11934 (secure 0 0 0) 0x7f82640add80 con 0x7f8274109df0 2024-09-21T03:44:51.616 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:44:51.616 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":36,"btime":"2024-09-21T03:44:22:526380+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2583791645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2583791645},{"type":"v1","addr":"172.21.15.112:6835","nonce":2583791645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3042717510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3042717510},{"type":"v1","addr":"172.21.15.112:6837","nonce":3042717510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/1272407908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":1272407908},{"type":"v1","addr":"172.21.15.112:6839","nonce":1272407908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14844,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/395166022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":395166022},{"type":"v1","addr":"172.21.15.112:6841","nonce":395166022}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24599,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3187781490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3187781490},{"type":"v1","addr":"172.21.15.187:6833","nonce":3187781490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24616,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/4285434211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":4285434211},{"type":"v1","addr":"172.21.15.203:6833","nonce":4285434211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T03:27:01.886242+0000","modified":"2024-09-21T03:44:22.336872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24685,"mds_1":24674,"mds_2":24667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24626":{"gid":24626,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2552593896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2552593896},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552593896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24647":{"gid":24647,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/2217535204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":2217535204},{"type":"v1","addr":"172.21.15.187:6837","nonce":2217535204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24658":{"gid":24658,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3541929468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3541929468},{"type":"v1","addr":"172.21.15.203:6835","nonce":3541929468}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.203:6837/4062574800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":4062574800},{"type":"v1","addr":"172.21.15.203:6837","nonce":4062574800}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6839/910214120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":910214120},{"type":"v1","addr":"172.21.15.187:6839","nonce":910214120}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24685":{"gid":24685,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.203:6839/482815172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":482815172},{"type":"v1","addr":"172.21.15.203:6839","nonce":482815172}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24685,"qdb_cluster":[24667,24674,24685]},"id":1}]} 2024-09-21T03:44:51.616 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 36 2024-09-21T03:44:51.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.615+0000 7f827bc44640 1 -- 172.21.15.112:0/3714250679 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f825c0b66b0 msgr2=0x7f825c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.615+0000 7f827bc44640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f825c0b66b0 0x7f825c0b8bc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f826c005d00 tx=0x7f826c005c70 comp rx=0 tx=0).stop 2024-09-21T03:44:51.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.615+0000 7f827bc44640 1 -- 172.21.15.112:0/3714250679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8274109df0 msgr2=0x7f827419fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:51.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.615+0000 7f827bc44640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8274109df0 0x7f827419fd00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f826400c970 tx=0x7f826400ce30 comp rx=0 tx=0).stop 2024-09-21T03:44:51.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.616+0000 7f827bc44640 1 -- 172.21.15.112:0/3714250679 shutdown_connections 2024-09-21T03:44:51.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.616+0000 7f827bc44640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f825c0b66b0 0x7f825c0b8bc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.616+0000 7f827bc44640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8274109df0 0x7f827419fd00 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.616+0000 7f827bc44640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82741094a0 0x7f827419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.616+0000 7f827bc44640 1 --2- 172.21.15.112:0/3714250679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8274103b60 0x7f827419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:51.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.616+0000 7f827bc44640 1 -- 172.21.15.112:0/3714250679 >> 172.21.15.112:0/3714250679 conn(0x7f827407b850 msgr2=0x7f8274105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:51.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.616+0000 7f827bc44640 1 -- 172.21.15.112:0/3714250679 shutdown_connections 2024-09-21T03:44:51.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:51.616+0000 7f827bc44640 1 -- 172.21.15.112:0/3714250679 wait complete. 2024-09-21T03:44:52.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:51 smithi112 ceph-mon[31442]: pgmap v813: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.7 MiB/s wr, 59 op/s 2024-09-21T03:44:52.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:51 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3714250679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:52.062 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:51 smithi203 ceph-mon[34723]: pgmap v813: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.7 MiB/s wr, 59 op/s 2024-09-21T03:44:52.063 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:51 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3714250679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:52.081 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:51 smithi187 ceph-mon[34603]: pgmap v813: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 1.7 MiB/s wr, 59 op/s 2024-09-21T03:44:52.081 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:51 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3714250679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:44:52.365 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:44:52.664 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:53.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.181+0000 7f7eb5a63640 1 -- 172.21.15.112:0/679386415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7eb01094f0 msgr2=0x7f7eb0109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.181+0000 7f7eb5a63640 1 --2- 172.21.15.112:0/679386415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7eb01094f0 0x7f7eb0109900 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009900 tx=0x7f7ea402a410 comp rx=0 tx=0).stop 2024-09-21T03:44:53.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.181+0000 7f7eb5a63640 1 -- 172.21.15.112:0/679386415 shutdown_connections 2024-09-21T03:44:53.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.181+0000 7f7eb5a63640 1 --2- 172.21.15.112:0/679386415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7eb0109e40 0x7f7eb0110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.181+0000 7f7eb5a63640 1 --2- 172.21.15.112:0/679386415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7eb01094f0 0x7f7eb0109900 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.181+0000 7f7eb5a63640 1 --2- 172.21.15.112:0/679386415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb0103bb0 0x7f7eb0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.181+0000 7f7eb5a63640 1 -- 172.21.15.112:0/679386415 >> 172.21.15.112:0/679386415 conn(0x7f7eb00ff600 msgr2=0x7f7eb0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.181+0000 7f7eb5a63640 1 -- 172.21.15.112:0/679386415 shutdown_connections 2024-09-21T03:44:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.181+0000 7f7eb5a63640 1 -- 172.21.15.112:0/679386415 wait complete. 2024-09-21T03:44:53.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eb5a63640 1 Processor -- start 2024-09-21T03:44:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eb5a63640 1 -- start start 2024-09-21T03:44:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eb5a63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7eb0103bb0 0x7f7eb019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eb5a63640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7eb01094f0 0x7f7eb019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eb5a63640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb0109e40 0x7f7eb019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eb5a63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0113b00 con 0x7f7eb0103bb0 2024-09-21T03:44:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eb5a63640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0113980 con 0x7f7eb0109e40 2024-09-21T03:44:53.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eb5a63640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0113c80 con 0x7f7eb01094f0 2024-09-21T03:44:53.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eaf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7eb0103bb0 0x7f7eb019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eaf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7eb0103bb0 0x7f7eb019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49234/0 (socket says 172.21.15.112:49234) 2024-09-21T03:44:53.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eaf7fe640 1 -- 172.21.15.112:0/3564827988 learned_addr learned my addr 172.21.15.112:0/3564827988 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:53.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eaffff640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb0109e40 0x7f7eb019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.182+0000 7f7eaeffd640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7eb01094f0 0x7f7eb019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.183+0000 7f7eaf7fe640 1 -- 172.21.15.112:0/3564827988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7eb01094f0 msgr2=0x7f7eb019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.183+0000 7f7eaf7fe640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7eb01094f0 0x7f7eb019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.183+0000 7f7eaf7fe640 1 -- 172.21.15.112:0/3564827988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb0109e40 msgr2=0x7f7eb019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.183+0000 7f7eaf7fe640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb0109e40 0x7f7eb019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.183+0000 7f7eaf7fe640 1 -- 172.21.15.112:0/3564827988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb01a04a0 con 0x7f7eb0103bb0 2024-09-21T03:44:53.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.183+0000 7f7eaf7fe640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7eb0103bb0 0x7f7eb019a530 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000e9f0 tx=0x7f7ea000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:53.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.183+0000 7f7eacff9640 1 -- 172.21.15.112:0/3564827988 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000cd80 con 0x7f7eb0103bb0 2024-09-21T03:44:53.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.183+0000 7f7eb5a63640 1 -- 172.21.15.112:0/3564827988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb01a0790 con 0x7f7eb0103bb0 2024-09-21T03:44:53.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.183+0000 7f7eacff9640 1 -- 172.21.15.112:0/3564827988 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ea00108e0 con 0x7f7eb0103bb0 2024-09-21T03:44:53.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.183+0000 7f7eacff9640 1 -- 172.21.15.112:0/3564827988 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0018980 con 0x7f7eb0103bb0 2024-09-21T03:44:53.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.183+0000 7f7eb5a63640 1 -- 172.21.15.112:0/3564827988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb01a6ef0 con 0x7f7eb0103bb0 2024-09-21T03:44:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.184+0000 7f7eb5a63640 1 -- 172.21.15.112:0/3564827988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb0103fc0 con 0x7f7eb0103bb0 2024-09-21T03:44:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.186+0000 7f7eacff9640 1 -- 172.21.15.112:0/3564827988 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7ea0010430 con 0x7f7eb0103bb0 2024-09-21T03:44:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.186+0000 7f7eacff9640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7e980b6680 0x7f7e980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.186+0000 7f7eaeffd640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7e980b6680 0x7f7e980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.186+0000 7f7eacff9640 1 -- 172.21.15.112:0/3564827988 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f7ea00e7ea0 con 0x7f7eb0103bb0 2024-09-21T03:44:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.187+0000 7f7eaeffd640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7e980b6680 0x7f7e980b8b90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009230 tx=0x7f7ea4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.190+0000 7f7eacff9640 1 -- 172.21.15.112:0/3564827988 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ea00b2800 con 0x7f7eb0103bb0 2024-09-21T03:44:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.373+0000 7f7eb5a63640 1 -- 172.21.15.112:0/3564827988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7eb019b660 con 0x7f7eb0103bb0 2024-09-21T03:44:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.374+0000 7f7eacff9640 1 -- 172.21.15.112:0/3564827988 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23156 (secure 0 0 0) 0x7f7ea00b7740 con 0x7f7eb0103bb0 2024-09-21T03:44:53.377 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T03:44:53.377 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":96,"fsid":"063b32f6-77c8-11ef-baf5-efdc52797490","created":"2024-09-21T03:18:41.875282+0000","modified":"2024-09-21T03:44:14.915070+0000","last_up_change":"2024-09-21T03:26:13.929302+0000","last_in_change":"2024-09-21T03:25:50.261138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:22:30.533177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T03:26:58.087349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T03:26:59.970444+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec165c44-c973-4f2b-b1ca-a88a9bf48968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6803","nonce":3601862796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6805","nonce":3601862796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6809","nonce":3601862796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3601862796},{"type":"v1","addr":"172.21.15.112:6807","nonce":3601862796}]},"public_addr":"172.21.15.112:6803/3601862796","cluster_addr":"172.21.15.112:6805/3601862796","heartbeat_back_addr":"172.21.15.112:6809/3601862796","heartbeat_front_addr":"172.21.15.112:6807/3601862796","state":["exists","up"]},{"osd":1,"uuid":"200471f7-a0d9-4b5e-8be4-0033c6a82688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6801","nonce":279076548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6803","nonce":279076548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6807","nonce":279076548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":279076548},{"type":"v1","addr":"172.21.15.187:6805","nonce":279076548}]},"public_addr":"172.21.15.187:6801/279076548","cluster_addr":"172.21.15.187:6803/279076548","heartbeat_back_addr":"172.21.15.187:6807/279076548","heartbeat_front_addr":"172.21.15.187:6805/279076548","state":["exists","up"]},{"osd":2,"uuid":"0afbab16-dfb7-45d0-895c-846a437a19b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6801","nonce":388961217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6803","nonce":388961217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6807","nonce":388961217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":388961217},{"type":"v1","addr":"172.21.15.203:6805","nonce":388961217}]},"public_addr":"172.21.15.203:6801/388961217","cluster_addr":"172.21.15.203:6803/388961217","heartbeat_back_addr":"172.21.15.203:6807/388961217","heartbeat_front_addr":"172.21.15.203:6805/388961217","state":["exists","up"]},{"osd":3,"uuid":"495ece6a-e62f-4deb-8600-ece833469091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6811","nonce":2827467739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6813","nonce":2827467739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6817","nonce":2827467739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2827467739},{"type":"v1","addr":"172.21.15.112:6815","nonce":2827467739}]},"public_addr":"172.21.15.112:6811/2827467739","cluster_addr":"172.21.15.112:6813/2827467739","heartbeat_back_addr":"172.21.15.112:6817/2827467739","heartbeat_front_addr":"172.21.15.112:6815/2827467739","state":["exists","up"]},{"osd":4,"uuid":"0c46289b-1c19-4c58-bd00-5e96505df718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6809","nonce":3494239939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6811","nonce":3494239939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6815","nonce":3494239939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3494239939},{"type":"v1","addr":"172.21.15.187:6813","nonce":3494239939}]},"public_addr":"172.21.15.187:6809/3494239939","cluster_addr":"172.21.15.187:6811/3494239939","heartbeat_back_addr":"172.21.15.187:6815/3494239939","heartbeat_front_addr":"172.21.15.187:6813/3494239939","state":["exists","up"]},{"osd":5,"uuid":"f4727682-1223-49ae-a893-2559f553b8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6809","nonce":3996799741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6811","nonce":3996799741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6815","nonce":3996799741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3996799741},{"type":"v1","addr":"172.21.15.203:6813","nonce":3996799741}]},"public_addr":"172.21.15.203:6809/3996799741","cluster_addr":"172.21.15.203:6811/3996799741","heartbeat_back_addr":"172.21.15.203:6815/3996799741","heartbeat_front_addr":"172.21.15.203:6813/3996799741","state":["exists","up"]},{"osd":6,"uuid":"a3b2e075-188b-4640-827b-735e57c61d82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6819","nonce":92569356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6821","nonce":92569356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6825","nonce":92569356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":92569356},{"type":"v1","addr":"172.21.15.112:6823","nonce":92569356}]},"public_addr":"172.21.15.112:6819/92569356","cluster_addr":"172.21.15.112:6821/92569356","heartbeat_back_addr":"172.21.15.112:6825/92569356","heartbeat_front_addr":"172.21.15.112:6823/92569356","state":["exists","up"]},{"osd":7,"uuid":"530f9be1-24fa-4f16-8225-fddebd4aba99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6817","nonce":4003842172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6819","nonce":4003842172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6823","nonce":4003842172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4003842172},{"type":"v1","addr":"172.21.15.187:6821","nonce":4003842172}]},"public_addr":"172.21.15.187:6817/4003842172","cluster_addr":"172.21.15.187:6819/4003842172","heartbeat_back_addr":"172.21.15.187:6823/4003842172","heartbeat_front_addr":"172.21.15.187:6821/4003842172","state":["exists","up"]},{"osd":8,"uuid":"3c3869ee-6cae-409d-98bb-9ee286713295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6817","nonce":3883207460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6819","nonce":3883207460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6823","nonce":3883207460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3883207460},{"type":"v1","addr":"172.21.15.203:6821","nonce":3883207460}]},"public_addr":"172.21.15.203:6817/3883207460","cluster_addr":"172.21.15.203:6819/3883207460","heartbeat_back_addr":"172.21.15.203:6823/3883207460","heartbeat_front_addr":"172.21.15.203:6821/3883207460","state":["exists","up"]},{"osd":9,"uuid":"c47f58d2-8b8a-4f98-b14d-879143b50781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6827","nonce":2553277143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6829","nonce":2553277143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6833","nonce":2553277143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2553277143},{"type":"v1","addr":"172.21.15.112:6831","nonce":2553277143}]},"public_addr":"172.21.15.112:6827/2553277143","cluster_addr":"172.21.15.112:6829/2553277143","heartbeat_back_addr":"172.21.15.112:6833/2553277143","heartbeat_front_addr":"172.21.15.112:6831/2553277143","state":["exists","up"]},{"osd":10,"uuid":"54812bd2-e71b-40eb-8411-d281bdbe0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6825","nonce":499555149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6827","nonce":499555149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6831","nonce":499555149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":499555149},{"type":"v1","addr":"172.21.15.187:6829","nonce":499555149}]},"public_addr":"172.21.15.187:6825/499555149","cluster_addr":"172.21.15.187:6827/499555149","heartbeat_back_addr":"172.21.15.187:6831/499555149","heartbeat_front_addr":"172.21.15.187:6829/499555149","state":["exists","up"]},{"osd":11,"uuid":"4157bc4f-e275-4f4d-8d38-f934f9f61a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6825","nonce":520562655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6827","nonce":520562655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6831","nonce":520562655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":520562655},{"type":"v1","addr":"172.21.15.203:6829","nonce":520562655}]},"public_addr":"172.21.15.203:6825/520562655","cluster_addr":"172.21.15.203:6827/520562655","heartbeat_back_addr":"172.21.15.203:6831/520562655","heartbeat_front_addr":"172.21.15.203:6829/520562655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:21:37.985814+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:02.817430+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:28.002197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:22:53.261312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:17.323318+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:23:42.392083+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:07.975183+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:33.090938+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:24:57.550728+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:22.184661+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:25:46.447515+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:26:11.625308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/4206336392":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6800/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1495858515":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/3545481828":"2024-09-22T03:19:06.754886+0000","172.21.15.112:6800/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/839399771":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/1177659103":"2024-09-22T03:19:06.754886+0000","172.21.15.112:0/3522404182":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/914655252":"2024-09-22T03:19:31.820776+0000","172.21.15.112:6801/3643082307":"2024-09-22T03:20:32.227264+0000","172.21.15.112:0/1049357498":"2024-09-22T03:20:32.227264+0000","172.21.15.112:6801/2996077964":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/597633710":"2024-09-22T03:19:31.820776+0000","172.21.15.112:0/2169183307":"2024-09-22T03:20:32.227264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:44:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 -- 172.21.15.112:0/3564827988 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7e980b6680 msgr2=0x7f7e980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7e980b6680 0x7f7e980b8b90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009230 tx=0x7f7ea4031400 comp rx=0 tx=0).stop 2024-09-21T03:44:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 -- 172.21.15.112:0/3564827988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7eb0103bb0 msgr2=0x7f7eb019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7eb0103bb0 0x7f7eb019a530 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000e9f0 tx=0x7f7ea000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 -- 172.21.15.112:0/3564827988 shutdown_connections 2024-09-21T03:44:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7e980b6680 0x7f7e980b8b90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb0109e40 0x7f7eb019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7eb01094f0 0x7f7eb019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 --2- 172.21.15.112:0/3564827988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7eb0103bb0 0x7f7eb019a530 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 -- 172.21.15.112:0/3564827988 >> 172.21.15.112:0/3564827988 conn(0x7f7eb00ff600 msgr2=0x7f7eb0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 -- 172.21.15.112:0/3564827988 shutdown_connections 2024-09-21T03:44:53.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:53.377+0000 7f7eb5a63640 1 -- 172.21.15.112:0/3564827988 wait complete. 2024-09-21T03:44:54.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:53 smithi112 ceph-mon[31442]: pgmap v814: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 751 KiB/s wr, 34 op/s 2024-09-21T03:44:54.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:53 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3564827988' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:54.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:53 smithi203 ceph-mon[34723]: pgmap v814: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 751 KiB/s wr, 34 op/s 2024-09-21T03:44:54.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:53 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3564827988' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:54.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:53 smithi187 ceph-mon[34603]: pgmap v814: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 751 KiB/s wr, 34 op/s 2024-09-21T03:44:54.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:53 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3564827988' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:44:54.197 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T03:44:54.446 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:55.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.992+0000 7f7b8bf45640 1 -- 172.21.15.112:0/1350596622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84103230 msgr2=0x7f7b84103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.992+0000 7f7b8bf45640 1 --2- 172.21.15.112:0/1350596622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84103230 0x7f7b84103640 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009ca0 tx=0x7f7b7402a8e0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.993+0000 7f7b8bf45640 1 -- 172.21.15.112:0/1350596622 shutdown_connections 2024-09-21T03:44:55.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.993+0000 7f7b8bf45640 1 --2- 172.21.15.112:0/1350596622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b84108fc0 0x7f7b8410f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.993+0000 7f7b8bf45640 1 --2- 172.21.15.112:0/1350596622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b84108670 0x7f7b84108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.993+0000 7f7b8bf45640 1 --2- 172.21.15.112:0/1350596622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84103230 0x7f7b84103640 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.993+0000 7f7b8bf45640 1 -- 172.21.15.112:0/1350596622 >> 172.21.15.112:0/1350596622 conn(0x7f7b84079ea0 msgr2=0x7f7b8407a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:55.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.993+0000 7f7b8bf45640 1 -- 172.21.15.112:0/1350596622 shutdown_connections 2024-09-21T03:44:55.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.993+0000 7f7b8bf45640 1 -- 172.21.15.112:0/1350596622 wait complete. 2024-09-21T03:44:55.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.993+0000 7f7b8bf45640 1 Processor -- start 2024-09-21T03:44:55.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.993+0000 7f7b8bf45640 1 -- start start 2024-09-21T03:44:55.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b8bf45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84103230 0x7f7b8419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:55.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b8bf45640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b84108670 0x7f7b8419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:55.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b8bf45640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b84108fc0 0x7f7b841a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:55.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b8bf45640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84112d40 con 0x7f7b84103230 2024-09-21T03:44:55.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b8bf45640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84112bc0 con 0x7f7b84108fc0 2024-09-21T03:44:55.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b8bf45640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84112ec0 con 0x7f7b84108670 2024-09-21T03:44:55.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b89ce1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84103230 0x7f7b8419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:55.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b89ce1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84103230 0x7f7b8419a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49250/0 (socket says 172.21.15.112:49250) 2024-09-21T03:44:55.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b89ce1640 1 -- 172.21.15.112:0/3076477020 learned_addr learned my addr 172.21.15.112:0/3076477020 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:55.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b894e0640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b84108670 0x7f7b8419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:55.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b8a4e2640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b84108fc0 0x7f7b841a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:55.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b89ce1640 1 -- 172.21.15.112:0/3076477020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b84108670 msgr2=0x7f7b8419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b89ce1640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b84108670 0x7f7b8419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b89ce1640 1 -- 172.21.15.112:0/3076477020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b84108fc0 msgr2=0x7f7b841a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b89ce1640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b84108fc0 0x7f7b841a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b89ce1640 1 -- 172.21.15.112:0/3076477020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b841a0810 con 0x7f7b84103230 2024-09-21T03:44:55.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.994+0000 7f7b89ce1640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84103230 0x7f7b8419a8a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7b7402ae00 tx=0x7f7b74003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:55.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.995+0000 7f7b82ffd640 1 -- 172.21.15.112:0/3076477020 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74004240 con 0x7f7b84103230 2024-09-21T03:44:55.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.995+0000 7f7b82ffd640 1 -- 172.21.15.112:0/3076477020 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b74003e50 con 0x7f7b84103230 2024-09-21T03:44:55.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.995+0000 7f7b8bf45640 1 -- 172.21.15.112:0/3076477020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b841a0aa0 con 0x7f7b84103230 2024-09-21T03:44:55.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.995+0000 7f7b82ffd640 1 -- 172.21.15.112:0/3076477020 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7403a400 con 0x7f7b84103230 2024-09-21T03:44:55.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.995+0000 7f7b8bf45640 1 -- 172.21.15.112:0/3076477020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b841a7260 con 0x7f7b84103230 2024-09-21T03:44:55.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.996+0000 7f7b8bf45640 1 -- 172.21.15.112:0/3076477020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b84103640 con 0x7f7b84103230 2024-09-21T03:44:55.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.997+0000 7f7b82ffd640 1 -- 172.21.15.112:0/3076477020 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7b740305e0 con 0x7f7b84103230 2024-09-21T03:44:55.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.998+0000 7f7b82ffd640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7b6c0b6680 0x7f7b6c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:55.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.998+0000 7f7b894e0640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7b6c0b6680 0x7f7b6c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:55.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.998+0000 7f7b82ffd640 1 -- 172.21.15.112:0/3076477020 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8470+0+0 (secure 0 0 0) 0x7f7b74108f30 con 0x7f7b84103230 2024-09-21T03:44:55.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:54.999+0000 7f7b894e0640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7b6c0b6680 0x7f7b6c0b8b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7b8406b0f0 tx=0x7f7b78008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:55.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.001+0000 7f7b82ffd640 1 -- 172.21.15.112:0/3076477020 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7b740cf4b0 con 0x7f7b84103230 2024-09-21T03:44:55.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.217+0000 7f7b8bf45640 1 -- 172.21.15.112:0/3076477020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7b8410bd50 con 0x7f7b84103230 2024-09-21T03:44:55.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.220+0000 7f7b82ffd640 1 -- 172.21.15.112:0/3076477020 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f7b740d3890 con 0x7f7b84103230 2024-09-21T03:44:55.222 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:44:55.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 -- 172.21.15.112:0/3076477020 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7b6c0b6680 msgr2=0x7f7b6c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7b6c0b6680 0x7f7b6c0b8b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7b8406b0f0 tx=0x7f7b78008040 comp rx=0 tx=0).stop 2024-09-21T03:44:55.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 -- 172.21.15.112:0/3076477020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84103230 msgr2=0x7f7b8419a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:55.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84103230 0x7f7b8419a8a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7b7402ae00 tx=0x7f7b74003ba0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 -- 172.21.15.112:0/3076477020 shutdown_connections 2024-09-21T03:44:55.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f7b6c0b6680 0x7f7b6c0b8b90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b84108fc0 0x7f7b841a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b84108670 0x7f7b8419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 --2- 172.21.15.112:0/3076477020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b84103230 0x7f7b8419a8a0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:55.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 -- 172.21.15.112:0/3076477020 >> 172.21.15.112:0/3076477020 conn(0x7f7b84079ea0 msgr2=0x7f7b84101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:55.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 -- 172.21.15.112:0/3076477020 shutdown_connections 2024-09-21T03:44:55.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:55.223+0000 7f7b8bf45640 1 -- 172.21.15.112:0/3076477020 wait complete. 2024-09-21T03:44:55.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:55 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a[31438]: 2024-09-21T03:44:55.218+0000 7f49da9b7640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:44:55.661 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i[64347]: ignoring --setuser ceph since I am not root 2024-09-21T03:44:55.661 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i[64347]: ignoring --setgroup ceph since I am not root 2024-09-21T03:44:55.661 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-mds[64351]: -- 172.21.15.203:0/569667592 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b95b3f860 con 0x564b99ce7180 2024-09-21T03:44:55.662 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i[64347]: starting mds.i at 2024-09-21T03:44:55.662 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c[64079]: ignoring --setuser ceph since I am not root 2024-09-21T03:44:55.662 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c[64079]: ignoring --setgroup ceph since I am not root 2024-09-21T03:44:55.662 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-mds[64083]: -- 172.21.15.203:0/1653176364 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55efb10b1c20 con 0x55efb5259600 2024-09-21T03:44:55.662 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c[64079]: starting mds.c at 2024-09-21T03:44:55.663 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f[63809]: ignoring --setuser ceph since I am not root 2024-09-21T03:44:55.663 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f[63809]: ignoring --setgroup ceph since I am not root 2024-09-21T03:44:55.663 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-mds[63813]: -- 172.21.15.203:0/4095387333 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fb54d4f860 con 0x55fb58ef7180 2024-09-21T03:44:55.663 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f[63809]: starting mds.f at 2024-09-21T03:44:55.663 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f[63809]: 2024-09-21T03:44:55.344+0000 7f8b36fa5640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2024-09-21T03:44:55.680 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b[63369]: ignoring --setuser ceph since I am not root 2024-09-21T03:44:55.680 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b[63369]: ignoring --setgroup ceph since I am not root 2024-09-21T03:44:55.680 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-mds[63373]: -- 172.21.15.187:0/1369877350 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587020c1860 con 0x558706269180 2024-09-21T03:44:55.680 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b[63369]: starting mds.b at 2024-09-21T03:44:55.680 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b[63369]: 2024-09-21T03:44:55.297+0000 7f2c5e2a4640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2024-09-21T03:44:55.681 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h[63089]: ignoring --setuser ceph since I am not root 2024-09-21T03:44:55.681 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h[63089]: ignoring --setgroup ceph since I am not root 2024-09-21T03:44:55.681 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-mds[63093]: -- 172.21.15.187:0/49342454 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b01fbf5c20 con 0x55b023d9d180 2024-09-21T03:44:55.681 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h[63089]: starting mds.h at 2024-09-21T03:44:55.681 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e[62819]: ignoring --setuser ceph since I am not root 2024-09-21T03:44:55.682 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e[62819]: ignoring --setgroup ceph since I am not root 2024-09-21T03:44:55.682 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-mds[62823]: -- 172.21.15.187:0/3569331571 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557f9956b860 con 0x557f9d713180 2024-09-21T03:44:55.682 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e[62819]: starting mds.e at 2024-09-21T03:44:55.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:55 smithi112 ceph-mon[31442]: pgmap v815: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 132 KiB/s wr, 15 op/s 2024-09-21T03:44:55.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:55 smithi112 ceph-mon[31442]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:44:55.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:55 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3076477020' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:44:55.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:55 smithi112 ceph-mon[31442]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:44:55.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:55 smithi112 ceph-mon[31442]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:44:55.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:55 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3076477020' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:44:55.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:55 smithi112 ceph-mon[31442]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T03:44:55.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:55 smithi112 ceph-mon[31442]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T03:44:55.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:44:56.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-mon[34723]: pgmap v815: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 132 KiB/s wr, 15 op/s 2024-09-21T03:44:56.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-mon[34723]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:44:56.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3076477020' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:44:56.161 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-mon[34723]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:44:56.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-mon[34723]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:44:56.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3076477020' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:44:56.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-mon[34723]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T03:44:56.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:55 smithi203 ceph-mon[34723]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T03:44:56.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-mon[34603]: pgmap v815: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 132 KiB/s wr, 15 op/s 2024-09-21T03:44:56.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-mon[34603]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T03:44:56.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3076477020' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:44:56.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-mon[34603]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:44:56.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-mon[34603]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:44:56.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3076477020' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:44:56.180 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-mon[34603]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T03:44:56.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:55 smithi187 ceph-mon[34603]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T03:44:56.215 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.850+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3991984998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c108d20 msgr2=0x7f2f6c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.850+0000 7f2f70b9c640 1 --2- 172.21.15.112:0/3991984998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c108d20 0x7f2f6c10f550 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009ca0 tx=0x7f2f6402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.850+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3991984998 shutdown_connections 2024-09-21T03:44:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.850+0000 7f2f70b9c640 1 --2- 172.21.15.112:0/3991984998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c108d20 0x7f2f6c10f550 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.850+0000 7f2f70b9c640 1 --2- 172.21.15.112:0/3991984998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c1083d0 0x7f2f6c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.850+0000 7f2f70b9c640 1 --2- 172.21.15.112:0/3991984998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f6c102ae0 0x7f2f6c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.850+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3991984998 >> 172.21.15.112:0/3991984998 conn(0x7f2f6c079760 msgr2=0x7f2f6c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:56.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.850+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3991984998 shutdown_connections 2024-09-21T03:44:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.850+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3991984998 wait complete. 2024-09-21T03:44:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.851+0000 7f2f70b9c640 1 Processor -- start 2024-09-21T03:44:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.851+0000 7f2f70b9c640 1 -- start start 2024-09-21T03:44:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f70b9c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c102ae0 0x7f2f6c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f70b9c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f6c1083d0 0x7f2f6c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f70b9c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c108d20 0x7f2f6c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f70b9c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c1129f0 con 0x7f2f6c102ae0 2024-09-21T03:44:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f70b9c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c112870 con 0x7f2f6c108d20 2024-09-21T03:44:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f70b9c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c112b70 con 0x7f2f6c1083d0 2024-09-21T03:44:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f6a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c102ae0 0x7f2f6c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f6a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c102ae0 0x7f2f6c199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49716/0 (socket says 172.21.15.112:49716) 2024-09-21T03:44:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f6a59c640 1 -- 172.21.15.112:0/3268134934 learned_addr learned my addr 172.21.15.112:0/3268134934 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f6ad9d640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c108d20 0x7f2f6c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f69d9b640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f6c1083d0 0x7f2f6c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f6a59c640 1 -- 172.21.15.112:0/3268134934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f6c1083d0 msgr2=0x7f2f6c199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f6a59c640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f6c1083d0 0x7f2f6c199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f6a59c640 1 -- 172.21.15.112:0/3268134934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c108d20 msgr2=0x7f2f6c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f6a59c640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c108d20 0x7f2f6c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f6a59c640 1 -- 172.21.15.112:0/3268134934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c19f2f0 con 0x7f2f6c102ae0 2024-09-21T03:44:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f69d9b640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f6c1083d0 0x7f2f6c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:44:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f6a59c640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c102ae0 0x7f2f6c199380 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00b4a0 tx=0x7f2f5c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f637fe640 1 -- 172.21.15.112:0/3268134934 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c014070 con 0x7f2f6c102ae0 2024-09-21T03:44:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f637fe640 1 -- 172.21.15.112:0/3268134934 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5c010ac0 con 0x7f2f6c102ae0 2024-09-21T03:44:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3268134934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c19f5e0 con 0x7f2f6c102ae0 2024-09-21T03:44:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f637fe640 1 -- 172.21.15.112:0/3268134934 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c017c30 con 0x7f2f6c102ae0 2024-09-21T03:44:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.852+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3268134934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c1a5d40 con 0x7f2f6c102ae0 2024-09-21T03:44:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.854+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3268134934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c103d10 con 0x7f2f6c102ae0 2024-09-21T03:44:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.855+0000 7f2f637fe640 1 -- 172.21.15.112:0/3268134934 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2f5c017070 con 0x7f2f6c102ae0 2024-09-21T03:44:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.856+0000 7f2f637fe640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f580b6700 0x7f2f580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.856+0000 7f2f69d9b640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f580b6700 0x7f2f580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.856+0000 7f2f637fe640 1 -- 172.21.15.112:0/3268134934 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8684+0+0 (secure 0 0 0) 0x7f2f5c0e6a50 con 0x7f2f6c102ae0 2024-09-21T03:44:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.856+0000 7f2f69d9b640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f580b6700 0x7f2f580b8c10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c107040 tx=0x7f2f54005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:56.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:56.860+0000 7f2f637fe640 1 -- 172.21.15.112:0/3268134934 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f5c0acf00 con 0x7f2f6c102ae0 2024-09-21T03:44:57.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.075+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3268134934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2f6c19a4b0 con 0x7f2f6c102ae0 2024-09-21T03:44:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.227+0000 7f2f637fe640 1 -- 172.21.15.112:0/3268134934 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) ==== 104+0+0 (secure 0 0 0) 0x7f2f5c0b12e0 con 0x7f2f6c102ae0 2024-09-21T03:44:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3268134934 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f580b6700 msgr2=0x7f2f580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f580b6700 0x7f2f580b8c10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c107040 tx=0x7f2f54005f90 comp rx=0 tx=0).stop 2024-09-21T03:44:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3268134934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c102ae0 msgr2=0x7f2f6c199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c102ae0 0x7f2f6c199380 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00b4a0 tx=0x7f2f5c00b960 comp rx=0 tx=0).stop 2024-09-21T03:44:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3268134934 shutdown_connections 2024-09-21T03:44:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f2f580b6700 0x7f2f580b8c10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f6c108d20 0x7f2f6c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f6c1083d0 0x7f2f6c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 --2- 172.21.15.112:0/3268134934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f6c102ae0 0x7f2f6c199380 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3268134934 >> 172.21.15.112:0/3268134934 conn(0x7f2f6c079760 msgr2=0x7f2f6c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3268134934 shutdown_connections 2024-09-21T03:44:57.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:57.230+0000 7f2f70b9c640 1 -- 172.21.15.112:0/3268134934 wait complete. 2024-09-21T03:44:57.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.187:6834/2609511481,v1:172.21.15.187:6837/2609511481] up:boot 2024-09-21T03:44:57.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.187:6835/636751204,v1:172.21.15.187:6838/636751204] up:boot 2024-09-21T03:44:57.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.187:6836/1795197104,v1:172.21.15.187:6839/1795197104] up:boot 2024-09-21T03:44:57.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.203:6834/2666260286,v1:172.21.15.203:6837/2666260286] up:boot 2024-09-21T03:44:57.394 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.203:6836/4242752476,v1:172.21.15.203:6839/4242752476] up:boot 2024-09-21T03:44:57.394 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: mds.? [v2:172.21.15.203:6835/1308634246,v1:172.21.15.203:6838/1308634246] up:boot 2024-09-21T03:44:57.394 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T03:44:57.394 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:44:57.394 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:44:57.394 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:44:57.394 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:44:57.394 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:44:57.395 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:44:57.395 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:57 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3268134934' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:44:57.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.187:6834/2609511481,v1:172.21.15.187:6837/2609511481] up:boot 2024-09-21T03:44:57.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.187:6835/636751204,v1:172.21.15.187:6838/636751204] up:boot 2024-09-21T03:44:57.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.187:6836/1795197104,v1:172.21.15.187:6839/1795197104] up:boot 2024-09-21T03:44:57.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.203:6834/2666260286,v1:172.21.15.203:6837/2666260286] up:boot 2024-09-21T03:44:57.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.203:6836/4242752476,v1:172.21.15.203:6839/4242752476] up:boot 2024-09-21T03:44:57.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: mds.? [v2:172.21.15.203:6835/1308634246,v1:172.21.15.203:6838/1308634246] up:boot 2024-09-21T03:44:57.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T03:44:57.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:44:57.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:44:57.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:44:57.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:44:57.663 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:44:57.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:44:57.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:57 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3268134934' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:44:57.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.187:6834/2609511481,v1:172.21.15.187:6837/2609511481] up:boot 2024-09-21T03:44:57.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.187:6835/636751204,v1:172.21.15.187:6838/636751204] up:boot 2024-09-21T03:44:57.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.187:6836/1795197104,v1:172.21.15.187:6839/1795197104] up:boot 2024-09-21T03:44:57.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.203:6834/2666260286,v1:172.21.15.203:6837/2666260286] up:boot 2024-09-21T03:44:57.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.203:6836/4242752476,v1:172.21.15.203:6839/4242752476] up:boot 2024-09-21T03:44:57.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: mds.? [v2:172.21.15.203:6835/1308634246,v1:172.21.15.203:6838/1308634246] up:boot 2024-09-21T03:44:57.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T03:44:57.681 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:44:57.682 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:44:57.682 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:44:57.682 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:44:57.682 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:44:57.682 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: from='mgr.14226 172.21.15.112:0/1402235283' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:44:57.682 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:57 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3268134934' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:44:57.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:44:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:44:58.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:58 smithi112 ceph-mon[31442]: pgmap v817: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 156 KiB/s wr, 16 op/s 2024-09-21T03:44:58.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:58 smithi112 ceph-mon[31442]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:44:58.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:58 smithi112 ceph-mon[31442]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:44:58.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:58 smithi112 ceph-mon[31442]: Cluster is now healthy 2024-09-21T03:44:58.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:58 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/3268134934' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:44:58.485 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:58 smithi112 ceph-mon[31442]: fsmap 12 up:standby 2024-09-21T03:44:58.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:58 smithi203 ceph-mon[34723]: pgmap v817: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 156 KiB/s wr, 16 op/s 2024-09-21T03:44:58.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:58 smithi203 ceph-mon[34723]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:44:58.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:58 smithi203 ceph-mon[34723]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:44:58.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:58 smithi203 ceph-mon[34723]: Cluster is now healthy 2024-09-21T03:44:58.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:58 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/3268134934' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:44:58.662 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:58 smithi203 ceph-mon[34723]: fsmap 12 up:standby 2024-09-21T03:44:58.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:58 smithi187 ceph-mon[34603]: pgmap v817: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 156 KiB/s wr, 16 op/s 2024-09-21T03:44:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:58 smithi187 ceph-mon[34603]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:44:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:58 smithi187 ceph-mon[34603]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:44:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:58 smithi187 ceph-mon[34603]: Cluster is now healthy 2024-09-21T03:44:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:58 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/3268134934' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:44:58.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:58 smithi187 ceph-mon[34603]: fsmap 12 up:standby 2024-09-21T03:44:58.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.694+0000 7efd4554f640 1 -- 172.21.15.112:0/3779297955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd40108cf0 msgr2=0x7efd4010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.694+0000 7efd4554f640 1 --2- 172.21.15.112:0/3779297955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd40108cf0 0x7efd4010f520 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7efd38009ca0 tx=0x7efd3802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.694+0000 7efd4554f640 1 -- 172.21.15.112:0/3779297955 shutdown_connections 2024-09-21T03:44:58.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.694+0000 7efd4554f640 1 --2- 172.21.15.112:0/3779297955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd40108cf0 0x7efd4010f520 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.694+0000 7efd4554f640 1 --2- 172.21.15.112:0/3779297955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd401083a0 0x7efd401087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.694+0000 7efd4554f640 1 --2- 172.21.15.112:0/3779297955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd40102a60 0x7efd40102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.694+0000 7efd4554f640 1 -- 172.21.15.112:0/3779297955 >> 172.21.15.112:0/3779297955 conn(0x7efd40079730 msgr2=0x7efd40079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:58.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.694+0000 7efd4554f640 1 -- 172.21.15.112:0/3779297955 shutdown_connections 2024-09-21T03:44:58.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.694+0000 7efd4554f640 1 -- 172.21.15.112:0/3779297955 wait complete. 2024-09-21T03:44:58.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.695+0000 7efd4554f640 1 Processor -- start 2024-09-21T03:44:58.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.695+0000 7efd4554f640 1 -- start start 2024-09-21T03:44:58.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.695+0000 7efd4554f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd40102a60 0x7efd4019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.695+0000 7efd4554f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd401083a0 0x7efd4019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.695+0000 7efd4554f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd40108cf0 0x7efd4019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.695+0000 7efd4554f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd40112960 con 0x7efd401083a0 2024-09-21T03:44:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.695+0000 7efd4554f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd401127e0 con 0x7efd40108cf0 2024-09-21T03:44:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.695+0000 7efd4554f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efd40112ae0 con 0x7efd40102a60 2024-09-21T03:44:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd367fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd401083a0 0x7efd4019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd367fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd401083a0 0x7efd4019b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49730/0 (socket says 172.21.15.112:49730) 2024-09-21T03:44:58.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd367fc640 1 -- 172.21.15.112:0/1389736834 learned_addr learned my addr 172.21.15.112:0/1389736834 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:44:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd3f7fe640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd40108cf0 0x7efd4019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd3effd640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd40102a60 0x7efd4019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd367fc640 1 -- 172.21.15.112:0/1389736834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd40102a60 msgr2=0x7efd4019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd367fc640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd40102a60 0x7efd4019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd367fc640 1 -- 172.21.15.112:0/1389736834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd40108cf0 msgr2=0x7efd4019bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd367fc640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd40108cf0 0x7efd4019bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd367fc640 1 -- 172.21.15.112:0/1389736834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd401a63c0 con 0x7efd401083a0 2024-09-21T03:44:58.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd3effd640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd40102a60 0x7efd4019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:44:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd367fc640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd401083a0 0x7efd4019b6d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c970 tx=0x7efd2800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd3cff9640 1 -- 172.21.15.112:0/1389736834 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28014070 con 0x7efd401083a0 2024-09-21T03:44:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd3cff9640 1 -- 172.21.15.112:0/1389736834 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd28004440 con 0x7efd401083a0 2024-09-21T03:44:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd4554f640 1 -- 172.21.15.112:0/1389736834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd401a66b0 con 0x7efd401083a0 2024-09-21T03:44:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd3cff9640 1 -- 172.21.15.112:0/1389736834 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28003e50 con 0x7efd401083a0 2024-09-21T03:44:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.696+0000 7efd4554f640 1 -- 172.21.15.112:0/1389736834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd401a6bf0 con 0x7efd401083a0 2024-09-21T03:44:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.697+0000 7efd4554f640 1 -- 172.21.15.112:0/1389736834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd40102e70 con 0x7efd401083a0 2024-09-21T03:44:58.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.699+0000 7efd3cff9640 1 -- 172.21.15.112:0/1389736834 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7efd28017070 con 0x7efd401083a0 2024-09-21T03:44:58.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.699+0000 7efd3cff9640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd2c0b6820 0x7efd2c0b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:44:58.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.700+0000 7efd3effd640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd2c0b6820 0x7efd2c0b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:44:58.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.700+0000 7efd3cff9640 1 -- 172.21.15.112:0/1389736834 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8684+0+0 (secure 0 0 0) 0x7efd280e7ad0 con 0x7efd401083a0 2024-09-21T03:44:58.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.700+0000 7efd3effd640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd2c0b6820 0x7efd2c0b8d30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efd4006b9e0 tx=0x7efd30005db0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:44:58.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.702+0000 7efd3cff9640 1 -- 172.21.15.112:0/1389736834 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efd2801e030 con 0x7efd401083a0 2024-09-21T03:44:58.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:58.884+0000 7efd4554f640 1 -- 172.21.15.112:0/1389736834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efd4006b290 con 0x7efd401083a0 2024-09-21T03:44:59.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.237+0000 7efd3cff9640 1 -- 172.21.15.112:0/1389736834 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7efd280adf80 con 0x7efd401083a0 2024-09-21T03:44:59.240 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.240+0000 7efd4554f640 1 -- 172.21.15.112:0/1389736834 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd2c0b6820 msgr2=0x7efd2c0b8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.240+0000 7efd4554f640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd2c0b6820 0x7efd2c0b8d30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efd4006b9e0 tx=0x7efd30005db0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.240+0000 7efd4554f640 1 -- 172.21.15.112:0/1389736834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd401083a0 msgr2=0x7efd4019b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.240+0000 7efd4554f640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd401083a0 0x7efd4019b6d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c970 tx=0x7efd2800ce30 comp rx=0 tx=0).stop 2024-09-21T03:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.241+0000 7efd4554f640 1 -- 172.21.15.112:0/1389736834 shutdown_connections 2024-09-21T03:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.241+0000 7efd4554f640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7efd2c0b6820 0x7efd2c0b8d30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.241+0000 7efd4554f640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd40108cf0 0x7efd4019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.241+0000 7efd4554f640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd401083a0 0x7efd4019b6d0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.241+0000 7efd4554f640 1 --2- 172.21.15.112:0/1389736834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd40102a60 0x7efd4019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:44:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.241+0000 7efd4554f640 1 -- 172.21.15.112:0/1389736834 >> 172.21.15.112:0/1389736834 conn(0x7efd40079730 msgr2=0x7efd40104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:44:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.241+0000 7efd4554f640 1 -- 172.21.15.112:0/1389736834 shutdown_connections 2024-09-21T03:44:59.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:44:59.241+0000 7efd4554f640 1 -- 172.21.15.112:0/1389736834 wait complete. 2024-09-21T03:44:59.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:44:59 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1389736834' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:44:59.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:44:59 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1389736834' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:44:59.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:44:59 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1389736834' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:45:00.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:45:00.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-mon[31442]: pgmap v818: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 157 KiB/s wr, 15 op/s 2024-09-21T03:45:00.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/1389736834' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:45:00.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-mon[31442]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T03:45:00.298 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:45:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:00 smithi203 ceph-mon[34723]: pgmap v818: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 157 KiB/s wr, 15 op/s 2024-09-21T03:45:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:00 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/1389736834' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:45:00.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:00 smithi203 ceph-mon[34723]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T03:45:00.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:00 smithi187 ceph-mon[34603]: pgmap v818: 97 pgs: 97 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 157 KiB/s wr, 15 op/s 2024-09-21T03:45:00.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:00 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/1389736834' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:45:00.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:00 smithi187 ceph-mon[34603]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T03:45:00.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.755+0000 7f89ac911640 1 -- 172.21.15.112:0/1828631553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a8102ae0 msgr2=0x7f89a8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.755+0000 7f89ac911640 1 --2- 172.21.15.112:0/1828631553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a8102ae0 0x7f89a8102ef0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8998009900 tx=0x7f899802a410 comp rx=0 tx=0).stop 2024-09-21T03:45:00.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.755+0000 7f89ac911640 1 -- 172.21.15.112:0/1828631553 shutdown_connections 2024-09-21T03:45:00.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.755+0000 7f89ac911640 1 --2- 172.21.15.112:0/1828631553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89a8108d20 0x7f89a810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.755+0000 7f89ac911640 1 --2- 172.21.15.112:0/1828631553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89a81083d0 0x7f89a81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.755+0000 7f89ac911640 1 --2- 172.21.15.112:0/1828631553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a8102ae0 0x7f89a8102ef0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.755+0000 7f89ac911640 1 -- 172.21.15.112:0/1828631553 >> 172.21.15.112:0/1828631553 conn(0x7f89a80797b0 msgr2=0x7f89a8079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:00.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.755+0000 7f89ac911640 1 -- 172.21.15.112:0/1828631553 shutdown_connections 2024-09-21T03:45:00.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.755+0000 7f89ac911640 1 -- 172.21.15.112:0/1828631553 wait complete. 2024-09-21T03:45:00.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.756+0000 7f89ac911640 1 Processor -- start 2024-09-21T03:45:00.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.756+0000 7f89ac911640 1 -- start start 2024-09-21T03:45:00.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89ac911640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a8102ae0 0x7f89a81993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89ac911640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89a81083d0 0x7f89a8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89ac911640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89a8108d20 0x7f89a819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89ac911640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89a81129e0 con 0x7f89a8102ae0 2024-09-21T03:45:00.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89ac911640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8112860 con 0x7f89a8108d20 2024-09-21T03:45:00.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89ac911640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8112b60 con 0x7f89a81083d0 2024-09-21T03:45:00.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a8102ae0 0x7f89a81993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a8102ae0 0x7f89a81993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49748/0 (socket says 172.21.15.112:49748) 2024-09-21T03:45:00.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a659c640 1 -- 172.21.15.112:0/4204128294 learned_addr learned my addr 172.21.15.112:0/4204128294 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T03:45:00.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a5d9b640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89a81083d0 0x7f89a8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a6d9d640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89a8108d20 0x7f89a819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a659c640 1 -- 172.21.15.112:0/4204128294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89a81083d0 msgr2=0x7f89a8199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a659c640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89a81083d0 0x7f89a8199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a659c640 1 -- 172.21.15.112:0/4204128294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89a8108d20 msgr2=0x7f89a819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:00.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a659c640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89a8108d20 0x7f89a819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:00.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a659c640 1 -- 172.21.15.112:0/4204128294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a819f320 con 0x7f89a8102ae0 2024-09-21T03:45:00.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a5d9b640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89a81083d0 0x7f89a8199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:45:00.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.757+0000 7f89a659c640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a8102ae0 0x7f89a81993b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8998005940 tx=0x7f89980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.758+0000 7f89977fe640 1 -- 172.21.15.112:0/4204128294 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899803f030 con 0x7f89a8102ae0 2024-09-21T03:45:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.758+0000 7f89977fe640 1 -- 172.21.15.112:0/4204128294 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8998003cb0 con 0x7f89a8102ae0 2024-09-21T03:45:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.758+0000 7f89977fe640 1 -- 172.21.15.112:0/4204128294 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998035070 con 0x7f89a8102ae0 2024-09-21T03:45:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.758+0000 7f89ac911640 1 -- 172.21.15.112:0/4204128294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a819f5b0 con 0x7f89a8102ae0 2024-09-21T03:45:00.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.758+0000 7f89ac911640 1 -- 172.21.15.112:0/4204128294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a806b820 con 0x7f89a8102ae0 2024-09-21T03:45:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.759+0000 7f89ac911640 1 -- 172.21.15.112:0/4204128294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a8103d10 con 0x7f89a8102ae0 2024-09-21T03:45:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.760+0000 7f89977fe640 1 -- 172.21.15.112:0/4204128294 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8998042030 con 0x7f89a8102ae0 2024-09-21T03:45:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.761+0000 7f89977fe640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89900b8910 0x7f89900bae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:45:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.761+0000 7f89a5d9b640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89900b8910 0x7f89900bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:45:00.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.761+0000 7f89977fe640 1 -- 172.21.15.112:0/4204128294 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8130+0+0 (secure 0 0 0) 0x7f8998107f70 con 0x7f89a8102ae0 2024-09-21T03:45:00.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.762+0000 7f89a5d9b640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89900b8910 0x7f89900bae20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f89a8079e30 tx=0x7f899c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:45:00.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.764+0000 7f89977fe640 1 -- 172.21.15.112:0/4204128294 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f89980d2a70 con 0x7f89a8102ae0 2024-09-21T03:45:00.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:00.951+0000 7f89ac911640 1 -- 172.21.15.112:0/4204128294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f89a806c240 con 0x7f89a8102ae0 2024-09-21T03:45:01.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:45:00.804+0000 7f3805f5e640 -1 client.14238: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-21T03:45:01.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:45:00.805+0000 7f3805f5e640 -1 client.14238: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-21T03:45:01.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: Exception in thread Thread-21: 2024-09-21T03:45:01.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: Traceback (most recent call last): 2024-09-21T03:45:01.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-21T03:45:01.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self.run() 2024-09-21T03:45:01.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-21T03:45:01.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self.function(*self.args, **self.kwargs) 2024-09-21T03:45:01.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-21T03:45:01.039 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: self.prune_snapshots(sched) 2024-09-21T03:45:01.039 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-21T03:45:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.253+0000 7f89977fe640 1 -- 172.21.15.112:0/4204128294 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7f89980d79b0 con 0x7f89a8102ae0 2024-09-21T03:45:01.255 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-09-21T03:45:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.255+0000 7f89ac911640 1 -- 172.21.15.112:0/4204128294 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89900b8910 msgr2=0x7f89900bae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.255+0000 7f89ac911640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89900b8910 0x7f89900bae20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f89a8079e30 tx=0x7f899c00b390 comp rx=0 tx=0).stop 2024-09-21T03:45:01.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.255+0000 7f89ac911640 1 -- 172.21.15.112:0/4204128294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a8102ae0 msgr2=0x7f89a81993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:45:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.255+0000 7f89ac911640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a8102ae0 0x7f89a81993b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8998005940 tx=0x7f89980042f0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.255+0000 7f89ac911640 1 -- 172.21.15.112:0/4204128294 shutdown_connections 2024-09-21T03:45:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.255+0000 7f89ac911640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.112:6800/4024464582,v1:172.21.15.112:6801/4024464582] conn(0x7f89900b8910 0x7f89900bae20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.255+0000 7f89ac911640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89a8108d20 0x7f89a819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.255+0000 7f89ac911640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89a81083d0 0x7f89a8199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.255+0000 7f89ac911640 1 --2- 172.21.15.112:0/4204128294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89a8102ae0 0x7f89a81993b0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:45:01.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.255+0000 7f89ac911640 1 -- 172.21.15.112:0/4204128294 >> 172.21.15.112:0/4204128294 conn(0x7f89a80797b0 msgr2=0x7f89a8106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:45:01.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.256+0000 7f89ac911640 1 -- 172.21.15.112:0/4204128294 shutdown_connections 2024-09-21T03:45:01.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T03:45:01.256+0000 7f89ac911640 1 -- 172.21.15.112:0/4204128294 wait complete. 2024-09-21T03:45:01.537 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:01 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/4204128294' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:45:01.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:01 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/4204128294' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:45:01.679 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:01 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/4204128294' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:45:02.065 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:45:02.091 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:45:02.119 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:45:02.147 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T03:45:02.148 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:45:02.162 DEBUG:teuthology.orchestra.run.smithi187:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:45:02.177 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:45:02.203 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T03:45:02.203 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T03:45:02.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:45:02.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:02 smithi112 systemd[1]: Stopping Ceph mon.a for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:02.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:02 smithi112 ceph-mon[31442]: pgmap v820: 65 pgs: 65 active+clean; 309 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:45:02.287 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:02 smithi112 ceph-mon[31442]: from='client.? 172.21.15.112:0/4204128294' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:45:02.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:02 smithi112 ceph-mon[31442]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T03:45:02.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:02 smithi203 ceph-mon[34723]: pgmap v820: 65 pgs: 65 active+clean; 309 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:45:02.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:02 smithi203 ceph-mon[34723]: from='client.? 172.21.15.112:0/4204128294' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:45:02.411 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:02 smithi203 ceph-mon[34723]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T03:45:02.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:02 smithi187 ceph-mon[34603]: pgmap v820: 65 pgs: 65 active+clean; 309 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:45:02.429 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:02 smithi187 ceph-mon[34603]: from='client.? 172.21.15.112:0/4204128294' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:45:02.430 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:02 smithi187 ceph-mon[34603]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T03:45:02.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:02 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a[31438]: 2024-09-21T03:45:02.502+0000 7f49e01c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:02.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:02 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a[31438]: 2024-09-21T03:45:02.502+0000 7f49e01c2640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T03:45:02.787 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:02 smithi112 podman[111520]: 2024-09-21 03:45:02.748595245 +0000 UTC m=+0.361681499 container died aaae2dc90557dd4784f23ccfde80e59ce857bda592cbf99a86f9d71ac2780e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:45:03.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:03 smithi112 podman[111520]: 2024-09-21 03:45:03.254575994 +0000 UTC m=+0.867662289 container cleanup aaae2dc90557dd4784f23ccfde80e59ce857bda592cbf99a86f9d71ac2780e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-21T03:45:03.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Sep 21 03:45:03 smithi112 bash[111520]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:45:03.643 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T03:45:03.696 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:03.697 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T03:45:03.697 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T03:45:03.697 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.b 2024-09-21T03:45:03.993 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:03 smithi187 systemd[1]: Stopping Ceph mon.b for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:04.360 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:03 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-b[34599]: 2024-09-21T03:45:03.990+0000 7f842c277640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:04.361 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:03 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-b[34599]: 2024-09-21T03:45:03.990+0000 7f842c277640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:45:04.361 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:04 smithi187 podman[68656]: 2024-09-21 03:45:04.072725792 +0000 UTC m=+0.186792732 container died 8c10b0113f0a4465f7a3b6274ba47560909db3aa943b5dcc8541945515a549e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:45:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:04 smithi187 podman[68656]: 2024-09-21 03:45:04.458296577 +0000 UTC m=+0.572363515 container cleanup 8c10b0113f0a4465f7a3b6274ba47560909db3aa943b5dcc8541945515a549e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T03:45:04.680 INFO:journalctl@ceph.mon.b.smithi187.stdout:Sep 21 03:45:04 smithi187 bash[68656]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-b 2024-09-21T03:45:04.883 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T03:45:04.930 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:04.930 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T03:45:04.930 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T03:45:04.931 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.c 2024-09-21T03:45:05.271 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:04 smithi203 systemd[1]: Stopping Ceph mon.c for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:05 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-c[34706]: 2024-09-21T03:45:05.268+0000 7fb8c10fc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:05 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-c[34706]: 2024-09-21T03:45:05.268+0000 7fb8c10fc640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:45:05.661 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:05 smithi203 podman[68985]: 2024-09-21 03:45:05.401926789 +0000 UTC m=+0.249463618 container died 27cd630757a959bbff34a9d186ba0f1e6218ae7ad57eab91e6c8c7fee8c4f774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-c, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T03:45:06.112 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:05 smithi203 podman[68985]: 2024-09-21 03:45:05.792632384 +0000 UTC m=+0.640169203 container cleanup 27cd630757a959bbff34a9d186ba0f1e6218ae7ad57eab91e6c8c7fee8c4f774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T03:45:06.113 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 21 03:45:05 smithi203 bash[68985]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mon-c 2024-09-21T03:45:06.287 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T03:45:06.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:06.355 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T03:45:06.355 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T03:45:06.355 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:45:06.696 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:06 smithi112 systemd[1]: Stopping Ceph mgr.x for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:07.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:06 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:45:06.692+0000 7f38b49ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:07.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:06 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x[31669]: 2024-09-21T03:45:06.692+0000 7f38b49ea640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T03:45:07.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:06 smithi112 podman[111641]: 2024-09-21 03:45:06.735350347 +0000 UTC m=+0.197409825 container died 0d4f99a98dc822d3cc18a024cf8b396cd6624acded96bbbcc533320e99e75179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:45:07.480 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:07 smithi112 podman[111641]: 2024-09-21 03:45:07.235411661 +0000 UTC m=+0.697471134 container cleanup 0d4f99a98dc822d3cc18a024cf8b396cd6624acded96bbbcc533320e99e75179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T03:45:07.481 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Sep 21 03:45:07 smithi112 bash[111641]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-x 2024-09-21T03:45:07.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T03:45:07.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:07.645 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T03:45:07.645 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T03:45:07.645 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.y 2024-09-21T03:45:07.930 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:45:07 smithi187 systemd[1]: Stopping Ceph mgr.y for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:08.381 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:45:07 smithi187 podman[68779]: 2024-09-21 03:45:07.967272566 +0000 UTC m=+0.129543602 container died 6187016459bf7b09211af00f60bbd8ae163353ca4487e793e8dd84d99aa4fa60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-y, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:45:08.679 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:45:08 smithi187 podman[68779]: 2024-09-21 03:45:08.471492864 +0000 UTC m=+0.633763891 container cleanup 6187016459bf7b09211af00f60bbd8ae163353ca4487e793e8dd84d99aa4fa60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-y, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:45:08.680 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Sep 21 03:45:08 smithi187 bash[68779]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-y 2024-09-21T03:45:08.946 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T03:45:08.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:08.980 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T03:45:08.980 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T03:45:08.981 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.z 2024-09-21T03:45:09.321 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:45:09 smithi203 systemd[1]: Stopping Ceph mgr.z for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:09.627 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:45:09 smithi203 podman[69103]: 2024-09-21 03:45:09.319062237 +0000 UTC m=+0.146761313 container died 77c02ea8cb23adc0e3f55d7fbeb67027fc228cfead46fb620140c43ffed181df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-z, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:45:09.911 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:45:09 smithi203 podman[69103]: 2024-09-21 03:45:09.706427362 +0000 UTC m=+0.534126440 container cleanup 77c02ea8cb23adc0e3f55d7fbeb67027fc228cfead46fb620140c43ffed181df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-z, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T03:45:09.911 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 21 03:45:09 smithi203 bash[69103]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mgr-z 2024-09-21T03:45:10.089 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T03:45:10.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:10.123 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T03:45:10.124 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T03:45:10.124 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.0 2024-09-21T03:45:10.433 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:10 smithi112 systemd[1]: Stopping Ceph osd.0 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:10.787 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:10 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0[48575]: 2024-09-21T03:45:10.431+0000 7f16ddf63640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:10.788 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:10 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0[48575]: 2024-09-21T03:45:10.431+0000 7f16ddf63640 -1 osd.0 99 *** Got signal Terminated *** 2024-09-21T03:45:10.788 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:10 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0[48575]: 2024-09-21T03:45:10.431+0000 7f16ddf63640 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:45:16.287 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:16 smithi112 podman[111761]: 2024-09-21 03:45:16.010267523 +0000 UTC m=+5.690373933 container died 36ccb388144724c69cc9af664b88b7602a5be151f15b14825ed074ec0b2c452d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T03:45:16.610 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:16 smithi112 podman[111761]: 2024-09-21 03:45:16.407170903 +0000 UTC m=+6.087277320 container cleanup 36ccb388144724c69cc9af664b88b7602a5be151f15b14825ed074ec0b2c452d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T03:45:16.610 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:16 smithi112 bash[111761]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0 2024-09-21T03:45:17.037 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:16 smithi112 podman[111776]: 2024-09-21 03:45:16.60860056 +0000 UTC m=+0.594608407 container remove 36ccb388144724c69cc9af664b88b7602a5be151f15b14825ed074ec0b2c452d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T03:45:17.297 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:17 smithi112 podman[111879]: 2024-09-21 03:45:16.943932605 +0000 UTC m=+0.020247799 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:45:17.297 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:17 smithi112 podman[111879]: 2024-09-21 03:45:17.059789253 +0000 UTC m=+0.136104429 container create 3935458371c7de567dc2f00188a207cd1c82b278fecb280c345dfbbe10dc19d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T03:45:17.297 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:17 smithi112 podman[111879]: 2024-09-21 03:45:17.295739018 +0000 UTC m=+0.372054208 container init 3935458371c7de567dc2f00188a207cd1c82b278fecb280c345dfbbe10dc19d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T03:45:17.787 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:17 smithi112 podman[111879]: 2024-09-21 03:45:17.299301472 +0000 UTC m=+0.375616651 container start 3935458371c7de567dc2f00188a207cd1c82b278fecb280c345dfbbe10dc19d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T03:45:17.788 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:17 smithi112 podman[111879]: 2024-09-21 03:45:17.420884172 +0000 UTC m=+0.497199376 container attach 3935458371c7de567dc2f00188a207cd1c82b278fecb280c345dfbbe10dc19d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:45:17.788 INFO:journalctl@ceph.osd.0.smithi112.stdout:Sep 21 03:45:17 smithi112 podman[111879]: 2024-09-21 03:45:17.493461045 +0000 UTC m=+0.569776222 container died 3935458371c7de567dc2f00188a207cd1c82b278fecb280c345dfbbe10dc19d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T03:45:18.159 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T03:45:18.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:18.200 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T03:45:18.200 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T03:45:18.200 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.1 2024-09-21T03:45:18.511 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:18 smithi187 systemd[1]: Stopping Ceph osd.1 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:18.930 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:18 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1[39251]: 2024-09-21T03:45:18.508+0000 7f8301513640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:18.930 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:18 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1[39251]: 2024-09-21T03:45:18.508+0000 7f8301513640 -1 osd.1 99 *** Got signal Terminated *** 2024-09-21T03:45:18.930 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:18 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1[39251]: 2024-09-21T03:45:18.508+0000 7f8301513640 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:45:24.336 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:24 smithi187 podman[68899]: 2024-09-21 03:45:24.07453665 +0000 UTC m=+5.688381848 container died 1088e34bc0e2cdca80ec306e1b66903102d2b3db00f8bea7a8338fe167c51a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T03:45:24.680 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:24 smithi187 podman[68899]: 2024-09-21 03:45:24.442046516 +0000 UTC m=+6.055891703 container cleanup 1088e34bc0e2cdca80ec306e1b66903102d2b3db00f8bea7a8338fe167c51a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T03:45:24.680 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:24 smithi187 bash[68899]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1 2024-09-21T03:45:25.062 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:24 smithi187 podman[68914]: 2024-09-21 03:45:24.710091667 +0000 UTC m=+0.635804285 container remove 1088e34bc0e2cdca80ec306e1b66903102d2b3db00f8bea7a8338fe167c51a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T03:45:25.340 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:25 smithi187 podman[69003]: 2024-09-21 03:45:24.963876791 +0000 UTC m=+0.015900430 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:45:25.340 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:25 smithi187 podman[69003]: 2024-09-21 03:45:25.086140715 +0000 UTC m=+0.138164354 container create d20b9dc9a389b04685d161b48f632df0f3b6c440a2f95eacf374068137ecc0b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T03:45:25.680 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:25 smithi187 podman[69003]: 2024-09-21 03:45:25.33891254 +0000 UTC m=+0.390936178 container init d20b9dc9a389b04685d161b48f632df0f3b6c440a2f95eacf374068137ecc0b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T03:45:25.680 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:25 smithi187 podman[69003]: 2024-09-21 03:45:25.342274001 +0000 UTC m=+0.394297642 container start d20b9dc9a389b04685d161b48f632df0f3b6c440a2f95eacf374068137ecc0b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T03:45:25.680 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:25 smithi187 podman[69003]: 2024-09-21 03:45:25.413864812 +0000 UTC m=+0.465888462 container attach d20b9dc9a389b04685d161b48f632df0f3b6c440a2f95eacf374068137ecc0b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:45:25.681 INFO:journalctl@ceph.osd.1.smithi187.stdout:Sep 21 03:45:25 smithi187 podman[69003]: 2024-09-21 03:45:25.535011727 +0000 UTC m=+0.587035369 container died d20b9dc9a389b04685d161b48f632df0f3b6c440a2f95eacf374068137ecc0b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T03:45:26.128 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T03:45:26.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:26.167 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T03:45:26.168 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T03:45:26.168 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.2 2024-09-21T03:45:26.661 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:26 smithi203 systemd[1]: Stopping Ceph osd.2 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:27.161 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:26 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2[39869]: 2024-09-21T03:45:26.735+0000 7f721de66640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:27.161 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:26 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2[39869]: 2024-09-21T03:45:26.735+0000 7f721de66640 -1 osd.2 99 *** Got signal Terminated *** 2024-09-21T03:45:27.161 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:26 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2[39869]: 2024-09-21T03:45:26.735+0000 7f721de66640 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:45:32.539 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:32 smithi203 podman[69230]: 2024-09-21 03:45:32.284483757 +0000 UTC m=+5.921861304 container died cfb905ef1a412a6d549e08c3f729666144ff66a51ebe32aac520ddcc4808d1b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T03:45:32.911 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:32 smithi203 podman[69230]: 2024-09-21 03:45:32.610639111 +0000 UTC m=+6.248016658 container cleanup cfb905ef1a412a6d549e08c3f729666144ff66a51ebe32aac520ddcc4808d1b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T03:45:32.911 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:32 smithi203 bash[69230]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2 2024-09-21T03:45:32.911 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:32 smithi203 podman[69257]: 2024-09-21 03:45:32.761935621 +0000 UTC m=+0.475489401 container remove cfb905ef1a412a6d549e08c3f729666144ff66a51ebe32aac520ddcc4808d1b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T03:45:33.483 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:33 smithi203 podman[69348]: 2024-09-21 03:45:33.079966143 +0000 UTC m=+0.015860284 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:45:33.483 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:33 smithi203 podman[69348]: 2024-09-21 03:45:33.254407694 +0000 UTC m=+0.190301814 container create f3617fcaa179520b2227f3ae2c153d0ea8daba18863913f24aac22496b326003 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2-deactivate, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T03:45:33.483 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:33 smithi203 podman[69348]: 2024-09-21 03:45:33.481888198 +0000 UTC m=+0.417782325 container init f3617fcaa179520b2227f3ae2c153d0ea8daba18863913f24aac22496b326003 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T03:45:33.793 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:33 smithi203 podman[69348]: 2024-09-21 03:45:33.485373055 +0000 UTC m=+0.421267180 container start f3617fcaa179520b2227f3ae2c153d0ea8daba18863913f24aac22496b326003 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T03:45:33.794 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:33 smithi203 podman[69348]: 2024-09-21 03:45:33.567359177 +0000 UTC m=+0.503253322 container attach f3617fcaa179520b2227f3ae2c153d0ea8daba18863913f24aac22496b326003 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T03:45:34.080 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 21 03:45:33 smithi203 podman[69348]: 2024-09-21 03:45:33.792276438 +0000 UTC m=+0.728170569 container died f3617fcaa179520b2227f3ae2c153d0ea8daba18863913f24aac22496b326003 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:45:34.366 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T03:45:34.408 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:34.408 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T03:45:34.408 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T03:45:34.409 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.3 2024-09-21T03:45:34.738 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:34 smithi112 systemd[1]: Stopping Ceph osd.3 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:35.038 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:34 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3[54831]: 2024-09-21T03:45:34.735+0000 7fc46d603640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:35.039 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:34 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3[54831]: 2024-09-21T03:45:34.735+0000 7fc46d603640 -1 osd.3 99 *** Got signal Terminated *** 2024-09-21T03:45:35.039 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:34 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3[54831]: 2024-09-21T03:45:34.735+0000 7fc46d603640 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:45:37.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:37 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:37.367+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:38.537 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:38 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3[54831]: 2024-09-21T03:45:38.077+0000 7fc465be6640 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:13.684621+0000 front 2024-09-21T03:45:13.684649+0000 (oldest deadline 2024-09-21T03:45:37.184783+0000) 2024-09-21T03:45:38.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:38 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:38.317+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:39.287 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:39 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3[54831]: 2024-09-21T03:45:39.029+0000 7fc465be6640 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:13.684621+0000 front 2024-09-21T03:45:13.684649+0000 (oldest deadline 2024-09-21T03:45:37.184783+0000) 2024-09-21T03:45:39.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:39 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:39.413+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:39.662 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:39 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:39.311+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:39.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:39 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:39.275+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:40.446 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:40 smithi112 podman[111984]: 2024-09-21 03:45:40.156048374 +0000 UTC m=+5.547518512 container died 52a95e62ac65db6ed3b8d43d816d735d110f219c389629d86ac69624f3b38349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T03:45:40.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:40 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:40.407+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:40.661 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:40 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:40.329+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:40.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:40 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:40.226+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:40.787 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:40 smithi112 podman[111984]: 2024-09-21 03:45:40.636268506 +0000 UTC m=+6.027738631 container cleanup 52a95e62ac65db6ed3b8d43d816d735d110f219c389629d86ac69624f3b38349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T03:45:40.787 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:40 smithi112 bash[111984]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3 2024-09-21T03:45:41.244 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:40 smithi112 podman[111999]: 2024-09-21 03:45:40.921334885 +0000 UTC m=+0.758996036 container remove 52a95e62ac65db6ed3b8d43d816d735d110f219c389629d86ac69624f3b38349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T03:45:41.245 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:41 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:41.019+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:41.537 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:41 smithi112 podman[112090]: 2024-09-21 03:45:41.145935866 +0000 UTC m=+0.015496015 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:45:41.537 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:41 smithi112 podman[112090]: 2024-09-21 03:45:41.246967631 +0000 UTC m=+0.116527788 container create dcc2c74a420111c77e0e58852c6b7ae2732c95e808fd7f3c6ac83ad093a45237 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:45:41.538 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:41 smithi112 podman[112090]: 2024-09-21 03:45:41.483063044 +0000 UTC m=+0.352623201 container init dcc2c74a420111c77e0e58852c6b7ae2732c95e808fd7f3c6ac83ad093a45237 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:45:41.538 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:41 smithi112 podman[112090]: 2024-09-21 03:45:41.486361967 +0000 UTC m=+0.355922117 container start dcc2c74a420111c77e0e58852c6b7ae2732c95e808fd7f3c6ac83ad093a45237 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T03:45:41.661 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:41 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:41.327+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:41.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:41 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:41.455+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:41.679 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:41 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:41.207+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:41.979 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:41 smithi112 podman[112090]: 2024-09-21 03:45:41.566346166 +0000 UTC m=+0.435906317 container attach dcc2c74a420111c77e0e58852c6b7ae2732c95e808fd7f3c6ac83ad093a45237 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-21T03:45:41.979 INFO:journalctl@ceph.osd.3.smithi112.stdout:Sep 21 03:45:41 smithi112 podman[112090]: 2024-09-21 03:45:41.677070611 +0000 UTC m=+0.546630763 container died dcc2c74a420111c77e0e58852c6b7ae2732c95e808fd7f3c6ac83ad093a45237 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T03:45:41.979 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:41 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:41.566+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:42.287 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:42 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:42.024+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:42.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T03:45:42.389 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:42.389 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T03:45:42.389 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T03:45:42.389 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.4 2024-09-21T03:45:42.429 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:42 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:42.168+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:42.661 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:42 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:42.370+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:42.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:42 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:42.466+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:42.930 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:42 smithi187 systemd[1]: Stopping Ceph osd.4 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:42.930 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:42 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4[45392]: 2024-09-21T03:45:42.677+0000 7f5dee7c4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:42.930 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:42 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4[45392]: 2024-09-21T03:45:42.677+0000 7f5dee7c4640 -1 osd.4 99 *** Got signal Terminated *** 2024-09-21T03:45:42.930 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:42 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4[45392]: 2024-09-21T03:45:42.677+0000 7f5dee7c4640 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:45:43.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:42 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:42.594+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:43.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:43 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:43.041+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:43.661 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:43 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:43.379+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:43.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:43 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:43.478+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:43.679 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:43 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:43.197+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:44.012 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:43 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:43.611+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:44.287 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:44 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:44.010+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:44.429 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:44 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:44.165+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:44.911 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:44 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:44.412+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:44.912 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:44 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:44.435+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:45.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:44 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:44.661+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:45.179 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:44 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4[45392]: 2024-09-21T03:45:44.867+0000 7f5de6da7640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.398858+0000 front 2024-09-21T03:45:21.398834+0000 (oldest deadline 2024-09-21T03:45:44.299078+0000) 2024-09-21T03:45:45.429 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:45 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:45.177+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:45.442 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:45 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:45.394+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:45.442 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:45 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:45.168+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:45.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:45 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:45.051+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:45.829 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:45 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:45.499+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:45.911 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:45 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:45.440+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:46.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:45 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:45.694+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:46.180 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:45 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4[45392]: 2024-09-21T03:45:45.826+0000 7f5de6da7640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.398858+0000 front 2024-09-21T03:45:21.398834+0000 (oldest deadline 2024-09-21T03:45:44.299078+0000) 2024-09-21T03:45:46.307 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:46 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:46.090+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:46.307 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:46 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:46.090+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:46.461 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:46 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:46.192+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:46.467 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:46 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:46.420+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:46.468 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:46 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:46.191+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:46.852 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:46 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:46.459+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:46.911 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:46 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:46.466+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:47.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:46 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:46.702+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:47.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:46 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:46.702+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:47.179 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:46 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4[45392]: 2024-09-21T03:45:46.850+0000 7f5de6da7640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.398858+0000 front 2024-09-21T03:45:21.398834+0000 (oldest deadline 2024-09-21T03:45:44.299078+0000) 2024-09-21T03:45:47.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:47 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:47.153+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:47.497 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:47 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:47.214+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:47.498 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:47 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:47.214+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:47.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:47 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:47.067+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:47.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:47 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:47.067+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:47.911 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:47 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:47.488+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:47.911 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:47 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:47.415+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:47.929 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:47 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:47.496+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:48.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:47 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:47.741+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:48.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:47 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:47.741+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:48.440 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:48 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:48.167+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:48.470 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:48 smithi187 podman[69117]: 2024-09-21 03:45:48.209297057 +0000 UTC m=+5.628889639 container died 478dd9d59f1db7e49ae6867f180a11e712b4513db5519a0f51d6093763cdb722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T03:45:48.470 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:48 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:48.252+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:48.470 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:48 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:48.252+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:48.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:48 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:48.097+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:48.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:48 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:48.097+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:48.855 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:48 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:48.468+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:48.855 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:48 smithi187 podman[69117]: 2024-09-21 03:45:48.569529128 +0000 UTC m=+5.989121700 container cleanup 478dd9d59f1db7e49ae6867f180a11e712b4513db5519a0f51d6093763cdb722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T03:45:48.856 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:48 smithi187 bash[69117]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4 2024-09-21T03:45:48.911 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:48 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:48.438+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:48.912 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:48 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:48.478+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:49.127 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:48 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:48.787+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:49.127 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:48 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:48.787+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:49.179 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:48 smithi187 podman[69133]: 2024-09-21 03:45:48.854043355 +0000 UTC m=+0.642170310 container remove 478dd9d59f1db7e49ae6867f180a11e712b4513db5519a0f51d6093763cdb722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:45:49.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:49 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:49.123+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:49.476 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:49 smithi187 podman[69226]: 2024-09-21 03:45:49.179819035 +0000 UTC m=+0.110456531 container create 3b744f7cd736576d0789e48a6027b7abeaca44d1f3223e5bba3505f4c28a59cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T03:45:49.477 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:49 smithi187 podman[69226]: 2024-09-21 03:45:49.084894524 +0000 UTC m=+0.015531975 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:45:49.477 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:49 smithi187 podman[69226]: 2024-09-21 03:45:49.382592362 +0000 UTC m=+0.313229814 container init 3b744f7cd736576d0789e48a6027b7abeaca44d1f3223e5bba3505f4c28a59cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T03:45:49.477 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:49 smithi187 podman[69226]: 2024-09-21 03:45:49.385832555 +0000 UTC m=+0.316470017 container start 3b744f7cd736576d0789e48a6027b7abeaca44d1f3223e5bba3505f4c28a59cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T03:45:49.477 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:49 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:49.229+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:49.477 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:49 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:49.229+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:49.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:49 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:49.124+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:49.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:49 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:49.124+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:49.911 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:49 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:49.452+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:49.911 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:49 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:49.447+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:49.912 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:49 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:49.447+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:18.634715+0000 front 2024-09-21T03:45:18.634915+0000 (oldest deadline 2024-09-21T03:45:49.235318+0000) 2024-09-21T03:45:49.930 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:49 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:49.473+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:49.930 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:49 smithi187 podman[69226]: 2024-09-21 03:45:49.482391081 +0000 UTC m=+0.413028552 container attach 3b744f7cd736576d0789e48a6027b7abeaca44d1f3223e5bba3505f4c28a59cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T03:45:49.931 INFO:journalctl@ceph.osd.4.smithi187.stdout:Sep 21 03:45:49 smithi187 podman[69226]: 2024-09-21 03:45:49.57376258 +0000 UTC m=+0.504400032 container died 3b744f7cd736576d0789e48a6027b7abeaca44d1f3223e5bba3505f4c28a59cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T03:45:50.158 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:49 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:49.826+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:50.158 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:49 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:49.826+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:50.229 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:50 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:50.182+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:50.229 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:50 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:50.182+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:50.253 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T03:45:50.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:50.334 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T03:45:50.334 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T03:45:50.334 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.5 2024-09-21T03:45:50.453 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:50 smithi203 systemd[1]: Stopping Ceph osd.5 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:50.454 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:50 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:50.409+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:50.454 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:50 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:50.409+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:45:50.454 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:50 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:50.162+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:50.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:50 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:50.156+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:50.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:50 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:50.156+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:50.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:50 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:50.441+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:50.911 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:50 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:50.451+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:50.911 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:50 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:50.451+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:18.634715+0000 front 2024-09-21T03:45:18.634915+0000 (oldest deadline 2024-09-21T03:45:49.235318+0000) 2024-09-21T03:45:50.911 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:50 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:50.640+0000 7f068854d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:50.912 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:50 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:50.640+0000 7f068854d640 -1 osd.5 99 *** Got signal Terminated *** 2024-09-21T03:45:50.912 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:50 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:50.640+0000 7f068854d640 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:45:51.144 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:50 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:50.844+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:51.145 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:50 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:50.844+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:51.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:51 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:51.136+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:51.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:51 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:51.361+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:51.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:51 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:51.361+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:45:51.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:51 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:51.142+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:51.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:51 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:51.142+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:51.679 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:51 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:51.394+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:51.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:51 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:51.218+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:51.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:51 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:51.218+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:51.911 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:51 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:51.409+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:51.911 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:51 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:51.409+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:18.634715+0000 front 2024-09-21T03:45:18.634915+0000 (oldest deadline 2024-09-21T03:45:49.235318+0000) 2024-09-21T03:45:52.131 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:51 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:51.863+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:52.131 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:51 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:51.863+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:52.435 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:52 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:52.182+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:52.435 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:52 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:52.336+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:52.435 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:52 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:52.336+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:45:52.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:52 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:52.129+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:52.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:52 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:52.129+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:52.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:52 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:52.353+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:52.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:52 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:52.241+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:52.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:52 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:52.241+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:52.911 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:52 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:52.433+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:52.911 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:52 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:52.433+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:18.634715+0000 front 2024-09-21T03:45:18.634915+0000 (oldest deadline 2024-09-21T03:45:49.235318+0000) 2024-09-21T03:45:53.129 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:52 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:52.826+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:53.129 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:52 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:52.826+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:53.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:53 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:53.126+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:53.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:53 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:53.126+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:53.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:53 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:53.218+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:53.662 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:53 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:53.391+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:53.662 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:53 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:53.391+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:18.634715+0000 front 2024-09-21T03:45:18.634915+0000 (oldest deadline 2024-09-21T03:45:49.235318+0000) 2024-09-21T03:45:53.662 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:53 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:53.391+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.235655+0000 front 2024-09-21T03:45:29.235727+0000 (oldest deadline 2024-09-21T03:45:53.335728+0000) 2024-09-21T03:45:53.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:53 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:53.307+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:53.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:53 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:53.307+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:45:53.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:53 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:53.307+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:45:53.679 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:53 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:53.376+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:53.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:53 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:53.196+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:53.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:53 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:53.196+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:54.121 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:53 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:53.799+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:54.122 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:53 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:53.799+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:54.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:54 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:54.361+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:54.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:54 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:54.361+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:45:54.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:54 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:54.174+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:54.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:54 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:54.174+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:54.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:54 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:54.174+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:45:54.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:54 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:54.119+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:54.538 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:54 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:54.119+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:54.538 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:54 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:54.119+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:31.303705+0000 front 2024-09-21T03:45:31.303863+0000 (oldest deadline 2024-09-21T03:45:53.603657+0000) 2024-09-21T03:45:54.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:54 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:54.234+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:54.661 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:54 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:54.364+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:54.662 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:54 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:54.364+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:18.634715+0000 front 2024-09-21T03:45:18.634915+0000 (oldest deadline 2024-09-21T03:45:49.235318+0000) 2024-09-21T03:45:54.662 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:54 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:54.364+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.235655+0000 front 2024-09-21T03:45:29.235727+0000 (oldest deadline 2024-09-21T03:45:53.335728+0000) 2024-09-21T03:45:54.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:54 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:54.307+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:54.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:54 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:54.307+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:45:54.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:54 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:54.307+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:45:55.146 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:54 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:54.843+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:55.146 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:54 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:54.843+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:55.429 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:55.365+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:55.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:55.365+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:45:55.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:55.130+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:55.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:55.130+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:55.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:55 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:55.130+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:45:55.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:55 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:55.144+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:55.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:55 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:55.144+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:55.538 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:55 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:55.144+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:31.303705+0000 front 2024-09-21T03:45:31.303863+0000 (oldest deadline 2024-09-21T03:45:53.603657+0000) 2024-09-21T03:45:55.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:55.204+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:55.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:55.204+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:45:55.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:55.301+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:55.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:55.301+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:45:55.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:55.301+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:45:55.662 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:55.393+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.134071+0000 front 2024-09-21T03:45:15.134044+0000 (oldest deadline 2024-09-21T03:45:38.634182+0000) 2024-09-21T03:45:55.662 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:55.393+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:18.634715+0000 front 2024-09-21T03:45:18.634915+0000 (oldest deadline 2024-09-21T03:45:49.235318+0000) 2024-09-21T03:45:55.663 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:55 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5[46434]: 2024-09-21T03:45:55.393+0000 7f0684b38640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.235655+0000 front 2024-09-21T03:45:29.235727+0000 (oldest deadline 2024-09-21T03:45:53.335728+0000) 2024-09-21T03:45:56.167 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:55 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:55.826+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:56.168 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:55 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:55.826+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:56.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:56 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:56.398+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:56.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:56 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:56.398+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:45:56.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:56 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:56.167+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:56.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:56 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:56.167+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:56.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:56 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:56.167+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:45:56.460 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:56 smithi203 podman[69449]: 2024-09-21 03:45:56.194423434 +0000 UTC m=+5.657304585 container died 2bc8155096eb8421ce433e3069b8b02b70a85659c5bceec7bd0ba571d9d6cda4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T03:45:56.461 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:56 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:56.344+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:56.461 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:56 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:56.344+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:45:56.461 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:56 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:56.344+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:45:56.461 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:56 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:56.179+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:56.461 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:56 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:56.179+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:45:56.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:56 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:56.165+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:56.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:56 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:56.165+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:56.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:56 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:56.165+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:31.303705+0000 front 2024-09-21T03:45:31.303863+0000 (oldest deadline 2024-09-21T03:45:53.603657+0000) 2024-09-21T03:45:56.852 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:56 smithi203 podman[69449]: 2024-09-21 03:45:56.582892987 +0000 UTC m=+6.045774141 container cleanup 2bc8155096eb8421ce433e3069b8b02b70a85659c5bceec7bd0ba571d9d6cda4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:45:56.852 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:56 smithi203 bash[69449]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5 2024-09-21T03:45:57.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:56 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:56.875+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:57.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:56 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:56.875+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:57.161 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:56 smithi203 podman[69464]: 2024-09-21 03:45:56.8509858 +0000 UTC m=+0.649866529 container remove 2bc8155096eb8421ce433e3069b8b02b70a85659c5bceec7bd0ba571d9d6cda4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T03:45:57.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:57 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:57.391+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:57.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:57 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:57.391+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:45:57.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:57 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:57.167+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:57.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:57 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:57.167+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:57.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:57 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:57.167+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:45:57.505 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:57 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:57.209+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:57.506 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:57 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:57.209+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:45:57.506 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:57 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:57.366+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:57.506 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:57 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:57.366+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:45:57.506 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:57 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:57.366+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:45:57.507 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:57 smithi203 podman[69555]: 2024-09-21 03:45:57.103800257 +0000 UTC m=+0.016161688 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:45:57.507 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:57 smithi203 podman[69555]: 2024-09-21 03:45:57.210398703 +0000 UTC m=+0.122760162 container create 65a3afb3ed5843a9802ddfe686ab8dccb3b98db24726d46c00b0084402d61a94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T03:45:57.507 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:57 smithi203 podman[69555]: 2024-09-21 03:45:57.429514204 +0000 UTC m=+0.341875628 container init 65a3afb3ed5843a9802ddfe686ab8dccb3b98db24726d46c00b0084402d61a94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T03:45:57.507 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:57 smithi203 podman[69555]: 2024-09-21 03:45:57.432894865 +0000 UTC m=+0.345256284 container start 65a3afb3ed5843a9802ddfe686ab8dccb3b98db24726d46c00b0084402d61a94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T03:45:57.507 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:57 smithi203 podman[69555]: 2024-09-21 03:45:57.504336184 +0000 UTC m=+0.416697614 container attach 65a3afb3ed5843a9802ddfe686ab8dccb3b98db24726d46c00b0084402d61a94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:45:57.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:57 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:57.185+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:57.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:57 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:57.185+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:57.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:57 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:57.185+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:31.303705+0000 front 2024-09-21T03:45:31.303863+0000 (oldest deadline 2024-09-21T03:45:53.603657+0000) 2024-09-21T03:45:57.909 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 21 03:45:57 smithi203 podman[69555]: 2024-09-21 03:45:57.62242321 +0000 UTC m=+0.534784644 container died 65a3afb3ed5843a9802ddfe686ab8dccb3b98db24726d46c00b0084402d61a94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T03:45:58.168 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:57 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:57.916+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:58.169 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:57 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:57.916+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:58.169 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:57 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:57.916+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:45:58.182 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T03:45:58.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:45:58.221 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T03:45:58.221 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T03:45:58.222 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.6 2024-09-21T03:45:58.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:58 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:58.355+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:58.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:58 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:58.355+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:45:58.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:58 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:58.355+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:45:58.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:58 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:58.188+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:58.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:58 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:58.188+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:45:58.429 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:58 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:58.404+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:58.429 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:58 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:58.404+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:45:58.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:58 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:58.174+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:58.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:58 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:58.174+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:58.430 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:58 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:58.174+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:45:58.534 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:58 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:58.167+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:58.534 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:58 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:58.167+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:58.534 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:58 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:58.167+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:31.303705+0000 front 2024-09-21T03:45:31.303863+0000 (oldest deadline 2024-09-21T03:45:53.603657+0000) 2024-09-21T03:45:58.534 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:58 smithi112 systemd[1]: Stopping Ceph osd.6 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:45:58.787 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:58 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:58.531+0000 7f32ccf06640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:45:58.787 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:58 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:58.531+0000 7f32ccf06640 -1 osd.6 99 *** Got signal Terminated *** 2024-09-21T03:45:58.787 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:58 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:58.531+0000 7f32ccf06640 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:45:59.217 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:58 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:58.872+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:45:59.218 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:58 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:58.872+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:45:59.218 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:58 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:58.872+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:45:59.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:59 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:59.215+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:45:59.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:59 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:59.215+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:45:59.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:45:59 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:45:59.215+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:31.303705+0000 front 2024-09-21T03:45:31.303863+0000 (oldest deadline 2024-09-21T03:45:53.603657+0000) 2024-09-21T03:45:59.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:59 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:59.189+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:45:59.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:45:59 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:45:59.189+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:45:59.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:59 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:59.377+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:45:59.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:59 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:59.377+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:45:59.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:45:59 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:45:59.377+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:45:59.679 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:59 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:59.387+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:45:59.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:45:59 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:45:59.387+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:45:59.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:59 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:59.191+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:45:59.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:59 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:59.191+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:45:59.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:45:59 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:45:59.191+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:00.259 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:59 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:59.896+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:00.260 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:59 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:59.896+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:00.260 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:45:59 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:45:59.896+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:00.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:00.258+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:46:00.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:00.258+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:46:00.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:00.258+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:31.303705+0000 front 2024-09-21T03:45:31.303863+0000 (oldest deadline 2024-09-21T03:45:53.603657+0000) 2024-09-21T03:46:00.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:00 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:00.189+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:00.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:00 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:00.189+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:00.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:00 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:00.335+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:00.663 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:00 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:00.335+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:00.663 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:00 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:00.335+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:00.679 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:00 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:00.418+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:00.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:00 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:00.418+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:00.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:00 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:00.180+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:00.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:00 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:00.180+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:00.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:00 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:00.180+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:01.264 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:00.894+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:01.264 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:00.894+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:01.264 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:00 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:00.894+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:01.464 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:01 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:01.187+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:01.464 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:01 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:01.187+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:01.465 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:01 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:01.187+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:01.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:01 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:01.261+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:46:01.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:01 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:01.261+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:46:01.538 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:01 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:01.261+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:31.303705+0000 front 2024-09-21T03:45:31.303863+0000 (oldest deadline 2024-09-21T03:45:53.603657+0000) 2024-09-21T03:46:01.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:01 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:01.184+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:01.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:01 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:01.184+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:01.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:01 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:01.348+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:01.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:01 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:01.348+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:01.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:01 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:01.348+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:01.930 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:01 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:01.462+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:01.930 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:01 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:01.462+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:02.240 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:01 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:01.861+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:02.240 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:01 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:01.861+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:02.240 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:01 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:01.861+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:02.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:02 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:02.237+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:46:02.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:02 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:02.237+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:46:02.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:02 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:02.237+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:31.303705+0000 front 2024-09-21T03:45:31.303863+0000 (oldest deadline 2024-09-21T03:45:53.603657+0000) 2024-09-21T03:46:02.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:02 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:02.195+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:02.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:02 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:02.195+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:02.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:02 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:02.304+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:02.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:02 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:02.304+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:02.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:02 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:02.304+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:02.679 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:02 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:02.453+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:02.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:02 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:02.453+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:02.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:02 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:02.453+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.454637+0000 front 2024-09-21T03:45:37.454777+0000 (oldest deadline 2024-09-21T03:46:01.554820+0000) 2024-09-21T03:46:02.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:02 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:02.235+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:02.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:02 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:02.235+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:02.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:02 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:02.235+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:03.256 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:02 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:02.857+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:03.256 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:02 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:02.857+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:03.256 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:02 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:02.857+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:03.256 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:02 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:02.857+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:03.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:03 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:03.254+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:10.101009+0000 front 2024-09-21T03:45:10.101098+0000 (oldest deadline 2024-09-21T03:45:40.701807+0000) 2024-09-21T03:46:03.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:03 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:03.254+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.702569+0000 front 2024-09-21T03:45:20.702421+0000 (oldest deadline 2024-09-21T03:45:45.402428+0000) 2024-09-21T03:46:03.537 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:03 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6[61418]: 2024-09-21T03:46:03.254+0000 7f32c94f1640 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:31.303705+0000 front 2024-09-21T03:45:31.303863+0000 (oldest deadline 2024-09-21T03:45:53.603657+0000) 2024-09-21T03:46:03.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:03 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:03.203+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:03.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:03 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:03.203+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:03.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:03 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:03.203+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:03.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:03 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:03.333+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:03.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:03 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:03.334+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:03.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:03 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:03.334+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:03.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:03 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:03.472+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:03.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:03 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:03.472+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:03.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:03 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:03.472+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.454637+0000 front 2024-09-21T03:45:37.454777+0000 (oldest deadline 2024-09-21T03:46:01.554820+0000) 2024-09-21T03:46:03.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:03 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:03.248+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:03.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:03 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:03.248+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:03.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:03 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:03.248+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:03.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:03 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:03.248+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:04.287 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:04 smithi112 podman[112205]: 2024-09-21 03:46:04.084720064 +0000 UTC m=+5.663561708 container died 5b32ed713d87d661598cc2707fcab0864f9b7e774eaf5bd183aa2f533882e450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T03:46:04.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:03 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:03.896+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:04.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:03 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:03.896+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:04.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:03 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:03.896+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:04.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:03 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:03.896+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:04.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:04 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:04.225+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:04.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:04 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:04.225+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:04.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:04 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:04.225+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:04.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:04 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:04.381+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:04.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:04 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:04.382+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:04.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:04 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:04.382+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:04.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:04 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:04.426+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:04.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:04 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:04.426+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:04.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:04 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:04.426+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.454637+0000 front 2024-09-21T03:45:37.454777+0000 (oldest deadline 2024-09-21T03:46:01.554820+0000) 2024-09-21T03:46:04.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:04 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:04.238+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:04.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:04 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:04.239+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:04.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:04 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:04.239+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:04.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:04 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:04.239+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:04.788 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:04 smithi112 podman[112205]: 2024-09-21 03:46:04.598618353 +0000 UTC m=+6.177460040 container cleanup 5b32ed713d87d661598cc2707fcab0864f9b7e774eaf5bd183aa2f533882e450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:46:04.788 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:04 smithi112 bash[112205]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6 2024-09-21T03:46:05.219 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:04 smithi112 podman[112220]: 2024-09-21 03:46:04.808355382 +0000 UTC m=+0.721127206 container remove 5b32ed713d87d661598cc2707fcab0864f9b7e774eaf5bd183aa2f533882e450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T03:46:05.219 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:04 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:04.905+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:05.219 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:04 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:04.905+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:05.219 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:04 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:04.905+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:05.219 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:04 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:04.905+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:05.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:05 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:05.385+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:05.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:05 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:05.385+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:05.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:05 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:05.385+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.454637+0000 front 2024-09-21T03:45:37.454777+0000 (oldest deadline 2024-09-21T03:46:01.554820+0000) 2024-09-21T03:46:05.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:05 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:05.262+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:05.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:05 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:05.262+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:05.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:05 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:05.262+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:05.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:05 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:05.262+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:05.521 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:05 smithi112 podman[112313]: 2024-09-21 03:46:05.217425463 +0000 UTC m=+0.110460906 container create a30e947dde38a42fe0c3e2895b1ec7c217045034b4ffdd0fa790f20b00da9641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T03:46:05.521 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:05 smithi112 podman[112313]: 2024-09-21 03:46:05.122560661 +0000 UTC m=+0.015596110 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:46:05.521 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:05 smithi112 podman[112313]: 2024-09-21 03:46:05.428167323 +0000 UTC m=+0.321202762 container init a30e947dde38a42fe0c3e2895b1ec7c217045034b4ffdd0fa790f20b00da9641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T03:46:05.522 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:05 smithi112 podman[112313]: 2024-09-21 03:46:05.431940781 +0000 UTC m=+0.324976220 container start a30e947dde38a42fe0c3e2895b1ec7c217045034b4ffdd0fa790f20b00da9641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T03:46:05.522 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:05 smithi112 podman[112313]: 2024-09-21 03:46:05.519838985 +0000 UTC m=+0.412874436 container attach a30e947dde38a42fe0c3e2895b1ec7c217045034b4ffdd0fa790f20b00da9641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T03:46:05.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:05 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:05.208+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:05.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:05 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:05.208+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:05.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:05 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:05.208+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:05.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:05 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:05.350+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:05.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:05 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:05.350+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:05.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:05 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:05.350+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:05.787 INFO:journalctl@ceph.osd.6.smithi112.stdout:Sep 21 03:46:05 smithi112 podman[112313]: 2024-09-21 03:46:05.624126566 +0000 UTC m=+0.517162007 container died a30e947dde38a42fe0c3e2895b1ec7c217045034b4ffdd0fa790f20b00da9641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:46:06.259 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:05 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:05.897+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:06.259 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:05 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:05.897+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:06.259 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:05 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:05.897+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:06.259 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:05 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:05.897+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:06.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T03:46:06.326 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:46:06.327 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T03:46:06.327 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T03:46:06.327 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.7 2024-09-21T03:46:06.640 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:06 smithi187 systemd[1]: Stopping Ceph osd.7 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:46:06.641 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:06 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:06.385+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:06.641 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:06 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:06.385+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:06.641 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:06 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:06.385+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.454637+0000 front 2024-09-21T03:45:37.454777+0000 (oldest deadline 2024-09-21T03:46:01.554820+0000) 2024-09-21T03:46:06.641 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:06 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:06.637+0000 7fdfccf29640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:46:06.641 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:06 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:06.637+0000 7fdfccf29640 -1 osd.7 99 *** Got signal Terminated *** 2024-09-21T03:46:06.641 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:06 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:06.637+0000 7fdfccf29640 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:46:06.642 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:06 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:06.279+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:06.642 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:06 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:06.279+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:06.642 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:06 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:06.279+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:06.642 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:06 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:06.279+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:06.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:06 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:06.203+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:06.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:06 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:06.203+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:06.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:06 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:06.203+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:06.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:06 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:06.346+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:06.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:06 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:06.346+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:06.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:06 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:06.346+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:07.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:06 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:06.892+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:07.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:06 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:06.892+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:07.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:06 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:06.892+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:07.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:06 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:06.892+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:07.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:07 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:07.207+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:07.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:07 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:07.207+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:07.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:07 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:07.207+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:07.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:07 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:07.328+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:07.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:07 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:07.328+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:07.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:07 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:07.328+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:07.679 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:07 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:07.421+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:07.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:07 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:07.421+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:07.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:07 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:07.421+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.454637+0000 front 2024-09-21T03:45:37.454777+0000 (oldest deadline 2024-09-21T03:46:01.554820+0000) 2024-09-21T03:46:07.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:07 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:07.326+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:07.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:07 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:07.326+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:07.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:07 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:07.326+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:07.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:07 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:07.326+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:08.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:07 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:07.879+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:08.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:07 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:07.879+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:08.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:07 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:07.879+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:08.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:07 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:07.879+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:08.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:08 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:08.175+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:08.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:08 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:08.175+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:08.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:08 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:08.175+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:08.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:08 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:08.336+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:08.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:08 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:08.336+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:08.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:08 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:08.336+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:08.679 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:08 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:08.406+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:08.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:08 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:08.406+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:08.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:08 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:08.406+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.454637+0000 front 2024-09-21T03:45:37.454777+0000 (oldest deadline 2024-09-21T03:46:01.554820+0000) 2024-09-21T03:46:08.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:08 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:08.337+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:08.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:08 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:08.337+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:08.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:08 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:08.337+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:08.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:08 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:08.337+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:09.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:08.865+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:09.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:08.865+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:09.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:08.865+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:09.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:08.865+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:09.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:08 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:08.865+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:09.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:09 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:09.166+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:09.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:09 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:09.166+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:09.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:09 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:09.166+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:09.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:09 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:09.382+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:09.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:09 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:09.382+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:09.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:09 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:09.382+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:09.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:09 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:09.433+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:09.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:09 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:09.433+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:09.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:09 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:09.433+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.454637+0000 front 2024-09-21T03:45:37.454777+0000 (oldest deadline 2024-09-21T03:46:01.554820+0000) 2024-09-21T03:46:09.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:09 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:09.333+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:09.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:09 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:09.333+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:09.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:09 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:09.333+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:09.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:09 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:09.333+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:10.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:09 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:09.824+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:10.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:09 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:09.824+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:10.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:09 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:09.824+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:10.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:09 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:09.824+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:10.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:09 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:09.824+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:10.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:10 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:10.174+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:10.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:10 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:10.174+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:10.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:10 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:10.174+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:10.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:10 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:10.174+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.386425+0000 front 2024-09-21T03:45:44.386303+0000 (oldest deadline 2024-09-21T03:46:09.686154+0000) 2024-09-21T03:46:10.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:10 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:10.337+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:10.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:10 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:10.337+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:10.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:10 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:10.337+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:10.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:10 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:10.337+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:10.679 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:10 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:10.451+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:10.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:10 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:10.451+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:10.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:10 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:10.451+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.454637+0000 front 2024-09-21T03:45:37.454777+0000 (oldest deadline 2024-09-21T03:46:01.554820+0000) 2024-09-21T03:46:10.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:10 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:10.451+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:45.556508+0000 front 2024-09-21T03:45:45.556485+0000 (oldest deadline 2024-09-21T03:46:10.256663+0000) 2024-09-21T03:46:10.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:10 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:10.308+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:10.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:10 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:10.308+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:10.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:10 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:10.308+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:10.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:10 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:10.308+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:11.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:10 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:10.834+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:11.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:10 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:10.834+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:11.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:10 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:10.834+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:11.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:10 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:10.834+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:11.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:10 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:10.834+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:11.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:11 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:11.328+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:11.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:11 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:11.328+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:11.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:11 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:11.328+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:11.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:11 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:11.328+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:11.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:11 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:11.206+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:11.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:11 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:11.206+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:11.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:11 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:11.206+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:11.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:11 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:11.206+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.386425+0000 front 2024-09-21T03:45:44.386303+0000 (oldest deadline 2024-09-21T03:46:09.686154+0000) 2024-09-21T03:46:11.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:11 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:11.433+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.451394+0000 front 2024-09-21T03:45:19.451637+0000 (oldest deadline 2024-09-21T03:45:44.751525+0000) 2024-09-21T03:46:11.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:11 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:11.433+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.053796+0000 front 2024-09-21T03:45:28.053657+0000 (oldest deadline 2024-09-21T03:45:53.953629+0000) 2024-09-21T03:46:11.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:11 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:11.433+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.454637+0000 front 2024-09-21T03:45:37.454777+0000 (oldest deadline 2024-09-21T03:46:01.554820+0000) 2024-09-21T03:46:11.680 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:11 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7[51976]: 2024-09-21T03:46:11.433+0000 7fdfc550c640 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:45.556508+0000 front 2024-09-21T03:45:45.556485+0000 (oldest deadline 2024-09-21T03:46:10.256663+0000) 2024-09-21T03:46:11.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:11 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:11.352+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:11.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:11 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:11.352+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:11.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:11 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:11.352+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:11.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:11 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:11.352+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:12.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:11 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:11.815+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:12.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:11 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:11.815+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:12.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:11 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:11.815+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:12.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:11 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:11.815+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:12.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:11 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:11.815+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:12.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:12 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:12.343+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:12.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:12 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:12.343+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:12.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:12 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:12.343+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:12.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:12 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:12.343+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:12.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:12 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:12.158+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:12.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:12 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:12.158+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:12.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:12 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:12.158+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:12.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:12 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:12.158+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.386425+0000 front 2024-09-21T03:45:44.386303+0000 (oldest deadline 2024-09-21T03:46:09.686154+0000) 2024-09-21T03:46:12.482 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:12 smithi187 podman[69342]: 2024-09-21 03:46:12.187188705 +0000 UTC m=+5.666914458 container died 0a4f5915555ad9b5bd5c1833b785f219b97278e4cbd4f1e6c4ef1e6b266d5b78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T03:46:12.482 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:12 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:12.361+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:12.483 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:12 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:12.361+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:12.483 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:12 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:12.361+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:12.483 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:12 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:12.361+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:12.841 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:12 smithi187 podman[69342]: 2024-09-21 03:46:12.579823344 +0000 UTC m=+6.059549083 container cleanup 0a4f5915555ad9b5bd5c1833b785f219b97278e4cbd4f1e6c4ef1e6b266d5b78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T03:46:12.841 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:12 smithi187 bash[69342]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7 2024-09-21T03:46:13.175 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:12 smithi187 podman[69358]: 2024-09-21 03:46:12.839488919 +0000 UTC m=+0.655460489 container remove 0a4f5915555ad9b5bd5c1833b785f219b97278e4cbd4f1e6c4ef1e6b266d5b78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:46:13.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:12 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:12.840+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:13.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:12 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:12.840+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:13.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:12 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:12.840+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:13.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:12 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:12.840+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:13.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:12 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:12.840+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:13.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:13 smithi187 podman[69444]: 2024-09-21 03:46:13.173583762 +0000 UTC m=+0.098790828 container create 694e60e4c48891f7abdab40805124dbcf17be53992aeafd5dfb79ffa5370ed80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:46:13.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:13 smithi187 podman[69444]: 2024-09-21 03:46:13.090331416 +0000 UTC m=+0.015538451 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:46:13.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:13 smithi187 podman[69444]: 2024-09-21 03:46:13.392645596 +0000 UTC m=+0.317852620 container init 694e60e4c48891f7abdab40805124dbcf17be53992aeafd5dfb79ffa5370ed80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T03:46:13.430 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:13 smithi187 podman[69444]: 2024-09-21 03:46:13.395972629 +0000 UTC m=+0.321179650 container start 694e60e4c48891f7abdab40805124dbcf17be53992aeafd5dfb79ffa5370ed80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T03:46:13.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:13 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:13.357+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:13.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:13 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:13.357+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:13.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:13 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:13.357+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:13.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:13 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:13.357+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:13.431 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:13 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:13.357+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:13.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:13 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:13.390+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:13.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:13 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:13.390+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:13.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:13 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:13.390+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:13.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:13 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:13.390+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:13.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:13 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:13.184+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:13.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:13 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:13.184+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:13.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:13 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:13.184+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:13.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:13 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:13.184+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.386425+0000 front 2024-09-21T03:45:44.386303+0000 (oldest deadline 2024-09-21T03:46:09.686154+0000) 2024-09-21T03:46:13.929 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:13 smithi187 podman[69444]: 2024-09-21 03:46:13.49294613 +0000 UTC m=+0.418153156 container attach 694e60e4c48891f7abdab40805124dbcf17be53992aeafd5dfb79ffa5370ed80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:46:13.930 INFO:journalctl@ceph.osd.7.smithi187.stdout:Sep 21 03:46:13 smithi187 podman[69444]: 2024-09-21 03:46:13.590593699 +0000 UTC m=+0.515800743 container died 694e60e4c48891f7abdab40805124dbcf17be53992aeafd5dfb79ffa5370ed80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-7-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T03:46:14.240 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T03:46:14.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:46:14.279 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T03:46:14.280 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T03:46:14.280 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.8 2024-09-21T03:46:14.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:13.867+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:14.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:13.867+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:14.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:13.867+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:14.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:13.867+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:14.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:13 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:13.867+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:14.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:14 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:14.138+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:14.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:14 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:14.138+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:14.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:14 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:14.138+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:14.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:14 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:14.138+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.386425+0000 front 2024-09-21T03:45:44.386303+0000 (oldest deadline 2024-09-21T03:46:09.686154+0000) 2024-09-21T03:46:14.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:14 smithi203 systemd[1]: Stopping Ceph osd.8 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:46:14.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:14 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:14.359+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:14.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:14 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:14.359+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:14.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:14 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:14.359+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:14.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:14 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:14.359+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:14.488 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:14 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:14.363+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:14.488 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:14 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:14.363+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:14.488 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:14 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:14.363+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:14.488 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:14 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:14.363+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:14.488 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:14 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:14.363+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:14.911 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:14 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:14.577+0000 7f5bc3836640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:46:14.911 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:14 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:14.577+0000 7f5bc3836640 -1 osd.8 99 *** Got signal Terminated *** 2024-09-21T03:46:14.911 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:14 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:14.577+0000 7f5bc3836640 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:46:15.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:14 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:14.820+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:15.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:14 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:14.820+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:15.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:14 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:14.820+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:15.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:14 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:14.820+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:15.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:14 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:14.820+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:15.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:15 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:15.184+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:15.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:15 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:15.184+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:15.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:15 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:15.184+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:15.662 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:15 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:15.184+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.386425+0000 front 2024-09-21T03:45:44.386303+0000 (oldest deadline 2024-09-21T03:46:09.686154+0000) 2024-09-21T03:46:15.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:15 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:15.352+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:15.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:15 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:15.352+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:15.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:15 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:15.352+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:15.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:15 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:15.352+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:15.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:15 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:15.412+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:15.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:15 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:15.412+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:15.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:15 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:15.412+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:15.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:15 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:15.413+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:15.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:15 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:15.413+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:16.188 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:15 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:15.845+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:16.189 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:15 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:15.845+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:16.189 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:15 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:15.845+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:16.189 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:15 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:15.845+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:16.189 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:15 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:15.845+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:16.414 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:16 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:16.317+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:16.414 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:16 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:16.317+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:16.414 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:16 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:16.317+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:16.414 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:16 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:16.317+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:16.415 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:16 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:16.143+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:16.415 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:16 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:16.143+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:16.415 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:16 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:16.143+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:16.415 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:16 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:16.143+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.386425+0000 front 2024-09-21T03:45:44.386303+0000 (oldest deadline 2024-09-21T03:46:09.686154+0000) 2024-09-21T03:46:16.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:16 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:16.367+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:16.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:16 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:16.367+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:16.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:16 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:16.367+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:16.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:16 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:16.367+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:16.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:16 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:16.367+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:17.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:16 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:16.808+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:17.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:16 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:16.808+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:17.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:16 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:16.808+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:17.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:16 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:16.808+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:17.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:16 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:16.808+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:17.288 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:16 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:16.808+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:51.742671+0000 front 2024-09-21T03:45:51.742597+0000 (oldest deadline 2024-09-21T03:46:16.343303+0000) 2024-09-21T03:46:17.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:17 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:17.353+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:17.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:17 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:17.353+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:17.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:17 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:17.353+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:17.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:17 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:17.353+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:17.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:17 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:17.138+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:17.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:17 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:17.138+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:17.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:17 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:17.138+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:17.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:17 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:17.138+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.386425+0000 front 2024-09-21T03:45:44.386303+0000 (oldest deadline 2024-09-21T03:46:09.686154+0000) 2024-09-21T03:46:17.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:17 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:17.138+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:54.387074+0000 front 2024-09-21T03:45:54.387044+0000 (oldest deadline 2024-09-21T03:46:16.687184+0000) 2024-09-21T03:46:17.679 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:17 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:17.345+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:17.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:17 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:17.345+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:17.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:17 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:17.345+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:17.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:17 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:17.345+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:17.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:17 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:17.345+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:18.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:17 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:17.765+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:18.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:17 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:17.765+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:18.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:17 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:17.765+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:18.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:17 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:17.765+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:18.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:17 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:17.765+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:18.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:17 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:17.765+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:51.742671+0000 front 2024-09-21T03:45:51.742597+0000 (oldest deadline 2024-09-21T03:46:16.343303+0000) 2024-09-21T03:46:18.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:18 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:18.111+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:18.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:18 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:18.111+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:18.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:18 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:18.111+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:18.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:18 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:18.111+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.386425+0000 front 2024-09-21T03:45:44.386303+0000 (oldest deadline 2024-09-21T03:46:09.686154+0000) 2024-09-21T03:46:18.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:18 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:18.111+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:54.387074+0000 front 2024-09-21T03:45:54.387044+0000 (oldest deadline 2024-09-21T03:46:16.687184+0000) 2024-09-21T03:46:18.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:18 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:18.306+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:18.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:18 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:18.306+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:18.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:18 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:18.306+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:18.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:18 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:18.306+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:18.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:18 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:18.306+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:18.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:18 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:18.338+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:18.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:18 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:18.338+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:18.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:18 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:18.338+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:18.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:18 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:18.338+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:18.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:18 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:18.338+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:18.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:18 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:18.338+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:19.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:18 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:18.734+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:19.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:18 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:18.734+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:19.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:18 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:18.734+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:19.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:18 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:18.734+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:19.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:18 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:18.734+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:19.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:18 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:18.734+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:51.742671+0000 front 2024-09-21T03:45:51.742597+0000 (oldest deadline 2024-09-21T03:46:16.343303+0000) 2024-09-21T03:46:19.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:19 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:19.150+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:20.482467+0000 front 2024-09-21T03:45:20.482594+0000 (oldest deadline 2024-09-21T03:45:45.083025+0000) 2024-09-21T03:46:19.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:19 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:19.150+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:30.384310+0000 front 2024-09-21T03:45:30.383991+0000 (oldest deadline 2024-09-21T03:45:54.484107+0000) 2024-09-21T03:46:19.411 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:19 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:19.150+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.185570+0000 front 2024-09-21T03:45:39.185426+0000 (oldest deadline 2024-09-21T03:46:02.685206+0000) 2024-09-21T03:46:19.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:19 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:19.150+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.386425+0000 front 2024-09-21T03:45:44.386303+0000 (oldest deadline 2024-09-21T03:46:09.686154+0000) 2024-09-21T03:46:19.412 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:19 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8[52613]: 2024-09-21T03:46:19.150+0000 7f5bbfe21640 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:54.387074+0000 front 2024-09-21T03:45:54.387044+0000 (oldest deadline 2024-09-21T03:46:16.687184+0000) 2024-09-21T03:46:19.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:19 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:19.309+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:19.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:19 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:19.309+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:19.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:19 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:19.309+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:19.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:19 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:19.309+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:19.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:19 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:19.309+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:19.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:19 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:19.317+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:19.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:19 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:19.317+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:19.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:19 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:19.317+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:19.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:19 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:19.317+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:19.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:19 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:19.317+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:19.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:19 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:19.317+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:20.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:19 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:19.691+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:20.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:19 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:19.691+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:20.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:19 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:19.691+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:20.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:19 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:19.691+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:20.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:19 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:19.691+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:20.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:19 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:19.691+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:51.742671+0000 front 2024-09-21T03:45:51.742597+0000 (oldest deadline 2024-09-21T03:46:16.343303+0000) 2024-09-21T03:46:20.472 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:20 smithi203 podman[69657]: 2024-09-21 03:46:20.195442835 +0000 UTC m=+5.718851932 container died 25474c3d780fc50323846395065c634376dca61f7267386ff57554b6f5b926f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T03:46:20.472 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:20 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:20.325+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:20.472 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:20 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:20.325+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:20.472 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:20 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:20.325+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:20.472 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:20 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:20.325+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:20.473 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:20 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:20.325+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:20.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:20 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:20.305+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:20.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:20 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:20.305+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:20.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:20 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:20.305+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:20.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:20 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:20.305+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:20.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:20 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:20.305+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:20.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:20 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:20.305+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:20.911 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:20 smithi203 podman[69657]: 2024-09-21 03:46:20.561089641 +0000 UTC m=+6.084498732 container cleanup 25474c3d780fc50323846395065c634376dca61f7267386ff57554b6f5b926f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T03:46:20.911 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:20 smithi203 bash[69657]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8 2024-09-21T03:46:20.911 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:20 smithi203 podman[69673]: 2024-09-21 03:46:20.712568351 +0000 UTC m=+0.511136718 container remove 25474c3d780fc50323846395065c634376dca61f7267386ff57554b6f5b926f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T03:46:21.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:20 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:20.665+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:21.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:20 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:20.665+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:21.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:20 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:20.665+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:21.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:20 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:20.665+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:21.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:20 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:20.665+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:21.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:20 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:20.665+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:51.742671+0000 front 2024-09-21T03:45:51.742597+0000 (oldest deadline 2024-09-21T03:46:16.343303+0000) 2024-09-21T03:46:21.409 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:21 smithi203 podman[69761]: 2024-09-21 03:46:21.014254513 +0000 UTC m=+0.016449988 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:46:21.409 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:21 smithi203 podman[69761]: 2024-09-21 03:46:21.113436219 +0000 UTC m=+0.115631724 container create 3662cb31cbe42685ac11e3cd001b7ae3e0ae5694b970914e53f66d2ac6914561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:46:21.409 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:21 smithi203 podman[69761]: 2024-09-21 03:46:21.324213129 +0000 UTC m=+0.326408585 container init 3662cb31cbe42685ac11e3cd001b7ae3e0ae5694b970914e53f66d2ac6914561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T03:46:21.409 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:21 smithi203 podman[69761]: 2024-09-21 03:46:21.327496956 +0000 UTC m=+0.329692413 container start 3662cb31cbe42685ac11e3cd001b7ae3e0ae5694b970914e53f66d2ac6914561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T03:46:21.409 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:21 smithi203 podman[69761]: 2024-09-21 03:46:21.407549773 +0000 UTC m=+0.409745240 container attach 3662cb31cbe42685ac11e3cd001b7ae3e0ae5694b970914e53f66d2ac6914561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T03:46:21.410 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:21 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:21.303+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:21.410 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:21 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:21.303+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:21.410 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:21 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:21.303+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:21.410 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:21 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:21.303+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:21.410 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:21 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:21.303+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:21.661 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 21 03:46:21 smithi203 podman[69761]: 2024-09-21 03:46:21.513485145 +0000 UTC m=+0.515680611 container died 3662cb31cbe42685ac11e3cd001b7ae3e0ae5694b970914e53f66d2ac6914561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T03:46:21.679 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:21 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:21.352+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:21.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:21 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:21.352+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:21.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:21 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:21.352+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:21.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:21 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:21.352+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:21.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:21 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:21.352+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:21.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:21 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:21.352+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:22.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:21 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:21.708+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:22.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:21 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:21.708+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:22.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:21 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:21.708+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:22.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:21 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:21.708+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:22.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:21 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:21.708+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:22.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:21 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:21.708+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:51.742671+0000 front 2024-09-21T03:45:51.742597+0000 (oldest deadline 2024-09-21T03:46:16.343303+0000) 2024-09-21T03:46:22.203 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T03:46:22.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:46:22.243 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T03:46:22.243 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T03:46:22.243 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.9 2024-09-21T03:46:22.452 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:22 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:22.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:22.453 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:22 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:22.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:22.453 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:22 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:22.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:22.453 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:22 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:22.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:22.453 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:22 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:22.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:22.536 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:22 smithi112 systemd[1]: Stopping Ceph osd.9 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:46:22.679 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:22 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:22.380+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:22.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:22 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:22.380+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:22.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:22 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:22.380+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:22.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:22 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:22.380+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:22.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:22 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:22.380+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:22.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:22 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:22.380+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:22.787 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:22 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:22.534+0000 7f2096573640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:46:22.788 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:22 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:22.534+0000 7f2096573640 -1 osd.9 99 *** Got signal Terminated *** 2024-09-21T03:46:22.788 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:22 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:22.534+0000 7f2096573640 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:46:22.788 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:22 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:22.700+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:22.788 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:22 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:22.700+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:22.788 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:22 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:22.700+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:22.789 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:22 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:22.700+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:22.789 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:22 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:22.700+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:22.789 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:22 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:22.700+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:51.742671+0000 front 2024-09-21T03:45:51.742597+0000 (oldest deadline 2024-09-21T03:46:16.343303+0000) 2024-09-21T03:46:23.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:23 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:23.227+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:23.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:23 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:23.227+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:23.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:23 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:23.227+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:23.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:23 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:23.227+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:23.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:23 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:23.227+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:23.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:23 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:23.341+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:23.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:23 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:23.341+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:23.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:23 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:23.341+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:23.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:23 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:23.341+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:23.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:23 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:23.341+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:23.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:23 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:23.341+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:24.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:23 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:23.696+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:24.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:23 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:23.696+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:24.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:23 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:23.696+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:24.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:23 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:23.696+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:24.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:23 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:23.696+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:24.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:23 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:23.696+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:51.742671+0000 front 2024-09-21T03:45:51.742597+0000 (oldest deadline 2024-09-21T03:46:16.343303+0000) 2024-09-21T03:46:24.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:24 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:24.238+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:24.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:24 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:24.238+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:24.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:24 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:24.238+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:24.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:24 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:24.238+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:24.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:24 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:24.238+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:24.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:24 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:24.386+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:24.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:24 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:24.386+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:24.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:24 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:24.386+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:24.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:24 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:24.386+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:24.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:24 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:24.386+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:24.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:24 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:24.386+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:25.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:24 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:24.720+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:25.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:24 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:24.720+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:25.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:24 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:24.720+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:25.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:24 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:24.720+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:25.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:24 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:24.720+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:25.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:24 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:24.720+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:51.742671+0000 front 2024-09-21T03:45:51.742597+0000 (oldest deadline 2024-09-21T03:46:16.343303+0000) 2024-09-21T03:46:25.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:25 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:25.267+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:25.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:25 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:25.267+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:25.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:25 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:25.267+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:25.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:25 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:25.267+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:25.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:25 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:25.267+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:25.679 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:25 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:25.371+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:25.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:25 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:25.371+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:25.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:25 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:25.371+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:25.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:25 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:25.371+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:25.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:25 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:25.371+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:25.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:25 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:25.371+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:26.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:25 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:25.681+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:26.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:25 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:25.681+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:26.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:25 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:25.681+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:26.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:25 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:25.681+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:26.039 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:25 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:25.681+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:26.039 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:25 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:25.681+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:51.742671+0000 front 2024-09-21T03:45:51.742597+0000 (oldest deadline 2024-09-21T03:46:16.343303+0000) 2024-09-21T03:46:26.562 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:26 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:26.271+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:26.563 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:26 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:26.271+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:26.563 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:26 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:26.271+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:26.563 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:26 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:26.271+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:26.563 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:26 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:26.271+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:26.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:26 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:26.402+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:26.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:26 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:26.402+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:26.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:26 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:26.402+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:26.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:26 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:26.402+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:26.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:26 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:26.402+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:26.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:26 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:26.402+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:27.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:26 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:26.726+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.235388+0000 front 2024-09-21T03:45:15.235601+0000 (oldest deadline 2024-09-21T03:45:41.135575+0000) 2024-09-21T03:46:27.037 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:26 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:26.726+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.136127+0000 front 2024-09-21T03:45:21.136151+0000 (oldest deadline 2024-09-21T03:45:45.836171+0000) 2024-09-21T03:46:27.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:26 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:26.726+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:28.137121+0000 front 2024-09-21T03:45:28.137179+0000 (oldest deadline 2024-09-21T03:45:56.937869+0000) 2024-09-21T03:46:27.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:26 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:26.726+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:37.439030+0000 front 2024-09-21T03:45:37.439107+0000 (oldest deadline 2024-09-21T03:46:02.139157+0000) 2024-09-21T03:46:27.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:26 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:26.726+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.240849+0000 front 2024-09-21T03:45:47.240963+0000 (oldest deadline 2024-09-21T03:46:08.340879+0000) 2024-09-21T03:46:27.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:26 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:26.726+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:51.742671+0000 front 2024-09-21T03:45:51.742597+0000 (oldest deadline 2024-09-21T03:46:16.343303+0000) 2024-09-21T03:46:27.038 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:26 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9[68070]: 2024-09-21T03:46:26.726+0000 7f2092b5e640 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.144938+0000 front 2024-09-21T03:46:02.144954+0000 (oldest deadline 2024-09-21T03:46:26.245000+0000) 2024-09-21T03:46:27.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:27 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:27.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:27.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:27 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:27.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:27.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:27 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:27.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:27.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:27 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:27.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:27.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:27 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:27.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:27.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:27 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:27.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:27.930 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:27 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:27.432+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:27.930 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:27 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:27.432+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:27.930 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:27 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:27.432+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:27.930 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:27 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:27.432+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:27.930 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:27 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:27.432+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:27.931 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:27 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:27.432+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:28.287 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:28 smithi112 podman[112431]: 2024-09-21 03:46:28.083170911 +0000 UTC m=+5.651416211 container died 0092d6555270c427eca29b0bf1fe61d631c8303c7bbacd8d30e30edcca4c94bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:46:28.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:28 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:28.268+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:28.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:28 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:28.268+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:28.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:28 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:28.268+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:28.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:28 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:28.268+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:28.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:28 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:28.268+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:28.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:28 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:28.268+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:28.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:28 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:28.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:28.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:28 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:28.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:28.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:28 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:28.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:28.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:28 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:28.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:28.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:28 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:28.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:28.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:28 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:28.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:28.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:28 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:28.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.837540+0000 front 2024-09-21T03:46:02.837564+0000 (oldest deadline 2024-09-21T03:46:28.137387+0000) 2024-09-21T03:46:28.787 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:28 smithi112 podman[112431]: 2024-09-21 03:46:28.543217105 +0000 UTC m=+6.111462378 container cleanup 0092d6555270c427eca29b0bf1fe61d631c8303c7bbacd8d30e30edcca4c94bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:46:28.787 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:28 smithi112 bash[112431]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9 2024-09-21T03:46:29.150 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:28 smithi112 podman[112449]: 2024-09-21 03:46:28.819596382 +0000 UTC m=+0.740675584 container remove 0092d6555270c427eca29b0bf1fe61d631c8303c7bbacd8d30e30edcca4c94bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T03:46:29.402 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:29 smithi112 podman[112534]: 2024-09-21 03:46:29.052011399 +0000 UTC m=+0.015786418 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:46:29.402 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:29 smithi112 podman[112534]: 2024-09-21 03:46:29.1620172 +0000 UTC m=+0.125792220 container create 537e3bcaeaf03c1b98350d3b7c0b78acca35b0f930146401f44b92012d180c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:46:29.403 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:29 smithi112 podman[112534]: 2024-09-21 03:46:29.397999733 +0000 UTC m=+0.361774770 container init 537e3bcaeaf03c1b98350d3b7c0b78acca35b0f930146401f44b92012d180c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:46:29.403 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:29 smithi112 podman[112534]: 2024-09-21 03:46:29.401246979 +0000 UTC m=+0.365021998 container start 537e3bcaeaf03c1b98350d3b7c0b78acca35b0f930146401f44b92012d180c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T03:46:29.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:29 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:29.278+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:29.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:29 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:29.278+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:29.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:29 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:29.278+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:29.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:29 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:29.278+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:29.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:29 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:29.278+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:29.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:29 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:29.278+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:29.679 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:29 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:29.426+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:29.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:29 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:29.426+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:29.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:29 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:29.426+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:29.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:29 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:29.426+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:29.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:29 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:29.426+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:29.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:29 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:29.426+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:29.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:29 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:29.426+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.837540+0000 front 2024-09-21T03:46:02.837564+0000 (oldest deadline 2024-09-21T03:46:28.137387+0000) 2024-09-21T03:46:29.787 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:29 smithi112 podman[112534]: 2024-09-21 03:46:29.506309696 +0000 UTC m=+0.470084716 container attach 537e3bcaeaf03c1b98350d3b7c0b78acca35b0f930146401f44b92012d180c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:46:29.787 INFO:journalctl@ceph.osd.9.smithi112.stdout:Sep 21 03:46:29 smithi112 podman[112534]: 2024-09-21 03:46:29.591589647 +0000 UTC m=+0.555364665 container died 537e3bcaeaf03c1b98350d3b7c0b78acca35b0f930146401f44b92012d180c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T03:46:30.184 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T03:46:30.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:46:30.233 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T03:46:30.233 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T03:46:30.233 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.10 2024-09-21T03:46:30.559 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:30 smithi187 systemd[1]: Stopping Ceph osd.10 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:46:30.559 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:30 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:30.411+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:30.559 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:30 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:30.411+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:30.559 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:30 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:30.411+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:30.560 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:30 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:30.411+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:30.560 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:30 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:30.411+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:30.560 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:30 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:30.411+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:30.560 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:30 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:30.411+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.837540+0000 front 2024-09-21T03:46:02.837564+0000 (oldest deadline 2024-09-21T03:46:28.137387+0000) 2024-09-21T03:46:30.560 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:30 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:30.556+0000 7f23270e2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:46:30.560 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:30 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:30.556+0000 7f23270e2640 -1 osd.10 99 *** Got signal Terminated *** 2024-09-21T03:46:30.560 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:30 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:30.556+0000 7f23270e2640 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:46:30.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:30 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:30.255+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:30.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:30 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:30.255+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:30.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:30 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:30.255+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:30.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:30 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:30.255+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:30.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:30 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:30.255+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:30.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:30 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:30.255+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:31.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:31 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:31.300+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:31.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:31 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:31.300+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:31.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:31 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:31.300+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:31.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:31 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:31.300+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:31.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:31 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:31.300+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:31.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:31 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:31.300+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:31.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:31 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:31.421+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:31.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:31 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:31.421+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:31.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:31 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:31.421+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:31.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:31 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:31.421+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:31.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:31 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:31.421+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:31.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:31 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:31.421+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:31.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:31 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:31.421+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.837540+0000 front 2024-09-21T03:46:02.837564+0000 (oldest deadline 2024-09-21T03:46:28.137387+0000) 2024-09-21T03:46:32.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:32 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:32.339+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:32.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:32 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:32.339+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:32.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:32 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:32.339+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:32.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:32 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:32.339+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:32.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:32 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:32.339+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:32.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:32 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:32.339+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:32.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:32 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:32.403+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:32.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:32 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:32.403+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:32.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:32 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:32.403+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:32.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:32 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:32.403+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:32.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:32 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:32.403+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:32.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:32 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:32.403+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:32.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:32 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:32.403+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.837540+0000 front 2024-09-21T03:46:02.837564+0000 (oldest deadline 2024-09-21T03:46:28.137387+0000) 2024-09-21T03:46:33.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:33 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:33.302+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:33.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:33 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:33.302+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:33.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:33 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:33.302+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:33.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:33 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:33.302+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:33.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:33 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:33.302+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:33.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:33 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:33.302+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:33.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:33 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:33.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:33.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:33 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:33.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:33.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:33 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:33.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:33.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:33 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:33.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:33.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:33 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:33.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:33.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:33 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:33.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:33.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:33 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:33.410+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.837540+0000 front 2024-09-21T03:46:02.837564+0000 (oldest deadline 2024-09-21T03:46:28.137387+0000) 2024-09-21T03:46:34.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:34 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:34.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:34.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:34 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:34.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:34.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:34 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:34.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:34.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:34 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:34.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:34.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:34 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:34.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:34.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:34 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:34.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:34.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:34 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:34.275+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-09-21T03:46:10.676657+0000 front 2024-09-21T03:46:10.676670+0000 (oldest deadline 2024-09-21T03:46:33.576458+0000) 2024-09-21T03:46:34.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:34 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:34.377+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:34.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:34 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:34.377+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:34.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:34 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:34.377+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:34.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:34 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:34.377+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:34.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:34 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:34.377+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:34.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:34 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:34.377+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:34.681 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:34 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:34.377+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.837540+0000 front 2024-09-21T03:46:02.837564+0000 (oldest deadline 2024-09-21T03:46:28.137387+0000) 2024-09-21T03:46:35.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:35 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:35.263+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:35.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:35 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:35.263+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:35.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:35 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:35.263+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:35.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:35 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:35.263+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:35.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:35 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:35.263+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:35.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:35 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:35.263+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:35.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:35 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:35.263+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-09-21T03:46:10.676657+0000 front 2024-09-21T03:46:10.676670+0000 (oldest deadline 2024-09-21T03:46:33.576458+0000) 2024-09-21T03:46:35.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:35 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:35.418+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.228772+0000 front 2024-09-21T03:45:15.228750+0000 (oldest deadline 2024-09-21T03:45:36.928663+0000) 2024-09-21T03:46:35.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:35 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:35.418+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:21.029886+0000 front 2024-09-21T03:45:21.029920+0000 (oldest deadline 2024-09-21T03:45:46.929873+0000) 2024-09-21T03:46:35.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:35 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:35.418+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:27.431049+0000 front 2024-09-21T03:45:27.430985+0000 (oldest deadline 2024-09-21T03:45:53.331021+0000) 2024-09-21T03:46:35.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:35 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:35.418+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-09-21T03:45:39.533706+0000 front 2024-09-21T03:45:39.533765+0000 (oldest deadline 2024-09-21T03:46:03.033833+0000) 2024-09-21T03:46:35.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:35 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:35.418+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:47.134690+0000 front 2024-09-21T03:45:47.134725+0000 (oldest deadline 2024-09-21T03:46:12.434835+0000) 2024-09-21T03:46:35.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:35 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:35.418+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:52.435151+0000 front 2024-09-21T03:45:52.435266+0000 (oldest deadline 2024-09-21T03:46:17.735385+0000) 2024-09-21T03:46:35.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:35 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10[58342]: 2024-09-21T03:46:35.418+0000 7f23236cd640 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.837540+0000 front 2024-09-21T03:46:02.837564+0000 (oldest deadline 2024-09-21T03:46:28.137387+0000) 2024-09-21T03:46:36.180 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:36 smithi187 podman[69562]: 2024-09-21 03:46:36.040498712 +0000 UTC m=+5.604303899 container died bf24e307b9c5cad836efbeb4ea2660f1ea1f38a1d6906112ece0b2149378ca90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T03:46:36.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:36 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:36.257+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:36.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:36 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:36.257+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:36.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:36 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:36.257+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:36.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:36 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:36.257+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:36.663 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:36 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:36.257+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:36.663 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:36 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:36.257+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:36.663 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:36 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:36.257+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-09-21T03:46:10.676657+0000 front 2024-09-21T03:46:10.676670+0000 (oldest deadline 2024-09-21T03:46:33.576458+0000) 2024-09-21T03:46:36.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:36 smithi187 podman[69562]: 2024-09-21 03:46:36.398668382 +0000 UTC m=+5.962473581 container cleanup bf24e307b9c5cad836efbeb4ea2660f1ea1f38a1d6906112ece0b2149378ca90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:46:36.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:36 smithi187 bash[69562]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10 2024-09-21T03:46:37.002 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:36 smithi187 podman[69578]: 2024-09-21 03:46:36.683313786 +0000 UTC m=+0.640949088 container remove bf24e307b9c5cad836efbeb4ea2660f1ea1f38a1d6906112ece0b2149378ca90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:46:37.329 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:37 smithi187 podman[69665]: 2024-09-21 03:46:37.000758839 +0000 UTC m=+0.109509455 container create f07f6ac2a950ec6404b350a8cfae42e36b731fd1cb2c120ee9b1cc34e8f1e274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:46:37.330 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:37 smithi187 podman[69665]: 2024-09-21 03:46:36.906971309 +0000 UTC m=+0.015721935 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:46:37.330 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:37 smithi187 podman[69665]: 2024-09-21 03:46:37.244906847 +0000 UTC m=+0.353657464 container init f07f6ac2a950ec6404b350a8cfae42e36b731fd1cb2c120ee9b1cc34e8f1e274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T03:46:37.330 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:37 smithi187 podman[69665]: 2024-09-21 03:46:37.248839029 +0000 UTC m=+0.357589657 container start f07f6ac2a950ec6404b350a8cfae42e36b731fd1cb2c120ee9b1cc34e8f1e274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:46:37.330 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:37 smithi187 podman[69665]: 2024-09-21 03:46:37.328000817 +0000 UTC m=+0.436751454 container attach f07f6ac2a950ec6404b350a8cfae42e36b731fd1cb2c120ee9b1cc34e8f1e274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:46:37.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:37 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:37.245+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:37.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:37 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:37.245+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:37.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:37 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:37.245+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:37.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:37 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:37.245+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:37.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:37 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:37.245+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:37.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:37 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:37.245+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:37.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:37 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:37.245+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-09-21T03:46:10.676657+0000 front 2024-09-21T03:46:10.676670+0000 (oldest deadline 2024-09-21T03:46:33.576458+0000) 2024-09-21T03:46:37.680 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:37 smithi187 podman[69665]: 2024-09-21 03:46:37.435667342 +0000 UTC m=+0.544417972 container died f07f6ac2a950ec6404b350a8cfae42e36b731fd1cb2c120ee9b1cc34e8f1e274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T03:46:37.978 INFO:journalctl@ceph.osd.10.smithi187.stdout:Sep 21 03:46:37 smithi187 podman[69688]: 2024-09-21 03:46:37.964969237 +0000 UTC m=+0.517641712 container remove f07f6ac2a950ec6404b350a8cfae42e36b731fd1cb2c120ee9b1cc34e8f1e274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-10-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:46:37.984 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T03:46:38.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:46:38.022 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T03:46:38.023 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T03:46:38.023 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.11 2024-09-21T03:46:38.341 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:38 smithi203 systemd[1]: Stopping Ceph osd.11 for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:46:38.341 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:38 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:38.254+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:38.342 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:38 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:38.254+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:38.342 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:38 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:38.254+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:38.342 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:38 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:38.254+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:38.342 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:38 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:38.254+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:38.342 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:38 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:38.254+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:38.342 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:38 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:38.254+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-09-21T03:46:10.676657+0000 front 2024-09-21T03:46:10.676670+0000 (oldest deadline 2024-09-21T03:46:33.576458+0000) 2024-09-21T03:46:38.342 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:38 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:38.338+0000 7f7033de9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:46:38.343 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:38 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:38.338+0000 7f7033de9640 -1 osd.11 99 *** Got signal Terminated *** 2024-09-21T03:46:38.343 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:38 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:38.338+0000 7f7033de9640 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:46:39.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:39 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:39.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:39.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:39 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:39.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:39.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:39 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:39.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:39.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:39 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:39.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:39.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:39 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:39.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:39.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:39 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:39.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:39.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:39 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:39.248+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-09-21T03:46:10.676657+0000 front 2024-09-21T03:46:10.676670+0000 (oldest deadline 2024-09-21T03:46:33.576458+0000) 2024-09-21T03:46:40.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:40 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:40.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:40.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:40 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:40.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:40.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:40 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:40.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:40.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:40 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:40.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:40.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:40 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:40.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:40.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:40 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:40.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:40.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:40 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:40.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-09-21T03:46:10.676657+0000 front 2024-09-21T03:46:10.676670+0000 (oldest deadline 2024-09-21T03:46:33.576458+0000) 2024-09-21T03:46:41.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:41 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:41.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:41.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:41 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:41.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:41.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:41 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:41.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:41.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:41 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:41.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:41.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:41 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:41.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:41.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:41 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:41.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:41.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:41 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:41.201+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-09-21T03:46:10.676657+0000 front 2024-09-21T03:46:10.676670+0000 (oldest deadline 2024-09-21T03:46:33.576458+0000) 2024-09-21T03:46:42.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:42 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:42.203+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:42.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:42 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:42.203+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:42.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:42 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:42.203+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:42.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:42 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:42.203+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:42.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:42 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:42.203+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:42.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:42 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:42.203+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:42.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:42 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:42.203+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-09-21T03:46:10.676657+0000 front 2024-09-21T03:46:10.676670+0000 (oldest deadline 2024-09-21T03:46:33.576458+0000) 2024-09-21T03:46:43.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:43 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:43.204+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-09-21T03:45:15.267092+0000 front 2024-09-21T03:45:15.267307+0000 (oldest deadline 2024-09-21T03:45:38.667659+0000) 2024-09-21T03:46:43.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:43 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:43.205+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-09-21T03:45:19.168999+0000 front 2024-09-21T03:45:19.168827+0000 (oldest deadline 2024-09-21T03:45:49.769562+0000) 2024-09-21T03:46:43.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:43 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:43.205+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-21T03:45:29.770036+0000 front 2024-09-21T03:45:29.770151+0000 (oldest deadline 2024-09-21T03:45:53.270098+0000) 2024-09-21T03:46:43.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:43 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:43.205+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-09-21T03:45:44.972589+0000 front 2024-09-21T03:45:44.972740+0000 (oldest deadline 2024-09-21T03:46:09.672585+0000) 2024-09-21T03:46:43.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:43 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:43.205+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-21T03:45:53.074234+0000 front 2024-09-21T03:45:53.074184+0000 (oldest deadline 2024-09-21T03:46:17.774312+0000) 2024-09-21T03:46:43.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:43 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:43.205+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-09-21T03:46:02.475600+0000 front 2024-09-21T03:46:02.475582+0000 (oldest deadline 2024-09-21T03:46:27.175584+0000) 2024-09-21T03:46:43.662 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:43 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11[59295]: 2024-09-21T03:46:43.205+0000 7f702c3cc640 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-09-21T03:46:10.676657+0000 front 2024-09-21T03:46:10.676670+0000 (oldest deadline 2024-09-21T03:46:33.576458+0000) 2024-09-21T03:46:44.050 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:43 smithi203 podman[69877]: 2024-09-21 03:46:43.750652086 +0000 UTC m=+5.531201793 container died 25840ab2e263335b95ec382849225a966d04d4c71c15015d0f9ee5c5220e2401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T03:46:44.375 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:44 smithi203 podman[69877]: 2024-09-21 03:46:44.151065541 +0000 UTC m=+5.931615248 container cleanup 25840ab2e263335b95ec382849225a966d04d4c71c15015d0f9ee5c5220e2401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T03:46:44.376 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:44 smithi203 bash[69877]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11 2024-09-21T03:46:44.661 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:44 smithi203 podman[69891]: 2024-09-21 03:46:44.374321605 +0000 UTC m=+0.614424313 container remove 25840ab2e263335b95ec382849225a966d04d4c71c15015d0f9ee5c5220e2401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:46:44.979 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:44 smithi203 podman[69979]: 2024-09-21 03:46:44.590319219 +0000 UTC m=+0.016149822 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:46:44.980 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:44 smithi203 podman[69979]: 2024-09-21 03:46:44.725509273 +0000 UTC m=+0.151339981 container create b828050899705dc92e27c8775aa19c2ebee59a29c3883a733df5cdf374f10fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:46:45.411 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:44 smithi203 podman[69979]: 2024-09-21 03:46:44.978016971 +0000 UTC m=+0.403847564 container init b828050899705dc92e27c8775aa19c2ebee59a29c3883a733df5cdf374f10fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:46:45.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:44 smithi203 podman[69979]: 2024-09-21 03:46:44.981873325 +0000 UTC m=+0.407703920 container start b828050899705dc92e27c8775aa19c2ebee59a29c3883a733df5cdf374f10fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:46:45.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:45 smithi203 podman[69979]: 2024-09-21 03:46:45.077928904 +0000 UTC m=+0.503759535 container attach b828050899705dc92e27c8775aa19c2ebee59a29c3883a733df5cdf374f10fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T03:46:45.412 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 21 03:46:45 smithi203 podman[69979]: 2024-09-21 03:46:45.171051117 +0000 UTC m=+0.596881712 container died b828050899705dc92e27c8775aa19c2ebee59a29c3883a733df5cdf374f10fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T03:46:45.755 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T03:46:45.793 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:46:45.794 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T03:46:45.794 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T03:46:45.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.a 2024-09-21T03:46:46.080 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:46:45 smithi112 systemd[1]: Stopping Ceph mds.a for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:46:46.537 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:46:46 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-a[74882]: 2024-09-21T03:46:46.078+0000 7f75a98b4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:46:46.538 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:46:46 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-a[74882]: 2024-09-21T03:46:46.078+0000 7f75a98b4640 -1 mds.a *** got signal Terminated *** 2024-09-21T03:46:49.988 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:46:49 smithi112 podman[112637]: 2024-09-21 03:46:49.676138287 +0000 UTC m=+3.694154906 container died a02fbf2e7356a0ffc52c4b70efe08322e9f27eb147fb8dd6b34a4bdc243798e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:46:50.287 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:46:50 smithi112 podman[112637]: 2024-09-21 03:46:50.161445955 +0000 UTC m=+4.179462616 container cleanup a02fbf2e7356a0ffc52c4b70efe08322e9f27eb147fb8dd6b34a4bdc243798e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True) 2024-09-21T03:46:50.287 INFO:journalctl@ceph.mds.a.smithi112.stdout:Sep 21 03:46:50 smithi112 bash[112637]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-a 2024-09-21T03:46:50.538 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T03:46:50.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:46:50.575 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T03:46:50.575 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T03:46:50.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.d 2024-09-21T03:46:50.787 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:46:50 smithi112 systemd[1]: Stopping Ceph mds.d for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:46:51.287 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:46:50 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-d[73837]: 2024-09-21T03:46:50.890+0000 7f3a991b2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:46:51.287 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:46:50 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-d[73837]: 2024-09-21T03:46:50.890+0000 7f3a991b2640 -1 mds.d *** got signal Terminated *** 2024-09-21T03:46:52.286 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:46:51 smithi112 podman[112770]: 2024-09-21 03:46:51.98288998 +0000 UTC m=+1.189759798 container died 694f730c71badc224c2ade3a3e13f7f09faf954f4e0f7ba9b0cbb9899d2e8ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T03:46:52.538 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:46:52 smithi112 podman[112770]: 2024-09-21 03:46:52.367662466 +0000 UTC m=+1.574532273 container cleanup 694f730c71badc224c2ade3a3e13f7f09faf954f4e0f7ba9b0cbb9899d2e8ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T03:46:52.538 INFO:journalctl@ceph.mds.d.smithi112.stdout:Sep 21 03:46:52 smithi112 bash[112770]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-d 2024-09-21T03:46:52.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T03:46:52.819 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:46:52.820 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T03:46:52.820 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T03:46:52.820 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.g 2024-09-21T03:46:53.037 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:46:52 smithi112 systemd[1]: Stopping Ceph mds.g for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:46:53.537 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:46:53 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-g[74577]: 2024-09-21T03:46:53.220+0000 7fc691b14640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:46:53.538 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:46:53 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-g[74577]: 2024-09-21T03:46:53.220+0000 7fc691b14640 -1 mds.g *** got signal Terminated *** 2024-09-21T03:46:55.166 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:46:54 smithi112 podman[112890]: 2024-09-21 03:46:54.881610783 +0000 UTC m=+1.846046627 container died 5d392b3753e0eacc4ef2483c19f17d83deee250d4e353f6fa82ea2fac6b5891d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-g, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T03:46:55.499 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:46:55 smithi112 podman[112890]: 2024-09-21 03:46:55.263199905 +0000 UTC m=+2.227635745 container cleanup 5d392b3753e0eacc4ef2483c19f17d83deee250d4e353f6fa82ea2fac6b5891d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-g, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T03:46:55.500 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:46:55 smithi112 bash[112890]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-g 2024-09-21T03:46:55.643 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T03:46:55.817 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:46:55 smithi112 podman[112902]: 2024-09-21 03:46:55.497668015 +0000 UTC m=+0.616444688 container remove 5d392b3753e0eacc4ef2483c19f17d83deee250d4e353f6fa82ea2fac6b5891d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-g, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-21T03:46:55.817 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:46:55 smithi112 systemd[1]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.g.service: Deactivated successfully. 2024-09-21T03:46:55.818 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:46:55 smithi112 systemd[1]: Stopped Ceph mds.g for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:46:55.818 INFO:journalctl@ceph.mds.g.smithi112.stdout:Sep 21 03:46:55 smithi112 systemd[1]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.g.service: Consumed 1.070s CPU time. 2024-09-21T03:46:55.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:46:55.850 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T03:46:55.850 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T03:46:55.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.j 2024-09-21T03:46:56.185 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:46:55 smithi112 systemd[1]: Stopping Ceph mds.j for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:46:56.537 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:46:56 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-j[73449]: 2024-09-21T03:46:56.181+0000 7fd561f96640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:46:56.537 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:46:56 smithi112 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-j[73449]: 2024-09-21T03:46:56.181+0000 7fd561f96640 -1 mds.j *** got signal Terminated *** 2024-09-21T03:47:00.478 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:47:00 smithi112 podman[113011]: 2024-09-21 03:47:00.188442896 +0000 UTC m=+4.110988521 container died 671905f460a214c77f98205e87d90c4906069b211fd172edbefbdeade1f84e08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-j, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:47:00.787 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:47:00 smithi112 podman[113011]: 2024-09-21 03:47:00.575197516 +0000 UTC m=+4.497743140 container cleanup 671905f460a214c77f98205e87d90c4906069b211fd172edbefbdeade1f84e08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-j, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:47:00.787 INFO:journalctl@ceph.mds.j.smithi112.stdout:Sep 21 03:47:00 smithi112 bash[113011]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-j 2024-09-21T03:47:00.955 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T03:47:00.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:47:00.994 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T03:47:00.994 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T03:47:00.994 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.b 2024-09-21T03:47:01.292 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:47:01 smithi187 systemd[1]: Stopping Ceph mds.b for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:47:01.550 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:47:01 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b[63369]: 2024-09-21T03:47:01.290+0000 7f2c5c2a0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:47:01.550 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:47:01 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b[63369]: 2024-09-21T03:47:01.290+0000 7f2c5c2a0640 -1 mds.b *** got signal Terminated *** 2024-09-21T03:47:03.680 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:47:03 smithi187 podman[69767]: 2024-09-21 03:47:03.315837715 +0000 UTC m=+2.122940743 container died dc4f3bac727d7fe4ba16f7983674415b6c34db1cf21bbbd51e0a8d11e5faa8a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:47:04.050 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:47:03 smithi187 podman[69767]: 2024-09-21 03:47:03.773216803 +0000 UTC m=+2.580319833 container cleanup dc4f3bac727d7fe4ba16f7983674415b6c34db1cf21bbbd51e0a8d11e5faa8a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T03:47:04.051 INFO:journalctl@ceph.mds.b.smithi187.stdout:Sep 21 03:47:03 smithi187 bash[69767]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-b 2024-09-21T03:47:04.194 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T03:47:04.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:47:04.231 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T03:47:04.231 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T03:47:04.231 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.e 2024-09-21T03:47:04.680 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:47:04 smithi187 systemd[1]: Stopping Ceph mds.e for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:47:04.680 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:47:04 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e[62819]: 2024-09-21T03:47:04.543+0000 7f7ada845640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:47:04.680 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:47:04 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e[62819]: 2024-09-21T03:47:04.543+0000 7f7ada845640 -1 mds.e *** got signal Terminated *** 2024-09-21T03:47:07.613 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:47:07 smithi187 podman[69901]: 2024-09-21 03:47:07.318646165 +0000 UTC m=+2.867372610 container died 0e278118c1b8a4110a424ea51bc7b6065789f8e3df6f19f3ebdeb14531cdb5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T03:47:07.930 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:47:07 smithi187 podman[69901]: 2024-09-21 03:47:07.693698032 +0000 UTC m=+3.242424465 container cleanup 0e278118c1b8a4110a424ea51bc7b6065789f8e3df6f19f3ebdeb14531cdb5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T03:47:07.930 INFO:journalctl@ceph.mds.e.smithi187.stdout:Sep 21 03:47:07 smithi187 bash[69901]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-e 2024-09-21T03:47:08.086 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T03:47:08.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:47:08.119 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T03:47:08.119 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T03:47:08.119 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.h 2024-09-21T03:47:08.433 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:47:08 smithi187 systemd[1]: Stopping Ceph mds.h for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:47:08.930 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:47:08 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h[63089]: 2024-09-21T03:47:08.431+0000 7fee90e0a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:47:08.930 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:47:08 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h[63089]: 2024-09-21T03:47:08.431+0000 7fee90e0a640 -1 mds.h *** got signal Terminated *** 2024-09-21T03:47:11.680 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:47:11 smithi187 podman[70022]: 2024-09-21 03:47:11.318562893 +0000 UTC m=+2.989920610 container died 0471c51f1b1fd225e9cf93c67193b68e495f97ee97f68dc3293bcd36d744fe81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T03:47:11.941 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:47:11 smithi187 podman[70022]: 2024-09-21 03:47:11.763875401 +0000 UTC m=+3.435233159 container cleanup 0471c51f1b1fd225e9cf93c67193b68e495f97ee97f68dc3293bcd36d744fe81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-21T03:47:11.941 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:47:11 smithi187 bash[70022]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h 2024-09-21T03:47:12.190 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T03:47:12.211 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:47:11 smithi187 podman[70034]: 2024-09-21 03:47:11.940005299 +0000 UTC m=+0.617205891 container remove 0471c51f1b1fd225e9cf93c67193b68e495f97ee97f68dc3293bcd36d744fe81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-h, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T03:47:12.212 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:47:12 smithi187 systemd[1]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.h.service: Deactivated successfully. 2024-09-21T03:47:12.212 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:47:12 smithi187 systemd[1]: Stopped Ceph mds.h for 063b32f6-77c8-11ef-baf5-efdc52797490. 2024-09-21T03:47:12.212 INFO:journalctl@ceph.mds.h.smithi187.stdout:Sep 21 03:47:12 smithi187 systemd[1]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.h.service: Consumed 20.340s CPU time. 2024-09-21T03:47:12.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:47:12.222 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T03:47:12.222 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T03:47:12.223 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.k 2024-09-21T03:47:12.571 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:47:12 smithi187 systemd[1]: Stopping Ceph mds.k for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:47:12.930 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:47:12 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-k[62547]: 2024-09-21T03:47:12.642+0000 7fc3b2dc4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:47:12.930 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:47:12 smithi187 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-k[62547]: 2024-09-21T03:47:12.642+0000 7fc3b2dc4640 -1 mds.k *** got signal Terminated *** 2024-09-21T03:47:16.930 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:47:16 smithi187 podman[70144]: 2024-09-21 03:47:16.657017064 +0000 UTC m=+4.224035434 container died 3fbc25157879db5c90ca2f170f9fd194719943a67d21a1d5509096add98e7980 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T03:47:17.295 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:47:17 smithi187 podman[70144]: 2024-09-21 03:47:17.026096788 +0000 UTC m=+4.593115188 container cleanup 3fbc25157879db5c90ca2f170f9fd194719943a67d21a1d5509096add98e7980 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-k, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T03:47:17.295 INFO:journalctl@ceph.mds.k.smithi187.stdout:Sep 21 03:47:17 smithi187 bash[70144]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-k 2024-09-21T03:47:17.451 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T03:47:17.483 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:47:17.484 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T03:47:17.484 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T03:47:17.484 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.c 2024-09-21T03:47:17.875 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:47:17 smithi203 systemd[1]: Stopping Ceph mds.c for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:47:18.161 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:47:17 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c[64079]: 2024-09-21T03:47:17.872+0000 7f037da7a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:47:18.161 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:47:17 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c[64079]: 2024-09-21T03:47:17.872+0000 7f037da7a640 -1 mds.c *** got signal Terminated *** 2024-09-21T03:47:19.760 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:47:19 smithi203 podman[70081]: 2024-09-21 03:47:19.435390924 +0000 UTC m=+1.748322990 container died ad91d241c5b11163d5c67eb9af288f49293bf8c1c3b3915e1f9cbcca769bcb46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T03:47:20.051 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:47:19 smithi203 podman[70081]: 2024-09-21 03:47:19.864922242 +0000 UTC m=+2.177854294 container cleanup ad91d241c5b11163d5c67eb9af288f49293bf8c1c3b3915e1f9cbcca769bcb46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T03:47:20.051 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 21 03:47:19 smithi203 bash[70081]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-c 2024-09-21T03:47:20.188 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T03:47:20.224 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:47:20.224 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T03:47:20.224 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T03:47:20.224 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.f 2024-09-21T03:47:20.661 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:47:20 smithi203 systemd[1]: Stopping Ceph mds.f for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:47:20.661 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:47:20 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f[63809]: 2024-09-21T03:47:20.534+0000 7f8b34fa1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:47:20.661 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:47:20 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f[63809]: 2024-09-21T03:47:20.534+0000 7f8b34fa1640 -1 mds.f *** got signal Terminated *** 2024-09-21T03:47:23.650 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:47:23 smithi203 podman[70214]: 2024-09-21 03:47:23.372903688 +0000 UTC m=+2.929481923 container died 7ca03a4afbb15b99dc668c0d961cf2e2655609a2c9bc053868f28dc039fd0cb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:47:23.911 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:47:23 smithi203 podman[70214]: 2024-09-21 03:47:23.727597108 +0000 UTC m=+3.284175333 container cleanup 7ca03a4afbb15b99dc668c0d961cf2e2655609a2c9bc053868f28dc039fd0cb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T03:47:23.911 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 21 03:47:23 smithi203 bash[70214]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-f 2024-09-21T03:47:24.167 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T03:47:24.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:47:24.202 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T03:47:24.202 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T03:47:24.202 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.i 2024-09-21T03:47:24.525 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:47:24 smithi203 systemd[1]: Stopping Ceph mds.i for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:47:24.911 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:47:24 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i[64347]: 2024-09-21T03:47:24.523+0000 7fab1c971640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:47:24.911 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:47:24 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i[64347]: 2024-09-21T03:47:24.523+0000 7fab1c971640 -1 mds.i *** got signal Terminated *** 2024-09-21T03:47:27.662 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:47:27 smithi203 podman[70332]: 2024-09-21 03:47:27.372404376 +0000 UTC m=+2.948750732 container died 2b2a8383862e4093e8f7c4ca6f69435e764c657fd09a33d44791999b4c108349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T03:47:27.993 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:47:27 smithi203 podman[70332]: 2024-09-21 03:47:27.764986584 +0000 UTC m=+3.341332939 container cleanup 2b2a8383862e4093e8f7c4ca6f69435e764c657fd09a33d44791999b4c108349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T03:47:27.993 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 21 03:47:27 smithi203 bash[70332]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-i 2024-09-21T03:47:28.155 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T03:47:28.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:47:28.189 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T03:47:28.189 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T03:47:28.190 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.l 2024-09-21T03:47:28.521 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:47:28 smithi203 systemd[1]: Stopping Ceph mds.l for 063b32f6-77c8-11ef-baf5-efdc52797490... 2024-09-21T03:47:28.911 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:47:28 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-l[63537]: 2024-09-21T03:47:28.518+0000 7fbf64549640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:47:28.911 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:47:28 smithi203 ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-l[63537]: 2024-09-21T03:47:28.518+0000 7fbf64549640 -1 mds.l *** got signal Terminated *** 2024-09-21T03:47:31.355 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:47:31 smithi203 podman[70450]: 2024-09-21 03:47:31.061947466 +0000 UTC m=+2.649948621 container died 5e23b10be529d244764870c4fa452421c85589bab3b4497da9d0af2de52a6a11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-l, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:47:31.661 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:47:31 smithi203 podman[70450]: 2024-09-21 03:47:31.427997985 +0000 UTC m=+3.015999182 container cleanup 5e23b10be529d244764870c4fa452421c85589bab3b4497da9d0af2de52a6a11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T03:47:31.661 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 21 03:47:31 smithi203 bash[70450]: ceph-063b32f6-77c8-11ef-baf5-efdc52797490-mds-l 2024-09-21T03:47:31.909 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-063b32f6-77c8-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T03:47:31.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:47:31.942 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T03:47:31.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:47:32.144 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:47:34.642 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:47:34.852 INFO:teuthology.orchestra.run.smithi187.stdout:Deleting cluster with fsid: 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:47:37.407 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:47:37.610 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:47:40.030 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:47:40.063 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:47:40.096 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:47:40.123 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T03:47:40.125 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456/remote/smithi112/crash 2024-09-21T03:47:40.126 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:47:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:47:40.159 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:47:40.161 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456/remote/smithi187/crash 2024-09-21T03:47:40.162 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:47:40.214 INFO:teuthology.orchestra.run.smithi187.stderr:tar: /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:47:40.214 INFO:teuthology.orchestra.run.smithi187.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:47:40.216 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456/remote/smithi203/crash 2024-09-21T03:47:40.217 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:47:40.255 INFO:teuthology.orchestra.run.smithi203.stderr:tar: /var/lib/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:47:40.256 INFO:teuthology.orchestra.run.smithi203.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:47:40.256 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T03:47:40.257 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T03:47:40.293 INFO:tasks.cephadm:Compressing logs... 2024-09-21T03:47:40.294 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:47:40.336 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:47:40.339 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:47:40.361 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T03:47:40.361 INFO:teuthology.orchestra.run.smithi112.stderr:: No such file or directory 2024-09-21T03:47:40.361 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T03:47:40.361 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:47:40.362 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:47:40.363 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T03:47:40.363 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:47:40.363 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T03:47:40.365 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T03:47:40.365 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mon.a.log: 84.2% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log.gzgzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T03:47:40.366 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T03:47:40.366 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:47:40.366 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.6.log: /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.g.log: 90.2% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:47:40.367 INFO:teuthology.orchestra.run.smithi187.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T03:47:40.367 INFO:teuthology.orchestra.run.smithi187.stderr:: No such file or directory 2024-09-21T03:47:40.367 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T03:47:40.368 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T03:47:40.368 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:47:40.370 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T03:47:40.370 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T03:47:40.370 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T03:47:40.370 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.b.log: /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:47:40.370 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:47:40.374 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.4.log: /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:47:40.375 INFO:teuthology.orchestra.run.smithi203.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T03:47:40.375 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:47:40.375 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:47:40.375 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T03:47:40.375 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T03:47:40.375 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:47:40.375 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T03:47:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T03:47:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T03:47:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:47:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T03:47:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T03:47:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T03:47:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T03:47:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T03:47:40.376 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:47:40.379 INFO:teuthology.orchestra.run.smithi112.stderr: 95.0% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T03:47:40.379 INFO:teuthology.orchestra.run.smithi112.stderr: 95.1% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T03:47:40.379 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T03:47:40.379 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T03:47:40.380 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T03:47:40.380 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mgr.x.log: 90.9% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:47:40.380 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 86.8% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:47:40.380 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T03:47:40.380 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:47:40.381 INFO:teuthology.orchestra.run.smithi187.stderr: 95.0% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T03:47:40.381 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T03:47:40.381 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:47:40.381 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log: /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mgr.y.log: 81.4% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:47:40.381 INFO:teuthology.orchestra.run.smithi187.stderr: 90.9% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:47:40.381 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T03:47:40.382 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T03:47:40.382 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.3.log: 95.0% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T03:47:40.385 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.h.log: 86.9% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:47:40.385 INFO:teuthology.orchestra.run.smithi112.stderr: 95.1% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T03:47:40.386 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T03:47:40.386 INFO:teuthology.orchestra.run.smithi187.stderr: 92.8% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T03:47:40.386 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:47:40.393 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/cephadm.log: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:47:40.412 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:47:40.497 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.9% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T03:47:40.506 INFO:teuthology.orchestra.run.smithi203.stderr: 81.4% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:47:40.511 INFO:teuthology.orchestra.run.smithi203.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:47:40.585 INFO:teuthology.orchestra.run.smithi203.stderr: 95.0% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T03:47:40.611 INFO:teuthology.orchestra.run.smithi203.stderr: 90.4% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:47:40.637 INFO:teuthology.orchestra.run.smithi203.stderr: 90.9% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:47:40.669 INFO:teuthology.orchestra.run.smithi112.stderr: 90.5% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T03:47:40.686 INFO:teuthology.orchestra.run.smithi203.stderr: 86.9% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:47:41.353 INFO:teuthology.orchestra.run.smithi112.stderr: 90.4% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T03:47:42.093 INFO:teuthology.orchestra.run.smithi187.stderr: 93.4% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T03:47:47.643 INFO:teuthology.orchestra.run.smithi187.stderr: 95.3% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T03:47:48.357 INFO:teuthology.orchestra.run.smithi112.stderr: 94.3% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T03:47:48.973 INFO:teuthology.orchestra.run.smithi112.stderr: 93.9% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T03:47:49.830 INFO:teuthology.orchestra.run.smithi187.stderr: 94.3% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T03:47:50.068 INFO:teuthology.orchestra.run.smithi203.stderr: 91.6% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T03:47:50.272 INFO:teuthology.orchestra.run.smithi187.stderr: 95.4% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T03:47:50.852 INFO:teuthology.orchestra.run.smithi112.stderr: 94.2% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T03:47:51.636 INFO:teuthology.orchestra.run.smithi112.stderr: 94.0% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T03:47:51.638 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T03:47:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m11.288s 2024-09-21T03:47:51.638 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m38.174s 2024-09-21T03:47:51.639 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m1.478s 2024-09-21T03:47:51.831 INFO:teuthology.orchestra.run.smithi187.stderr: 94.0% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T03:47:53.643 INFO:teuthology.orchestra.run.smithi187.stderr: 94.2% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T03:47:53.925 INFO:teuthology.orchestra.run.smithi187.stderr: 94.2% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T03:48:35.213 INFO:teuthology.orchestra.run.smithi203.stderr: 94.3% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T03:49:03.776 INFO:teuthology.orchestra.run.smithi203.stderr: 94.1% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T03:49:09.792 INFO:teuthology.orchestra.run.smithi203.stderr: 95.5% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T03:49:20.859 INFO:teuthology.orchestra.run.smithi187.stderr: 93.3% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T03:49:20.861 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-09-21T03:49:20.861 INFO:teuthology.orchestra.run.smithi187.stderr:real 1m40.508s 2024-09-21T03:49:20.861 INFO:teuthology.orchestra.run.smithi187.stderr:user 2m41.504s 2024-09-21T03:49:20.861 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m5.317s 2024-09-21T03:49:35.891 INFO:teuthology.orchestra.run.smithi203.stderr: 94.3% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T03:49:51.212 INFO:teuthology.orchestra.run.smithi203.stderr: 94.3% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T03:55:08.832 INFO:teuthology.orchestra.run.smithi203.stderr: 93.4% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T03:55:55.939 INFO:teuthology.orchestra.run.smithi203.stderr: 93.3% -- replaced with /var/log/ceph/063b32f6-77c8-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T03:55:55.941 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-09-21T03:55:55.941 INFO:teuthology.orchestra.run.smithi203.stderr:real 8m15.588s 2024-09-21T03:55:55.942 INFO:teuthology.orchestra.run.smithi203.stderr:user 6m12.761s 2024-09-21T03:55:55.942 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m15.152s 2024-09-21T03:55:55.942 INFO:tasks.cephadm:Archiving logs... 2024-09-21T03:55:55.943 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456/remote/smithi112/log 2024-09-21T03:55:55.949 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:56:01.542 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456/remote/smithi187/log 2024-09-21T03:56:01.543 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:56:22.465 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456/remote/smithi203/log 2024-09-21T03:56:22.466 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:57:16.251 INFO:tasks.cephadm:Removing cluster... 2024-09-21T03:57:16.252 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 --force 2024-09-21T03:57:16.454 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:57:16.764 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 --force 2024-09-21T03:57:16.968 INFO:teuthology.orchestra.run.smithi187.stdout:Deleting cluster with fsid: 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:57:17.412 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 063b32f6-77c8-11ef-baf5-efdc52797490 --force 2024-09-21T03:57:17.758 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 063b32f6-77c8-11ef-baf5-efdc52797490 2024-09-21T03:57:18.391 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T03:57:18.391 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:57:18.409 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:57:18.429 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:57:18.445 INFO:tasks.cephadm:Teardown complete 2024-09-21T03:57:18.445 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T03:57:18.455 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T03:57:18.456 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:57:18.458 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:57:18.471 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:57:18.591 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T03:57:18.603 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T03:57:18.604 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:57:18.606 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:57:18.609 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:57:18.619 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:57:18.623 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:57:18.624 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:57:18.624 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-21T03:57:18.624 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 -418us[ -413us] +/- 38ms 2024-09-21T03:57:18.624 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 +951us[ +951us] +/- 66ms 2024-09-21T03:57:18.624 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -261us[ -256us] +/- 68ms 2024-09-21T03:57:18.624 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:57:18.625 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:57:18.629 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:57:18.629 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-09-21T03:57:18.629 INFO:teuthology.orchestra.run.smithi187.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 -402us[ -375us] +/- 38ms 2024-09-21T03:57:18.629 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 +976us[+1003us] +/- 66ms 2024-09-21T03:57:18.630 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -249us[ -223us] +/- 68ms 2024-09-21T03:57:18.630 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:57:18.661 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:57:18.661 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-09-21T03:57:18.661 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 -424us[ -432us] +/- 38ms 2024-09-21T03:57:18.661 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 +941us[ +941us] +/- 66ms 2024-09-21T03:57:18.661 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -275us[ -283us] +/- 68ms 2024-09-21T03:57:18.661 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:57:18.662 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T03:57:18.672 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T03:57:18.672 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T03:57:18.682 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:57:18.762 DEBUG:teuthology.orchestra.run.smithi187:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:57:18.835 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:57:18.921 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:57:18.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:57:18.950 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:57:18.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:57:18.980 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:57:19.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:57:19.009 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T03:57:19.035 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T03:57:19.063 INFO:teuthology.task.internal:Duration was 2741.087953 seconds 2024-09-21T03:57:19.064 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T03:57:19.092 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T03:57:19.092 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:57:19.095 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:57:19.098 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:57:19.131 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:57:19.139 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:57:19.164 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:57:19.478 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T03:57:19.478 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T03:57:19.478 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:57:19.498 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2024-09-21T03:57:19.499 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:57:19.544 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-09-21T03:57:19.544 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:57:19.578 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T03:57:19.578 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:57:19.580 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:57:19.587 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:57:19.658 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T03:57:19.658 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:57:19.660 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:57:19.663 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:57:20.989 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T03:57:21.001 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T03:57:21.001 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:57:21.026 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:57:21.057 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:57:21.088 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T03:57:21.099 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:57:21.102 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:57:21.105 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:57:21.122 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-09-21T03:57:21.129 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2024-09-21T03:57:21.167 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-09-21T03:57:21.188 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:57:21.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:57:21.207 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:57:21.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:57:21.224 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:57:21.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:57:21.245 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T03:57:21.260 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T03:57:21.262 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456/remote/smithi112 2024-09-21T03:57:21.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:57:21.332 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456/remote/smithi187 2024-09-21T03:57:21.332 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:57:21.374 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913456/remote/smithi203 2024-09-21T03:57:21.375 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:57:21.413 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T03:57:21.413 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:57:21.416 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:57:21.418 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:57:21.464 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T03:57:21.478 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T03:57:21.478 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T03:57:21.503 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T03:57:21.504 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:57:21.506 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:57:21.510 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:57:21.519 INFO:teuthology.orchestra.run.smithi112.stdout: 265701 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:57 /home/ubuntu/cephtest 2024-09-21T03:57:21.523 INFO:teuthology.orchestra.run.smithi187.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:57 /home/ubuntu/cephtest 2024-09-21T03:57:21.526 INFO:teuthology.orchestra.run.smithi203.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:57 /home/ubuntu/cephtest 2024-09-21T03:57:21.527 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T03:57:21.613 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T03:57:21.647 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} duration: 2741.0879526138306 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi112 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=2d4b4ffd0efb4be58a5a7ea219dab69f status: fail success: false 2024-09-21T03:57:21.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:57:21.733 INFO:teuthology.run:FAIL